When FileInfo was imported in https://github.com/mono/mono/commit/bdd0de7ee52d9675dada0d19985b9daf4edab5bc the check in Delete()
got removed that raised an UnauthorizedAccessException when the FileInfo object holds a path to a directory instead of a file.
It happened to pass the MonoTests.System.IO.FileInfoTest.Delete_Directory test on OSX because unlink returns EPERM there
which got turned into the expected UnauthorizedAccessException, but on Linux it returns EISDIR so the test would fail.
Fix is to put the explicit check for directory back.
NativeMethods.GetExitCodeProcess would inherently be broken on Unix platforms for non-child processes as it can't succesfully get the exit code. This would in turn break Process.HasExited, which should not break, as it is possible to check if a non-child process is alive or dead.
It's not easy to trigger but following repro works fairly reliably
using System;
using System.Threading;
class MainClass
{
public static void Main (string [] args)
{
var r = new Random ();
for (int i = 0, total = 10000000; i < total; ++i) {
if (i % 50 == 0)
Console.WriteLine ("{0}/{1}", i, total);
var c1 = new CancellationTokenSource ();
var wh = c1.Token.WaitHandle;
GC.Collect ();
c1.CancelAfter (r.Next(5));
c1.Dispose ();
}
}
}
It crashes like
Unhandled Exception:
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Threading.ManualResetEvent'.
at System.Threading.WaitHandle.CheckDisposed () <0x1f95be0 + 0x0005c> in <filename unknown>:0
at System.Threading.EventWaitHandle.Set () <0x1f8d320 + 0x0007d> in <filename unknown>:0
at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle:Set ()
at System.Threading.CancellationTokenSource.NotifyCancellation (Boolean throwOnFirstException) <0x1d92c70 + 0x00084> in <filename unknown>:0
at System.Threading.CancellationTokenSource.Cancel (Boolean throwOnFirstException) <0x1d92330 + 0x00030> in <filename unknown>:0
at System.Threading.CancellationTokenSource.Cancel () <0x1d92300 + 0x00025> in <filename unknown>:0
at System.Threading.CancellationTokenSource.TimerCallbackLogic (System.Object obj) <0x1d925c0 + 0x0005d> in <filename unknown>:0
This change is fixing the DateTime parser in cases of date/time strings for cultures which has the date and time separators are same. This happen with Norway, Serbia and Finland cultures
[tfs-changeset: 1513801]
The race condition would manifest in the following code:
```
for (int i = 0, total = 100000; i < total; ++i) {
if (i % 50 == 0)
Console.WriteLine ("{0}/{1}", i, total);
var c1 = new CancellationTokenSource ();
var wh = c1.Token.WaitHandle;
c1.CancelAfter (1);
Thread.Sleep (1);
c1.Dispose ();
}
```
And we would observe the following exception:
```
Unhandled Exception: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Threading.ManualResetEvent'.
at System.Threading.WaitHandle.CheckDisposed () [0x00016] in /Users/builder/data/lanes/1196/33e00ac6/source/mono/mcs/class/corlib/System.Threading/WaitHandle.cs:426
at System.Threading.EventWaitHandle.Set () [0x0000c] in /Users/builder/data/lanes/1196/33e00ac6/source/mono/mcs/class/corlib/System.Threading/EventWaitHandle.cs:205
at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle:Set ()
at System.Threading.CancellationTokenSource.NotifyCancellation (Boolean throwOnFirstException) [0x00051] in /Users/builder/data/lanes/1196/33e00ac6/source/mono/external/referencesource/mscorlib/system/threading/CancellationTokenSource.cs:723
at System.Threading.CancellationTokenSource.Cancel (Boolean throwOnFirstException) [0x00006] in /Users/builder/data/lanes/1196/33e00ac6/source/mono/external/referencesource/mscorlib/system/threading/CancellationTokenSource.cs:409
at System.Threading.CancellationTokenSource.Cancel () <0x7d75ada8 + 0x00017> in <filename unknown>:0
at System.Threading.CancellationTokenSource.TimerCallbackLogic (System.Object obj) [0x00012] in /Users/builder/data/lanes/1196/33e00ac6/source/mono/external/referencesource/mscorlib/system/threading/CancellationTokenSource.cs:538
```
This would be a race condition between the TimerCallbackLogic call to Cancel and the call to Dispose on another thread. You could trigger it more reliably by putting a Thread.Sleep at CancellationTokenSource.cs:599.
Mono now uses ExceptionHandlingClauseOptions from methodbody.cs and ExceptionHandler from methobuilder.cs.
Remaining types from thoses files were ignored with ifdef.
Mono's implementation of RSAOAEPKeyExchangeDeformatter did the throwing
but we're now using MS implementation of it - so it make sense to move
the throw (it's needed) in the utils.cs file
MS implementation is incorrect and the .NET framework never hits this code
path as it's RSA implementation calls CryptoAPI (bypassing those managed
classes).
mscorlib/system/security/cryptography/pkcs1maskgenerationmethod.cs
-> use Mono MGF1 implementation
mscorlib/system/security/cryptography/utils.cs
-> remove duplicated code from Mono's PKCS1
-> fix RsaOaepEncrypt and RsaOaepDecrypt to call Mono code
* `ServerCertValidationCallback': Enable this for Mono.
* `ServicePointManager': Use 'MONO_FEATURE_WEB_STACK' here, we'll
eventually bring this together with the web stack.
We want the helper APIs at the top of the file, but not the actual
'ServicePointManager' class.
This is the same issue as in https://github.com/mono/referencesource/pull/7,
i.e. there's an overflow when casting a too large value from double->long
and the referencesource implementation relies on this being a large negative
value in order for the ArgumentOutOfRangeException to be properly thrown.
As Mono's runtime on ARM in this case returns -1, the exception isn't thrown.
The fix is to introduce an explicit overflow check, catch an eventual
OverflowException and raise the proper exception instead. This can be JIT'd
to better code than checking if the value fits into long every time.
It fixes a couple of DateTime unit tests that currently fail on ARM.
When setting Interval = double.MaxValue like in the TimerTest.Interval_TooHigh_ThrowOnEnabled and
TimerTest.Interval_TooHigh_Enabled_Throw Mono unit tests there was a regression on ARM in that
the ArgumentOutOfRangeException wasn't thrown as expected.
Turns out, on ARM the call to (int)Math.Ceiling(double.MaxValue) as used in UpdateTimer() and Enabled setter
doesn't produce -2147483648 (i.e. Int32.MinValue) like on x86/amd64 but rather 2147483647 (i.e. Int32.MaxValue),
which is perfectly valid as the cast overflows and the result is an undefined value.
This means that the value passed to the underlying System.Threading.Timer is not negative
and no ArgumentOutOfRangeException is thrown there.
The fix is to properly check for interval>Int32.MaxValue like is done already in the constructor.
I pulled that logic out and introduced a separate method CalculateRoundedInterval().
Note: The thrown exception changes from an ArgumentOutOfRangeException inside of System.Threading.Timer
to an ArgumentException inside of CalculateRoundedInterval().
I think this has minimal impact as no-one should rely on this exception anyway and it actually matches
what MSDN (https://msdn.microsoft.com/en-us/library/system.timers.timer.interval(v=vs.110).aspx)
says should happen in this case (even though the MS.NET 4.5 implementation doesn't comply).
Mono's implementation also did the same before: https://github.com/mono/mono/blob/4cb3f77b4bbf703b1cda59db2f5aee206e35d31a/mcs/class/System/System.Timers/Timer.cs#L95-L96
Even after we imported System.Xml referencesource, XmlSerializer is still
based on existing code. And we have some special trick for XElement
serialization support in System.Xml, by allowing access from
System.Xml.Linq.dll to use special type converter attribute.
It does not exist in referencesource and thus we lost serializability on
XElement, so we need to add the missing code to bring it back.
The former condition is to disable <msxsl:script> element.
The latter is to disable XslCompiledTransform and co (though they are
mostly removed at *.dll.sources level).
Somewhere in System.Xml requires NameTable to be canonically reused when
deserializing XmlSchema, so that certain deserialization part can check
only reference equality.
Since we have hybrid System.Xml implementation for XmlSerializer, this
is not completely achieved and with reference comparison it causes some
schema deserialization failure. To avoid this problem, just disable
reference comparison. It should not affect outside XmlSchema serialization.
DISABLE_CAS_USE is required to avoid NotImplementedExceptions in CAS land.
MONO_HYBRID_XML is to preserve Mono's XmlSerializer which also allows
interpreter-based XML serialization (as opposed to run-time code generation
based XML serialization, which is not doable in iOS).
Though some Xml.Serialization classes are useful even with mono
implementation, so we partially import them (namely schema importer).
This will let us have both the Mono and the Microsoft ThreadPool at the same time.
To switch to the Microsoft one, set the MONO_THREADPOOL environment variable to "microsoft". It will default to the Mono one.
All changes are conditional to MONO_FEATURE_NEW_TLS.
In addition to that, all changes to their code are also conditional to 'MONO'; this
mostly applies to SecureChannel, which needed several changes to replace their native
SSPI implementation with Mono's new managed version.
* Net/SecureProtocols/SslStream.cs: Make partial and comment out 'SslStream.TransportContext'.
* Net/SecureProtocols/SslState.cs: Make partial and add new internal constructor; pass the
'SSPIConfiguration' argument down to SecureChannel's .ctor.
* Net/SecureProtocols/_SslStream.cs: Make partial.
* Net/SecureProtocols/_SecureChannel.cs: Add Mono-specific SSPI APIs.
* Net/_SslSessionCache.cs: Make conditional.
- SafeRegistryHandle.ReleaseHandle is only called on code that is ignored on Mono.
- Mono is using AppDomain.FriendlyName directly instead of UnsafeNativeMethods.GetModuleFileName.
* Builds without [MethodImpl(MethodImplOptions.AggressiveInlining)]
when target framework version is lower than 4.5.
* Use Thread.Volatile instead of Volatile.Read when target framework
version is lower than 4.5.
* Change location of internal Array constants. Mono runtime cannot handle
array type with constants.
* Removes some warnings from the build for unused fields.
This patch does the following things:
* Adds support to build without Code Access Security, by using the new
define DISABLE_CAS_USE. The reason is that this is both slow, and
Mono does not actually offer a Code Access Secured environment. So
avoid paying the price.
* Removes some warnings from the build for unused variables.
* Allows the code to be bootstrapped with a 2.0 compiler (which Mono
does during the bootstrap phase currently).
Socket.AcceptAsync should use SocketAsyncEventArgs.AcceptSocket
Socket.AcceptAsync shouldn't create new socket if SocketAsyncEventArgs.AcceptSocket isn't null.
The MoveTo target is in the current working directory instead of in the temp dir, so it runs into permission issues.
Use the temp dir instead. Fixes part 2 of #40624
We don't have .NEt 4.6 installed on all of our build machines, to
running mcs.exe on Windows fails, as it was built with .NET 4.6.
Instead, we will run mcs.exe via Mono on Windows (which is what happens
on OSX, although there the low-level Process class handles it).
To accomplish this, we run mcs.bat instead of mcs.exe.
The child process would simpky execute ls, then exiting right away. Writing to its stdin would then crash with "EPIPE: An attempt is made to write to a pipe that is not open for reading by any process." as the child process exited.
Some serialization attributes are unlikely to have direct reference,
from IL, but will be needed at runtime to correctly serialize (or
deserialize) objects.
* OnDeserializedAttribute
* OnDeserializingAttribute
* OnSerializedAttribute
* OnSerializingAttribute
This, along with aad627c5e8, fixed bug
40574. The later part will have new unit tests in XI.
references:
https://bugzilla.xamarin.com/show_bug.cgi?id=40574
The problem is that private fields in described types doesn't start from _ prefix, but it should be so according to reference source.
That's way MONO can't deserialize that types correctly if source stream was created by MS .NET binary serializer.
This change is released under the MIT license.
Switching maccore to use mono-4.5.0-branch introduced this bug where
AesManaged is not available in System.Core for both tvOS and watchOS
profiles (iOS is fine).
This results in unit tests failures on bots building maccore/master.
This fix the problem by importing the "main" monotouch_System.Core.
dll.sources into the other profiles - since they all share the same
feature set (in this case) and make it less likely to forget something
else in the future.
reference:
https://bugzilla.xamarin.com/show_bug.cgi?id=40570
[sgen] Add regression test for the interaction between the bridge and handles.
The bridge code was triggering handles to be considered deallocated thus leading
them to be overwritten.
[corlib] Ifdef out calls to SynchronizationAttribute on monotouch
When WaitHandle was imported from referencesource in f6fa8f0, the code was refactored and basically all WaitHandle.Wait* calls end up in WaitOneNative or WaitMultiple now.
On monotouch those methods threw a NotSupportedException when exitContext == true even before the import, so this behavior got inherited to some other methods that didn't throw after the exception was removed from them in 93e31cb. This broke tests in maccore and other code that passed exitContext=true.
To fix this, we just ifdef out the calls to SynchronizationAttribute on monotouch since they don't make sense there anyway since remoting isn't available. The commit that initially added the exception in 2ce9fc1 only did that to make the behavior consistent when the linker is enabled. By removing the calls we get the same result.
Do the same for Monitor so we're consistent.
commit message from @akoeplinger
[verifier] Do method visibility checks for virtual final methods. Enable type visibility for all methods. Fixes#5644.
I cannot repro the issue mentioned in the comments of mono_method_can_access_method.
* FIXME:
* with generics calls to explicit interface implementations can be expressed
* directly: the method is private, but we must allow it. This may be opening
* a hole or the generics code should handle this differently.
* Maybe just ensure the interface type is public.
@monojenkins merge
This particular piece of documentation is added manually, because it's not in the regular BCL library, but is provided by Xamarin on the iOS platform. Because it was missing an entry in the index file, it was excluded from the published documentation when doing mdoc assemble.
When WaitHandle was imported from referencesource in f6fa8f0, the code was refactored and basically all WaitHandle.Wait* calls end up in WaitOneNative or WaitMultiple now.
On monotouch those methods threw a NotSupportedException when exitContext == true even before the import, so this behavior got inherited to some other methods that didn't throw after the exception was removed from them in 93e31cb. This broke tests in maccore and other code that passed exitContext=true.
To fix this, we just ifdef out the calls to SynchronizationAttribute on monotouch since they don't make sense there anyway since remoting isn't available. The commit that initially added the exception in 2ce9fc1 only did that to make the behavior consistent when the linker is enabled. By removing the calls we get the same result.
Do the same for Monitor so we're consistent.
I cannot repro the issue mentioned in the comments of mono_method_can_access_method.
* FIXME:
* with generics calls to explicit interface implementations can be expressed
* directly: the method is private, but we must allow it. This may be opening
* a hole or the generics code should handle this differently.
* Maybe just ensure the interface type is public.
Remove calls to mono_lock_free_queue_node_init for nodes that just came out of mono_lock_free_queue_dequeue.
That call is superfluous as dequeue will set the returned node 'next' field to INVALID_NEXT, which is the
same thing the mono_lock_free_queue_node_init call did.
[utils] Fix MONO_SIGNAL_USE_UCONTEXT_T not being defined on tvOS simulator
In the maccore tvossimulator tests we were seeing the following assertion with the Mono 4.5 preview:
```
error: * Assertion: should not be reached at /Users/builder/data/lanes/1381/0d6923b4/source/maccore/external/mono/mono/utils/mono-context.c:194
0 dont link 0x000000010175702d mono_handle_native_sigsegv + 253
1 libsystem_platform.dylib 0x000000010a0cfeaa _sigtramp + 26
2 ??? 0x0000000001bfc4db 0x0 + 29344987
3 libsystem_c.dylib 0x0000000109e56cbc abort + 129
4 dont link 0x00000001018d1503 _ZL12log_callbackPKcS0_S0_iPv + 67
5 dont link 0x00000001018c26d0 monoeg_assertion_message + 192
6 dont link 0x00000001018af6de mono_sigctx_to_monoctx + 30
7 dont link 0x000000010183f473 sgen_suspend_thread + 323
8 dont link 0x000000010183f58e sgen_thread_handshake + 126
9 dont link 0x000000010183fb20 sgen_client_stop_world + 1296
10 dont link 0x0000000101871283 sgen_stop_world + 67
11 dont link 0x0000000101870d43 sgen_perform_collection + 99
12 dont link 0x0000000101872576 sgen_gc_collect + 102
13 dont link 0x00000001018d1886 _ZL7pump_gcPv + 54
14 libsystem_pthread.dylib 0x000000010a0b7c13 _pthread_body + 131
15 libsystem_pthread.dylib 0x000000010a0b7b90 _pthread_body + 0
16 libsystem_pthread.dylib 0x000000010a0b5375 thread_start + 13
```
This was caused by a4e257a07f, the issue is that the check was changed from UCONTEXT_REG_RAX to MONO_SIGNAL_USE_UCONTEXT_T, but that is only available when HAVE_SIGACTION is defined which it is not in this case.
Talked with Zoltan, fixing this by enabling MONO_SIGNAL_USE_UCONTEXT_T on all apple platforms for now.
/cc @vargaz @xmcclure
@monojenkins merge
1. Mark mono_array_clone external only. Runtime should use
mono_array_clone_checked.
2. Add MonoError* arg to mono_array_clone_in_domain.
3. Separate ves_icall_System_Array_Clone
Mark it external only. Runtime should use
mono_property_set_value_checked.
Note: mono_property_set_value_checked always catches exceptions from
managed code and threads them out via MonoError.
We need to make sure all external references have been resolved so that we
don't invoke the dynamic linker in the mono_sample_hit () callback which is
called in async-signal context.
There is a slight problem when we're using CLOCK_PROCESS_CPUTIME_ID: If we're
shutting down and there's largely no activity in the process other than waiting
for the sampler thread to shut down, it can take upwards of 20 seconds
(depending on a lot of factors) for us to shut down because the sleep
progresses very slowly as a result of the low CPU activity.
We fix this by repeatedly sending the profiler signal to the sampler thread in
order to interrupt the sleep.
We do not need to do this on platforms where we use a regular sleep based on a
monotonic clock. The sleep will return in a reasonable amount of time in those
cases.
The old code had quite a few problems:
1. It would result in stack corruption under heavy load, somehow. This typically
resulted in an assertion when starting the unwinding process because the
unwinding flags would be inconsistent.
2. It sometimes wrote complete garbage into the statistical buffers.
3. As a result of using the helper thread to assemble collected samples into log
buffers, it did not work on platforms where the helper thread is disabled.
4. Performance was poor under heavy load because waking up the helper thread to
assemble events would always trigger a counters sample + dump.
5. The code was very complicated and hard to understand/maintain.
(Points (1) and (2) can probably be attributed to point (5).)
In the new sampling world order, we use a lock-free allocator and a lock-free
reuse queue to acquire memory for sample hit events in the sample hit callback.
We then fill it with the info we collect during the async stack walk. Finally,
we ship it off to a separate dumper thread (via another lock-free queue) where
it gets turned into a log buffer and then shipped off to the writer thread.
This new approach appears very stable; I've yet to make it crash in any of the
stress tests I typically run. In a heavy workload, the helper thread's CPU
usage was previously around 25-30%. It is now down to basically 0% while the
dumper thread is only around 15%. CPU usage by threads that are collecting
samples is a bit higher than before, but only until the reuse queue converges
on a size that is big enough to satisfy all sample event requests. At that
point, no more allocations from the lock-free allocator happen, and performance
improves.
Our previous default (1000 Hz) was quite extreme and resulted in very high
overhead from the profiler for moderate workloads.
Also, fix the profiler to actually treat the 'freq' argument as a frequency
when passing it to the runtime. It was being incorrectly treated as a sleep
interval.
In the maccore tvossimulator tests we were seeing the following assertion with the Mono 4.5 preview:
```
error: * Assertion: should not be reached at /Users/builder/data/lanes/1381/0d6923b4/source/maccore/external/mono/mono/utils/mono-context.c:194
0 dont link 0x000000010175702d mono_handle_native_sigsegv + 253
1 libsystem_platform.dylib 0x000000010a0cfeaa _sigtramp + 26
2 ??? 0x0000000001bfc4db 0x0 + 29344987
3 libsystem_c.dylib 0x0000000109e56cbc abort + 129
4 dont link 0x00000001018d1503 _ZL12log_callbackPKcS0_S0_iPv + 67
5 dont link 0x00000001018c26d0 monoeg_assertion_message + 192
6 dont link 0x00000001018af6de mono_sigctx_to_monoctx + 30
7 dont link 0x000000010183f473 sgen_suspend_thread + 323
8 dont link 0x000000010183f58e sgen_thread_handshake + 126
9 dont link 0x000000010183fb20 sgen_client_stop_world + 1296
10 dont link 0x0000000101871283 sgen_stop_world + 67
11 dont link 0x0000000101870d43 sgen_perform_collection + 99
12 dont link 0x0000000101872576 sgen_gc_collect + 102
13 dont link 0x00000001018d1886 _ZL7pump_gcPv + 54
14 libsystem_pthread.dylib 0x000000010a0b7c13 _pthread_body + 131
15 libsystem_pthread.dylib 0x000000010a0b7b90 _pthread_body + 0
16 libsystem_pthread.dylib 0x000000010a0b5375 thread_start + 13
```
This was caused by a4e257a07f, the issue is that the check was changed from UCONTEXT_REG_RAX to MONO_SIGNAL_USE_UCONTEXT_T, but that is only available when HAVE_SIGACTION is defined which it is not in this case.
Discussed with Zoltan, fixing this by enabling MONO_SIGNAL_USE_UCONTEXT_T on all apple platforms for now.
1. MonoError-ize mono_remoting_wrapper icall.
This is a bit this method is allowed to use
mono_error_raise_exception: it is called from a
MONO_WRAPPER_REMOTING_INVOKE method which is considered a "protected
wrapper" in which the usual pending exception interruption check is a no-op.
2. Use mono_set_pending_exception in mono_marshal_set_domain_by_id icall
fixup f5dc934
The following test would trigger a SIGSEGV during a GC collection:
```
object _Lock;
C5.HashedLinkedList<byte[]> _Set;
List<byte[]> _Candidates;
void Run ()
{
_Lock = new object ();
_Set = new C5.HashedLinkedList<byte[]>(new Core.CommonUtils.ByteArrayComparer());
BuildList ();
for (var i = 0; i < 10; ++i) {
Task.Run (() => WorkOnSet ());
}
}
void BuildList() {
_Candidates = new List<byte[]>();
Random r = new Random();
for(var i = 0; i < 20; ++i) {
var key = new byte[16];
r.NextBytes(key);
_Candidates.Add(key);
}
}
void WorkOnSet() {
Random r = new Random();
for (;;) {
byte[] src = _Candidates [r.Next (_Candidates.Count)];
byte[] key = new byte[src.Length];
for (int i = 0; i < src.Length; ++i)
key [i] = src [i];
lock(_Lock) {
if(r.Next (2) != 0)
_Set.Add(key);
else
_Set.Remove(key);
}
}
}
```
The binary protocol just before the crash would be as follow:
```
collection_requested generation 0 requested_size 4096 force false
collection_begin index 14 generation 0
copy from 0x3150658 to 0x47e0e40 vtable 0x189f1780 size 24
gray_enqueue queue 0xa4a2d0 cursor 0x3b68178 value 0x47e0e40
gray_dequeue queue 0xa4a2d0 cursor 0x3b68178 value 0x47e0e40
scan_begin obj 0x47e0e40 vtable 0x189f1780 size 24
scan_process_reference obj 0x47e0e40 ptr 0x47e0e48 value 0x3150618
scan_process_reference obj 0x47e0e40 ptr 0x47e0e4c value 0x3150638
scan_process_reference obj 0x47e0e40 ptr 0x47e0e50 value 0x0 <----- here the value is 0x0
scan_process_reference obj 0x47e0db0 ptr 0x47e0dc0 value 0x47e0e40
collection_end 14 generation 0 scanned 0 unique 0 0.00%
wbarrier ptr 0x3af8710 value 0x47e0e40 value_vtable 0x189f1780
collection_requested generation 0 requested_size 4096 force false
collection_begin index 15 generation 0
collection_end 15 generation 0 scanned 0 unique 0 0.00%
collection_requested generation 0 requested_size 4096 force false
collection_begin index 16 generation 0
card_scan start 0x47e0df8 size 520
scan_begin obj 0x47e0e40 vtable 0x189f1780 size 24
scan_process_reference obj 0x47e0e40 ptr 0x47e0e48 value 0x47b4d40
scan_process_reference obj 0x47e0e40 ptr 0x47e0e4c value 0x47b03c0
scan_process_reference obj 0x47e0e40 ptr 0x47e0e50 value 0x3100800 <----- here the value is 0x3100800, but no WB in between
```
Object at 0x47e0e40 is a C5.HashSet<T>.Bucket with T a C5.KeyValuePair<byte[],C5.HashedLinkedList<byte[]>.Node>. The memory layout of C5.HashSet<T>.Bucket would be as follow:
class Bucket {
item: offset 8 size 8
overflow: offset 16 size 4
hashval: offset 20 size 4
}
So the bug would manifest itself as a missing write barrier when writing to Bucket.overflow, leading to the premature sweeping of a Bucket object. The Bucket.overflow value would then point to the middle of an object, triggering a SIGSEGV when trying to access its vtable.
When running basic make check in mono/mini there are some hidden errors in the check-seq-points test. Current test compares output between JIT and AOT compile assemblies but doesn't pay attention to the result of the as and ld calls building the AOT:ed modules. It turns out that the linkage of the AOT compiled module on win32 x86 actually fails due to naming of symbols included in .def or .globl assembler expressions. The fix makes sure that local function symbols or global symbol definitions follows the cdecl win32 x86 convention and are prefixed with _. The fix tries to resolve this using two different strategies. Since there are many code paths in the AOT compiler emitting local/global symbols + referencing the emitted symbols, it doesn't look like there is one place to fix this (unless rename the symbol altogether and that could cause regressions elsewhere).
First, local function symbols are emitted using .def expression when used as debug symbols. By prefixing debug symbols with _ will resolve most of the local function symbols. By mangling the debug symbols we can fix both emitted symbols and references to these symbols in two location (get_debug_sym, get_plt_entry_debug_sym).
Second, in order to make sure all other symbols (local/global) ending up in .def or .globl expressions are correctly mangled, the functions to emit local/global symbols in aot-compiler.c have been extended to check mangling on passed in symbols before emitting. If passed in symbol doesn't follow the name mangling rule, the symbols will be mangled and the mangled symbol will be emitted together with a label to the mangled symbol. This is done to make sure we have a correct label for the emitted symbol that can be used by the linker. Since the rest of the code is unaware of the mangled name, it will continue reference the unmangled name and since there is already a label to the unmangled symbol, the mangling will be transparent for the rest of the code.
NOTE, the fix only applies to windows X86 build, other configurations are not affected by this issue.
This PR also includes a fix detected on windows when debugging and fixing above issue. The AOT compiler tries to rename linked module to the final target name, for example from basic.exe.dll.tmp to basic.exe.dll. If the file already exist, the rename will fail on windows causing a silent failure in the AOT compiler and this will "leak" the tmp file + ending up with the old basic.exe.dll. The fix is to check result of the rename function call and if it fails with "file already exist error", unlink target and redo rename operation. The rename can still fail due to other reasons ending up in the same problem, but to make a good fix, the error handling of the method needs to be revisited in order to properly clean up tmp files and allocations in error cases. This will be handled in a different PR.
Fix undefined behaviour in left-shift.
This PR fixes a left-shift that relied on undefined behaviour.
Clang emitted the following warning (prior to this patch):
```
sgen-descriptor.c:189:54: warning: shifting a negative signed value is undefined [-Wshift-negative-value]
return desc | VECTOR_SUBTYPE_REFS | ((gssize)(-1) << 16);
~~~~~~~~~~~~ ^
```
Mark mono_store_remote_field_new external only.
Runtime should use mono_store_remote_field_new_checked.
Change mono_marshal_get_stfld_remote_wrapper to use
mono_store_remote_field_new_icall instead of a native call.
Add mono_load_remote_field_new_checked. Mark mono_load_remote_field_new
external only. Add icall mono_load_remote_field_new_icall and use it in
remoting wrappers.
[appdomain] Cleanup MonoError code
1. Assert in mono_get_corlib_version instead of raising an exn.
2. Don't raise in mono_domain_create_appdomain external API function.
3. Assert in mono_domain_assembly_postload_search instead of raising.
4. Don't raise in mono_domain_fire_assembly_load.
5. Use mono_error_set_pending_exception in
ves_icall_System_AppDomain_InternalGetProcessGuid
6. Dont' raise in unload_thread_main, populate unload_data instead.
Added support for SHA256, 384 and 512 in AuthenticodeDeformatter.
Due to the changes introduced by Microsoft to require signing with stronger algorithms, SHA-256, SHA-384 and SHA-512 support was needed.
This change is released under the MIT license.
There could be a race between System.Net.Sockets.Socket:Close_internal and the io threadpool cleanup, which would lead to a native crash. That's because the io threadpool cleanup would clean the mutex, condition variables as well as close the sockets.
The easiest way to fix that is by cleaning only what's going to stop the runtime from shutting down, and in this case the background selector thread. The same is done for the threadpool where we just shutdown the monitor thread and the worker threads.
Due to the changes introduced by Microsoft to require signing with stronger algorithms, SHA-256, SHA-384 and SHA-512 support was needed.
This change is released under the MIT license.
[marshal] Ensure marshal code emit the right indirect store when dealing with ref types.
The marshal code was using CEE_STIND_I for a lot of ref stores. This would not
generate a WB and is a heap corruption vector.
Part of man-page showing this interface:
PTHREAD_GETNAME_NP(3) Library Functions Manual PTHREAD_GETNAME_NP(3)
NAME
pthread_getname_np - get and set descriptive name of a thread
LIBRARY
POSIX Threads Library (libpthread, -lpthread)
SYNOPSIS
#include <pthread.h>
int
pthread_getname_np(pthread_t thread, char *name, size_t len);
int
pthread_setname_np(pthread_t thread, const char *name, void *arg);
-- pthread_setname_np(3)
The .orig files are traditionally files of patch(1) and diff(1) tools.
Remove stale file Makefile.orig as it is generating false positives for
grabbing patches from a directory tree.
[metadata] Don't require HandleRef in domain.c
In 1da6b76297 loading of System.Runtime.InteropServices.HandleRef
was changed from optional to required. This caused errors in the maccore tests:
```
MONO_PATH=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/64/Build /Users/builder/data/lanes/1411/5ca65b46/source/maccore/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/bin/arm64-darwin-mono-sgen --debug -O=gsharedvt --aot=mtriple=arm64-ios,data-outfile=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/System.arm64.aotdata,static,asmonly,direct-icalls,full,dwarfdebug,no-direct-calls,outfile=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/System.dll.arm64.s "/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/64/Build/System.dll"
Runtime critical type System.Runtime.InteropServices.HandleRef not found
AOT Compilation exited with code 134, command:
```
@monojenkins merge
In 1da6b76297 loading of System.Runtime.InteropServices.HandleRef
was changed from optional to required. This caused errors in the maccore tests:
```
MONO_PATH=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/64/Build /Users/builder/data/lanes/1411/5ca65b46/source/maccore/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/bin/arm64-darwin-mono-sgen --debug -O=gsharedvt --aot=mtriple=arm64-ios,data-outfile=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/System.arm64.aotdata,static,asmonly,direct-icalls,full,dwarfdebug,no-direct-calls,outfile=/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/System.dll.arm64.s "/Users/builder/data/lanes/1411/5ca65b46/source/maccore/msbuild/tests/MyKeyboardExtension/obj/iPhone/Debug/mtouch-cache/64/Build/System.dll"
Runtime critical type System.Runtime.InteropServices.HandleRef not found
AOT Compilation exited with code 134, command:
```
gmcs and ilasm2 are no longer distributed with Mono.
The run-test step had several issues that prevented it from running,
e.g. the comments caused "unexpected end of line" issues on OSX.
Improve shell portability on NetBSD
The "test" command, as well as the "[" command, are not required to know
the "==" operator.
Comparing with empty strings ("") might lead to unspecified results.
Replace unportable shell loops:
for I in {2..6}; do ..; done
with POSIX ones:
I=2; while [ $I -le 6 ]; do ..; I=$((I + 1)); done
The "test" command, as well as the "[" command, are not required to know
the "==" operator.
Comparing with empty strings ("") might lead to unspecified results.
Replace unportable shell loops:
for I in {2..6}; do ..; done
with POSIX ones:
I=2; while [ $I -le 6 ]; do ..; I=$((I + 1)); done
The <sys/errno.h> header on NetBSD declares only a list of ERROR values.
This fixes the following build issue:
In file included from ../../mono/utils/mono-proclib.h:7:0,
from mono-proclib.c:8:
mono-proclib.c: In function 'mono_cpu_usage':
mono-proclib.c:893:54: error: 'errno' undeclared (first use in this function)
g_error ("getrusage() failed, errno is %d (%s)\n", errno, strerror (errno));
^
../../eglib/src/glib.h:600:70: note: in definition of macro 'g_error'
#define g_error(...) do { g_log (G_LOG_DOMAIN, G_LOG_LEVEL_ERROR, __VA_ARGS__); for (;;); } while (0)
^
mono-proclib.c:893:54: note: each undeclared identifier is reported only once for each function it appears in
g_error ("getrusage() failed, errno is %d (%s)\n", errno, strerror (errno));
^
../../eglib/src/glib.h:600:70: note: in definition of macro 'g_error'
#define g_error(...) do { g_log (G_LOG_DOMAIN, G_LOG_LEVEL_ERROR, __VA_ARGS__); for (;;); } while (0)
The problem here was that the function was not correctly setting is_generic when handling partially inflated methods.
The working scenario was method 'Foo<A>::Bar<B>' inflated with (Something<Else>;)[1] resulting in Foo<Something<Else>>::Bar<B>.
This would work because the is_generic check would verify that the supplied context has no method_inst, thus had to remain a GMD.
The problematic one would be inflating Foo<Something<A>>::Bar<B> with (Else;) resulting too in Foo<Something<Else>>::Bar<B>.
To handle this, we would first extract the root method definition [2] and then inflate the original method context with the supplied
context. So we end up with Foo<A>::Bar<B> and inflate (Something<A>;B) with (Else;) resulting in (Something<Else>;B) *1*.
Finally, we would inflate Foo<A>::Bar<B> with that new context, resulting in Foo<Something<Else>>::Bar<B>.
The resulting method is the same in both scenarios, but context used for inflation is different in that in the second case
it includes the GMD method instantiation (The B in *1*).
This would trip up the computation of whether we ended up with a GMD or not.
The right check, though, is not whether the inflation context has a method instantiation or not, but whether the resulting method
has the same method context as the root GMD[2].
This affected #29585 since disabling some optimizations triggered things to be inflated in different ways.
[1] (Something<Else>;) means a context with a class instantiation and no method instantiation
[2] The GMD defined on the GTD.
<configuration><configSections><section> in a .config file takes a
type name as argument. Some of the allowed types are in System.dll.
The type lookup is done in System.Configuration.dll and does not find
these types. Check System.dll as a fallback to fix this.
This appears to have been copied from Boehm, but here we are not responsible for zeroing.
Also, 's->chars [len]' is not the same as 'align_up (s + size) - 2', so this was corrupting canaries.
The first nursery collection after the start of a concurrent collection allocated mod union card tables for all blocks. Allocate it only for blocks that have dirty cards. Blocks without a mod_union are not scanned in the major collector. This can reduce the pause time of those nursery collections by half.
[sgen] Scan pinned objects in nursery as part of concurrent mark
Objects pinned in the nursery can hide behind them a large graph of objects in the heap which is never scanned because we don't follow references in the nursery. After precleaning, we scan the latest set of pinned objects (while locking in order to make sure that the set doesn't change, so none are moved).
One overlooked aspect of mach is that its calls are interruptible and
we never really accounted for that.
This problem turned up when sampling at high frequency as they would
start to randomly fail.
'make check-seq-points' on Cygwin 2.4.1 64-bit.
* Added removal of *.exe.dll to the clean_aot() function to delete AOT output
from previous 'make check-seq-points' runs.
* Use ';'' as separator in MONO_PATH if running under Cygwin rather than ':'.
Objects pinned in the nursery can hide behind them a large graph of objects in the heap which is never scanned because we don't follow references in the nursery. After precleaning, we scan the latest set of pinned objects (while locking in order to make sure that the set doesn't change, so none are moved).
NativeMethods.GetExitCodeProcess would inherently be broken on Unix platforms for non-child processes as it can't succesfully get the exit code. This would in turn break Process.HasExited, which should not break, as it is possible to check if a non-child process is alive or dead.
Instead of opening a file for the sole purpose of calling
F_GETPATH on the resulting descriptor, we can just use the
native realpath() function; as a side benefit, this also works
on *BSD where F_GETPATH isn't supported.
Per current .NET, exception serialization may be implemented using the
SerializeObjectState event. Use of this approach is mandatory for
certain classes, such as HttpRequestException subclasses. Mono does
not support it. The solution is to switch out our System.Exception
implementation for the referencesource one.
To accommodate this, classes that access System.Exception internal
fields directly must switch to the new field names; a new support
method is added to System.Environment; the "security level" methods
and icall are added to MonoCMethods; and the field order in
_MonoException must be adjusted. A new test is also added in the
System.Runtime.Serialization assembly.
jlorenss/win x64 support
Initial fixes needed in order to get pass on mono regression tests for windows x64 visual studio build,
--regression basic.exe basic-float.exe basic-long.exe basic-calls.exe objects.exe arrays.exe basic-math.exe exceptions.exe iltests.exe devirtualization.exe generics.exe basic-simd.exe
Before fixes there were failures in basic.exe, exceptions.exe and objects.exe for windows x64 Visual studio build.
Failures in basic.exe and exceptions.exe was due to a mix up of signal defines on windows platform in Visual Studio build where SIGILL and SIGFPE was defined differently in different source files causing incorrect signal handler to be invoked at runtime, causing incorrect thrown .NET exception. Fix is to make sure signal.h is always included giving the same value for SIGILl and SIGFPE over all source files.
Failures in objects.exe test_0_mul_ovf_regress_36052 caused a stack overflow crash when running togehter with branch optimization mode. This was due to incorrect lowering of x64 test instruction when optimization moved over to use R8 instead of RAX and current lowering didn't handle that correctly meaning that the actual test was performed against wrong register causing a loop that caused a stack overflow error. The specific amd64 codegen , amd64_test_reg_imm, is currently only used on x64 windows platform for the stackalloc call.
In SSCX.OSX509Certificates.GetCertificate(), we cannot simply use
'X509Certificate.Handle' because it may point to something that's
not a SecCertificate.
(cherry picked from commit 3fc068f7e2)
In the lazy sweep case, the sweep job doesn't sweep the blocks. Instead, the blocks are swept when they are actually needed (when we want to allocate in the block, scan the block etc.). This delays the mutator or increases the nursery pauses. Try to sweep as many blocks as possible concurrently, immediately after we finish the sweep job.
Currently only triggered by objects.exe regression test on windows x64 with branch opt, since stackalloc gets lowered to this test and branch will start using R8 instead of RAX for the test.
Caused incorrect exception behavior and failures in basic.exe, exceptions.exe regression test on x64 windows. This was due to the fact that some source files didn't include signal.h while others did. In case where signal.h was not included incompatible defines were used for SIGILL and SIGFPE in mini-amd64.h.
x86 cleanups for register arguments
Best I can tell, cinfo->args and cfg->args need to be independent before implementing calling conventions where some arguments are in registers, at least if we want to do it cleanly. Also, storing the argument offset in a cfg->args instruction that claims to be for a register seems hacky and confusing to me.
We decompose them as part of the cprop pass to handle more immediates and to further propagate copies.
The native backend of some architectures (for example x86) may support long shifts with specialized instructions. Even there, this still makes immediate shifts 2 times faster because long ops are poorly handled by the jit.
On arm, where we were emulating, this makes long immediate shifts 3-5 times faster.
[sgen] Set ARCH_MIN_MS_BLOCK_SIZE to 64k on both ppc32 and ppc64.
This fixes a problem that was happening on Debian ppc32 build machines. Since
they are set up as a 32-bit userland on a 64-bit kernel, Mono would be compiled
with an ARCH_MIN_MS_BLOCK_SIZE value of 16k. However, at runtime, the system
page size would be 64k (due to the 64-bit kernel with PPC_64K_PAGES set).
Signed-off-by: `Alex Rønne Petersen <[email protected]>`
[metadata] Ifdef out an icall that is not available in the DISABLE_REFLECTION_EMIT case
It caused errors like the following on the maccore builds:
```
CCLD libmonosgen-2.0.la
Undefined symbols for architecture armv7k:
"_mono_reflection_get_custom_attrs_blob_checked", referenced from:
_ves_icall_System_Reflection_Emit_CustomAttributeBuilder_GetBlob in libmonoruntimesgen.a(libmonoruntimesgen_la-icall.o)
ld: symbol(s) not found for architecture armv7k
```
---
This is the last patch required for getting maccore master to build again with mono master.
@monojenkins merge
This is the support that makes it so that mkbundle can create self-contained executables without requiring a native toolchain to be present. Also is the foundation to allow cross-compilation of self-contained executables.
This works by concatenating the runtime with the various dependencies and configuration files, attaching a directory at the end of the file and a signature to flag this as a runtime that has embedded resources in the executable file itself.
This works by modifying the runtime startup to check for a special signature at the end of the main executable, if it is not present, execution resumes as usual. Otherwise a directory is read from the end of the file which contains both the kind of file (assembly, configuration file, and so on), as well as pointer to the contents in the file. Then the regular Mono APIs to load resources are used to load these.This is the support that makes it so that mkbundle can create self-contained executables without requiring a native toolchain to be present. Also is the foundation to allow cross-compilation of self-contained executables.
This works by concatenating the runtime with the various dependencies and configuration files, attaching a directory at the end of the file and a signature to flag this as a runtime that has embedded resources in the executable file itself.
This works by modifying the runtime startup to check for a special signature at the end of the main executable, if it is not present, execution resumes as usual. Otherwise a directory is read from the end of the file which contains both the kind of file (assembly, configuration file, and so on), as well as pointer to the contents in the file. Then the regular Mono APIs to load resources are used to load these.
It caused errors like the following on the maccore builds:
```
CCLD libmonosgen-2.0.la
Undefined symbols for architecture armv7k:
"_mono_reflection_get_custom_attrs_blob_checked", referenced from:
_ves_icall_System_Reflection_Emit_CustomAttributeBuilder_GetBlob in libmonoruntimesgen.a(libmonoruntimesgen_la-icall.o)
ld: symbol(s) not found for architecture armv7k
```
This fixes a problem that was happening on Debian ppc32 build machines. Since
they are set up as a 32-bit userland on a 64-bit kernel, Mono would be compiled
with an ARCH_MIN_MS_BLOCK_SIZE value of 16k. However, at runtime, the system
page size would be 64k (due to the 64-bit kernel with PPC_64K_PAGES set).
Signed-off-by: Alex Rønne Petersen <[email protected]>
Apparently Windows doesn't free this debug info in Vista+ when calling
DeleteCriticalSection [1], so just avoid allocating it in the first place by
calling InitializeCriticalSectionEx with the CRITICAL_SECTION_NO_DEBUG_INFO
flag. This function is only available in Vista+, but we've dropped support for
XP anyway.
[1] http://stackoverflow.com/a/889853
We don't evacuate from full blocks. When it comes to blocks that have free slots, instead of evacuating all of them to a new set of blocks, aim to keep the fuller blocks around and evacuate the emptier ones into them.
[profiler] Fix incorrect initial writer semaphore value.
This was used during debugging. I forgot to revert it back to 0 before posting
the pull request (#2838). This led to ~3-5% more CPU usage by the writer thread
than actually necessary; it now sits around 0-1% for typical workloads.
The overloaded GetValidator() which takes both 'MonoTlsSettings'
and 'MonoTlsProvider' is actually used by the test suite, so let's
keep it public.
(cherry picked from commit 0977d531d2)
babysitter generates some temp paths in python when running under cygwin, which breaks down
when those paths are passed to the native Windows mono.exe during testing:
Exception in TestRunnerThread ---> System.IO.DirectoryNotFoundException: Could not find a part of the path "C:\cygdrive\c\WINDOWS\TEMP\tmpC4K5yK\babysitter_report_current_test_file.txt".
This was used during debugging. I forgot to revert it back to 0 before posting
the pull request (#2838). This led to ~3-5% more CPU usage by the writer thread
than actually necessary; it now sits around 0-1% for typical workloads.
[profiler] Fix usage of the lock-free queue API.
* Free nodes hazardously.
* No need to unpoison nodes since we're not reusing them.
* Fix memory leak when emptying coverage queues.
1. icall `mono_image_module_basic_init`
2. icall `mono_reflection_sighelper_get_signature_local`
3. icall `mono_reflection_sighelper_get_signature_field`
4. API function mono_module_get_object
5. API function mono_type_get_object
6. API function mono_method_get_object
7. API function mono_field_get_object
8. API function mono_property_get_object
9. API function mono_event_get_object
1. static function `create_custom_attr` (set MonoError exception instance)
2. icall `mono_reflection_resolve_custom_attribute_data` (set pending
exception from MonoError). Also rename it to
ves_icall_System_Reflection_CustomAttributeData_ResolveArgumentsInternal
3. icall `mono_reflection_bind_generic_method_parameters` (set pending
exception from MonoError) Not renaming this one because it's actually
used for two icalls:
System.Reflection.Emit.MethodBuilder.MakeGenericMethod and System.Reflection.MonoMethod.MakeGenericMethod_impl
Change icall name to
ves_icall_System_Reflection_Emit_TypeBuilder_create_generic_class
Give mono_reflection_create_generic_class a MonoError argument,
and get rid of the redundant mono_reflection_create_generic_class_checked
Some Jenkins workers (e.g. the s390x one) have an earlier python version installed where the babysitter doesn't work.
Fall back to timeout in those cases.
* Mono.Security.Interface.ICertificateValidator2:
new internal interface, deriving from ICertificateValidator.
* Mono.Security.Interface.CertificateValidationHelper:
the internal GetValidator() now returns ICertificateValidator2.
* Mono.Security.Interface.MonoTlsProvider:
make HasCustomSystemCertificateValidator and InvokeSystemCertificateValidator()
internal and use ICertificateValidator2.
* Mono.Net.Security.ChainValidator:
implement ICertificateValidator2; we may now be called with a
fully-built X509Chain.
* corlib, System, Mono.Security:
add [assembly: InternalsVisibleTo].
[profiler] Use a semaphore to signal the writer thread.
We would previously constantly try to dequeue from the writer queue which would
result in extremely high CPU usage, making many apps (e.g. Xamarin Studio)
nearly unusable.
[corlib] Type.GetType parser and resolver improvements
We have two type parsers the one invoked by `Type.GetType(string)` (and some others) which is implemented in C in `reflection.c`, and the one invoked by `Type.GetType(string, Func<...>, Func<...>)` implemented in managed code as `System.TypeSpec`.
In both cases `string→Type` conversion goes in two stages: parsing and resolution. Parsing just changes the string into an AST. And resolution crawls the AST and constructs the appropriate types.
This fixes [Bugzilla #40060](https://bugzilla.xamarin.com/show_bug.cgi?id=40060)
1. Fix the resolver on the native side for types like `"System.Generic.Collections.Dictionary[System.Int32,MyType]"` where the generic type is from corlib and a type argument is from the current assembly.
2. Fix `get_caller_no_reflection` to work correctly when the call stack has a System.Reflection method as the immediate caller of the icall.
3. Fix the resolver on the native side to work when it's called from corlib's System namespace, not just System.Reflection.
4. Fix the managed parser to work with types like `"MyGeneric[MyType]"` where the generic argument is not an assembly qualified name or there's a mixture of aqn and non-aqn type arguments. Also rationalize how the code handles assembly qualification in general.
5. Test cases
We would previously constantly try to dequeue from the writer queue which would
result in extremely high CPU usage, making many apps (e.g. Xamarin Studio)
nearly unusable.
1. Assert in mono_get_corlib_version instead of raising an exn.
2. Don't raise in mono_domain_create_appdomain external API function.
3. Assert in mono_domain_assembly_postload_search instead of raising.
4. Don't raise in mono_domain_fire_assembly_load.
5. Use mono_error_set_pending_exception in
ves_icall_System_AppDomain_InternalGetProcessGuid
6. Dont' raise in unload_thread_main, populate unload_data instead.
.. attribute for CurrentSolutionConfigurationContents .
<CurrentSolutionConfigurationContents>
<SolutionConfiguration xmlns="">
<ProjectConfiguration Project="{82323650-6C2A-4FAC-91C6-A715C2FE9CB2}" AbsolutePath="/Users/ankit/test/msbuild/bug40094/App1/App1\App1\App1.csproj">
Release|AnyCPU
</ProjectConfiguration>
...
Related to fix for bxc #40094
Previously for types like "Foo[P,Q]" the recursive call of the parser
would incorrectly consume the delimiter after a type argument (ie the
',' or ']') which would leave the caller potentially past the end of the
string which would signal a failed parse. New behavior is to peek at
the delimiter but leave it up to the caller to consume.
Also `Type.GetType(name, null, null)` should behave like
`Type.GetType(name)` (previously it dropped generic type arguments).
type_from_parsed_name is a helper for
ves_icall_System_Type_internal_from_name which is called directly from
System.Type as well as indirectly by System.TypeNameParse. We should
skip over all those classes to find the managed caller.
This fixes the remainder of [40060](https://bugzilla.xamarin.com/show_bug.cgi?id=40060)
In order to find the user code that called some corlib API
that calls an icall, we write:
dest = m = mono_method_get_last_managed ();
mono_stack_walk_no_il (get_caller_no_reflection, &dest);
if (!dest) dest = m;
The get_caller_no_reflection walk works in two stages: in the first
stage we look for the target method (the initial value of dest) when we
find the target we set dest to NULL and then the second stage continues
walking upward past the reflection APIs.
However if the target method is itself in System.Reflection, the
old implementation would skip over it even in the first stage and we'd
never find the target method and exhaust the whole stack.
The fix is to move the check for the target method before the check for
the Reflection API methods.
And in type_from_parsed_name pass the calling image as the metadata
context even if the initial class we're looking for is in corlib.
This lets calls like
Type.GetType ("System.Collections.Generic.Dictionary<int, MyClass<int>>")
succeed. (Previously we would firs try to find
System.Collections.Generic.Dictionary in the calling assembly,
which would fail outside of corlib, and then we'd retry with the
rootimage as corlib which would find Dictionary, but fail to find
MyClass. Note that this was only broken if the outermost type was in
corlib. It always worked deeper in generic type arguments provided
we startd in the calling assmebly:
Type.GetType ("MyClass<System.Collections.Generic.Dictionary<int, MyClass<int>>>")
This fixes part of [40060](https://bugzilla.xamarin.com/show_bug.cgi?id=40060)
According to http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0068b/CIHEEIDJ.html the MRC instruction is documented as:
MRC{cond} coproc, opcode1, Rd, CRn, CRm{, opcode2}
with coproc being "the name of the coprocessor the instruction isfor. The standard name is pn, where n is an integer in the range 0-15."
GCC seemed to accept the operand without the 'p' prefix but clang raised an "invalid operand for instruction" error.
In some platforms, C enums might be considered unsigned, and in that case we'd get the following warning:
mono/mono/metadata/marshal.c:5108:12: warning: comparison of constant -1 with expression of type 'MonoMarshalConv' is always false
[-Wtautological-constant-out-of-range-compare]
if (conv == -1) {
~~~~ ^ ~~
The explicit cast should do the right thing.
Log buffers can end up with a zero thread ID when we synthesize events on the
writer thread, leading to mprof-report thinking a thread exists with that ID.
We simply ignore the "zero thread" in the thread summary.
If we don't do this, mprof-report will pick up on the fact that the helper and
writer threads exist and print them in the thread summary with no names. It's
OK to zero out the thread ID as the kinds of events we write from these threads
cannot be meaningfully associated with the log buffer's thread ID (or in the
case of counters, any thread ID).
We only set the native name as MS.NET leaves the managed name for the main
thread as null. At least by setting the native name, we get a nice name in GDB.
The profiler helper thread does a full attach for on-demand heapshots. Since we
collect no actual data for the helper thread, there's no reason to report it.
A double init could happen when profiling as we'd call mono_gc_base_init ()
from mono_profiler_load () early on, and then later call it again during the
normal startup sequence.
Previously, we would shut down the profiler so early that it would never see
unload events for threads, domains, etc. This sounds harmless at first, but
since the profiler dumps the remaining data in the thread-local log buffer to
disk from within the thread end callback, we would often miss events near the
end of a thread's lifetime.
Shutting down the profiler after the major runtime components doesn't appear to
have any negative effects, which makes sense as all managed threads should be
stopped by this point.
Strong Togglerefs must be considered as roots for the purpose of Ephemeron processing.
Otherwise you make it not possible to use a toggleref object as key.
Fix uninitialized MonoError in mono_basic_block_split
Follow-up on PR #2781 which removed a mono_error_init leading to memory unsafety. This was causing crashes when verifying assemblies in the 4.4 branch although it does not seem to have caused any known problems in master for some reason.
The recent changes to the mobile profiles introduced generated files in their .sources like CommonCrypto/MD5CryptoServiceProvider.g.cs
However, those are not generated when the mobile profiles aren't enabled at configure time.
This caused a problem during "make dist" since it tries to grab the entries in .sources from *all* profiles, causing errors like:
cp: cannot stat ‘CommonCrypto/MD5CryptoServiceProvider.g.cs’: No such file or directory
make[5]: *** [dist-local] Error 1
The easiest fix is to filter out those generated files.
Any native function that can modify errno should have the
SetLastError attribute set to true. Also, no need to specify
EntryPoint if it's the same as the function name.
[corlib] Fix exception in __ComObject finalizer during shutdown
* keep a reference to a __ComObject's proxy so it doesn't get garbage
collected
* don't create __ComObjects unnecessarily
* Fix for bug [24974](https://bugzilla.xamarin.com/show_bug.cgi?id=24974)
We hit this on Jenkins over the weekend:
https://jenkins.mono-project.com/job/test-mono-mainline/label=osx-amd64/3845/parsed_console/log_content.html#WARNING1
```
Unhandled Exception:
System.InvalidOperationException: Cannot process request because the process (51792) has exited.
at System.Diagnostics.Process.GetProcessHandle (Int32 access, Boolean throwIfExited) <0x10d7ea000 + 0x001b6> in <filename unknown>:0
at System.Diagnostics.Process.GetProcessHandle (Int32 access) <0x10d7f2fd0 + 0x0001e> in <filename unknown>:0
at System.Diagnostics.Process.Kill () <0x10d7f2ee0 + 0x00031> in <filename unknown>:0
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Kill ()
at TestRunner+<Main>c__AnonStorey0.<>m__0 () <0x10a9b50b0 + 0x00e30> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) <0x10c55bc90 + 0x0009a> in <filename unknown>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x10c55a210 + 0x001c6> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x10c55a1e0 + 0x00020> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) <0x10c55a130 + 0x00059> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart () <0x10c55be50 + 0x0002e> in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidOperationException: Cannot process request because the process (51792) has exited.
at System.Diagnostics.Process.GetProcessHandle (Int32 access, Boolean throwIfExited) <0x10d7ea000 + 0x001b6> in <filename unknown>:0
at System.Diagnostics.Process.GetProcessHandle (Int32 access) <0x10d7f2fd0 + 0x0001e> in <filename unknown>:0
at System.Diagnostics.Process.Kill () <0x10d7f2ee0 + 0x00031> in <filename unknown>:0
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Kill ()
at TestRunner+<Main>c__AnonStorey0.<>m__0 () <0x10a9b50b0 + 0x00e30> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) <0x10c55bc90 + 0x0009a> in <filename unknown>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x10c55a210 + 0x001c6> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x10c55a1e0 + 0x00020> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) <0x10c55a130 + 0x00059> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart () <0x10c55be50 + 0x0002e> in <filename unknown>:0
make[1]: *** [test-generic-sharing-managed] Error 1
```
We can't do anything about those errors, so just catch and ignore it.
[Profiler] Keep a reference to assemblies while coverage is running
In certain circumstances, for example when running under nunit-console, assemblies could be unloaded while the profiler still had a reference to them in order to generate the coverage data. In these cases, the memory referred to by the profiler would be invalid and would crash.
Fix this by incrementing the reference count of the assemblies preventing them from being unloaded while coverage is being collected.
Fixes BXC #39325
- Back out MONO_PROFILER_LIB_DIR change
- Profiler loading traces should use log mask “dll” not “asm”
- mono_dl_open_runtime_lib should preempt standard paths when loading
profiler
- mono_dl_open_runtime_lib should check in profiler library directory
- mono_dl_open_runtime_lib contained a memory leak
- With above changes, ptestrunner.pl can remove some environment
massaging
Fixes#4242, see bug comments. JsonSerializer seems to have been already
fixed in abe9dc62. The JsonDeserializer number conversion is now much
stricter, allowing only integer number style for integers and float number
style for floats.
[profiler] Use a background thread to send out sampling signals.
Previously, when using an interval timer, the initial profiling signal could be
delivered to *any* thread in the process. This is not normally a problem, but
if the signaled thread has not even finished its initialization inside libc, it
can happen that it hasn't even set up thread-local storage yet. We then blow up
spectacularly when trying to back up errno in the SIGPROF signal handler, as
it's a TLS variable. Even the SIGSEGV handler blows up immediately after as it
can't access JIT TLS data.
Since there appears to be no reliable and portable way for a library like Mono
to control which exact thread gets the initial timer signal, instead switch to
using a background thread that uses a high-resolution sleep and attempts to
switch itself to real time scheduling if possible. This way, we have full
control over which threads we send SIGPROF to, letting us avoid any threads
which aren't in a 'good' state for profiling.
This commit also gets rid of the multiplexing that was going on in the SIGPROF
signal handler, since we now send out the signals from a background thread. As
this was the only use of the async job API, that API has been removed. This
indirectly fixes the signal storming issue that sometimes popped up, where for
some reason multiple threads would think that they're the initiating thread,
resulting in way too many signals going out.
A nice side-effect of doing the signaling ourselves is that we can now use
real-time signals on systems that have them (e.g. Linux), resulting in a nearly
100% signal delivery rate in all cases. Previously, we would lose a tremendous
amount of signals when an application was under heavy load.
Previously, when using an interval timer, the initial profiling signal could be
delivered to *any* thread in the process. This is not normally a problem, but
if the signaled thread has not even finished its initialization inside libc, it
can happen that it hasn't even set up thread-local storage yet. We then blow up
spectacularly when trying to back up errno in the SIGPROF signal handler, as
it's a TLS variable. Even the SIGSEGV handler blows up immediately after as it
can't access JIT TLS data.
Since there appears to be no reliable and portable way for a library like Mono
to control which exact thread gets the initial timer signal, instead switch to
using a background thread that uses a high-resolution sleep and attempts to
switch itself to real time scheduling if possible. This way, we have full
control over which threads we send SIGPROF to, letting us avoid any threads
which aren't in a 'good' state for profiling.
This commit also gets rid of the multiplexing that was going on in the SIGPROF
signal handler, since we now send out the signals from a background thread. As
this was the only use of the async job API, that API has been removed. This
indirectly fixes the signal storming issue that sometimes popped up, where for
some reason multiple threads would think that they're the initiating thread,
resulting in way too many signals going out.
A nice side-effect of doing the signaling ourselves is that we can now use
real-time signals on systems that have them (e.g. Linux), resulting in a nearly
100% signal delivery rate in all cases. Previously, we would lose a tremendous
amount of signals when an application was under heavy load.
Various cleanups on profiler
3 commits:
Assist on profiler module loading
- Add a MONO_PROFILER_LIB_DIR from which profiler modules are
dynamically loading, in case it is not the normal library path
- Add information about profiler loading to mono_trace
Comments and clarity in proflog.c
- Add comments on structures
- Rename 'data' and 'data_end' to 'cursor' and 'buf_end' in LogBuffer/StatsBuffer as the old names were confusing
- Add new assert when processing samples
Improve a safety check when writing data into StatBuffer
- The safety check should occur such that if the new value for
StatBuffer::cursor is beyond the bounds of the StatBuffer, the cursor
is not updated.
Fix hazard free
Replace usage of mono_thread_hazardous_free_or_queue and remove it.
mono_thread_hazardous_free_or_queue has a fundamentally broken design.
It allows arbitrary free code to run in the context of its caller.
The sync/async split on where it's called is not enough to know whether
we can run free'ing code or not.
Since in sync context we called free functions that could take locks,
it happened that those locks conflicted with the ones already taken.
In particular, mono_jit_info_table_free did take a domain lock and
quite a few of the sync context calls would hold locks that must
not be held when taking a domain lock.
This, is practice, means that we'd need to partition the free calls
into 3 groups: async context, reentrant context (no runtime locks held)
and sync context (maybe some locks held).
There was no case where reentrant context would be usable meaning,
in practice, that all calls happens in what effectively is async context
where free functions can't be called.
The new design is a lot more straightforward:
- No implicit free queue pumping
- A pair of free functions with well defined behavior.
mono_thread_hazardous_try_free - to be used where the caller expects the free function to be called
mono_thread_hazardous_queue_free - to be used where the caller don't expect the free function to be called [1]
- Explicit pumping on places that known to be ok
Thread detach
Finalizer thread
[1] This might sound like a weird condition, but a lot of lock-free code
have compensation logic that trigger free'ing during lookups.
We add a callback to the HP so the GC can monitor the queue size and act on it.
The queue size was picked to keep the overhead limited even under high load.
- Add comments on structures
- Rename 'data' and 'data_end' to 'cursor' and 'buf_end' in
LogBuffer/StatsBuffer as the old names were confusing
- Add new assert when processing samples
- Add a MONO_PROFILER_LIB_DIR from which custom profilers are
dynamically loaded, in case it is not the normal library path
- Modify mono/profiler make check to use MONO_PROFILER_LIB_DIR
(this fixes a bug where it would sometimes load the wrong profiler)
- Add information about profiler loading to mono_trace
We used to log the major heap size before the sweep job finished. Remove the major heap logging at collection end time and add a separate log entry at sweep finish time.
mono_thread_hazardous_free_or_queue has a fundamentally broken design.
It allows arbitrary free code to run in the context of its caller.
The sync/async split on where it's called is not enough to know whether
we can run free'ing code or not.
Since in sync context we called free functions that could take locks,
it happened that those locks conflicted with the ones already taken.
In particular, mono_jit_info_table_free did take a domain lock and
quite a few of the sync context calls would hold locks that must
not be held when taking a domain lock.
This, is practice, means that we'd need to partition the free calls
into 3 groups: async context, reentrant context (no runtime locks held)
and sync context (maybe some locks held).
There was no case where reentrant context would be usable meaning,
in practice, that all calls happens in what effectively is async context
where free functions can't be called.
The new design is a lot more straightforward:
- No implicit free queue pumping
- A pair of free functions with well defined behavior.
mono_thread_hazardous_try_free - to be used where the caller expects the free function to be called
mono_thread_hazardous_queue_free - to be used where the caller don't expect the free function to be called [1]
- Explicit pumping on places that known to be ok
Thread detach
Finalizer thread
[1] This might sound like a weird condition, but a lot of lock-free code
have compensation logic that trigger free'ing during lookups.
[sgen] Clean up thread pool on shutdown.
Keeping threads around can cause processes to keep running when they shouldn't, particularly on Windows when the entry point returns.
(The reason this is important for me is that Wine provides _CorExeMain, which is an entry point that does return, unlike normal compiled C programs which appear to call exit() after main() returns. A Windows process doesn't exit until ALL threads have exited, so a thread that never exits means a hung process.)
It doesn't seem like we have other GC-specific cleanup code, so I don't know if this is a good approach. Feedback on that question would be appreciated.
Those two functions are intended to replace mono_thread_hazardous_free_or_queue.
The problem with that function is that it doesn't compose well given it forces
the whole call chain leader to a free call to follow its semantics.
Additionally, it's not enough safety to be in sync context to call the free functions
as they introduce reentrancy - they can be called from any other arbitrary free
call that doesn't claim to be in async context.
Given most data structures, in general, can't guarantee how they are used, they
must make the conservative assumption of being in async context, thus not allowing
any form of locking happening.
Make AOT compilation on 64 bit windows produce executables
This PR contains three commits (see commit descriptions) which:
- Fix 64-bit debug windows builds in MSVC 2015
- Make it possible to use the tool-prefix aot option with mono/mini make fullaotcheck (necessary if you are in cygwin and wish to invoke `x86_64-w64-mingw32-gcc`)
- Change mini so that `--aot=full` successfully emits 64-bit windows executables.
Building 32 bit executables will generally fail because we do not follow cdecl. The 64 bit executables do not run properly when executed.
[runtime] MonoError-ize mono_loookup_dynamic_token{,_class}
Push `MonoError` through `mono_lookup_dynamic_token_class` and `mono_lookup_dynamic_token`
Also simplify the implementations of those functions so that they call `mono_reflection_lookup_dynamic_token` directly, rather than via a function pointer. The old implementation was a holdover from when the runtime was implemented as separate libraries.
Android uses dlmalloc within bionic. For some reason, our x86 build
links against the libc implementation instead of our own.
This leads us allocating memory that is not executable which made
dynamic methods fail when executing.
List of original contributors:
Rodrigo Kumpera <[email protected]>
On some junk Samsung x86 device, pthread_attr_getstack is completely broken.
If the returned value is not sane, we parse /proc/self/maps to find the right
memory regions for the current stack.
Fixes BXC #21550 and DESK #83333.
List of original contributors:
Rodrigo Kumpera <[email protected]>
1) Free() throws on MS.NET when the handle was already freed, copied and adapted the referencesource implementation.
2) Explicit conversion from IntPtr to GCHandle throws InvalidOperationException when value is IntPtr.Zero on MS.NET, not ArgumentException.
This fixes two failing CoreCLR test cases.
With the concurrent collector, we used to reset cementing at the start of the finishing pause (we can uncement since we process all the remsets when the world is stopped, through the mod union entries). This processing means cardtable scan in the finishing pause. In order to reduce pause time and overhead, we extend cementing for the finishing pause. Now we don't uncement objects (during sgen_cement_reset) that are still pinned in the roots at the start of the concurrent collection. We won't mark mod_union entries for these objects and they will survive the collection in the nursery. They can become uncemented at the start of the next major collection, given that they are no longer pinned in the roots.
Call mono_reflection_lookup_dynamic_token directly from
mono_lookup_dynamic_token.
Delete mono_install_lookup_dynamic_token () and the
MonoLookupDynamicToken typedef.
[runtime] MonoError-ize mono_object_isinst{,_mbyref}
Also mark them external only. Runtime should use mono_object_isinst_mbyref_checked and mono_object_isinst_checked
We are quite frequently seeing an error on the ARM workers on Jenkins:
```
Test Case Failures:
1) MonoTests.System.Net.Sockets.SocketTest.SendAsyncFile : System.Exception : Could not abort registered blocking threads before closing socket.
at System.Net.Sockets.SafeSocketHandle.ReleaseHandle () [0x000be] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/System/System.Net.Sockets/SafeSocketHandle.cs:77
at System.Runtime.InteropServices.SafeHandle.DangerousReleaseInternal (Boolean dispose) <0x42cb0050 + 0x0020b> in <filename unknown>:0
at System.Runtime.InteropServices.SafeHandle.InternalDispose () <0x446ba758 + 0x00023> in <filename unknown>:0
at System.Runtime.InteropServices.SafeHandle.Dispose (Boolean disposing) [0x00000] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/external/referencesource/mscorlib/system/runtime/remoting/callcontext.cs:804
at System.Runtime.InteropServices.SafeHandle.Dispose () [0x00000] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/external/referencesource/mscorlib/system/runtime/remoting/callcontext.cs:795
at System.Net.Sockets.Socket.Dispose (Boolean disposing) [0x00047] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/System/System.Net.Sockets/Socket.cs:3321
at System.Net.Sockets.Socket.Dispose () [0x00000] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/System/System.Net.Sockets/Socket.cs:3327
at System.Net.Sockets.Socket.Close () [0x00007] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/System/System.Net.Sockets/Socket.cs:3256
at MonoTests.System.Net.Sockets.SocketTest.SendAsyncFile () [0x000e7] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/System/Test/System.Net.Sockets/SocketTest.cs:4331
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0003d] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armhf/mcs/class/corlib/System.Reflection/StrongNameKeyPair.cs:103
```
Let's see if implementing the suggestion from https://github.com/mono/mono/pull/1612/files#r26393224
fixes this. I couldn't reproduce the test failure even on the same ARM machine so this is a bit of
an opportunistic change, but it feels like the more "correct" solution anyway.
I decided to inline AbortRegisteredThreads as at the only place we're calling it we already know
blocking_threads can't be null and acquired the lock.
[threadpool-ms] MonoError-ize
1. Convert icalls that throw `NotImplementedException` to do it via mono_error_set_pending_exception
2. Add MonoError outargs to `mono_threadpool_ms_enqueue_work_item` and `mono_threadpool_ms_begin_invoke`
Return TRUE if pending exception was set, FALSE otherwise.
The idea is that you can write:
if (mono_error_set_pending_exception (error)) {
/* cleanup */
return;
}
Krait cpus don't ignore thumb ITT state in ARM mode (it's unspecified CPU behavior).
The linux kernel then didn't bother resetting that state when landing an ARM signal handler.
The end result was that Krait cpus would skip the first few instructions of the signal handler
and things would explode in the most amazing possible ways.
Patch by Mark Probst <[email protected]>.
Don't raise deep in the marshalling code, just assert.
(Raising means we'd have to convert the code to use MonoError and then
thread it out through all of the marshalling code resulting in a lot of
code churn)
When the concurrent mark ends, we have another concurrent phase where we scan the cardtable trying to clean as many cards as possible. The way we achieve this is by first copying all the mod union cards (for each block/los_obj) to a separate temporary card_table. When scanning we will only scan these cards. Therefore we will first preclean these cards in the actual mod_union card_tables, followed by a full memory barrier (which assures us that, when scanning, we will observe the changes that led to the corresponding card marks) and then by a normal mod_union scan, which can remark some of these cards, without any races with the preclean phase.
This way we avoid duplicating cardtable marking code, making sure we have consistent code.
For example, when scanning los objects in the mod union scan step, we didn't check whether the object is cemented or not, leading to additional unnecessary global remsets for the next nursery collection.
We add a new data structure which contains a bucket of entries, exponentially increasing in sizes. The structure is customizable through a few callbacks.
Setting is_slot_set_func enables us to also lookup empty slots, instead of only adding entries at the end of the list.
Setting set_slot_func enables the use of a customized function for setting the slot. If this function is not set, the slots and array fields will be set directly, therefore the structure won't be thread safe.
When doing the mod union scan, we use the block's cardtable (instead of the global one), which we also cannot prefetch, since that would mean dereferencing the block info.
Runtime should use g_assert and g_assert_not_reached or set a MonoError,
as needed.
Also remove REAL_PRINT_REG dead macros from mini-x86.c and mini-amd64.c
- COFF does not use .local or .size
- COFF does use .def to mark functions and .data to mark data
- The seh_handler functions should not be called on full aot builds (Zoltan says they are unnecessary in full aot, and they attempt to jit)
The executables do not yet run without crashing.
* Support MONO_FULLAOT_ADDITIONAL_ARGS env var for make fullaotcheck, appended to --aot parameters
* To support above, permit "blank" parameters (ie trailing commas on parameter list) to --aot=
* --aot tool-prefix parameter now supported for LD as well as AS
Add new internal 'X509Certificate2Impl' and 'X509ChainImpl' classes similar to
'X509CertificateImpl' in corlib and only initialize them when actually needed.
[SRE] MonoError-ize some icalls
* Cleanup some memory that would leak on failure; improve a few error messages
* Add MonoError to `mono_reflection_call_is_assignable_to` and `fieldbuilder_to_mono_class_field`.
* Update several SRE icalls to follow the pattern:
```c
void
the_icall (args... )
{
MonoError error;
(void) the_real_work_function (args..., &error);
mono_error_set_pending_exception (&error);
}
static gboolean
the_real_work_function (args..., MonoError *error)
{
... /* return TRUE on success, FALSE and set error on failure */ ...
}
```
Do this to:
1. `mono_reflection_register_with_runtime`
2. `mono_reflection_setup_internal_class`
3. `mono_reflection_create_internal_class`
4. `mono_reflection_generic_class_initialize`
5. `mono_reflection_event_builder_get_event_info`
6. `mono_reflection_initialize_generic_parameter`
7. `mono_reflection_create_dynamic_method`
It should not prevent the process from terminating. Fixes
issue where the thread is not shut down if ProjectInstances
are built directly instead of via the build manager.
Make `make fullaotcheck` in mono/mini runnable on Windows
Run with
MONO_EXECUTABLE=/path/to/msvc/mono.exe LIST_SEPARATOR=";"
This assumes you are running the test harness in Cygwin and you built mono with MSVC. Also, the test does not currently succeed.
Being able to run the test is a zeroth step toward making the test pass.
It was the only test in the class that required network access (for looking up xamarin.com in the redirect).
Replacing the redirect target with a second local HttpListener should be more resilient to network/DNS issues.
Changes to "checked build" feature
Here are some changes to checked builds I made while trying to debug some coop gc problems. The changes made are:
- Replace the checked build configure flags with a single --enable-checked-build= flag
- Runtime environment variable required for checked build asserts to run
- Report the thread transition history for more kinds of asserts
- For the `mono_threads_reset_blocking` functions, add an assert that guarantees proper nesting
In addition to this, I have documented the checked build feature as implemented here, see my pull request for the website: https://github.com/mono/website/pull/187
Overall my goals with this patch were
- Make it easier to do rapid-fire testing with the checked builds (runtime switch, default-off)
- Make the checked build implementation more internally consistent (more uniform names etc)
Notice I do make the change that instead of checked build sub-features being disabled with a blacklist, you now must enable them with a whitelist. I think this is an improvement because it makes it easier to just casually keep a checked build around (for one thing I would have the option to ship debug builds to users with the checked build modes included but dormant) but I could be talked out of it if this change turns out to be controversial. The only thing I feel strongly about is that if the configure flag is changed back to a blacklist the environment variable should switch to a blacklist also.
If we accept this change the CI bots will need to adopt the new configure flag and environment variable.
[System.Reflection.Emit] MonoError in mono_reflection_create_runtime_class
Depends on #2745
Bubble MonoError up to `mono_reflection_create_runtime_class`. Use `mono_error_set_pending_exception` instead of `mono_error_raise_exception` And try to release resources on the way out.
1. Don't raise while holding locks.
2. Change remove_instantiations_of_and_ensure_contents (called via
g_hash_table_foreach_remove) to populate a MonoError instead of
throwing. If there are multiple errors, we just stash the first one
in user_data and clean up the rest.
3. Use mono_erro_set_pending_exception instead of
mono_error_raise_exception since it's an icall.
[System.Reflection.Emit] Propagate MonoError through some more static functions
Some externally visible changes include:
* Bubble MonoError up to one more SRE icall: `System.Reflection.Emit.CustomAttributeBuilder.GetBlob()`
* `fix_partial_generic_class` and `ensure_generic_class_runtime_vtable` now expect their callers to set the class failure state which they do under slightly more circumstances now than previously (now: whenever there's an error set from the above functions; before: only when we had trouble initializing the generic type definition.)
* Previous API for `mono_reflection_bind_generic_parameters` didn't make it clear whether NULL was a valid return value or not. Now when we return NULL we always set the `error` out-arg. So non-NULL is success and NULL is failure.
The runtime should use mono_reflection_get_custom_attrs_blob_checked.
The icall is now ves_icall_System_Reflection_Emit_CustomAttributeBuilder_GetBlob.
This commit also changes the behavior of
mono_reflection_bind_generic_parameters slightly: if the passed in class
isn't generic, it sets error and returns NULL. (It wasn't clearly
previously whether NULL was an error condition or not - with this commit
we clarify that a NULL result is an error).
Because mono_msec_ticks would previously return the number of milliseconds since boot time, it would overflow after ~49 days. That would mean that anywhere you would use it, you would need to be careful of arithmetic overflow.
By simply returning a gint64 we will not overflow before ~30k years.
Fixes bug https://bugzilla.xamarin.com/show_bug.cgi?id=38666
Due to our usage of chained commands it'd use the exit code of the last command,
which is not what we want. We use chaining because e.g. in the ms-test-suite
case we still want to run the systemruntimebringup suite even if the conformance
suite had a failing test.
It's not supposed to contain a full path to a directory, but rather a relative path
to a subdirectory of the ApplicationBase. However, this isn't enforced in all cases
by Mono so setting a full path worked when the path doesn't contain a '.' because then
the code in [1] doesn't clear the variable and the path is added to the domain->search_path.
This caused the test to fail on Wrench on the mono-4.4.0 lane because the path contains
a dot while confusingly it worked fine on master.
Fixing this by appending the xbuild_14 path via an alternative (albeit slightly hacky)
method related to the standard MONO_PATH env var. Because the variable is only read
once at app startup we need to call into the runtime to explicitly set the new path
before trying to load the MSBuild 14 assemblies. This is fine since we only
want to test that the correct assembly version is loaded and nothing else.
[1] https://github.com/mono/mono/blob/37b2b9fbc25a2199aba1d794117924d4828360a7/mono/metadata/appdomain.c#L1319-L1337
It's flaky on Jenkins, removing the dependency on an external server and instead using a local HttpListener
to answer the requests should hopefully make it more reliable.
Moved StackTrace.GetFullNameForStackTrace into a new class
StackTraceHelper so the code can be reused by mono-symbolicate.
Reusing GetFullNameForStackTrace in mono-symbolicate allows the tool to
easily match a stack trace line with a method without doing a rather
complicate parsing of types and possible generic parameters
combinations.
This was inadvertently removed in https://github.com/mono/mono/commit/611a43ee0f672adbac3e25dc77731843a3d10cf1 from MulticastDelegate.CombineImpl().
Add the type check back in the Delegate base class, we already do the same in Remove().
Add unit tests to cover the behavior to the corlib test suite. (reverted from commit f29878dc713f4e148fe0a1e703fae8929e6fb6e0)
Needed to update some monodoc tests to match that an exception is now thrown again.
fixed replacement of $mono_libdir in config file
strncpy does not NUL-terminate the destination string in this case so one cannot use strcat to append
This fixes the error "malloc: top chunk is corrupt" as reported on mono-devel-list back in 2015-10-20 (subject: "malloc error executing OBS-built mono") at least for me.
This removed the last source of loader error from the runtime
The last source was on the AOT compiler in load_image.
I hooked enough MonoError to bubble the issue to a reasonably higher layer.
@vargaz please review.
This was an incorrect test. Tasks can't really be used like this
especially this once since it tries to Log an error, but it hasn't been
assigned a BuildEngine yet!
Other fixes in TaskLoggingHelper expose this issue.
Fix msbuild remapping
First commit:
```
Revert "Do not remap framework assembly if it's version is higher than the runtime version"
It would cause issues on mobile, cause the runtime version is 2.0.5.0 there and loading
e.g. System.dll v4.0 would no longer be mapped to that version anymore after this change.
```
---
Second commit:
```
[runtime] Stop remapping higher versions of
Microsoft.Build.Framework/Engine and remap Microsoft.Build.Utilities/Tasks
(.v3.5)
The previous remapping was added in 725ed3f824, but it didn't actually
fix the bug in https://bugzilla.xamarin.com/show_bug.cgi?id=32561.
The underlying cause of the bug is that in Mono 4.0 we removed the 2.0/3.5 profiles.
This meant that Microsoft.Build.Utilities.dll and Microsoft.Build.Utilities.v3.5.dll weren't installed
in the GAC anymore, because MSBuild 4.0 uses Microsoft.Build.Utilities.v4.0.dll, so the app failed
to load the 2.0 version of the assembly. The same applies for the .Tasks variants.
We remap earlier versions of the Microsoft.Build.Utilities/Tasks assemblies to the v4.0 version.
I've introduced another item in the AssemblyVersionMap struct which allows remapping to target a different
assembly name, because MSBuild made the questionable decision to put the version number in the assembly name.
While strictly speaking this is not what .NET is doing (it just fails there if .NET 2.0 isn't installed)
on Mono there's no way to install those older profiles anymore, so we remap them to v4.0 to keep apps working.
Newer versions of Microsoft.Build.Framework/Engine (like 12.0, 14.0) however have the same name, so we need to be
more careful here and only do a remapping to the 4.0 version if it's an earlier version (e.g. 2.0 -> 4.0).
This avoids issues with accidentally remapping 14.0 -> 4.0.
I tested the app from the bug report and it now works fine.
```
The previous remapping was added in 725ed3f824, but it didn't actually
fix the bug in https://bugzilla.xamarin.com/show_bug.cgi?id=32561.
The underlying cause of the bug is that in Mono 4.0 we removed the 2.0/3.5 profiles.
This meant that Microsoft.Build.Utilities.dll and Microsoft.Build.Utilities.v3.5.dll weren't installed
in the GAC anymore, because MSBuild 4.0 uses Microsoft.Build.Utilities.v4.0.dll, so the app failed
to load the 2.0 version of the assembly. The same applies for the .Tasks variants.
We remap earlier versions of the Microsoft.Build.Utilities/Tasks assemblies to the v4.0 version.
I've introduced another item in the AssemblyVersionMap struct which allows remapping to target a different
assembly name, because MSBuild made the questionable decision to put the version number in the assembly name.
While strictly speaking this is not what .NET is doing (it just fails there if .NET 2.0 isn't installed)
on Mono there's no way to install those older profiles anymore, so we remap them to v4.0 to keep apps working.
Newer versions of Microsoft.Build.Framework/Engine (like 12.0, 14.0) however have the same name, so we need to be
more careful here and only do a remapping to the 4.0 version if it's an earlier version (e.g. 2.0 -> 4.0).
This avoids issues with accidentally remapping 14.0 -> 4.0.
I tested the app from the bug report and it now works fine.
Saw a few failures on Jenkins on the slower ARM machines over the last days:
```
) MonoTests.System.Collections.Concurrent.BlockingCollectionTests.AddAnyCancellable : System.OperationCanceledException : The operation was canceled.
at System.Collections.Concurrent.BlockingCollection`1[T].TryAddToAnyCore (System.Collections.Concurrent.BlockingCollection`1[] collections, System.Collections.Concurrent.T item, Int32 millisecondsTimeout, CancellationToken externalCancellationToken) [0x00092] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armel/external/referencesource/System/sys/system/collections/concurrent/BlockingCollection.cs:1041
at System.Collections.Concurrent.BlockingCollection`1[T].TryAddToAny (System.Collections.Concurrent.BlockingCollection`1[] collections, System.Collections.Concurrent.T item, Int32 millisecondsTimeout, CancellationToken cancellationToken) [0x00006] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armel/external/referencesource/System/sys/system/collections/concurrent/BlockingCollection.cs:971
at System.Collections.Concurrent.BlockingCollection`1[T].AddToAny (System.Collections.Concurrent.BlockingCollection`1[] collections, System.Collections.Concurrent.T item, CancellationToken cancellationToken) <0x4421bf18 + 0x0002b> in <filename unknown>:0
at MonoTests.System.Collections.Concurrent.BlockingCollectionTests.AddAnyCancellable () [0x00031] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armel/mcs/class/System/Test/System.Collections.Concurrent/BlockingCollectionTests.cs:276
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0003d] in /media/ssd/jenkins/workspace/test-mono-mainline/label/debian-armel/mcs/class/corlib/System.Reflection/StrongNameKeyPair.cs:103
```
100ms is likely just too short, bumping it.
[runtime] Fix deadlock while unwinding the stack of suspended thread
A suspended thread, in non-coop case, can be holding runtime locks. When the suspender tries to unwind the stack for this thread, it might require taking of runtime locks (ex. in aot case we might search for the jit info in the aot image and then register it, which requires the domain lock). Mark the suspender thread as running in async context while it unwinds other threads.
In addition to --enable-checked-build, it is now mandatory to put the
requested checked-build modes in the environment variable
MONO_CHECK_MODE at runtime.
Also add a MONO_CHECK_THREAD_TRANSITION_HISTORY environement variable
for thread transition history length (instead of hardcoding 3).
This reverts commit 100bd760a4.
It would cause issues on mobile, cause the runtime version is 2.0.5.0 there and loading e.g. System.dll v4.0 would
no longer be mapped to that version anymore after this change.
Testing for the remap message via grep isn't ideal, a better way to verify the assembly
version would be to check asm.GetName().Version, but unfortunately in the test environment
when MONO_PATH is set the assembly loading always fallsback to a simple file name matching
in the directory so we end up with the wrong (v4.0) assembly in that case.
This was revealed by the CoreCLR tests/src/CoreMangLib/cti/system/convert/converttouint3219.exe test.
When we tried to convert a number larger than UInt32.MaxValue (in this case 4933891728) we would cast
the intermediate result to uint which overflows and makes the end result smaller which means we don't throw.
Storing the intermediate result in a long before checking for the correct range fixes this.
Added a few additional tests that I felt were missing and/or only present for other Convert.* methods.
I found two more cases that don't throw OverflowException but fixing it looked more involved so I punted
and just ignored them with a FIXME for now.
ARM platform tends to try very hard to save power by switching off CPUs. This could lead to inconsistencies in the value returned by mono_cpu_count
Recap from Alexander Köplinger <[email protected]>:
When we merged the change from PR #2722, we started seeing random failures on ARM in the MonoTests.System.Threading.ThreadPoolTests.SetAndGetMaxThreads and MonoTests.System.Threading.ManualResetEventSlimTests.Constructor_Defaults tests. Both of those tests are dealing with Environment.ProcessorCount to verify some implementation details.
It turns out that on the Jetson TK1 board we use on public Jenkins and on ARM kernels in general, the value returned by sched_getaffinity (or _SC_NPROCESSORS_ONLN) doesn't contain CPUs/cores that are powered off for power saving reasons. This is contrary to what happens on x86, where even cores in deep-sleep state are returned [1], [2]. This means that we would get a processor count of 1 at one point in time and a higher value when load increases later on as the system wakes CPUs.
Various runtime pieces like the threadpool and also user code however relies on the value returned by Environment.ProcessorCount e.g. for deciding how many parallel tasks to start, thereby limiting the performance when that code thinks we only have one CPU.
Talking to a few people, this was the reason why we changed to _SC_NPROCESSORS_CONF in https://github.com/mono/mono/pull/1688 and why we added a special case for Android in https://github.com/mono/mono/commit/de3addcde5bf875dcd43d3e2c6536d61871e7675 to get the "real" number of processors in the system.
Because of those issues Android/Dalvik also switched from _ONLN to _SC_NPROCESSORS_CONF for the Java API Runtime.availableProcessors() too [3], citing:
> Traditionally this returned the number currently online,
but many mobile devices are able to take unused cores offline to
save power, so releases newer than Android 4.2 (Jelly Bean) return the maximum number of
cores that could be made available if there were no power or heat
constraints.
The problem with sticking to _SC_NPROCESSORS_CONF however is that it breaks down in constrained environments like Docker or with an explicit CPU affinity set by the Linux `taskset` command, They'd get a higher CPU count than can be used, start more threads etc. which results in unnecessary context switches and overloaded systems. That's why we need to respect sched_getaffinity.
So while in an ideal world we would be able to rely on sched_getaffinity/_SC_NPROCESSORS_ONLN to return the number of theoretically available CPUs regardless of power saving measures everywhere, we can't do this on ARM.
I think the pragmatic solution is the following:
* use sched_getaffinity (+ fallback to _SC_NPROCESSORS_ONLN in case of error) on x86. This ensures we're inline with what OpenJDK [4] and CoreCLR [5] do
* use _SC_NPROCESSORS_CONF exclusively on ARM (I think we could eventually even get rid of the PLATFORM_ANDROID special case)
Thoughts?
Helpful links:
[1] https://sourceware.org/ml/libc-alpha/2013-07/msg00383.html
[2] https://lists.01.org/pipermail/powertop/2012-September/000433.html
[3] https://android.googlesource.com/platform/libcore/+/750dc634e56c58d1d04f6a138734ac2b772900b5%5E1..750dc634e56c58d1d04f6a138734ac2b772900b5/
[4] https://bugs.openjdk.java.net/browse/JDK-6515172
[5] https://github.com/dotnet/coreclr/blob/7058273693db2555f127ce16e6b0c5b40fb04867/src/pal/src/misc/sysinfo.cpp#L148
Do not remap framework assembly if it's version is higher than the ru…
…ntime version
Currently, assemblies like Microsoft.Build.Framework and
Microsoft.Build.Engine get remapped to the current runtime version, at
load time. This means that if an assembly references the above
assemblies with a version like `14.1.0.0`, then that would get remapped
to `4.0.0.0`. This is incorrect behavior and breaks msbuild. Instead,
now we do the remapping only if the requested version is lower than the
runtime version.
A suspended thread, in non-coop case, can be holding runtime locks. When the suspender tries to unwind the stack for this thread, it might require taking of runtime locks (ex. in aot case we might search for the jit info in the aot image and then register it, which requires the domain lock). Mark the suspender thread as running in async context while it unwinds other threads.
Currently, assemblies like Microsoft.Build.Framework and
Microsoft.Build.Engine get remapped to the current runtime version, at
load time. This means that if an assembly references the above
assemblies with a version like `14.1.0.0`, then that would get remapped
to `4.0.0.0`. This is incorrect behavior and breaks msbuild. Instead,
now we do the remapping only if the requested version is lower than the
runtime version.
@akoplinger: This will actually impact xbuild too right now because
we're defaulting to xbuild 14.0 in Mono 4.4 now so we'd see messages
like Mono: The request to load the assembly Microsoft.Build.Framework
v14.0.0.0 was remapped to v4.0.0.0 in the assembly loading log, which
will break in very subtle ways.
A few people told me it's surprising that acceptance-tests checks out repos into <mono repo>/../
and it can cause unwanted changes when a repo with the same name already exists there.
While this structure is used in several internal repos I agree for Mono it's better to keep
everything inside the repo itself.
Early ICALL_ADDR lookup on pinvoke to fix checked-build assert
The pinvoke implementation in the failure case was creating a managed
exception object while in blocking mode, a rules violation. This patch
puts a redundant lookup early so that any exceptions occur before
blocking mode is entered. (The "real" lookup is unchanged because the
current pattern is optimized well by the JIT.)
The pinvoke implementation in the failure case was creating a managed
exception object while in blocking mode, a rules violation. This patch
puts a redundant lookup early so that any exceptions occur before
blocking mode is entered. (The "real" lookup is unchanged because the
current pattern is optimized well by the JIT.)
[runtime] Set exit code on unhandled exception even if a managed handler is present
In case we would add a AppDomain.CurrentDomain.UnhandledException callback, we would not exit the VM in case of unhandled exception.
This goes against the expected behaviour, as we should indeed call these managed callbacks in all cases, but they should not stop the VM from exiting.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=39077
In case we would add a AppDomain.CurrentDomain.UnhandledException callback, we would not exit the VM in case of unhandled exception.
This goes against the expected behaviour, as we should indeed call these managed callbacks in all cases, but they should not stop the VM from exiting.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=39077
[runtime] MonoError cleanup in mono-security.c
Some easy MonoError cleanup in mono-security.c: use set_pending_exception, don't raise in helper functions, free memory on errors.
[runtime] Use MonoError in mono_runtime_object_init
Mark mono_runtime_object_init checked external only. Runtime should use mono_runtime_object_init_checked.
This is to guarantee that even in the case an exception is thrown between both calls, we switch back and forth correctly from RUNNING to BLOCKING state.
A connect flood will break the async loop and take down the entire HttpListener instance (will stop accepting connections).
To reproduce. Take a HttpListener run say 2 wrk test instances against it (1000 connections each works here). After the run the HttpListener has broken its async loop and will no longer accept any connections after. The reason for this is that it was not previously handling the case that AcceptAsync can return synchronously. In this case event will not be called with args.
Patch handles this case and the above issue is not reproducible with.
[System] Fix more hardcoded test ports
A test failed on Jenkins today with "address already in use".
Opportunistically replaced a few more hardcoded test ports to avoid failures in the future.
@monojenkins merge
A test failed on Jenkins today with "address already in use".
Opportunistically replaced a few more hardcoded test ports to avoid failures in the future.
We'd run into an error before:
```
In file included from ../../mono/utils/mono-context.h:234:0,
from ../../mono/utils/mono-stack-unwinding.h:10,
from ../../mono/utils/mono-threads.h:14,
from mono-mmap.c:36:
../../mono/arch/arm/arm-codegen.h:936:41: fatal error: mono/arch/arm/arm_dpimacros.h: No such file or directory
```
This was done for the other cases in 637ee7cbaa, but not when the monotouch
special case was added in cf754cdb63.
[reflection] MonoError for mono_reflection_type_get_handle
`mono_reflection_type_get_handle` is really a kind of normalization procedure that synthesizes real `MonoType`s from reflection's derived types such as `System.Reflection.Emit.ArrayType, ByRefType`, etc. So we call it all over the place in System.Reflection.Emit runtime code.
That would all be fine, except that users are allowed to subclass `System.Type`, so we have to call the `UnderlyingSystemType()` method to get a real type. Which means that an override can throw. Which in coop mode means we have to thread around a `MonoError`. So that's why this code is uglier.
But really this code is doing a lot of type inspection and high-level work that we should figure out how to migrate to managed.
In certain circumstances, for example when running under nunit-console, assemblies could be unloaded while the profiler still had a reference to them in order to generate the coverage data. In these cases, the memory referred to by the profiler would be invalid and would crash.
Fix this by incrementing the reference count of the assemblies preventing them from being unloaded while coverage is being collected.
Fixes BXC #39325
[utils] Get processor count from scheduler affinity if available
In case the user has set a scheduler CPU affinity, we should use it as the number of available CPU for the runtime. This is particularly important on docket which uses cgroups that set the container a cpu affinity, and by not using that, the runtime report the total number of CPU the machine has, and not the number of CPU the container has.
Fixes bug https://bugzilla.xamarin.com/show_bug.cgi?id=39279
Enable building of monotouch/xammac profiles without mono-extensions and expose mobile_static in configure.ac
Relies on https://github.com/xamarin/mono-extensions/pull/17
This enables us to compile the monotouch, xammac and mobile_static profiles on public Jenkins without mono-extensions.
[corlib] Fix three decimal tests to not rely on current culture for formatting
They'd fail on cultures where "." is not the decimal separator before.
I've also reenabled them on mobile cause the workaround for #27269 is no longer necessary since all bots are now on Mono 4.0+.
@monojenkins merge
We need this because when the runtime is built without --enable-minimal=com the runtime crashes
when trying to AOT because it tries to find the __ComObject (even though nothing in mobile_static
actually uses COM). With this shim we can AOT the mobile_static assemblies even with a "normal"
runtime.
They'd fail on cultures where "." is not the decimal separator before.
I've also reenabled them on mobile cause the workaround for #27269 is no longer necessary since all bots are now on Mono 4.0+.
In case the user has set a scheduler CPU affinity, we should use it as the number of available CPU for the runtime. This is particularly important on docket which uses cgroups that set the container a cpu affinity, and by not using that, the runtime report the total number of CPU the machine has, and not the number of CPU the container has.
Fixes bug https://bugzilla.xamarin.com/show_bug.cgi?id=39279
As it turns out, all these uses of FOREACH_THREAD_SAFE were inside regions
where the suspend lock is held. This effectively meant that no modifications
to the thread list could actually happen while the foreach was executing. Since
the guarantees provided by FOREACH_THREAD_SAFE are not needed in any of these
cases, switch to FOREACH_THREAD. The latter macro also happens to be much more
lightweight, so this should result in an overall (small) performance gain.
The caller of mono_lls_init () can now specify whether the free function can or
cannot lock, rather than the LLS assuming that it can't. The only reason this
limitation existed was because the LLS wasn't extended to allow specifying this
information in/after 56a6ae2bd0.
In addition, all LLS functions are now callable from both safe and async
context; the caller must specify the context as an additional argument. If a
free function is in use and it may lock, we'll simply enqueue it for execution
at a later point if an LLS function is invoked in async context.
1. The function suffered from the "too many bools" problem. Pass enum values
instead so that call sites are clearer.
2. Don't restrict the combination of arguments. This makes it easier to use
the function as part of data structures where the arguments aren't known
ahead of time (e.g. the LLS).
This fixes a bug that would manifest when running the profiler on a program
that rapidly spawns/joins threads.
The problem was this: Given T1 which is joining on T2, T2 which is returning
and removing itself from the thread list, and T3 which is currently in the
SIGPROF signal handler, it was possible for T3 to encounter a pointer to T2's
freed thread info structure in the thread list. This can happen because the
SIGPROF handler doesn't actually acquire the suspend lock (it can't) while
iterating the thread list.
The MONO_LLS_FOREACH_SAFE macro was supposed to protect against this case
specifically and, as a result, only provide a snapshot of the thread list.
However, it was missing some very important bits of the mono_lls_find ()
algorithm. The gory details can be found in the paper that the LLS is based on:
http://www.research.ibm.com/people/m/michael/spaa-2002.pdf
In particular, refer to the "Using Other Memory Management Methods" section
on page 6. It describes why lines E1-2 and E3-4 are necessary in order to
safely iterate the list in an implementation using hazard pointers (such as
ours). Basically, the iteration of the list has to be restarted if any sort of
modification to the list is detected during iteration. Otherwise, it's possible
to read and follow removed/freed memory, leading to all sorts of Bad Things™
happening.
Now, obviously we can't just restart a foreach loop without further thought;
that would cause all sorts of breakage due to the loop body being executed on
the same elements multiple times. As it happens, the LLS is an ordered data
structure: Elements are arranged in ascending order by key. We can exploit
this fact when iterating the list and restarting the loop. By simply storing
the last key we successfully processed, when restarting the loop, we can just
skip all keys that are not greater than the last key we successfully processed.
In addition to this, the MONO_LLS_FOREACH_SAFE macro now assists in freeing
removed elements that it encounters, just like mono_lls_find ().
The end result is that MONO_LLS_FOREACH_SAFE is now truly safe to use at any
point. It does have the potential to be ever so slightly slower, however, since
it does do significantly more work than it used to, and might even have to
restart the loop multiple times if lots of threads are modifying the thread
list at the same time. This doesn't seem to be a problem in practice, though.
Finally, to reduce the maintenance burden, the non-filtered versions of the
macros are now implemented in terms of the filtered versions, using a static
inline filter function that always returns true. When using the non-filtered
macros, the compiler will just eliminate the check entirely through its usual
optimization passes.
When generating varargs signatures for method defs, ilasm would discard the rest of the call conv. Things
like instance or explicit this would be discarded.
CoreClr allows storing a wider primitive byref into a narrower one. int32& -> int16&, for example.
This is a safe weakening as there's no risk of overflows, undefined behavior or unaligned access.
Ass oddly as it sounds, it's valid to emit the catch handler before the try block
and have it be the last sequence of instructions in the method body.
This patch checks for that condition before looking the BB in cil_offset_to_bb.
Even if we are interrupted we must throw the exception only after we have regained the lock. This prevents throwing an SynchronizationLockException in the finally clause of common scenarios like :
lock (obj) {
Monitor.Wait (obj);
}
The downside of this is that, if another thread doesn't give up the lock, we could fail to abort the thread. But this is what MS does.
A monitor enter internal method returns -1 for interrupted, 0 for timed out, 1 for acquired. When doing a monitor wait, we regain the lock using an infinite wait (which can't time out) and retrying in the case of interruption.
We don't register jit infos for tramps that don't have unwind info because we can't unwind from them. Explicitly set uw_info to NULL, instead of pointing to a zero length array in the aot image, to not confuse the runtime when registering the tramp.
Likely to fix#38267
In some cases, the null terminator in the SocketAddress may not be the last
character. This changes the algorithm to ignore the null terminator and
anything after it instead of assuming that the last character is the null
terminator.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=35004
Before when the variable wasn't set (= the default), the mobile_static profile would try to pass
the -O=gsharedvt flag even when gsharedvt is not enabled (e.g. when building without mono-extensions)
which causes AOTing to fail.
A better solution is to export the variable from configure.ac into the mcs build system depending
on whether gsharedvt is enabled at configure time.
The code in ManagedProtection.GetKey() instantiated a new RSACryptoServiceProvider singleton without proper
locking to avoid multiple threads running the initialization concurrently.
Since RSACryptoServiceProvider (or rather KeyPairPersistence underneath) uses an XML file with a name derived
from the parameters to store the data there could be a case where one thread writes while another tries to read
and gets garbage. This results in a CryptographicException later on.
Added locking to prevent this.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=38933
It failed on Jenkins today (https://jenkins.mono-project.com/job/test-mono-mainline/label=debian-amd64/3611/parsed_console/log_content.html#WARNING1)
and I could repro it after running the test in a loop.
While looking at the test, the "sum -= buf [0]" seemed suspicious to me. The final callback invocation after the last byte was read results
in buf [0] being 0 most of the time (because no bytes were read). However, due to the async nature of the test that final callback might happen
before another callback and buf then has a value, resulting in a wrong checksum if that value is substracted.
The fix is to remove the substraction. I have no idea why it was done in the first place, but it's been there since the test was added in 2003...
While at it, I've added another check for the number of bytes read and used proper Interlocked.* operations to ensure atomicity.
By using the <profile>_opt_<assembly>.sources file we can specify files
that are used when mono-extension is not enabled and add shims for the
classes or methods that are missing an implementation in that case.
This allows us to build the monotouch and xammac profiles on public Jenkins.
[threads] Fix mono_thread_info_sleep overflow
This overflow would happen of the machine was up for more than 29 days, leading to an overflow in the value returned by mono_100ms_ticks.
The simplest solution is simply to resort to mono_100ns_ticks which would only overflow after more than 29247 years. So even the platform implement this value as the time since 0000-00-00 00:00:00, we still have more than 27000 years if front of us. It's also highly unlikely that we would overflow the `end' value when adding `ms', as that would mean we are in the last 29 days of that 29247 years.
@monojenkins merge
[reflection] Get rid of loader error in reflection.c
Get rid of (direct and indirect) calls to `mono_loader_set_error_from_mono_error` in reflection.c:
- `mono_custom_attrs_from_index`
- `mono_custom_attrs_from_method`
- `mono_custom_attrs_from_class`
- `mono_custom_attrs_from_assembly`
- `mono_custom_attrs_from_module`
- `mono_custom_attrs_from_property`
- `mono_custom_attrs_from_event`
- `mono_custom_attrs_from_field`
- `mono_custom_attrs_from_param`
In all the functions, add a `_checked` variant that has a `MonoError*` out argument and for the MONO_API functions. The public API versions just clean up the error and return NULL. Loader error isn't touched anymore.
This overflow would happen of the machine was up for more than 29 days, leading to an overflow in the value returned by mono_100ms_ticks.
The simplest solution is simply to resort to mono_100ns_ticks which would only overflow after more than 29247 years. So even the platform implement this value as the time since 0000-00-00 00:00:00, we still have more than 27000 years if front of us. It's also highly unlikely that we would overflow the `end' value when adding `ms', as that would mean we are in the last 29 days of that 29247 years.
[msbuild] Getting build error "Error initializing task XmlPeek: Not registered task XmlPeek".
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=38638
Removes the uses of LogXXXFromResources from XmlPeek because
we do not use string resources in xbuild.
[runtime] Use MonoError for mono_class_vtable_full and mono_runtime_class_init_full
Switch from a boolean `raise_on_error` arg to a MonoError outarg and let
callers decide what to do with the error.
Also mark mono_runtime_class_init external only. Runtime should use
mono_runtime_class_init_full.
Switch from a boolean raise_on_error arg to a MonoError outarg and let
callers decide what to do with the error.
Also mark mono_runtime_class_init external only. Runtime should use
mono_runtime_class_init_full.
FileStream was writing to the wrong position When a value longer than
the FileStream buffer was written after a read.
This was fixed by doing a seek before doing the problematic write.
Fixes#11699
[SafeHandle] Avoid handle leakage in case of ThreadAbortException
If we get a ThreadAbortException while at SafeHandle.cs:124 (before this commit), we would not set release to true, even after incrementing the internal reference count of the SafeHandle. That could lead to potential leak, as the caller (following the common following pattern) would never call DangerousRelase, thus never calling PerformRelease. The finalizer or Dispose would not lead to the call of PerformRelease either, as the DangerousAddRef / DangerousRelase calls are unbalanced.
The common pattern for DangerousAddRef / DangerousRelease is the following:
bool release = false;
try {
safeHandle.DangerousAddRef (ref release);
/* do something with safeHandle.DangerousGetHandle () */
} finally {
if (release)
safeHandle.DangerousRelease ();
}
@monojenkins merge
[runtime] Don't access MonoClass::exception_type directly.
Use `mono_class_get_failure ()` and `mono_class_has_failed ()`
This is mostly just cleanup refactoring at this point that will make it easier to change how exception_type is used.
Also from this exercise we've learned that most of the class handling code just cares whether the class has an error or not, not what sort of error it was.
If we get a ThreadAbortException while at SafeHandle.cs:124 (before this commit), we would not set release to true, even after incrementing the internal reference count of the SafeHandle. That could lead to potential leak, as the caller (following the common following pattern) would never call DangerousRelase, thus never calling PerformRelease. The finalizer or Dispose would not lead to the call of PerformRelease either, as the DangerousAddRef / DangerousRelase calls are unbalanced.
The common pattern for DangerousAddRef / DangerousRelease is the following:
bool release = false;
try {
safeHandle.DangerousAddRef (ref release);
/* do something with safeHandle.DangerousGetHandle () */
} finally {
if (release)
safeHandle.DangerousRelease ();
}
In the case where we would call for example WaitHandle.WaitOne with a timeout, we could run in the case where we would wait way longer than the timeout would allow it (240s instead of 90s for example). This would be due to spurious wake, which would not be considered to be an error, but which would lead calling the `mono_os_cond_timedwait` function with the same timeout every time. That would mean that `mono_os_cond_timedwait` would return that it timedout, if and only if, it would timed out after N milliseconds, with N the millisecondsTimeout parameter to WaitOne in this example. That mean that any spurious wake would effectively lead to rewaiting with a full timeout every time.
That could lead to the following scenario:
- T1 calls waitHandle.WaitOne(timeout = 1000ms) at T=0ms
- T1 calls mono_os_cond_timedwait(timeout = 1000) at T=0ms
- GC suspend T1 at T=500ms <- here it's not necessarily the GC, it can be anything suspending/signaling the thread
- T1 spurious wake, and call mono_os_cond_timedwait(timeout = 1000) at T=500ms
- GC interrupt T1 at T=1250ms
- T1 spurious wake, and call mono_os_cond_timedwait(timeout = 1000) at T=1250ms
- etc.
As you can see, T1 should have returned from waitHandle.WaitOne before the second spurious wake, but as it recalled mono_os_cond_timedwait with a timeout of 1000ms again, it didn't.
The fix consists in keeping track of when the call is supposed to finish, and pass an adaptable timeout to mono_os_cond_timedwait, which decreases as time goes on, to enventually reach 0, and return in a timely manner.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=38382
We do not want to wait on a handle that have been cleaned up, as that will lead to a warning message from the io-layer, without any error in managed code. That mean we could not detect this error case and recover from it.
By doing this DangerousAddRef / DangerousRelease before going to native, we ensure that the WaitHandle is and will stay alive while waiting on it. This same is done in CoreCLR, but in the marshalling code.
When waiting on the current process, we would return WAIT_FAILED, as waitpid would return -1, with errno set to ECHILD.
But the semantic of WaitForSingleObjectEx and WaitForMultipleObjectsEx when waiting on current process is to return WAIT_TIMEOUT. This is then a special case that we need to add to this code.
dmcs was deprecated in favor of mcs a while back, yet people are still creating new scripts with it.
Adding a message that shows up on each dmcs invocation should hopefully make this more obvious.
We can use mono_class_load_from_name in all cases that we previously used to assert after loading.
All optional types have being moved over to use GENERATE_TRY_GET_CLASS_WITH_CACHE.
All but System.Runtime.ConstrainedExecution.CriticalFinalizerObject which is used within
sgen and we must ensure we don't try to lazy load it there.
I'm not 100% sure whether it should be optional, maybe it's a leftover from the 1.0 -> 2.0 timeframe?
The problem is that marking more types as required would be a burden on linkers as they might not force
it to be preserved.
I'm not particularly happy with the current large set of types that are loaded upfront. A significant
number of them could be lazy loaded.
This function generates probing functions that rely on mono_class_try_load_from_name.
In addition replace usage of hand crafted loading code from all around.
Those are variants of mono_class_from_name_checked that implement two common type loading scenarios in the rutime.
The first is mono_class_load_from_name, that loads critical types that the runtime cannot function without.
This means we abort if the type is missing or broken. We should use it in places where today we crash when
either of those would happen.
The other is mono_class_try_load_from_name, which is ok if the type is not found, but will abort if the type is broken.
The reasoning for aborting on a broken type is that it's unclear on whether the runtime should be using the type or not,
specially given most of those optional types change behavior of the application. This, in practice, forces the linker to
not produce a broken corlib.
[System.Drawing.Printing] CUPS default PageSize and InputSlot should …
[System.Drawing.Printing] CUPS default PageSize and InputSlot should override PPD defaults.
There is already a test case for that (PrintingServicesUnixTest.cs, Bug602934_PrinterSettingsReturnActualValues) which fails when user configured options are different from PPD settings. Patch also makes this test not to fail if "Resolution" option is not present in CUPS options.
This is useful for coop mode only, as coop needs additionnal thread state transition from BLOCKING to RUNNING and vice versa, and if both these transitions are not done properly, we finish in a incorrect state.
In non-coop mode, the mono_jit_detach only sets the domain to NULL, to avoid scanning threads unnecessarily, so not actually detaching properly is not hurting anything.
Previously, when we would invoke a native-to-managed wrapper from a thread that has not been attached to the runtime, we would simply ensure to attach this thread to the runtime. The issue arise when using coop and needing to switch a thread which is not executing runtime or managed code to BLOCKING state.
That would arise on iOS when using dispatch_async:
- the user would enqueue a Delegate, triggering a call to mono_delegate_to_ftnptr, generating this native-to-managed wrapper
- the wrapper would be called asynchronously on a thread managed by the OS/dispatch_async
- the thread would attach to the runtime, switching its state from STARTING -> RUNNING
- the thread would finish executing the delegate, thus 'detaching' from the runtime, but without switching its state from RUNNING -> BLOCKING
- during the next garbage collection, the GC would try to suspend this thread as it is still in RUNNING state, but because it's neither executing runtime code, neither managed code, it will never hit a safepoint, leading to a timeouting suspend.
To fix that issue, we need to support the following state switches:
- STARTING -> RUNNING -> BLOCKING: happens when it's the first time this thread calls a native-to-managed wrapper
- BLOCKING -> RUNNING -> BLOCKING: happens when it's not the first time this thread is called from a facility like dispatch_async, or if it is, for example, called as a callback from an external library
- RUNNING -> RUNNING -> RUNNING: if we call this wrapper from runtime or managed code, see for example System.Reflection.Emit.ILGeneratorTest.TestEmitCalliWithNullReturnType
This facility also need to take care of the current domain, to ensure that we switch properly between domains, and set it to the one that was used when creating this delegate, then restoring the original one.
The previous implementation of ProcessWrapper OutputStreamChanged and ErrorStreamChanged would manually launch a background thread to read on the process StandardOutput and StandardError, then simply passing the output to the OutputStreamChanged and ErrorStreamChanged events. This mean that even the newline characters would be passed to these events, leaving these events callbacks deal with splitting the output line by line.
On the other hand, Process.OutputDataReceived and Process.ErrorDataReceived already split the data line by line, and discard the newline character. That implies that, to keep the old ProcessWrapper behaviour, we need to add these newlines character before calling OutputStreamChanged and ErrorStreamChanged events callbacks.
We observe the following error when building XS:
"Threadpool worker" tid=0x0x7f011bfff700 this=0x0x7f01394bfb30 thread handle 0x432 state : not waiting owns ()
at <unknown> <0xffffffff>
at (wrapper managed-to-native) System.Threading.WaitHandle.WaitAll_internal (System.Threading.WaitHandle[],int,bool) <0x0005d>
at System.Threading.WaitHandle.WaitAll (System.Threading.WaitHandle[]) <0x00026>
at Microsoft.Build.Utilities.ProcessWrapper.<Start>m__0 (object,System.EventArgs) <0x0008f>
at System.Diagnostics.Process.OnExited () <0x000ed>
at System.Diagnostics.Process.RaiseOnExited () <0x0009f>
at System.Diagnostics.Process.CompletionCallback (object,bool) <0x00017>
at System.Threading.RegisteredWaitHandle.DoCallBack (object) <0x00088>
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (object) <0x00058>
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x001c6>
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x00020>
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0x0004c>
at System.Threading.ThreadPoolWorkQueue.Dispatch () <0x001d6>
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0x00008>
at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) <0x0005a>
And the process output would contains a bunch of "_wapi_handle_ref: Attempting to ref unused handle 0x1234" and "_wapi_handle_unref_full: Attempting to unref unused handle 0x1234".
The expected behaviour is for WaitHandle.(WaitAll|WaitAny|WaitOne|SignalAndWait) and EventWaitHandle.Set to throw an ObjectDisposedException in case the WaitHandle (or one of them if calling WaitAll, WaitAny or SignalAndWait) have been disposed.
This error would be due to a race between ProcessWrapper.Dispose, which closes endEventOut and endEventErr, and the above Process.Wrapper.<Start>m__0 (the base.Exited) callback, which waits on endEventOut and endEventErr.
There is already a test case for that (PrintingServicesUnixTest.cs,
Bug602934_PrinterSettingsReturnActualValues) which fails when user
configured options are different from PPD settings. Patch also makes
this test not to fail if "Resolution" option is not present in CUPS
options.
*Update: removed leftover debug Console.WriteLine from patch
The internal GetLegacyCertificatePolicy() now returns null unless this
property has explicitly been assigned to, so the legacy handler will not
be used in ChainValidationHelper.
(cherry picked from commit e69e878405)
Fix mobile/AOT build following System.Diagnostics.Process import
System.Diagnostics.Process was recently added to Mono. This introduced two problems for mobile builds:
- Support classes were not added to the mobile profile .sources.
- The imported referencesource code did not build on fullaot.
This PR fixes both issues.
[Microsoft.Build] Fix Microsoft.Build.Execution.BuildManagerTest.BasicManualParallelBuilds
The previous implementation of ProcessWrapper would launch its own threads to read on the process stdout and stderr. We can more simply use OutputDataReceived and ErrorDataReceived which will do the same under the hood. We use as well the Exited event to track the end of the Process.
The previous implementation of ProcessWrapper would launch its own threads to read on the process stdout and stderr. We can more simply use OutputDataReceived and ErrorDataReceived which will do the same under the hood. We use as well the Exited event to track the end of the Process.
If we construct RegisteredWaitHandle with executeOnlyOnce = true, and never call Unregister, we will keep _waitObject.SafeWaitHandle reference count unbalanced. This will lead to never releasing the waitObject handle.
As we only need to ensure that the waitObject handle is alive while in the Wait method, we use the SafeHandle mechanism only there. The 2 possibles scenarios in the Wait method are:
- waitObject has not been closed: we increment its ref count, so we ensure that it's alive as long as we use it here
- waitObject has been closed: a ObjectDisposedException is thrown, we catch it and keep on without trying to wait on the handle
- fixes#38818
- setting the *unchanged* scope to non-marked types is harmful,
because in case the type has declaring type, the scope of declaring
type is set as well. see
https://github.com/mono/cecil/blob/master/Mono.Cecil/TypeReference.cs#L132
public virtual IMetadataScope Scope {
get ...
set {
var declaring_type = this.DeclaringType;
if (declaring_type != null) {
declaring_type.Scope = value;
return;
}
scope = value;
}
}
so in our case, we first set correctly scope of
System.Collections.Generic.Dictionary`2 to mscorlib, then (re)set
scope of System.Collections.Generic.Dictionary`2/ValueCollection
and
System.Collections.Generic.Dictionary`2/ValueCollection/Enumerator
to System.Collection (note that these types are not marked and are
swept away by linker), which sets scope of its declaring type
System.Collections.Generic.Dictionary`2 back to System.Collection
and because the System.Collection is linked-out, the typeref is
incorrectly written with the scope set to System later
In the following case we would have a race between Wait and Unregister methods:
- T1 calls ThreadPool.RegisterWaitForSingleObject to wait on waitHandle
- registeredWaitHandle is constructed to wait on waitHandle
- registeredWaitHandle.Wait method is enqueued on the ThreadPool
- T1 calls registeredWaitHandle.Unregister
- T1 calls waitHandle.Close (happens in Process.StopWatchingForExit)
- TTP (the threadpool thread) calls registeredWaitHandle.Wait
- TTP calls WaitHandle.WaitAny (waitHandle, ...)
The last step try to access the waitHandle that has been previously closed, leading to a _wapi_handle_ref / _wapi_handle_unref_full error message.
The fix is to ensure that the RegisteredWaitHandle keeps the WaitHandle alive. To achieve this, we simply use the SafeWaitHandle inside the WaitHandle by: increasing its ref count when constructing the RegisteredWaitHandle, and decrementing it when unregistering it.
We also increase the count when using it in Wait, so if we execute Unregister and Wait concurrently, we guarantee that the WaitHandle is still valid in Wait, or get an exception if not.
FileInfo.MoveTo was not moving a file from one directory to another when
the file name did not change.
Fixes#18361.
An ArgumentException when destFileName is an empty array was added,
otherwise FileInfoTest.MoveTo_DestFileName_Empty would fail. Both
Path.GetFullPath and File.Move throw similar exceptions but
MoveTo_DestFileName_Empty expects the one from File.Move so it was added
to this method.
Mono has a list of assemblies considered "internal", whose versions are automatically remapped to the distro default version.
Since we no longer bundle Npgsql, it is up to individual distributions to package their own Npgsql - but Mono cannot load any version other than 4.0.0.0 from the GAC, due to this remapping:
Mono: Assembly Ref addref npghello[0x17e2fd0] -> mscorlib[0x1789c00]: 2
Mono: The request to load the assembly Npgsql v2.2.7.0 was remapped to v4.0.0.0
Mono: Assembly Loader probing location: '/usr/lib/mono/gac/Npgsql/4.0.0.0__5d8b90d52f46fda7/Npgsql.dll'.
Npgsql, Version=2.2.7.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7
This was added in 2687866e99, but on systems where /bin/sh (which make uses by default) isn't bash this doesn't work.
Fix is to use a standard switch-case which works in all shells.
Do not run 4GB string tests on low memory systems (fixes#38392)
"Low memory" here is defined as 32-bit, less than 6GB of physical RAM, or mobile (since mobile lacks the class for querying physical RAM).
Without this patch, one of the corlib tests allocates 4 GB of RAM, which causes OOM crashes on low-memory Jenkins instances and also mobile phones.
This reverts commit a7955a93f5.
The Process import commit was reverted in f74f703719,
this reverts the added tests as well since they fail on Jenkins now.
The implementation added in b5d0ffe7dd forgot to multiply vmstat.free_count (== number of free pages)
by the page size in the __APPLE__ case, making the reported number much smaller than the reality.
[reflection] Signal pending loader error in mono_reflection_get_custom_attrs_data
Fixes [Bugzilla 38222](https://bugzilla.xamarin.com/show_bug.cgi?id=38222)
While we're at it, convert `mono_reflection_get_custom_attrs_data` to use
`MonoError` and mark it external only. Runtime should use
`mono_reflection_get_custom_attrs_data_checked`
`sgen_ensure_free_space()` assumed that if `size` was larger than
`SGEN_MAX_SMALL_OBJ_SIZE`, it should find free space on the major
heap, so it wouldn't consider triggering a nursery allocation. Making
TLABs larger than that would thus never trigger nursery collections
anymore.
Now we pass in the generation that says where we want space.
The Mono cert store traditionally uses the SKI for the filename of a certificate, but this does not handle cases where a cert has been reissued with the same SKI but different serial (seen in the wild with StartCom certs). This patch appends the serial to all new imported certs, and attempts to interoperate cleanly with existing serial-less certificate stores to avoid duplication.
Closes: #30902
Allows opening a character device or named fifo more than once with code like:
using (var f = new FileStream ("/dev/zero", FileMode.Open))
{
...
}
using (var f = new FileStream ("/dev/zero", FileMode.Open))
{
...
}
[System] Fix a few more hardcoded ports in tests
They'd cause "address already in use" errors when two builds are running concurrently on the same machine on Jenkins.
@monojenkins merge
InternalThread is a fundamentally different type WRT to appdomain. All instances of Thread bound to the same
os thread have a reference to a shared InternalThread.
InternalThread is always allocated in the root domain, so it MUST NOT have references to other domains as
it will lead to crashes on unload.
This was the case with pending_exception. It could point to an exception meant to the domain unloading.
This bug became more and more frequent as we move to use mono_set_pending_exception in more places. I could
reproduce it in 10 minutes of looping appdomain-unload.exe.
Separate the code path for async and self abort/suspend. This make the code intention more clear. It also allows us to remove some unused parameters (can_raise_excetpion which is always TRUE). Add some comments and harmonise locking behaviours : async_suspend_internal and self_suspend_internal now both call UNLOCK_THREAD.
On Windows we saw a failure like the following in mcs tests:
```
cs1589-2.cs... REGRESSION (CORRECT ERROR -> WRONG ERROR MESSAGE)
Exp: Unable to include XML fragment `/foo/bar' of file `there-is-no-such-file'. Could not find file "*PATH*/there-is-no-such-file"
Was: Unable to include XML fragment `/foo/bar' of file `there-is-no-such-file'. Could not find file "C:\j\workspace\z\label\w64\mcs\errors\there-is-no-such-file"
```
Fixing it by replacing the backslash with forward slash before comparing it to the expected value.
When MONO_PATH is set, the corlib lookup returns early on in mono_assembly_load_corlib (),
without adding the Facades dir to default_path.
This causes any app/library that relies on the facades to fail at runtime.
Fixing this by always doing the Facades lookup when returning the found corlib image.
Fixes [Bugzilla 38222](https://bugzilla.xamarin.com/show_bug.cgi?id=38222)
While we're at it, convert mono_reflection_get_custom_attrs_data to use
MonoError and mark it external only. Runtime should use
mono_reflection_get_custom_attrs_data_checked
fix for the following problem:
> [13:33:37] src/System/Test/System.Diagnostics/ProcessTest.cs(1048,19): error CS0103: The name `GetCrossPlatformStartInfo' does not exist in the current context
[threadpool] Improve the monitor thread heuristic
Because the ThreadPool heuristic is optimized for short lived work item, it has more difficulty when executing long lived work item. That case can lead to a starvation of the worker threads: they are all doing work, while there is outstanding request, and these requests are not satisfied because we reached the max number of working thread.
To fix that issue, we have the monitor thread, whose sole job is to unstuck this kind of starvation cases. Unfortunately, it only works when all the worker threads are in the ThreadState.WaitSleepJoin state, which excludes the case of calling an IO operation in a Task. That includes the case of FileStream.BeginRead/Write/... which implementation can be simplified as follows: FileStream.BeginRead(...) -> Task.Run(() => FileStream.Read(...)).
The way we implement it in this commit is: every MONITOR_INTERVAL (500ms here), we check if there is any outstanding request, and if so, we assume that we are in the starvation case, and we simply increase the max number of working thread. Also to reduce the number of false positives, we do that only if there has been no completed work item for more than at least MONITOR_INTERVAL (in case of low CPU usage, more otherwise, see monitor_sufficient_delay_since_last_dequeue). This case is typically the case where we have all working thread stuck in long running work items.
Finally we increase the monitor interval from 100ms to 500ms so we guarantee we do not create more than 2 threads per second in the monitor thread. That is the value used by the old threadpool.
Because the ThreadPool heuristic is optimized for short lived work item, it has more difficulty when executing long lived work item. That case can lead to a starvation of the worker threads: they are all doing work, while there is outstanding request, and these requests are not satisfied because we reached the max number of working thread.
To fix that issue, we have the monitor thread, whose sole job is to unstuck this kind of starvation cases. Unfortunately, it only works when all the worker threads are in the ThreadState.WaitSleepJoin state, which excludes the case of calling an IO operation in a Task. That includes the case of FileStream.BeginRead/Write/... which implementation can be simplified as follows: FileStream.BeginRead(...) -> Task.Run(() => FileStream.Read(...)).
The way we implement it in this commit is: every MONITOR_INTERVAL (500ms here), we check if there is any outstanding request, and if so, we assume that we are in the starvation case, and we simply increase the max number of working thread. Also to reduce the number of false positives, we do that only if there has been no completed work item for more than at least MONITOR_INTERVAL (in case of low CPU usage, more otherwise, see monitor_sufficient_delay_since_last_dequeue). This case is typically the case where we have all working thread stuck in long running work items.
Finally we increase the monitor interval from 100ms to 500ms so we guarantee we do not create more than 2 threads per second in the monitor thread. That is the value used by the old threadpool.
There's no managed code calling mono_domain_try_unload, so raising was
wrong. Thread the error out through exc which gets checked by callers.
Fixup for 45bd9bb5d5
The concurrent collector typically uses more memory because more memory is allocated during the concurrent phase, which also leads to more objects surviving the major collections and higher allowances. We improved the memory situation by forcing synchronous finishing of the concurrent mark if too much memory is allocated during the concurrent mark phase. Still, this doesn't tackle the underlying problem. Even if we have less memory allocated during the CMS phase, some is still allocated and leads to increased overall memory usage. And how much memory growth should we allow in the first place ? The even greater problem is that it appears to force a lot of synchronous finishes, significantly increasing finishing pause times.
This commit attempts to solve this problem by following an ideal mathematical approach. We use the following notations : H - heap size at start of major collection, Gs - heap growth during sync collection, Gc - heap growth during conc collection, s - survival rate of the collection, a - allowance ratio (0.33), Ts - trigger for next major sync, Tc - trigger for next major conc.
Assume we have the heap size H at the start of a major collection. For the two types of majors we would have :
Ts = (H + Gs) * s * (a + 1)
Tc = (H + Gc) * s * (a + 1)
Obviously the problem is that Gc > Gs, which leads to Tc > Ts and higher memory usage. In order to have the two triggers equal we need to find an X that we can subtract from Tc to obtain Ts. Considering Gs = 0 (only a few nursery objects are promoted during a major sync collection) and similar survival rates for the two types of collection, we get X = Gc * s * (a + 1). In order to get equal triggers we would need to subtract this value from the allowance computed after a concurrent collection.
The more the heap grows, the more X grows. If X gets bigger than the available starting allowance ((H + Gc) * s * a), it would mean that we are trying to set a trigger that it is lower than the actual heap size at the end of the collection (which means we already exceeded the target memory usage). In order to avoid this problem we need to enforce Tc - X > (H + Gc) * s. Doing the substitutions we get Gc < a * H. When the heap growth exceeds this value it means we need to force the finish of the concurrent collection.
* keep a reference to a __ComObject's proxy so it doesn't get garbage
collected
* don't create __ComObjects unnecessarily
* Fix for bug 24974
Change-Id: Ia09237b7f1af996405134c250bbc29220f81f067
Docker uses aufs or overlayfs to mount the root system in a
container. This change fixes Xamarin-31021 so that the drives
show up properly.
This change is released under the MIT license.
Change-Id: I5150b0e82f26e9dbd07bbcbc50e7ca50d856dde8
A kevent() syscall may "fail" with EINTR if the process receives
a signal before the timeout expires and before an event is placed
in the kqueue; in this case, rather than erroring out, the
syscall should simply be retried.
Keeping threads around can cause processes to keep running when they
shouldn't, particularly on Windows when the entry point returns.
This commit licensed as MIT/X11.
While serializing operations with XmlSerializerFormatAttribute,
attributes xmlns:xsi and xmlns:xsd were not added to the message body.
XmlBodyWriter now adds the missing attributes.
Fixes#34413.
When a frame marked with OPCODE 0, it is a continuation frame. This frame does not carry the message type and should be derived from the previous frame.
Without this patch you can't use "WebResource.axd" ressources
in a cluster (each mono instance will get a different link for the same ressource)
Signed-off-by: Etienne CHAMPETIER <[email protected]>
2014-04-01 11:42:09 +02:00
14475 changed files with 3780742 additions and 69850 deletions
This directory contains acceptance tests, handled by optional non git submodule based submodules. Run the tests via "make check-full" (there are also targets for individual test suites).
Acceptance tests
================
The SUBMODULES.json file stores information about the submodules, and make targets are used to check out submodules, check their versions, and update the submodule information.
This directory contains acceptance tests, which are third party test suites and frameworks that are used to validate Mono against a wider range of test cases that go beyond the Mono unit tests that run as part of CI.
By convention, submodules repositories are at the same level as the mono repository.
In order to make checking out those test suites optional we don't use traditional git submodules, but instead clone them on demand when needed. The custom submodule repositories are checked out into the acceptance-tests/external/ directory.
Make targets available:
## Usage
make check-<module> - Checks whenever <module> is checked out and matches the version in the info file.
make reset-<module> - Clones <module>if neccesary and checks out the revision specified in the info file.
make bump-<module> - Updates the revision stored in the info file for <module>.
make bump-current-<module> - Updates the revision stored in the info file for <module> to the current revision.
make commit-bump-<module> - Same as make bump-<module>, but commits the change.
make commit-bump-current-<module> - Same as make bump-current-<module>, but commits the change.
Running all test suites is possible via "make check-full". There are also targets for running individual test suites, see below.
Some of the test suites require an installed Mono (i.e. they don't work with the in-tree build), those will ask you to pass in the PREFIX variable pointing to the installation directory when invoking make. Note that this directory needs to be writable as we overwrite some files there as part of testing.
## Individual test suites and targets
*`make check-ms-test-suite` - Runs tests that were shared with Xamarin, those are not available publically and will be skipped when the repository is not accessible.
*`make check-roslyn` - Runs the Roslyn test suite.
*`make check-coreclr` - Runs the CoreCLR test suite.
*`make coreclr-runtest-coremanglib` - Runs only the CoreMangLib portion of the CoreCLR tests, those tests mostly target the BCL behavior.
*`make coreclr-runtest-basic` - Runs only the CoreCLR tests that target runtime behavior and stability.
*`make coreclr-compile-tests` - Convenience target that precompiles all the test cases in parallel.
*`make coreclr-gcstress` - Runs the CoreCLR GC stress tests.
## Updating submodules
The SUBMODULES.json file stores information about the submodules, and make targets are used to check out submodules, check their versions, and update the submodule information:
*`make validate-<module>` - Checks whenever `<module>` is checked out and matches the version in the info file.
*`make reset-<module>` - Clones `<module>` if neccesary and checks out the revision specified in the info file.
*`make bump-<module>` - Updates the revision stored in the info file for `<module>`.
*`make bump-current-<module>` - Updates the revision stored in the info file for `<module>` to the current revision.
*`make commit-bump-<module>` - Same as `make bump-<module>`, but commits the change.
*`make commit-bump-current-<module>` - Same as `make bump-current-<module>`, but commits the change.
Example: when making a change in the CoreCLR submodule (like disabling something on Mono), you'd commit the change there and then run `make bump-current-coreclr` or `make commit-bump-current-coreclr` to update the version in SUBMODULES.json.
@@ -1083,6 +1074,7 @@ if test x$host_win32 = xno; then
dnl hires monotonic clock support
AC_SEARCH_LIBS(clock_gettime, rt)
AC_CHECK_FUNCS(clock_nanosleep)
dnl dynamic loader support
AC_CHECK_FUNC(dlopen, DL_LIB="",
@@ -1156,6 +1148,7 @@ if test x$host_win32 = xno; then
AC_CHECK_FUNCS(getrlimit)
AC_CHECK_FUNCS(prctl)
AC_CHECK_FUNCS(sched_getaffinity)
AC_CHECK_FUNCS(sched_setaffinity)
AC_CHECK_FUNCS(sched_getcpu)
@@ -2323,6 +2316,9 @@ else
#include <winsock2.h>
#include <ws2tcpip.h>
], [
#ifndef inet_pton
(void) inet_pton;
#endif
inet_pton (0, NULL, NULL);
], [
# Yes, we have it...
@@ -2455,7 +2451,7 @@ fi
AC_ARG_ENABLE(bcl-opt, [ --disable-bcl-opt BCL is compiled with no optimizations (allows accurate BCL debugging)], test_bcl_opt=$enableval, test_bcl_opt=yes)
AC_ARG_ENABLE(perf-events, [ --enable-perf-events Enable using `perf` for profiling on Linux], test_perf_events=$enableval, test_perf_events=no)
if test "x$test_perf_events" == "xyes"; then
if test "x$test_perf_events" = "xyes"; then
AC_DEFINE(ENABLE_PERF_EVENTS, 1, [Enable using `perf` for profiling on Linux])
AC_SUBST(ENABLE_PERF_EVENTS)
fi
@@ -2611,20 +2607,50 @@ if test "x$enable_llvm" = "xyes"; then
if test "x$enable_llvm_version_check" = "xyes"; then
if test "$llvm_version" != "$expected_llvm_version"; then
AC_MSG_ERROR([Expected llvm version $expected_llvm_version, but llvm-config --version returned $llvm_version"])
fi
fi
else
AC_MSG_ERROR([Compiling with stock LLVM is not supported, please use the Mono LLVM repo at https://github.com/mono/llvm, with the GIT branch which matches this version of mono, i.e. 'mono-2-10' for Mono 2.10.])
fi
# The output of --cflags seems to include optimizations flags too
LLVM_CFLAGS=`$LLVM_CONFIG --cflags | sed -e 's/-O2//g' | sed -e 's/-O0//g' | sed -e 's/-fomit-frame-pointer//g' | sed -e 's/-fPIC//g'`
LLVM_CXXFLAGS=`$LLVM_CONFIG --cxxflags | sed -e 's/-O2//g' | sed -e 's/-O0//g' | sed -e 's/-fomit-frame-pointer//g' | sed -e 's/-fPIC//g'`
if test $llvm_api_version -gt 100; then
# The --cflags argument includes all kinds of warnings -pendantic etc.
if test "x$enable_llvm_version_check" == "xyes"; then
if test "$llvm_version" != "$expected_llvm_version"; then
AC_MSG_ERROR([Expected llvm version $expected_llvm_version, but llvm-config --version returned $llvm_version"])
fi
fi
else
AC_MSG_ERROR([Compiling with stock LLVM is not supported, please use the Mono LLVM repo at https://github.com/mono/llvm, with the GIT branch which matches this version of mono, i.e. 'mono-2-10' for Mono 2.10.])
@@ -2993,6 +3010,17 @@ if test "x$host" != "x$target"; then
;;
esac
;;
arm*-netbsd*-eabi*)
TARGET=ARM;
arch_target=arm;
AC_DEFINE(TARGET_ARM, 1, [...])
ACCESS_UNALIGNED="no"
CPPFLAGS="$CPPFLAGS -D__ARM_EABI__"
# Can't use tls, since it depends on the runtime detection of tls offsets
# in mono-compiler.h
with_tls=pthread
target_mach=no
;;
i686*-linux-*)
TARGET=X86;
arch_target=x86;
@@ -3020,6 +3048,7 @@ if test "x$host" != "x$target"; then
arch_target=amd64;
AC_DEFINE(TARGET_AMD64, 1, [...])
AC_DEFINE(TARGET_PS4, 1, [...])
AC_DEFINE(DISABLE_HW_TRAPS, 1, [...])
CPPFLAGS="$CPPFLAGS"
# Can't use tls, since it depends on the runtime detection of tls offsets
# in mono-compiler.h
@@ -3083,6 +3112,45 @@ SPARC64)
;;
esac
case "$HOST" in
X86)
AC_DEFINE(HOST_X86, 1, [...])
;;
AMD64)
AC_DEFINE(HOST_AMD64, 1, [...])
;;
ARM)
AC_DEFINE(HOST_ARM, 1, [...])
;;
ARM64)
AC_DEFINE(HOST_ARM64, 1, [...])
;;
POWERPC)
AC_DEFINE(HOST_POWERPC, 1, [...])
;;
POWERPC64)
AC_DEFINE(HOST_POWERPC, 1, [...])
AC_DEFINE(HOST_POWERPC64, 1, [...])
;;
S390X)
AC_DEFINE(HOST_S390X, 1, [...])
;;
MIPS)
AC_DEFINE(HOST_MIPS, 1, [...])
;;
IA64)
AC_DEFINE(HOST_IA64, 1, [...])
;;
SPARC)
AC_DEFINE(HOST_SPARC, 1, [...])
;;
SPARC64)
AC_DEFINE(HOST_SPARC64, 1, [...])
;;
esac
dnl *************
dnl *** VTUNE ***
dnl *************
@@ -3214,8 +3282,10 @@ case "$host" in
GTKX11="libgtk-x11-2.0.dylib"
;;
*-*-*netbsd*)
LIBC="libc.so.12"
INTL="libintl.so.0"
LIBC="libc.so"
INTL="libintl.so"
SQLITE="libsqlite.so"
SQLITE3="libsqlite3.so"
;;
*-*-kfreebsd*-gnu)
LIBC="libc.so.0.1"
@@ -3263,6 +3333,12 @@ AC_ARG_WITH([libgdiplus],
# default install location
libgdiplus_install_loc=libgdiplus${libsuffix}
case "$host" in
*-*-*linux*)
libgdiplus_install_loc=libgdiplus${libsuffix}.0
;;
esac
case $with_libgdiplus in
no|installed)
libgdiplus_loc=
@@ -3480,6 +3556,7 @@ AC_ARG_WITH(monotouch_watch, [ --with-monotouch_watch=yes,no If you want to b
AC_ARG_WITH(monotouch_tv, [ --with-monotouch_tv=yes,no If you want to build the Xamarin.TVOS assemblies (defaults to no)], [], [with_monotouch_tv=no])
AC_ARG_WITH(bitcode, [ --with-bitcode=yes,no If bitcode is enabled (defaults to no)], [], [with_bitcode=no])
AC_ARG_WITH(xammac, [ --with-xammac=yes,no If you want to build the Xamarin.Mac assemblies (defaults to no)], [], [with_xammac=no])
AC_ARG_WITH(mobile_static, [ --with-mobile_static=yes,no If you want to build the mobile_static assemblies (defaults to no)], [], [with_mobile_static=no])
OPROFILE=no
AC_ARG_WITH(oprofile,[ --with-oprofile=no,<oprofile install dir> Enable oprofile support (defaults to no)],[
AC_ARG_WITH(checked_build, [ --with-checked-build=yes|noEnable checked build (expensive asserts)) (defaults to no)],[
if test x$with_checked_build != xno ; then
AC_DEFINE(CHECKED_BUILD,1,[Enablecheckedbuild.])
AC_ARG_ENABLE(checked_build, [ --enable-checked-build=LISTTo enable checked build (expensive asserts), configure with a comma-separated LIST of checked build modules and then include that same list in the environment variable MONO_CHECK_MODE at runtime. Recognized checked build modules: all, gc, metadata, thread],[
AC_ARG_ENABLE(checked_build_test, [ --enable-checked-build-test=LIST drop support for LIST checked build tests. LIST is a comma-separated list from: gc, metadata, thread.],
[
for feature in `echo "$enable_checked_build_test" | sed -e "s/,/ /g"`; do
This license governs use of the accompanying software. If you use the software, you accept this license. If you do not accept the license, do not use the software.
1. Definitions
The terms “reproduce,” “reproduction,” “derivative works,” and “distribution” have the same meaning here as under U.S. copyright law.
A “contribution” is the original software, or any additions or changes to the software.
A “contributor” is any person that distributes its contribution under this license.
“Licensed patents” are a contributor’s patent claims that read directly on its contribution.
2. Grant of Rights
(A) Copyright Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free copyright license to reproduce its contribution, prepare derivative works of its contribution, and distribute its contribution or any derivative works that you create.
(B) Patent Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free license under its licensed patents to make, have made, use, sell, offer for sale, import, and/or otherwise dispose of its contribution in the software or derivative works of the contribution in the software.
3. Conditions and Limitations
(A) No Trademark License- This license does not grant you rights to use any contributors’ name, logo, or trademarks.
(B) If you bring a patent claim against any contributor over patents that you claim are infringed by the software, your patent license from such contributor to the software ends automatically.
(C) If you distribute any portion of the software, you must retain all copyright, patent, trademark, and attribution notices that are present in the software.
(D) If you distribute any portion of the software in source code form, you may do so only under this license by including a complete copy of this license with your distribution. If you distribute any portion of the software in compiled or object code form, you may only do so under a license that complies with this license.
(E) The software is licensed “as-is.” You bear the risk of using it. The contributors give no express warranties, guarantees or conditions. You may have additional consumer rights under your local laws which this license cannot change. To the extent permitted under your local laws, the contributors exclude the implied warranties of merchantability, fitness for a particular purpose and non-infringement.
(F) If you distribute the software or derivative works with programs you develop, you agree to indemnify, defend, and hold harmless all contributors from any claims, including attorneys’ fees, related to the distribution or use of your programs. For clarity, you have no such obligations to a contributor for any claims based solely on the unmodified contributions of that contributor.
(G) If you make any additions or changes to the original software, you may only distribute them under a new namespace. In addition, you will clearly identify your changes or additions as your own.
Loaded 100 of 14475 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.