Roslyn looks only inside `$msbuild_bin/Roslyn/` and does not fallback to
the framework path. msbuild worked around this by setting `CscToolPath`:
https://github.com/mono/msbuild/commit/157a0f4d1a7aadd4e561e3ca8c784cbc2529926f
But this kicks in only if the default msbuild target files are imported!
If the `Csc` task is being used without importing them, then we fail
with
`error MSB3082: Task failed because "csc.exe" was not found.`
`Microsoft.CodeAnalysis.VisualBasic.dll` is the only file not being
symlinked, after this change. So, instead we will symlink all the files.
Fixes bxc #60340 .
Cherry-pick 26ccf75f06
quote from the darwin man page:
> The functions getpwnam_r(), getpwuid_r(), and getpwuuid_r()
> return 0 if no error occurred, or an error number to indicate
> failure. It is not an error if a matching entry is not found.
> (Thus, if result is NULL and the return value is 0, no
> matching entry exists.)
When I simplified MONO_CORLIB_VERSION to only include major.minor
instead of major.minor.build in https://github.com/mono/mono/pull/5449
I forgot to apply the same change to the msvc copy of that logic.
(cherry picked from commit 43757d67ef)
TestHelper was previously excluded because the XA tests weren't built
as part of Mono but rather as part of XA test suite which included
its own version of the helper. Now, however (https://github.com/xamarin/xamarin-android/commit/e9daf5ea35709ce1bc6ba81c7c37d72d385bace2),
XA uses BCL+friends test assemblies built by Mono itself and so
we need to restore TestHelper in the build.
Up until NDK release 14 Android SDK included a separate set of C header
files for each supported platform. NDK 14 introduced a new set of those
headers (they are NOT the same as the old ones) which unifies support
for all of the platforms so that bug fixes are available for all the
API levels without having to backport etc. With NDK 15 the unified headers
become the default ones (including when creating standalone toolchains)
and with NDK 16 (in beta currently) the old per-platform headers are removed.
Unfortunately, the new headers introduce breaking changes which made it
impossible to build Mono with NDK configured to use them. This commit makes
a handful of changes to Mono which make it build with unified headers.
* [jit] Fix the saving of the 'cfg->ret_var_set' flag when inlining, it was set to FALSE before it was saved, causing invalid code to be generated in some cases when doing nested inlining. Fixes#59608. (#5654)
* Bump API snapshot submodule
In mono_image_create_token we are passed in a managed reflection object and we
need to create a token for it and optionally register it so that we can resolve
that token back to a runtime structure and managed object later.
For a MonoReflectionMethod object that refers to a method from outside the
current dynamic image, we need to generate a memberref token. It will be the
same token for the same underlying MonoMethod*. When we go to register the
token with mono_dynamic_image_register_token we need to pass a managed object.
Unfortunately two MonoReflectionMethod objects could refer to the same
MonoMethod* but differ in their ReflectedType, so just registering the
passed-in MonoReflectionMethod means we could assert in
mono_dynamic_image_register_token because previously we may have registered the
same method via a different ReflectedType.
So what we do is we cal mono_method_get_object_handle with NULL for the
reflected type - which corresponds to just using the DeclaringType of the given
MonoMethod*.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=59364
Previously, we mapped fieldref tokens to MonoReflectionField managed objects,
But the object is just used as a key for caching the token so we may as well
use the native field instead of a managed object.
Regression test for https://bugzilla.xamarin.com/show_bug.cgi?id=59364
We need to cause SRE to generate the same methodref/fieldref token from two
different reflection objects. That happens if you get the same method or field
via a base and a derived class so that the MemberInfo:DeclaredType is different
but the underlying MonoMethod* or MonoClassField* is the same.
Suppose we want to know C.IsAssignableFrom (T1):
```
class C { }
class G<T1, T2> where T1 : T2, T2 : C { }
```
Previously, we would get `False`, because we only checked the constraints of T1
if C was an interface or another gparam. Now Instead we always consider the
constraints in `A.IsAssignableFrom (B)` whenever `B` is a gparam.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=58809
Check that generic parameters's constraints properly affect the subclass and
interface relationships even when the constraints relate several of the generic
parameters themselves.
Originally mono_bounded_array_class_get () would populate
MonoClass:sizes.element_size as soon as the MonoClass for the array was needed.
This is a problem because we cannot call mono_class_array_element_size () on
the element class on a valuetype until mono_class_layout_fields () finishes
initializing it.
That means that structs which contain an array of themselves would hit the
recursive initialization check in mono_class_setup_fields and the MonoClass for
the element class would be marked as failed to load.
Instead we rely on the MonoClass:size_inited bit to tell us whether the array
class has been initialized and if not, we set the array element size in
mono_class_layout_fields.
This is possible because MonoClass:sizes.element_size is only really needed to
know how to allocate space for an array. When laying out a class that contains
an array we don't need the element size - they array is just a reference type.
Example:
```csharp
struct S {
static S[][] foo;
}
```
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=43563
If we detect that a recursive type initialization is happening, we should leave
without removing the class from the init_pending_tld_id list. Otherwise we get
a double-free when the outer recursive call finally finishes and tries to
remove the same element.
Also re-load the init_list from TLS since a recursive call to mono_class_init could
have modified it between the time the current mono_class_init call started and
when it is finishing.
* [Tests] Fix Mono.Data.Tds tests on iOS
The modified tests failed on iOS following PR:
https://github.com/xamarin/xamarin-macios/pull/2580 that adds support
for Xharness to run the Mono.Data.Tds on Xamarin.iOS. With the changes
the tests run a expected.
* Close Listener when done.
(cherry picked from commit a09b912b86)
* [bcl] Use RuntimeHelpers.GetHashCode () for getting the hash code of the delegate target to avoid calling user defined hash code implementations which can throw exceptions. Fixes a regression introduced by a48bc43985.
* add test for #59235https://bugzilla.xamarin.com/show_bug.cgi?id=59235
* [fixup] RuntimeHelpers.GetHashCode handles null
With the introduction of magic array interfaces, they must be handled as if they
were variant if they are being dispatched against an array.
This was not being done in common_trampoline and resulted in the IMT thunk never
getting rebuilt and getting decent perf.
With this patch, dispatch to simple methods such as Array::Count over a magic interface
in a covariant case (such as IList<object> dispatch onto string[]) will accrue gvi's.
On a trivial benchmark of executing Count 1M times, perf goes from 760ms to 50ms.
It's causing unnecessary complexity because we need to create a new
monolite when bumping the .build number in a major.minor.build
version like from 5.2.0 -> 5.2.1
corlib version bumps are very rare in those cases and even then
you can just increment the MONO_CORLIB_COUNTER variable.
(cherry picked from commit ec9a95dd53)
These should be automatically collected by the GC, but explicitly disposing
will keep memory usage more stable over time and keep pressure off the GC.
(cherry picked from commit 093915cf83)
(cherry picked from commit 9b8bb0edbe)
Originally we used a C90-style struct initializer `{ (index), (offset), (type)
}`. Unfortunately, SpecialStaticOffset is defined with fields in a different
order on big-endian and little-endian machines. Using a designated
initializer, we don't have to care about the order.
... assemblies.
msbuild tries to filter out the denied assemblies, which mono refuses to
load, during a build. It uses the `deniedAssembliesList.txt` installed
to `$mono_prefix/lib/mono/xbuild` to identify them.
This was generated with:
`$ mono tools/nuget-hash-extractor/nuget-hash-extractor.exe nugets guids_for_msbuild > mcs/tools/xbuild/data/deniedAssembliesList.txt`
This file needs to be kept in sync with any changes in the assemblies
denied by the runtime.
msbuild PR that adds support for this -
https://github.com/mono/msbuild/pull/23 .
cherry-pick 823bca4aed
.. directory, which is `$mono_prefix/lib/mono/msbuild/15.0/bin/Roslyn/`
Mono installs `csc.exe` but the corresponding
`Microsoft.Build.Tasks.CodeAnalysis.dll` required for supporting msbuild C#
projects is installed by msbuild. But these two can get out of sync,
like they did and had to be worked around in
https://github.com/mono/msbuild/pull/19 .
Instead, we now install all the relevant roslyn bits (C# and VB) from the same
source. Only the msbuild specific files are copied to the Roslyn folder.
VBCSCompiler, csi and their dependencies are symlinked from `lib/mono/4.5`.
Also:
- csi.exe is included since the tool is surfaced as an msbuild task,
and that requires the *Scripting.dll assemblies.
- VBCSCompiler* - shared compiler is installed
- vbc.exe doesn't work on mono currently, so skipping that
- And `Microsoft.DiaSymReader.Native.{amd64,x86}.dll` are dropped.
Cherry-picking 43cec6c0dd
Commit message: Allow for non-fatal shell execution, and turn “install_name_tool” staging failures to warnings as we find a more permanent solution to AOT dylibs not always having enough field space for the new path (all other dylibs made in the build with the XCode toolchain typically do, as they are built with the “-headerpad_max_install_names” flag.
interface Type, not MarshalByRefObject. (Fixes#17325)
If a RealProxy instance is created (using the protected RealProxy(Type)
constructor) for an interface, mono internally sets up a MonoRemoteClass which
looks like a proxy for a MarshalByRefObject instance that happens to implement
the given interface. However mono_class_proxy_vtable just does a memcpy() to
setup the MonoVTable:type field of the transparent proxy's vtable. That type
field is hit by the JIT when System.Object.GetType() is called - and on .NET
Framework GetType for a transparent proxy of an interface returns the interface
not (as Mono used to) typeof(MarshalByRefObject).
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=17325
It is crashing still way more than it works and now that the mcs
configuration works again we should default to that to make our CI
more pleasant.
(cherry picked from commit ee8c7c80b7)
With this, profilers can distinguish between the two, and also know what the
target method of a tail call is.
The call instrumentation flags have been renamed accordingly and a new flag has
been added for instrumenting tail calls. I've also added a flag for exceptional
leave events, and method_exception_leave is only raised if this flag is given
for a method.
Also refactored the relevant JIT code a bit so it's easier to follow.
Code coverage must now be explicitly enabled. This simplifies initialization
of some coverage-related structures and also lets us enable debug info which
is needed to produce good data.
Also change the signature of mono_profiler_get_coverage_data so it returns a
value indicating whether coverage data was available.
An inlined method does not require instrumentation (if it
did, it would not have been inlined). The flags that
control instrumentation options are valid for the caller
method and are not to be consulted.
This patch replaces an assertion with a check and early
return. A method that "wants to be instrumented" does not
get inlined, but a method that "doesn't want to be instrumented"
can get inlined. However, the flags that control instrumention
options have been calculated against the caller method, hence
they should not be consulted at all. The extra check renders
the assertion unnecessary.
- Fix the Linux build instructions.
- Remove eglib dependencies, as they complicate the build of an
extenral module.
- Make instance static instead of heap-allocated.
- Make the call counter 64 bits.
- Adjustments to accommodate the new profiling API.
* [w32file] Remove io_ops indirection
* [w32file] Replace use of w32handle by a GHashTable for File, Console and Pipe
* [Socket] Avoid relying on MonoIO.DuplicateHandle
* [w32socket] Remove force closing of sockets on shutdown
* fixup! [Socket] Avoid relying on MonoIO.DuplicateHandle
* [w32socket] Replace use of w32handle by a GHashTable
* [w32handle] Remove dead function mono_w32handle_new_fd
* fixup! [w32file] Replace use of w32handle by a GHashTable for File, Console and Pipe
* [w32handle] Remove dead variable mono_w32handle_fd_reserve
* [fdhandle] Factor w32file and w32socket common fdhandle code
* fixup! [Socket] Avoid relying on MonoIO.DuplicateHandle
* [w32socket] Fix GC safe/unsafe transitions
* fixup! [w32handle] Remove dead variable mono_w32handle_fd_reserve
* [bcl] Grab free ports randomly in NetworkHelpers
We're frequently seeing "address already in use" errors on Jenkins.
The theory is that when we're running tests and grab the next free
port via our custom NetworkHelpers we're getting a port which will
also be returned to a simultaneously running test (e.g. another chroot)
because we're closing the TcpListener and thus releasing the port until
we start using it in actual test code. By that time the other test
might've already opened the port, causing our test to fail.
Instead we now try to use a random port in the range 10000-60000
and try if it's available. This doesn't completely fix the inherent
race but should hopefully make it way less likely.
In principle, mono_verifier_verify_cattr_content should prevent malformed custom
attribute blobs from being passed in here.
In practice:
1. The verifier is not on by default
2. System.Reflection.Emit allows an arbitrary byte[] to be passed in which
means that code like this can cause mono to read past the end of the array.
```
// "1 1-byte constructor argument and then 65280 named properties follow"
assembly.SetCustomAttribute(constructor, new byte[] { 1, 0, 1, 0x00, 0xFF });
var attributes = assembly.GetCustomAttributes(true);
```
On llvm aborts can happen during finally blocks. This test was not failing before because the method in question had llvm disabled, which has recently changed.
[metadata] Fix racy next_generic_inst_id
Like suggested by @luhenry, I would like to propose this fix which was discussed in https://bugzilla.xamarin.com/show_bug.cgi?id=58423. As `next_generic_inst_id` is `int`, `InterlockedIncrement64 ()` seems to be the right choice.
Also, I cannot add any reviewers or assignees, which is why I would kindly ask @lambdageek to review this as well :)
These symbols weren't being properly exported in the final mono executable on
Mac because no code in the runtime called these functions. As long as they're
defined in an object file that contains used functions, they'll be exported.
When this feature is enabled for a method, the enter/leave event receives an
additional argument, a so-called 'call context'. This call context contains
enough information about the stack frame of the instrumented method to allow
the enter/leave callback to inspect the 'this' reference, method arguments,
local variables, and the return value (for non-void methods).
This feature enables some interesting scenarios that were not possible with
the regular enter/leave events. For example, a profiler could instrument
well-known methods in the managed thread pool code to get an idea of how an
application is using the thread pool, or it could instrument network-related
methods to gather statistics or even log all network traffic.
This is implemented by storing a MonoProfilerCallContext on the stack, whose
MonoContext field is populated by executing an OP_FILL_PROF_CALL_CTX opcode
which stores the stack pointer, frame pointer, and all callee-saved registers
to it. For the epilogue, a pointer to the return value (for non-void methods)
is also stored in the MonoProfilerCallContext. An address to this context is
then passed to mono_profiler_raise_method_enter/leave. Based on debug info,
all arguments and locals can then be looked up in the instrumented method's
stack frame.
For the interpreter, we just store an InterpFrame pointer on the
MonoProfilerCallContext and look everything up from that. We don't need debug
info in this case.
This feature is currently not supported with LLVM (for regular LLVM mode, it
will fall back to Mono's JIT, while for LLVM-only mode, it's not available).
I also refactored the interpreter code so that enter/leave events are generated
not only when interpreter debugging is enabled. Also, the interpreter will only
call mono_profiler_get_call_instrumentation_flags () once per method now.
Finally, I made the interpreter also generate exception leave events.
GenericTypeParameterBuilder:InternalResolve (Fixes#58454)
When the parameter belongs to a method builder, we need to pass the declaring
type (appropriately resolved) to GetMethodFromHandle to avoid an
ArgumentException.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=58454
* [runtime] Integrate eglib's configure into mono's configure to simplify and speed up the configure process.
* [runtime] Delete some duplicate eglib configure code.
Cause the creation of a token for an EnumBuilder object. Test both typedef
tokens (EnumBuilder from same assembly) and typeref token (EnumBuilder in another assembly)
The new argument controls what should happen when the given token is already
present.
In some situations it's expected that we either see the same object, or we
unconditionally want to replace it (for example if we previously registered a
TypeBuilder or a MethodBuilder, after the type or method is created, we will
re-register the RuntimeType or MonoMethod in its place). In other cases (for
example when a module references a MonoMethod from a different assembly), we
may still call `mono_image_create_token()` multiple times but we expect to see
the same MonoMethod object.
* [runtime] Move the sre source files to 'common_sources', they no longer have compile time gc dependencies after the Boehm GC root changes. sre.c still depends on the gc dependent caching code in reflection-cache.h.
* [runtime] Remove compile time gc dependencies from the reflection caching code.
* [runtime] Move reflection.c/sre.c to 'common_sources', they no longer have compile time gc dependencies.
* [runtime] Remove compile-time gc dependencies from domain.c/appdomain.c.
* [runtime] Move domain.c/appdomain.c to 'common_sources' since they no longer have compile time gc dependencies.
Instead of hijacking the return address when suspending a thread in a finally block for an abort, do an explicit check in managed code when we are returning from the handler in order to see if we need to abort. Hijacking the return address is arch specific and it doesn't work if we are suspended before it is initialized in handler start.
Fixes#5798
* [mempool] Annotate functions for ThreadSanitizer
Whitelist `mono_mempool_alloc`, `mono_mempool_new_size` and `mono_mempool_destroy` according to https://bugzilla.xamarin.com/show_bug.cgi?id=57936
* [fixup!] Test the NO_SANITIZE_THREAD macro
* [fixup!] Merge comments / documentation
This follows the reference source in understanding `EndpointBehaviors` to be an alias of `Behaviors` with a more general type.
Note that although an alias might not seem necessary to implement, when using netstandard1.3 the [`System.ServiceModel.Primitives`](https://www.nuget.org/packages/System.ServiceModel.Primitives) package exposes `EndpointBehaviors` but not `Behaviors`.
These are used by XA/XI/XM and because XM in particular needs to work with
both old and new versions of system Mono, we need to make sure these functions
still exist.
Using the cardtable scan and finish_gray_stack start and end events is cumbersome to parse and require a lot of logging, especially in the case of the parallel collector.
If a worker is stuck doing work in another context, it might not be very responsive in transitioning its own state from WORK_ENQUEUED to WORKING and finally to FINISHING. When joining worker threads, we check whether we have any threads working in the corresponding context and, if there aren't any, change their state directly from the gc thread.
We can do this because work is distributed from centralized pools (worker_distribute_gray_queue and the job_queue), instead of per worker. Therefore we don't need all the workers to perform actual work in the context.
fixes this compilation error
MonoAotOffsetsDumper.cs(279,59): error CS0201: Only assignment, call, increment, decrement, await, and new object expressions can be used as a statement
The runtime has ~always supported having multiple profilers loaded, so it
didn't make much sense that only the last --profile option given would be used.
I intend to move code coverage support to a separate profiler module, so these
events will eventually be removed from the log format. As such, I'd rather not
commit to having these event classes in the API.
This won't affect users of the library who don't enable code coverage.
So mono_profiler_init becomes e.g. mono_profiler_init_log.
This allows us to link together multiple profilers into a single module and
load them selectively. This is especially useful for XA/XI/XM.
Calling this function could cause us to miss some profiler events.
In the past, we needed to call this function so that SMR and mono-threads were
initialized. Since then, 8dec9848f6 changed
profiler initialization to happen later during startup, so this is no longer
necessary.
TypeTest.GetGenericMethodDefinitionOverInflatedMethodOnGTD accesses
List<T>.ConvertAll() via reflection but doesn't use the method
elsewhere so the linker can remove it. Added some usage to avoid that.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=56242
MONO_GC_EVENT_START is actually sent _after_ all the 'stop world' events. The
heapshot logic being in this switch case only worked by accident previously.
* Remove the mark/reclaim events as they've been broken for a long time and
aren't particularly useful.
* Reorder the enum member declarations to the order they actually arrive in.
ValueAdd was added over a decade ago and all the tests marked with this category are passing now.
We have *tons* of tests that rely on working DNS/Internet that aren't marked with
InetAccess (and they actually provide good value so we don't want to exclude them).
Since these categories don't really make sense and just causes confusion
it's better to just remove them.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=57691
For keys longer than 1024 bits, we encoded the length incorrectly as
0x80 (length & 0xff)
instead of the correct algorithm:
if (length < 0x80)
length
else if (length < 0x4000)
0x80|(length>>8) (length & 0xff)
else
0xc0|(length>>24) (length>>16)&0xff (length>>8)&0xff length&0xff
which is provided by mono_metadata_encode_value.
See ECMA-335 II.24.2.4 #US and #Blob heaps
N.B.: we now always allocate the memory for the key as (keylen + 4) which
over-allocates by a couple of bytes in the cases where the key is shorter.
Regression test for https://bugzilla.xamarin.com/show_bug.cgi?id=57691
We build a library that has an InternalsVisibleTo attribute that references an
executable that is signed with a long (>128bytes) public key. Previously we
were using an incorrect method of encoding the key length.
This commit does a few different things.
First, metadata event options have been removed. We now always emit these
events. This is done because they contain important data that must be present
in order for dynamic event mask changes at runtime to result in useful data.
Also, almost all uses of the profiler require this data to be present, so
letting it be turned arguably just adds confusion for users.
Second, the PROFLOG_*_FEATURE flags have been removed. These made the code a
bit confusing because it was unclear whether code in log.c should be checking
the PROFLOG_*_FEATURE flag or the PROFLOG_*_EVENTS flag for a particular kind
of event. For heapshot in particular, the heapshot mode is now a separate enum
configuration variable. For enter/leave and coverage, switching those on and
off in the log profiler at runtime is not something we want to support at this
time.
As a result of the above change, some options are now processed manually rather
than being part of the event_list array.
Third, the way we turn off GC root events when needed is now by setting the
callback to NULL. This is more efficient as it signals to the runtime that it
doesn't have to gather root information. This will especially matter later on
when the profiler root info revamp lands, as it collects a lot more data than
we do today. Also, GC roots are no longer gathered unless we're doing a
heapshot.
Fourth, the profiler now always starts the sampling thread in idle mode. This
is so that we can reconfigure and enable/disable it at runtime as needed.
Finally, the alloc alias option now includes GC move events, and GC resize
events are only emitted if the gc option is given.
We've seen this test failure a few times:
```
MESSAGE:
#2
Expected: greater than or equal to 00:00:01
But was: 00:00:00.9544900
+++++++++++++++++++
STACK TRACE:
at MonoTests.Microsoft.Build.Execution.BuildSubmissionTest.EndBuildWaitsForSubmissionCompletion () [0x000d8] in /mnt/jenkins/workspace/test-mono-mainline-linux/label/ubuntu-1404-amd64/mcs/class/Microsoft.Build/Test/Microsoft.Build.Execution/BuildSubmissionTest.cs:110
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in /mnt/jenkins/workspace/test-mono-mainline-linux/label/ubuntu-1404-amd64/mcs/class/corlib/System.Reflection/MonoMethod.cs:305
```
Rewriting the test to use Stopwatch instead of DateTime
for measuring elapsed time should make it more reliable.
Previously, the profiler's state was spread all over the place. Some stuff was
in the MonoProfiler structure, some stuff was in global variables littered
throughout log.c. Now everything is stored in MonoProfiler and ProfilerConfig.
The call to GetDiskFreeSpaceEx passes uninitialized pointers to GetDiskFreeSpaceEx
instead of pointers to stack variables. This will cause random memory corruption and random
crashes on Windows builds.
This is a regression caused by the refactoring work done in this PR:
https://github.com/mono/mono/pull/4255
Location of ml.exe and ml64.exe is different in VS2017 compared to VS2015.
By removing the partial path from the command, the VS environment will locate
the tool depending on selected build configuration and VS version.
This should work for both VS2015 and VS2017.
LanmanWorkstation and SamSs services don't depend on the Browser
service anymore, at least in the Windows Server build that our
Azure VM agents on Jenkins are using now. This started failing a
few days ago so I assume a new build was rolled out.
It shouldn't throw in that case.
Helps cover https://bugzilla.xamarin.com/show_bug.cgi?id=57629
(note that we don't explicitly test on a readonly FS as that is hard
to do in a unit test, but running this test on Android should suffice).
Instead of using fragile lockless code in the sigchld signal handler, wake up the finalizer thread and make it call back
into the process code to signal finished processes.
* Profiler callbacks can now be changed and disabled at any point.
* API users no longer have to set event flags. The API instead uses a counter
internally for each type of callback.
* Filter functions for enter/leave instrumentation can be installed, and they
can choose whether to instrument the prologue, epilogue, or both.
* Managed allocators can now be instrumented for allocation profiling.
* A profiler must now declare that it wishes to use allocation profiling and/or
sampling in its init function.
* Sampling parameters can now be changed at any point, and the sampling thread
can be put into an idle mode when no sampling is needed.
* Only one profiler can have control over the sampling parameters. Whichever
profiler enables sampling first gets control.
* Adding new events is now very easy: One line in profiler-events.h and one
line wherever the event should be raised.
* Lifted the restriction that enter/leave instrumentation would cause an abort
in full AOT mode.
* Support for call chain sampling has been removed.
* Support for the old, platform-specific code coverage mode has been removed.
* The new profiler module entry point is mono_profiler_init. If a module has
a mono_profiler_startup symbol (the old entry point), a warning will be
printed and the module won't be loaded.
* Updated the profiler test suite to work with instrumented managed allocators.
After the refactoring in 9aaa0083d0
CppCheck complained about using 'result' uninitialized (looks like
it can't deduce that g_error will abort the process anyway).
Console.Error.WriteLine ("Got a bad hardware address length for an AF_PACKET {0} {1}",...
This Console.WriteLine interferes with Apache Storm that reads from the console I/O.
See bug: https://bugzilla.xamarin.com/show_bug.cgi?id=57887
[sre] Deal with ResolveEventHandler returning an AssemblyBuilder (Fixes#57850, #57851)
Fix a couple of bugs in the situation where a `ResolveEventHandler` happens to return an `AssemblyBuilder`.
* [#57850](https://bugzilla.xamarin.com/show_bug.cgi?id=57850) - if the `ResolveEventHandler` returns a reflection only assembly, throw a `FileNotFoundException`
* [#57851](https://bugzilla.xamarin.com/show_bug.cgi?id=57851) - don't look for a `ReferenceAssemblyAttribute` on assemblies returned by a `ResolveEventHandler` if they happen to be an `AssemblyBuilder`. (the native code path to get there is to try and instantiate a custom attribute in some other assembly, hence the fairly convoluted test in `assemblyresolve_event5` and its helper files)
In particular this makes sure that AssemblyBuilderAccess.ReflectionOnly aren't
allowed to run. And AssemblyBuilderAccess.CollectAndRun aren't allowed to save.
mono_w32handle_new_fd: We must hold scan_mutex while initializing an unallocated handle otherwise
mono_w32handle_foreach might think it's a valid handle.
mono_w32handle_timedwait_signal_handle: We must increment the refcount before installing the interrupt handle
as the other end can be called right after we finish installing and would trigger a failure.
* [w32handle] Fix handle dumped ref
* [w32handle] Replace exposing mono_w32handle_{ref,unref} by mono_w32handle_{duplicate,close}, it is closer to what the win32 API exposes.
* [w32handle] Remove unrefing of handles on shutdown
If one of the handle is a file handle for example, then w32file has already be cleaned up, so we cannot guarantee that the handle we are going to close is not going to call into w32file code.
We're checking out a specific commit for both now so the git branch
is no longer required.
Also fixes warnings that a 2017-06 branch doesn't exist in mono-extensions.
While this is not an issue yet, sgen_thread_pool_wait_for_all_jobs is logically expected to be called by the main thread and resume execution after all jobs have been executed. Concurrent M&S can enqueue additional jobs as part of its finishing phase, which means that we could detect early that no more jobs are to be processed. Make sure this is not the case.
* Fix compiler warnings (w32)
- add explicit (long long (unsigned)) type casts
- change %ld to %lld once, to match the general style of these logging messages
* Fix compiler warnings (threadpool)
- add explicit (void *) type casts
* Fix compiler warnings (boehm)
- add an explicit (long) type cast; the values are defined as gint64, an upgrade to at least "%ld" seems like a good idea
- about dropping const: const does not seem to be necessary as g_getenv() returns pointers to freshly allocated buffers whenever called. const might have been used to protect the return value of g_getenv() on unix systems with earlier versions of Mono (as a comment in gmisc-unix.c suggests). keeping const here causes a compiler warning, as the const qualifier gets dropped when *env is being used with g_free(). that led to the decision to drop the qualifier altogether.
* Fix compiler warnings (sgen)
- specify the arguments (void) to avoid -Wstrict-prototypes warnings
* Fix compiler warnings (misc - metadata)
- specify the arguments (void) to avoid -Wstrict-prototypes warnings
* Fix compiler warnings (misc)
- specify the arguments (void) to avoid -Wstrict-prototypes warnings
- add an explicit (void *) type cast; changing the return type to char* might be more usable and could be discussed
- ret is not being used and can be removed; fixes a -Wunused-but-set-variable warning
* Fix compiler warnings (mini)
- specify the arguments (void) to avoid -Wstrict-prototypes warnings
* [fixup!] Correct indentation
* [fixup!] Correct (void *)
- use GUINT_TO_POINTER and MONO_NATIVE_THREAD_ID_TO_UINT instead of (void *) to cast mono_native_thread_id_get () to %p
* [fixup!] Correct indentation of boehm-gc.c and add FIXME
* include <inttypes.h> and test the G_GINT64_FORMAT macro
* [fixup!] Use G_G*_FORMAT and PRIx32 macros instead of type casts wherever possible
Reuse the existing icall infrastructure to give the same handle back.
This PR implements all OpenExisting methods (with the access rights parameter discarded) and the SafeHandle property.
In the runtime, we don't check the capacity parameter if we try to open a mmap file, and we fallthrough the handle refcount case.
- Had to borrow a gchandles implementation from mono/metadata/boehm-gc.c
- Added coop handles support for NULL GC
- Fixed typos in configure.ac
- To use: `./configure --with-libgc=none --without-sgen --enable-boehm`
You get a mono-boehm which actually is not collecting anything.
We need to run jobs on the thread pool threads for parallel minor, parallel major, concurrent mark and concurrent sweep, some of which can run simultaneously. Instead of making separate threads reserved for each type of work, we create a work context abstraction (this contains a set of pending jobs to run and the sgen-worker callbacks). When a thread pool thread looks for work, it searches for work (jobs and idle work) through all contexts and finishes work from each one of them, from the highest to the lowest priority.
This is implemented by having different thread pool contexts (which contain the job list, work callbacks that indicate whether it should work and its ineherent priority), overall unrelated to the GC. The sgen worker infrastructure also creates separate worker contexts (which contain the object ops to be used and information about each worker, like the state and the private gray queue). We use at most two worker contexts (for minor and for major) and each worker context makes use of a sgen thread pool context.
Concurrent mark doesn't normally scan objects in the nursery since they can be moved. At the end of the concurrent mark, we were locking over the entire gc pause in order to make sure that the we can scan the latest pinned object list (aka none of those objects can be moved). Lock only over the pinning phase of the collection, since it is enough.
This reduces regressions for switching from MS_BLOCK_SIZE compile time constant
to ms_block_size variable by 85% on the tests where this was showing up. In the
future we should implement a block allocator, which would, amongst other benefits,
enable us to remove the block-size/page-size dependency.
The block size must be a multiple of the system page size. Page size simply
cannot be determined reliably at compile time, so we must obtain the page size
through a system call on startup and use it as the block size. We still enforce
a minimum block size of 16kb on systems where the page size is smaller than
that.
The old code hardcoded page sizes for certain architectures with large pages.
THe problem is that those architectures don't necessarily have to have large
pages and on systems where this wasn't the case, we'd end up with a block size
much larger than needed.
Rebase of @alexrp's work.
Linux has this particular behavior of returning ENOFS when unlinking from an read only FS.
Even if the file doesn't exists! Darwin does the sane thing and return ENOENT in this case.
To workaround this, if unlink returns EROFS, we stat the file and deal with that.
This fixes an issue we see on the BCL test suite when run in the context
of Xamarin Android:
```
I/mono-stdout( 2534): Certificate1 [FAIL] : TearDown : System.UnauthorizedAccessException : Access to the path "temp.b64" is denied.
```
It seems like those files aren't generated anymore. Not sure why this
starts to be a problem now.
This avoids situations where we could end up filling the signal queue
completely, leading to the undocumented EAGAIN return value from pthread_kill,
which breaks ~everything in the STW/GC code because it thinks an error return
value from pthread_kill means the thread has exited.
The actual reason why this happened is that when a thread is suspended, we mask
all signals except the restart signal for that thread. This means that a
program with a large number of threads or where GC collections could take
longer than usual would end up with tens of thousands of sampling signals
queued up, eventually reaching the limit of the user-level signal queue (see
`ulimit -i` and `RLIMIT_SIGPENDING`).
We only do this on systems where we use real time signals. Non-RT signals do
not have this problem as the kernel will throttle them.
These will be used in the log profiler.
Also, make mono_thread_info_register_small_id () only allocate a new small ID
if one isn't already allocated for the current thread. This is necessary so
that we can allocate a small ID for a thread and not have it be overwritten
if/when that thread is later attached probably (which also results in this
function being called).
This hardens the profiler against cases where an unattached thread causes some
event callback to be invoked. This is particularly relevant when embedding.
If a program is idle, we likely won't get enough samples to prompt the dumper
thread to send its log buffer to the writer thread. This can make it seem like
the profiler isn't working when it actually is. So, just send the dumper
thread's log buffer every second to make sure data is delivered in a timely
fashion.
* [threads] Make mono_thread_attach_full static
* [threads] Remove prefix on MonoThreadInfoCallbacks callbacks
* [threads] Rename mono_threads*_init functions to mono_thread_info*_init
* [threads] Remove dead method mono_runtime_is_critical_method
* [threads] Move MonoThreadInfoCallbacks to metadata/threads
* [threads] Store MonoInternalThread in MonoThreadInfo for use when detaching
When the native thread would exit, the TLS keys are destroyed. There is a TLS destructor for MonoThreadInfo, but not for MonoInternalThread. If MonoInternalThread is destroyed before MonoThreadInfo, then MonoThreadInfoCallbacks.thread_detach wouldn't successfully detach the current MonoInternalThread as mono_thread_internal_current would return NULL. This would lead to MonoInternalThread still laying around, while their corresponding MonoThreadInfo has already been destroyed. This fixes this problem, since we now rely on MonoThreadInfo to store the MonoInternalThread for detach.
- Create a unique temporary directory for each test in TypeBuilderTest and delete
it on TearDown.
- Change tests to use tempDir instead of Path.GetTempPath ()
- Change IsSerializable () test to how it was 11 years ago and have it write to a
file with a simple name (but now in a directory where noone else is writing)
- Change FieldsWithSameName (), FieldsWithSameNameAndType () and
MethodsWithSameNameAndSig () to use the default temp assembly (now in a
unique tempDir) instead of using the CreateTempAssembly ()
- Remove CreateTempAssembly function. It is dead code.
* [threads] Use MonoThreadInfo.{stack_start_limit,stack_end} in more places
* [threads] Remove dead code
* [threads] Rename MonoThreadInfoCallbacks.thread_{register,detach,unregister} to MonoThreadInfoCallbacks.thread_{attach,detach,detach_with_lock} respectively
This error can occur on mobile devices during network operations
as evidenced in https://bugzilla.xamarin.com/show_bug.cgi?id=56653
and having an actual description of the error printed in this case
makes it easier for the user to discover what's going on.
mono_thread_info_suspend_lock () can be called from boehm-gc.c
on_gc_notification before the new thread's start_wrapper calls
mono_thread_info_attach but after pthread_create calls the start wrapper.
As a result, the asserts in mono_thread_info_suspend_lock_with_info can catch
the new thread in an inconsistent state.
They now are produced every time a GC spills heap events such as roots, moves and objects.
Additionally, I changed GC syncpoints to happen in case any GC events are active.
It's a safer choice with little penalty. Figuring out the actual combinations that would require them is non-trivial.
Suppose we have two versions of an assembly: AsmA 1.0.0.0 and AsmA 2.0.0.0 that
we want to load into the same domain at the same time (for example we want to
load AsmB and AsmC that statically reference the two versions, respectively, or
maybe we used reflection). Suppose version 1 got loaded first (and we found it
on our MONO_PATH in "somepathdir/AsmA.dll"). Now we want to load version 2.
The problem is that if version 1 is on MONO_PATH first, we'll find a filename
that matches ("somepathdir/AsmA.dll") and try to open that image. Then we'll
end up here in mono_assembly_open_predicate() after the mono_image_open_a_lot()
call hits the image cache. So we get back an image and image->assembly is
version 1.
The old behavior would just return any non-NULL image->assembly. The new
behavior is to check the predicate and reject opening the propose file if the
predicate doesn't match. (If we're not using the strict loader, or if there's
no predicate set, return the existing image->assembly as before).
They lived in https://github.com/mono/release before, moving them
into the main repo allows us to version them more easily.
Note: I intend to do more cleanup on those scripts in the future,
this is just the first step.
Group gc params configurations into general gc modes. The default mode (SGEN_MODE_NONE) is what we've been using so far, meaning old default features. The other modes can include experimental features.
We the profiler now emiting a lot less events by default, all the existing polling no longer makes sense as they won't matter in the common path.
Instead we make sure the finalization end callback is always installed and poll from it.
This introduces the risk of a heapdump never happening or experiencing long delays because there are either too many finalizers or one of them hung.
It's an acceptable risk.
* Fix System.Numerics xunit tests and add them to CI
Xunit.Extensions.NetCore.dll is available as a net46 version now
so we can just use it instead of building it from our buildtools fork.
This also allows us to remove the fork from our submodules.
We were missing running the System.Numerics xunit tests in CI,
this adds them so we catch breaks like these earlier.
* Skip tests on Windows, they seem to fail there
Regression test for https://bugzilla.xamarin.com/show_bug.cgi?id=57222
Although methods aren't affected by this bug, I added a regression test for
this case anyway in case the implementation changes.
Currently MethodBuilder.RuntimeResolve() goes through a different codepath - it
calls RuntimeType.GetMethod (MethodInfo) which calls into an icall which looks
for a method with a matching token rather than a matching name.
We can't pass RuntimeMethodHandle across AppDomains since that can lead to memory corruption.
The way this happens is that when Domain (A) passes a method handle (M) to Domain (B), (B) will get the reflection object for
which will add that MonoMethod to the per-domain reflection hashtable. This won't cause the assembly of (M) to be registered with (B).
When (A) is unloaded, it might be the only domain referencing the assembly of (M), cause it to unload resulting in (B) having a dangling
pointer to (M) on its reflection hashtable.
The fix is to serialize enough lookup data across the wire so we can do type/method lookup using reflection on the target domain.
It might be slower, but this only happens on some remoting slowpaths anyways.
Finally, we can't simply register the assembly of (M) into domain (B) as that list (MonoDomain::domain_assemblies) is used for type loading,
thus breaking the target domain in some scenarios.
RuntimeType:GetField (FieldInfo) looks up the field by name and attributes,
which doesn't work for SRE because it is legal to create multiple fields with
the same name and type and attributes. So lookup using the
RuntimeFieldHandle (aka MonoClassField*) instead.
* [System]: Correctly implement close and shutdown in SslStream.
* Cleanup.
* Cosmetic.
* More ReferenceSources/SR2.cs into common.sources to make it build.
When creating a bundle on Windows to run on Android, macOS/iOS or Linux we need
to use the target system's path separator when storing satellite assemblies in
the bundle instead of the host system's one. Failing to do so will embed (on
Windows) the satellite assemblies with paths like
fr_FR\myassembly.resources.dll
while at runtime Mono will try to look up the assembly by
fr_FR/myassembly.resources.dll
and fail to find it, obviously. The `--style` parameter is now consulted to see
what path separator character should be used.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=55603
MonoInternalThreadHandle is not really needed - all MonoInternalThread
instances are supposed to be pinned. But for uniformity, the icall marshaller
makes handles anyway. This function lets us get back to a raw ptr.
- Use coop handles for create_object_from_sockaddr
Eventually want to rewrite its callers to use
create_object_handle_from_sockaddr directly.
- Use coop handles for
- LocalEndPoint_internal
- RemoteEndPoint_internal
- inside create_sockaddr_from_object
Eventually callers should be updated to use create_sockaddr_form_handle
directly.
- Use coop handles for
- ReceiveFrom_internal
- Bind_internal
- Connect_internal
- SendTo_internal
- Remove dead code (create_object_from_sockaddr, create_sockaddr_from_object)
- Use coop handles for
- Receive_internal WSABUF[]
- Receive_internal byte[]
- Send_internal
- Send_array_internal
- SetSocketOption_internal
- GetSocketOption_arr_internal
- GetSocketOption_obj_internal
- IOControl_internal
- SendFile_internal
- Remove more dead code
- Use coop handles for
- Socket_internal
- Select_internal
- Trivial coop handles conversion for several icalls in System.Net.Sockets.Socket
None of these functions use managed objects, so just add a MonoError* argument.
Previously we only had "out" reference args working - they get a handle that's
initialized to NULL on entry and then writes back to the out arg on icall
return.
For "ref" reference args, do the same thing but initialize to the initial
argument value.
* [Mono.Options] Fix test and add suite to CI
We missed running Mono.Options tests on CI so we didn't catch
this error which was introduced with: https://github.com/mono/mono/pull/4719
```
CSC [net_4_x-darwin] net_4_x_Mono.Options_test.dll
Test/Mono.Options/CommandSetTest.cs(231,16): error CS7036: There is no argument given that corresponds to the required formal parameter 'error' of 'CommandSet.CommandSet(string, TextWriter, TextWriter, Converter<string, string>)'
```
* [Mono.Options] Stop hardcoding \n to fix tests on Windows
With the new bockbuild in place we can finally remove the
implicit dependency on a point-in-time state of mono-extensions
and instead explicitly pin the desired revision.
This removes all [no]foo weirdness and flag dancing in favor a simpler model.
Users can enable/disable both events and profiling modes. We expose pretty much all individual events.
A profiling feature is a group of flags that enable a specific profiling scenario.
Right now we have code coverage, perf sampling, gc alloc and heap dump as features.
All events can be prefixed with 'no' and cause them to be disabled. We'll later introduce some sanity
checking around misconfiguration.
Finally, this changes the default set of enabled events to threads and gc. Everything else must be added on top.
Experience has shown that the vast majority of those events are just noise when profiling and the current configuration
design made it quite awkward to introduce quieter modes.
We would previously mix 2 SocketAsyncResult, and in case we would succeed asynchronously, we would return one that would not be updated afterwards.
The solution consists in using only 1 SocketAsyncResult that we ensure to update accordingly.
After looking at https://github.com/mono/mono/pull/4949 I did a
quick pass through the other config sections and found two more
cases where we missed adding to the properties list.
Otherwise, while we are clearing the domain, another domain might be created that has the same id as the current one. This would lead to crashes due to releasing of data from the newly created domain (ex gchandles).
* [S.R.I.RuntimeInformation] Fix FrameworkDescription on Android
It turns out using `DllImport ["__Internal"]` doesn't work on
XA (and other cases where Mono is embedded) because it searches
for the method in the host which embeds Mono and doesn't find it,
resulting in an EntryPointNotFoundException.
We need to switch to an icall instead. However there's already
an existing icall+BCL wrapper: Mono.Runtime.GetDisplayName().
Added IVT from corlib to S.R.I.RuntimeInformation so we can call it.
* Update API snapshot
we got a couple of bug reports, all with the same failure:
https://bugzilla.xamarin.com/show_bug.cgi?id=44907https://bugzilla.xamarin.com/show_bug.cgi?id=46482https://bugzilla.xamarin.com/show_bug.cgi?id=51791
The bugs are private, therefore here what I wrote:
```
Thank you Matthias, this was very helpful.
>>> 09-30 12:24:50.347: A/DEBUG(6211): pid: 6191, tid: 6209, name: Thread-4 >>> com.distech.x50.ui.droid <<<
>>> 09-30 12:24:50.347: A/DEBUG(6211): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x8
>>> 09-30 12:24:50.347: A/DEBUG(6211): r0 d6b58728 r1 00004001 r2 00000000 r3 d6b58738
>>> 09-30 12:24:50.347: A/DEBUG(6211): r4 00000008 r5 ea97427c r6 00001f3c r7 00000015
>>> 09-30 12:24:50.347: A/DEBUG(6211): r8 00001f38 r9 00000015 sl d6b58728 fp d67fed90
>>> 09-30 12:24:50.347: A/DEBUG(6211): ip ea9742c0 sp d67fed60 lr ea7a4504 pc ea716a58 cpsr 800e0010
The crash happens on this assignment:
https://github.com/mono/mono/blob/de1865dad5c0350f391fedcaa08f02f610530d3f/mono/mini/mini-generic-sharing.c#L418
Here the according disassembly:
https://gist.github.com/lewurm/b1094749027c9e5ea19fdc4fac7905a7
The crash happens in the last loop iteration (`r9=i`, `r7=slot`). Looking
at the machine code it just _cannot_ happen, which is confirmed by the C
code as well: `*oti` successfully happens in the if check, but after
returning from `alloc_oti()`, `oti` doesn't contain a valid address anymore.
I suspect some weird hardware issue that fails to restore all registers
properly from the stack.
```
[...]
```
Thanks again Matthias. Unfortunately, I'm out of ideas, and I can't
blame anything but the hardware. The situation we see is too weird. We
segfault at offset `0xdda60: str sl, [r4]`, but really we should
already segfault at offset `0xdda2c: ldrge sl, [r4, #8]!"`. So I
suspect two things why this could happen:
(1) The instruction at `dda2c` fails to do the post-increment correctly
for *whatever* reason.
(2) Something along the execution path corrupts the stackslot, where
`r4` is saved, in such a way that it *exactly* masks it with `0xf`.
Everything else on the stack looks fine, so this is sort of very
unlikely to be honest.
This only happens on a very specific device: The Nexus 9 is the only
device that was ever shipped with the Tegra K1 T132. I suspect an issue
in the binary translation layer of `armv7` instruction set to the internal
micro-ops of the CPU. I tried to stress test the instruction in
question (see https://github.com/lewurm/ldrinsntest), however I was not
able to trigger a crash. So either, I'm missing some context in order
to trigger the bug or I'm on a completely wrong track.
That said, even if we could proof that it is indeed a hardware issue,
the workaround is also non-trivial (it would then either need a fix in
gcc or require a microcode update by the chip vendor).
```
And then we saw:
https://github.com/golang/go/issues/19809#issuecomment-290804472
Another hint that this device is buggy.
This change massively improves reflection query scalability.
Given this benchmark and my 4 cores laptop:
```
const int T_C = 4;
var list = new List<Thread> ();
bool done = false;
int count = 0;
for (int i = 0; i < T_C; ++i) {
var t = new Thread(() => {
while (!done) {
typeof (Driver)
.GetMethods (
BindingFlags.Public | BindingFlags.NonPublic |
BindingFlags.Instance | BindingFlags.Static);
Interlocked.Increment (ref count);
}
});
t.Start ();
list.Add (t);
}
Thread.Sleep (5000);
done = true;
foreach (var t in list)
t.Join ();
Console.WriteLine ("Got {0} lists done", count);
```
This change makes us go from 166.725 queries/s to 5.140.821 queries/s. Or 30x faster.
* [interpreter] Add beginnings of debugger support.
Handle debugger support similarly to the JIT:
- Add a MINT_SDB_SEQ_POINT instruction to the debugger IR. Replace it with a MINT_BREAKPOINT instruction when a breakpoint is placed.
- Store the sequence point information in the existing sequence point data structures.
* [interp] Add support for debugger stacktraces.
* [interp] Add support for debugger frame info.
* [interp] Connect seq points inside the same basic block so simple single stepping works.
* [interp] Link sequence points in different basic blocks the same way the JIT does it.
* [interp] Allocate some bitsets from a mempool.
* [interp] Rewrite the patching in the transform pass to make it a bit easier to understand.
* [interp] Add interruption support for sdb.
* [interp] Add support for mkrefany,refanytype and refanyval.
* [interp] Fix debugger failures.
* Add support for setting byref variables.
* Fix thread interruption when the top frame is an interpreter frame.
* Allow stack walks when a thread is suspended while transforming a method.
* Implement method entry/exit events.
* Fix ldstr+dynamic methods.
* [interp] Implement support for Debugger.Break.
* [interp] Change the offsets in MonoJitInfo/seq points into byte based from short based to reduce the amount of special casing needed in other parts of the code.
* [interp] Factor out the push/pop lmf code into a pair of helper functions.
* [mixed] Fix mixed mode EH.
* [interp] Implement support for MONO_VERBOSE_METHOD. Improve sequence point placement.
* [interp] Implement support for SetIP ().
Pass the target thread as an additional argument to mono_interp_set_resume_state (). Check the resume state after single
stepping/breakpoints in the interpreter.
* [interp] Disable the usage of the STLOC_NP opcodes when running under the debugger, it doesn't work if the ip is changed when the execution is stopped.
* [interp] Fix async stack walks in the debugger when the thread is in native code called from interpreter code.
* [interp] Fix the disable interpreter build.
* [interp] Fix the calculation of native offsets in the line number info.
* [interp] Fix the build.
* [interp] Make CMD_STACK_FRAME_GET_THIS behave the same when called on managed-to-native frames, with the interpreter, frame->has_ctx is set.
* [interp] Fix typos, remove some dead code, use g_print instead of printf in a few places, move some interpreter stubs to interp-stubs.c.
- Make a new mono_w32event_create_full on unix
- Convert the MonoString to utf8 (unix) or utf16 (win32) in the icall, then
call an implementation func.
- transition to GC safe mode around the win32 api call on Windows
When an AssemblyLoad event fires in a domain, a System.Reflection.Assembly
object is created in that domain for the assembly being loaded. The bug left
this object alive after the domain was unloaded, which lead to a GC crash.
After fixing my windows setup, I managed to properly test MS's behavior, which is a lot saner and it's the following:
Opening fails with ERROR_CANT_RESOLVE_FILENAME (1921).
Testing and Deleting works.
Fixing File.Delete required removing the lstat we do before deleting, which no longer serves any purpose.
Fixing File.Exists requires handling ELOOP when stat fails and we retry with lstat.
* Create mono_delegate_handle_to_ftnptr for the implementation
* Use coop handles for
System.Runtime.InteropServices.Marshal.GetFunctionPointerForDelegateInternal
* Move most of the logic to mono_ftnptr_to_delegate_handle
* Use handles for
System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointerInternal
Most of the removed files weren't touched in the last decade
and are unused so it's better to remove them to avoid confusion
about what they do.
I also went over and removed the most outdated info from readme
files, we probably want to do a more thorough overhaul in the future.
mscorlib.dll etc will be platform specific in the future. This also
means that we need to adapt monolite since it'll become platform
specific once we start the divergence.
monolite is essentially a repackaging of the "build" profile which
means we need to turn that profile into multi-platform profile
like net_4_x. We'll still have a symlink from build -> build-darwin
and the monolite which is downloaded or packaged in the tarball
is expanded into mcs/lib/monolite-<platform> depending on the platform
you're building on.
I had to move the get-monolite-latest target from the top-level
Makefile.am into mcs/class/Makefile so it has access to HOST_PLATFORM,
but it arguably should've been there anyway before.
Bring in the change that makes building the "tool" subdirectory
in BTLS off by default - we don't need it in Mono and it fails
to build on some platforms with older C++ compiler.
Follow-up to https://github.com/mono/mono/pull/4899. I audited
all the places in BCL where we made use of Cecil's AssemblyDefinition
and made sure we're properly disposing them and not keeping files
unnecessarily open.
I used InMemory=true in cases where tracking the lifetime would
be complicated. There are also some tests in mcs/tests which I
didn't bother to fix.
* Bug #53038: Warn on failure to create default Reflection.Emit SymbolWriter (on platforms like Android that don't ship one) instead of throwing and failing
Remove 10000 size check in mini_emit_memory_copy_bytes, this is no longer needed now that the memcpy expansion logic in mini_emit_memcpy_internal was fixes.
Use a define for 10000.
Remove useless debug spew.
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56275
Recent Cecil 0.10 changed to not reading the assembly in memory but
instead reads directly from the underlying stream [0]. This however
means that the file handle is not freed until the AssemblyDefinition
is disposed.
Xamarin.Android is using the pdb2mdb source as part of
Xamarin.Android.Build.Tasks.dll and hit an issue where the file
would be locked inside of VS.
[0] http://cecil.pe/post/149243207656/mono-cecil-010-beta-1
(cherry picked from commit 5077205a44)
This issue is actually a pair of bugs:
1. We don't perform any access checks for constructors (which can give a false impression of our behavior for this issue). This means that you can also invoke private constructors from other assemblies.
2. InternalsVisibleTo checks are done using strcmp instead of a case-insensitive comparison.
Problems with the current code:
It's wrong on 64bits if align == 4
It's wrong on 64bits if offset is not 0 or 4.
It generates bad code if align > 1 && align < SIZEOF_VOIDP
This PR implements unaligned based on the following set of assumptions that are true for the current set of platforms we support:
- A load/store of size X works with data aligned on X. IE, a short store can handle addresses aligned to 2
- Unaligned access is either disallowed or plain slower.
- Both source and dest abide to the supplied alignment.
The way we emit unaligned access is by emitting the widest possible access based on alignment and offset.
TBH: use unaligned access on targets like x86 that support it and are profitable.
When MONO_HANDLE_TRACK_SP is defined, each handle records the native C stack pointer
at the time of its nearest enclosing HANDLE_FUNCTION_ENTER. If a subsequent
handle is allocated with a lower stack pointer (in memory; ie higher in the
call tree) then a HANDLE_FUNCTION_RETURN is missing and we're leaking handles.
* [btls] Add "desktop" ARM Linux configurations (for IoT, mostly)
* [btls] Disable optimized asm on soft-float ARM
It depends on a NEON FPU, which we don't target on soft float
It was #ifdef'ed to only be available on monotouch before,
but the testing_aot_full profile in CI runs into this test failure:
```
MESSAGE:
System.ExecutionEngineException : Attempting to JIT compile method '(wrapper native-to-managed) Mono.Btls.MonoBtlsSslCtx:NativeVerifyCallback (intptr,int,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
+++++++++++++++++++
STACK TRACE:
at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_thread_interruption_checkpoint ()
at (wrapper managed-to-native) System.Runtime.InteropServices.Marshal:GetFunctionPointerForDelegateInternal (System.Delegate)
at System.Runtime.InteropServices.Marshal.GetFunctionPointerForDelegate[TDelegate] (TDelegate d) [0x00013] in <fa55a8c31ba0490383239e5850d81686>:0
at Mono.Btls.MonoBtlsSslCtx..ctor (Mono.Btls.MonoBtlsSslCtx+BoringSslCtxHandle handle) [0x00059] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Btls.MonoBtlsSslCtx..ctor () [0x0000b] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Btls.MonoBtlsContext.InitializeConnection () [0x00000] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Btls.MonoBtlsContext.StartHandshake () [0x00000] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncProtocolRequest asyncRequest, Mono.Net.Security.AsyncOperationStatus status) [0x00004] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (Mono.Net.Security.AsyncOperationStatus status) [0x0006b] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation () [0x0000d] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.AsyncProtocolRequest.StartOperation () [0x0003c] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.AsyncProtocolRequest.StartOperation (Mono.Net.Security.AsyncOperation operation) [0x00024] in <18810f39bffc41dcb0263eb2043cc10e>:0
at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Net.LazyAsyncResult lazyResult) [0x00057] in <18810f39bffc41dcb0263eb2043cc10e>:0
```
The attribute itself is already conditional on MONOTOUCH and
FULL_AOT_RUNTIME so simply removing the ifdef should fix the issue.
It's a build artifact anyway since we check in the generated csproj's
so you don't really need the order.xml.
After the new platform-specific BCL changes went in it also started
containing stuff like `-r:./../../class/lib/net_4_x-linux/mscorlib.dll`
which makes it even less useful to have checked in as it'd get
changed when you build on OSX for example.
It is accessed with a 64bits atomic operation, so it needs to be 64bits aligned, even on a 32bits platform. A CPU exception EXC_ARM_DA_ALIGN triggers on armv7 32bits.
Fix https://bugzilla.xamarin.com/show_bug.cgi?id=56202
It's both racy and not necessary.
Beyond that, it will produce the wrong exception if opening the file fails for a different reason that it's missing.
This is an issue in case of bad permissions or symlink loops.
* [interp] Add support for mkrefany,refanytype and refanyval.
* [interp] Avoid inlining empty methods marked with NoInlining.
* [interp] Fix the popping of arguments when inlining empty methods.
Minor fixes which make System.Drawing compatible with netstandard1.6:
- Use MemoryStream.ToArray instead of .GetBuffer
- Use RuntimeInformation.IsOSPlatform to determine the operating system
- Use Stream.Dispose instead of Stream.Close
- Disable serialization support for exceptions
- Don't use AppDomain
Related: commit 00c6a94a
When [Xamarin.Android attempts to build with mono/2017-04][0], the
build breaks when cross-linking Windows code via MXE due to the
presence of an undefined `inline`d function:
./.libs/libmini.a(libmini_la-tramp-amd64.o):…/xamarin-android/external/mono/mono/mini/tramp-amd64.c:746:
undefined reference to `mono_arch_unwindinfo_get_size'
Turn `mono_arch_unwindinfo_find_pc_rt_func_in_table()` and
`mono_arch_unwindinfo_get_size()` into `static inline` functions so
that they don't cause "undefined reference" linker errors.
Additionally, audit the rest of `mono` to ensure there aren't any
other `inline` functions that aren't `static`:
$ git grep '\<inline\>' mono | grep -v static | grep -E '(\.c|\.h):'
# ~50 matches, all of which are in comments or otherwise not relevant.
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/388/
This function calls mono_class_get_ref_info which allocates a coop handle.
However it is called from an icall that wasn't converted to use coop handles
yet, so nothing was saving/restoring the handle stack mark, and consequently
the handles leak.
We had the object ops set in the job when allocating the job (typically by the main gc thread). When enqueueing jobs, we leave their ops NULL, and when the worker actually executes the job, it will use the idle object ops, which can get dynamically optimized over the work cycle.
- ImageFormatConverter uses string.Compare to compare extensions for known file types. It uses CultureInfo.InvariantCulture,
which is not available on netstandard1.6. Because all the extensions use ASCII-characters, StringCOmparsion.OrdinalIgnoreCase
can be used, which is equivalent.
- Replace ToLower (CultureInfo.InvariantCulture) with .ToLowerInvariant ()
* netstandard1.6 compatibility: Use GetTypeInfo () where required
A lot of methods have moved from Type to TypeInfo in netstandard1.6, add calls to GetTypeInfo ()
where required to support compiling System.Drawing on netstandard1.6
* Add missing using statement
After https://github.com/mono/mono/pull/4837 was merged the
csproj update bot started sending PRs which hardcoded net_4_x-linux
as the OutputPath.
Updated the csproj generator to detect this and use a variable
in the OutputPath instead. It'll autodetect based on the current OS
and can be overriden by passing /p:HostPlatform=foobar to msbuild.
Ideally we'd have configurations for each platform in the csproj since
in the future the file lists will be different but this is a first step.
* [jit] Fix CPBLK for large sizes.
* [amd64] Fix the byref->byval passing of small arguments from gsharedvt methods, we were loading loading them as pointers, which meant the upper bits might contain garbage. Fixes#56247.
* [mcs] Rework %-recursive target to explicitely list subdirectories
* [mcs] Rename PLATFORM to BUILD_PLATFORM
* [mcs] Add platform specific files
This is to support adding corefx platform specific files. To do that, we now support `mcs/class/corlib/linux_net_4_x_corlib.dll.sources` for example, on top of `mcs/class/corlib/net_4_x_corlib.dll.sources` and `mcs/class/corlib/corlib.dll.sources`.
The 3 platforms that are supported are: `linux`, `darwin` and `win32`.
* [mcs] Build cross-platform bcl in separate folder
This reverts commit 47365cd54b.
We didn't do this in the private bockbuild and putting
'opt' from Mono into the PATH could be problematic.
This makes us consistent with the private bockbuild whitelist.
The thread_state indicates whether we have a pending exception that needs to be consumed (be it sync or async). Normally we check the state before calling mono_thread_execute_interruption, but make sure we don't consume an exception if the thread_state doesn't allow us to (we can have an abort requested but mono_thread_clear_interruption_requested can return FALSE if we are inside an abort protected block and shouldn't process it)
Thread.Yield doesn't guarantee that the thread doing GCs won't do it excessively, leading to significant variances in test run time.
Decrease number of iterations since these tests can take around 1 min.
* [System]: Add new 'common-networking.sources' and move stuff from 'common.sources'.
This moves all the networking-related sources which aren't supported on monotouch_watch
into a separate file, so we don't have to maintain a huge "Thing That Should Not Exist".
* Put back System.Net/NtlmClient.cs - we actually do need it.
* Add HttpListenerTimeoutManager.platformnotsupported.cs.
* Make it build.
Refactor it into mono_security_principal_windows_identity_get_current_token ()
and then call that from ves_icall_System_Security_Principal_WindowsIdentity_GetCurrentToken ()
- Fix typo in REFLECTION_EMIT_INIT/REFLECTION_EMIT_SAVE in
mono_dynamic_image_register_token
- Add mono_dynamic_image_get_registered_token ()
- N.B. locking change - ves_icall_ModuleBuilder_GetRegisteredToken used to take
the loader lock. Now it calls mono_dynamic_image_get_registered_token which
just takes the image lock.
Consider the old code (which just used mono_path_canonicalize)
Suppose foo.dll is a symlink to bar.dll
Suppose we've try to load foo.dll the first time:
do_mono_image_open is passed "foo.dll" it then sets image->name to "bar.dll"
which is then used by register_image to do a lookup and then inserts the
mapping from "bar.dll" to the new image (call it Image1).
Suppose we try to load foo.dll a second time:
we check for "foo.dll" in the loaded_images cache and get NULL.
So we call do_mono_image_open ("foo.dll", ...) again, and again image->name is
set to "bar.dll". (Call this Image2)
We call register_image again which looks up bar.dll and gets Image1. So it
increments the refcount for Image1 and unloads Image2 and so
mono_image_open_a_lot returns Image1.
With the new code:
The first time we try to load foo.dll everything is the same. The second time
we try to load foo.dll, we resolve the symlink from "foo.dll" to "bar.dll" in
mono_image_open_a_lot and return Image1 directly from loaded_images.
Use 'ObjCRuntimeInternal' as the namespace instead of 'ObjCRuntime' to avoid
these compiler-warnings when building xamarin-macios assemblies (which defines
the same types in the ObjCRuntime namespace, and to whom System.dll has an
InternalsVisibleTo attribute):
> xamarin-macios/src/build/watch/watch/Foundation/NSExtensionRequestHandling.g.cs(28,49): warning CS0436: The type `ObjCRuntime.INativeObject' conflicts with the imported type of same name'. Ignoring the imported type definition
> xamarin-macios/src/ObjCRuntime/INativeObject.cs(6,19): (Location of the symbol related to previous warning)
> xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.WatchOS/repl/System.dll (Location of the symbol related to previous warning)
If a remote invoke in another domain throws an exception, the xdomain-dispatch wrapper will serialize the exception and pass it over to the calling domain so that it can be rethrown there. Serializing the exception object can itself throw an exception which we didn't properly catch, leading to unwinding to the caller domain without changing the domain state back and wreaking havoc.
If the serialization of the original exception throws a new exception, this new exception takes its place, being passed instead to the caller domain.
Similar to 9635b9cea0 we
can now break the dependency on WinForms from resx2sr.
This allows us to add it to the "build" profile
and clean up the Makefile.
This runs the API diff against the monotouch and monodroid
profiles in addition to net_4_x to help us catch mistakes that
only affect the mobile profiles.
To make updating the API snapshot in case of an intentional change
easier without needing to build those profiles locally I added
a way to download the diff as a .patch file. You can just download
the file from the API diff Jenkins generates and apply it locally.
* [System]: Make HttpListener partial and move Mono-specific bits into HttpListener.Mono.cs.
* Remove ancient EMBEDDED_IN_1_0 conditional.
* Remove unused using statements.
* HttpListener.CreateSslStream() now uses RemoteCertificateValidationCallback.
* [System]: Fully stub-out SslStream on platforms where it's not supported.
* Remove 'SECURITY_DEP' conditional from LocalCertificateSelectionCallback.
* Add the internal delegates and remove 'SECURITY_DEP' from SslStream.platformnotsupported.cs.
* In SslStream.cs, we now provide the full API when '!SECURITY_DEP' using the
PlatformNotSupportedException()-based implementation from SslStream.platformnotsupported.cs
* [System]: Rename the internal class 'ChunkStream' into 'MonoChunkStream'.
Our 'ChunkStream' is used by 'WebConnection', so it will stay after using
HttpListener from corefx.
* [System]: Fully stub-out SslStream on platforms where it's not supported.
* Remove 'SECURITY_DEP' conditional from LocalCertificateSelectionCallback.
* Add the internal delegates and remove 'SECURITY_DEP' from SslStream.platformnotsupported.cs.
* In SslStream.cs, we now provide the full API when '!SECURITY_DEP' using the
PlatformNotSupportedException()-based implementation from SslStream.platformnotsupported.cs
* Fix typo.
if assembly name is NULL, mono_exception_from_name_two_strings_checked looks
for a TLE(string) constructor - which takes a message - which we immediately
overwrite.
So pass the empty string instead, which goes to the (Mono-specific)
TLE(string,string) constructor which intializes ClassName and AssemblyName
members of a TLE.
If a class T has a field e of enum type E, and E is either
incomplete (EnumBuilder that hasn't had CreateType called) or a broken class,
then mark T as failed.
I noticed this when I cleaned my Mono repo and reran autogen.sh.
I suddenly got a popup from XQuartz about "Unrecognized option: =".
Turns out this was introduced in 9e967f00b3,
the second half of the if is missing `test`, so it actually runs
`x = xyes` which starts XQuartz :D
Also cleaned up the USE_NLS checking since that ran immediately
before this and just printed "no" before which confused me.
- System.Text.EncodingHelper.InternalCodePage
- System.Runtime.InteropServices.Marshal.PtrToStringAnsi(intptr)
- Use coop handles for ves_icall_System_Configuration_DefaultConfig_get_machine_config_path
- Use coop handles for ves_icall_System_Configuration_InternalConfigurationHost_get_bundled_app_config
- Use coop handles for System.Web.Util.ICalls.GetMachineInstallDirectory
We trigger degraded mode when we are unable to properly build the nursery fragments due to excessive nursery pinning. This is done by initializing degraded_mode at the end of a minor. alloc_degraded is used to do allocation to major heap either as part of mature allocation or as part of degraded mode (when we bump the degraded_mode so we can detect when the next minor should be triggered).
When failing to allocate from the nursery after doing a minor (because the other threads might have already consumed the nursery) we need to do a mature allocation (as a fallback) instead of a degraded mode allocation so we don't uselessly enable the mode.
* [mixed] Add beginnings of unwinding support for interp->jit transitions by pushing an LMF frame on the stack when exiting interpreted code.
* [runtime] Add a mono_debug_lookup_source_location_by_il () helper function to lookup a source location using an IL offset.
* [mixed] Add support for mixed mode managed stack walks.
* [interp] Generate line number info for interpreted code, using the same MonoDebugMethodJitInfo structure used by the JIT.
* [interp] Pass the clause index to the MINT_ENDFINALLY opcode. Not yet used.
* [interp] Create a MonoJitInfo structure for each interpreted method. Not yet used.
* [interp] Add a mono_interp_set_resume_state () function which can be used to set the frame/ip the interpreter will resume execution from when execution returns to it.
* [mixed] Add support for mixed mode exception handling by extending the normal JIT exception handling code in mini-exceptions.c to handle interpreted frames as well.
* [jit] Add an Unwinder type to reduce the amount of code duplication when unwinding though interpreter frames.
* [interp] Fix the !ENABLE_INTERPRETER build.
* [interp] Disable an assert which is hit even when running without --interpreter.
* Fix a comment.
* [interp] Fix an assertion. Fix/add comments.
This adds to mini.h & fixes naming of the following functions:
emit_get_gsharedvt_info_klass
mono_emit_calli
get_memcpy_method
get_memset_method
emit_memory_barrier
emit_runtime_constant
mini_class_check_context_used
mono_emit_wb_aware_memcpy
emit_write_barrier
Those functions will be used later by the memory access functions to be extracted out of method-to-ir.c
With these targets we can generate a snapshot of the current
Mono API in C# source code form.
Whenever we make changes, we can rerun this to ensure we didn't
inadvertently change the public API.
It also outputs a diff in html form that we can show on CI.
The code expected that a custom attribute was specified as a MonoCMethod and
was calling mono_image_create_token(). However that function expects to be
called from the managed S.R.E.ModuleBuilder:GetToken (MemberInfo) which handles
all the builders in managed code. Consequently mono_image_create_token () was
asserting.
So now we encode the token for a ConstructorBuilder in place.
monodroid used to build its own version of mkbundle since it
requires slightly different code than the desktop. This
logic was protected in a #if block. Now that monodroid is
trying to make use of the shipped mkbundle rather than use
its own we need to turn this logic on at runtime.
A new parameter has been added
--bundled-header
When this option is passed, the code which skips including
mono/metadata/mono-config.h
will be skipped in favour of defining the required structures
and entry points directly.
* [System]: Cleanup internal TLS Provider registration.
Merge MonoTlsProviderFactory.Apple.cs and MonoTlsProviderFactory.Droid.cs
into MonoTlsProviderFactory.cs.
Disable the dynamic reflection-load on mobile while still keeping the
registration, so we can use IsSupported() and friends.
* Make it build.
If mono is compiled without DISABLE_STRICT_STRONG_NAMES, but someone passes
--assembly-loader=legacy at runtime, give them the lax strong name
checking behavior (a candidate assembly is good as long as the name is right, but ignoring
the public key token and version).
When strict strong name checking is enabled, Mono will check public key tokens
and versions match between a requested assembly reference and a candidate found
via its usual loading directories.
amend
* [Mono.Security.Interface]: Improve synergy between `MonoTlsProvider`, `SslStream` and `IMonoSslStream`.
* Add `IMonoSslStream.SslStream` property; the `IMonoSslStream` is now only created my the `SslStream`
constructor, so `SslStream` "owns" the `IMonoSslStream`.
* Add internal `MonoTlsProvider.CreateSslStreamInternal()`.
* `SslStream` does not need to implement any additional interfaces; we can do without after
getting rid of the wrapper classes.
* Add internal `SslStream.CreateMonoSslStream(Stream,bool,MonoTlsProvider,MonoTlsSettings)`.
* `MonoTlsProvider.CreateSslStream()` implementations should use this new method.
* Remove ancient obsolete MonoTlsProviderFactory APIs.
* `HttpListener` and `HttpConnection` now use `SslStream` instead of `IMonoSslStream`.
* [System]: Cleanup internal TLS Provider registration.
* Make it build.
errors are correctly dealt with.
Some network errors in iOS are exposed as Stream errors, specially those
related with the lost of the connection. This patch ensures that if such
an issue happens an exception is raised in the async read task.
Fixes bug https://bugzilla.xamarin.com/show_bug.cgi?id=53792
* Check strong names on a couple more codepaths during assembly loading. Fixes [bugzilla #580](https://bugzilla.xamarin.com/show_bug.cgi?id=580) and [bugzilla #55436](https://bugzilla.xamarin.com/show_bug.cgi?id=55436)
* Add `--apply-bindings=FILE` option. Allows us to specify a configuration file for assembly binding redirections when AOTing.
* Use the above flag when AOTing `System.ReflectionMetadata.dll` to pick up `csc.exe.config`
* Add all the Facades to the `framework_assemblies` list in the runtime. Facade and non-Facade framework assemblies behave differently:
* for Facades, don't do any version remapping, and any version is okay.
* for non-Facades, consult the version map to remap depdending on the --runtime used.
Previous it was just the desktop runtime, but now that we have special handling
for framework facade assemblies in the loader, we should do it on other
runtimes, too.
As long as the name and public key token matches.
This is different from the non-facade framework assemblies for which the
name and version must match but the public key token is ignored.
Apply bindings from configuration file FILE when running the AOT compiler.
This is useful if you need to AOT a module bar.dll that is loaded when the main
assembly foo.exe has a configuration file foo.exe.config that specifies
assembly bindings that bar.dll should respect.
* Create one filters file for each targets file
* Import each filters file into the project-level filters file
* Add folders for Common, Win32, and Posix targets files
This change extracts the eglib source files from the build instuctions
for builds via MSVC on Windows. It also separates the source files into
platform-specific and common .targets files. This provides more
flexibility for builds from MSVC in a few important ways:
1. We can do MSVC-based builds for platforms that will not be pushed to
public repos by checking out the Mono code externally and importing the
.targets files into a custom .vcxproj for the platform.
2. We can replace the platform specific .targets files with a custom set
of source files to implement a different platform abstraction layer
that will not be public without the need to merge changes to the
.vcxproj from the public repo.
This allows for ease of synchronizing changes to the public Mono
repo with private forks, while _not_ changing the behavior of the code
in the public repo.
* [aot] Emit class references in EH info in such as way that they can be skipped when reading them in async context.
* [aot] Rewrite the LLVM EH info loading code by moving allocations to callers so the lower level code is async safe.
* [aot] Make the memory allocation in the loading of llvm EH info async safe. Remove fields from MonoLLVMFDEInfo which are passed as parameters.
* [runtime] Add the ability to store the unwind info address/length in the MonoJitInfo structure instead of referencing it using the 'unwind_info' field.
* [aot] Make the loading of llvm EH information async safe.
Memory allocated with mono_gc_alloc_fixed is uncollectable both on sgen and boehm. This array serves no other purpose. It should probably be killed entirely, but it's part of the API.
replace it by its canonical open instantiation.
If we have
```
class Foo<A,B,C> { ... }
class Bar<T,U> { ... }
```
Then if someone tries to construct `Bar<Foo, int>` (or more precisely, the
MonoGenericInst corresponding to `<Foo, int>` instead return `Bar< Foo<A,B,C>, int >`.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=54485
The canonical instantiation of a generic type definition is the open
instantiation where the GTD is applied to its own generic parameters in order.
Suppose we have `class T<A,B> { ... }` then the canonical instantiation is just "T<A,B>".
When `PCL` is defined, the intent is that the resulting
`Mono.Options.dll` assembly will be usable on all PCL profiles.
This includes "silly" profiles such as Profile259, which lacks the
`System.Console` type, meaning e.g. `Console.Out` doesn't exist.
Unfortunately, the
`CommandSet(string, Converter<string, string, TextWriter, TextWriter)`
constructor would use `Console.Out` and `Console.Error` when the
`TextWriter` parameters were `null`, meaning this constructor can't
be provided for all PCL profiles.
Altering the constructor to instead use `TextWriter.Null` would allow
the constructor to be provided, but would be a *worse* experience,
IMHO: anybody (everybody!) expecting the default values to
"do something useful" will be sorely disappointed, as output and
error messages would instead be dropped. :-(
Instead, remove this constructor:
public CommandSet (string suite, MessageLocalizerConverter localizer = null, TextWriter output = null, TextWriter error = null);
and replace it with these *two* constructors:
public CommandSet (string suite, MessageLocalizerConverter localizer = null)
public CommandSet (string suite, TextWriter output, TextWriter error, MessageLocalizerConverter localizer = null)
The first constructor, which accepts no `TextWriter` parameters, is
only provided in non-PCL builds, allowing it to use `Console.Out`.
The second constructor is available in PCL profiles, and *requires*
that `output` and `error` be non-`null`.
Documentation has been updated accordingly.
The TrimSpecial() method and Windows-specific code was removed from
xammac 4.5 so reflection can't find it anymore, handle this case.
Also add a workaround for SqlParameterCollectionTest.CopyToTest()
which fails on watchos since the SqlCommand would throw a PNSE.
It's a workaround because it just makes the test ignored since we don't
set the env var that ConnectionManager.Instance.Sql reads in XI tests,
but it makes the test consistent with the other tests in the suite.
We should fix this properly later on.
* There is an existing check for the `sockaddr_in6` struct.
* Use it to wrap code that implements IPv6.
* This makes porting to platforms that don't support IPv6 easier.
* It should have no impact on platforms that do support IPv6.
When ConnectionManager.Instance.Sql.ConnectionString throws in
SetUp() we'd still call SqlConnection.ClearAllPools() in the TearDown()
which happens for example when the connection string environment
variable is not set.
This is a problem on WatchOS since SqlConnection.ClearAllPools()
throws PlatformNotSupportedException there, making the tests fail.
Instead we now check whether the connectionString is set and only
call the method in that case.
We previously recursed through the entire directed cyclic graph
of the type passed to SRE. When given a (legal) circular type reference,
we encountered stack overflows. This was particularly easy to trigger
with fsharpi and with FAKE.
By creating everything about the type before searching for the parent,
and by defering all type hierarchy establishment until the type must be
surfaced to manage code, we tease out the data dependencies caught up in
the cycle.
Note the bounded lifetime of the unmanaged object hidden in the SRE
module, which acts as a global reference for all types reachable from
the TypeBuilder passed to the runtime through the icall.
* Update nuget-buildtasks from upstream's `dev15.1` branch
dev15.1 branch + fixup import paths
This was prompted by needing to support nuget restore for projects with
`PackageReference`.
* [nuget] Bump reference to NuGet.BuildTasks to pick up a build fix
* Remove the use of ExpectedException in Mono.Posix Tests
ExpectedException is not available in later versions of
NUnit. To test the NetCore 2.0 version of Mono.Posix I
need a newer version of NUnit that does not contain ExpectedException.
Using Assert.Throws should be equivalent.
* Mono.Posix Tests move CanUseRealTimeSignals checks
Moving the CanUseRealTimeSignals check outside of the
Assert.Throws callback function.
This is a change in test behavior requested by @akoeplinger
in the PR review.
* Fixing Mono.Posix.UnixSignalTest.TestSignumPropertyThrows
This test throws InvalidOperationException and we need to check
for that in the Assert.Throws type.
mono_gc_wbarrier_value_copy_bitmap shown up when profiling Roslyn and this PR applies the following set of optimizations to it:
- Drop the bitmap, the extra branch in the loop costs more than the extra cardtable store itself
- Rename it to mono_gc_wbarrier_range_copy to reflect its new meaning
- Remove the JIT wrapper, this function doesn't need it
- Use mono_gc_get_range_copy_func as a way to punch through all layers and get the actual implementation
- Move the implementation to sgen-cardtable.c where everything can be inlined
- Exploit the fact we only need to mark the first card of a given range and thus we can hoist the address calculation outside of the loop
All of this speeds up a microbenchmark by 2x and Roslyn wallclock by around 2%.
This optimization exploits the fact that if the element type idepth is <= MONO_DEFAULT_SUPERTABLE_SIZE we can skip the idepth check.
This improves an array store only benchmark by 11%.
On Roslyn, all arrays falls into this case.
* [WebConnection] Make ReadDone and InitRead instance methods to avoid passing cnc around
* [WebConnection] Inline only call to InitConnection
* [WebConnection] Fix race condition between Close and BeginWrite
* ves_icall_System_Runtime_Remoting_Contexts_Context_RegisterContext
* ves_icall_System_Runtime_Remoting_Contexts_Context_ReleaseContext
Added new mono_threads_{register,release}_app_context to be used internally in
the runtime.
* [BTLS]: Cleanup certificate store initialization.
* Kill unused MonoBtlsX509Store.AddTrustedRoots().
* In server-mode, MonoBtlsProvider.SetupCertificateStore() now only adds
certificates explicitly trused via MonoTlsSettings.TrustAnchors.
* MonoTlsProvider.ValidateCertificate() - which is called from
X509Certificate2.Verify() via X509CertificateImplBtls.Verify() - now
uses MonoTlsSettings.DefaultSettings and assumes client-mode.
* [appletls]: Actually use the new code that was added in PR #4671.
* [btls]: Add new MonoTlsSettings.CertificateValidationTime property.
This allows you to set a custom time for certificate expiration checks.
* [appletls]: Bind SecTrustSetVerifyDate() and check MonoTlsSettings.CertificateValidationTime.
When framework assemblies are compiled for the mobile runtime, they are
generally signed with a different key than the normal desktop one.
Unfortunately each assembly has different logic for which key is
used (typically silverlight.pub or corefx.pub in place of ecma.pub), and short
of hardcoding a table in the runtime, there's no good way to know.
Main pointers :
- explicit minor and bridge configuration (add some for missing major)
- reduce duplicated testing or of low relevance (like split nursery)
- intensive testing of latest new configurations combined (parallel minor + major + future experiments)
- include testing with a small and a large nursery for additional variance
* [appletls]: Use SecIdentityCreate() to avoid using the Mac keychain when we have a private key.
Reading Certificates from the Mac Keychain
==========================================
Reading the private key from the keychain is a new feature introduced with
AppleTls on XamMac and iOS. On Desktop Mono, this new feature has several
known issues and it also did not received any testing yet. We go back to the old
way of doing things, which is to explicitly provide an X509Certificate2 with a
private key.
Keychain Dialog Popups
======================
When using Xamarin.Mac or Xamarin.iOS, we try to search the keychain
for the certificate and private key.
On Xamarin.iOS, this is easy because each app has its own keychain.
On Xamarin.Mac, the .app package needs to be trusted via code-sign
to get permission to access the user's keychain. [FIXME: I still have to
research how to actually do that.] Without this, you will get a popup
message each time, asking you whether you want to allow the app to access
the keychain, but you can make these go away by selecting "Trust always".
On Desktop Mono, this is problematic because selecting "Trust always"
give the 'mono' binary (and thus everything you'll ever run with Mono)
permission to retrieve the private key from the keychain.
This code would also trigger constant keychain popup messages,
which could only be suppressed by granting full trust. It also makes it
impossible to run Mono in headless mode.
SecIdentityCreate
=================
To avoid these problems, we are currently using an undocumented API
called SecIdentityRef() to avoid using the Mac keychain whenever a
X509Certificate2 with a private key is used.
On iOS and XamMac, you can still provide the X509Certificate without
a private key - in this case, a keychain search will be performed (and you
may get a popup message on XamMac).
The code was bailing straight to native which is significantly slower. This is the result of roslyn compiling corlib on my machine:
Before:
real 0m5.757s
user 0m22.059s
sys 0m1.736s
After:
real 0m5.816s
user 0m23.623s
sys 0m1.693s
Added a new EXTERNAL_FACADE_DRAWING_REFERENCE variable which
specifies the path to an assembly that contains drawing types,
e.g. in the monotouch/monodroid case System.Drawing is built
inside the XI/XA repos.
Also update netstandard typeforwards to the latest version
from the dotnet/standard repo.
resgen.exe used to load WinForms via reflection which is a pretty
large dependency. Instead we can just embed the few types that
are used for reading/writing resx files directly into the app.
Add System.Drawing.dll to the "build" profile so we can put
resgen there as well and allow us to no longer rely on system
resgen for mobile profiles like monotouch/monodroid.
The first iteration of this code suffered from spurious wakeups due to using a single cond-var for all compilation.
This introduce a per-job cond var and make threads wait on that. Spurious wakeups are quite problematic in heavily threaded
code such as Roslyn. On average, every wait would get 2 extra wakeups for no good reason.
With this change, on my laptop compilation times are the following:
With spurious wakeups:
real 0m6.134s
user 0m23.588s
sys 0m2.331s
Without spurious wakeups:
real 0m5.658s
user 0m22.851s
sys 0m1.694s
This represents a 8% wallclock speedup and 3% user time speedup.
As part of this change, the active threads accounting is gone as the number of waits due to cpu overload was close to zero.
Also remove Mono.Simd as we don't it on new platforms.
Cleanup monotouch_dirs to not depend on testing_aot_full_dirs since
it should be separate (even though it contains the same right now).
This shown up as a problem under parallel roslyn.
We hit frequently the case of multiple threads compiling the same method in parallel
and wasting all but one.
Another issue that happens, but infrequently, is having more threads JITing than there
are cores in the machine. Such thrashing doesn't help.
Test setup. 4/8 macbook pro compiling corlib.
Baseline:
real 0m7.665s
user 0m20.078s
sys 0m2.653s
Methods JITted using mono JIT : 22422
Total time spent JITting (sec) : 18.5365
With this patch:
real 0m6.149s
user 0m18.504s
sys 0m1.487s
Methods JITted using mono JIT : 16619
Total time spent JITting (sec) : 4.9420
New counters
JIT compile waited others : 7681
JIT compile 1+ jobs : 1
JIT compile overload wait : 67
JIT compile spurious wakeups : 14469
This results in a 20% wall clock reduction but only a 8% reduction on user.
We JIT 26% less methods, with very few duplications. Showing this drastically improves the situation.
JIT compilation time metrics are bogus due to .cctors and other sources of interference. So take it with a grain of salt.
Future work:
Based on the new counters, it's clear that the current wakeup design is suboptimal and we could further improve it by cutting on spurious wakeups.
Loading modules for other processes can fail under some linux sandboxing rules but that doesn't mean
we should not be able to compute the process name. So when module == NULL, we make module loading non-fatal.
It is used as the "verify" step on Jenkins/Wrench and I noticed
it didn't run recently.
Looks like it was broken a while ago by 695e8f53ae.
Moving the verifiable_files variable at the top is wrong since
the $$profile variable is only defined inside the for loop and
so we'd get an empty list.
When that was fixed it still didn't work as piping to `echo`
doesn't work. Replaced it with a Makefile solution instead.
While this was ok because workers and sweep could not run at the same time, this is no longer the case with parallel nursery collections. In the future, if many threads will be required for different types of jobs, we can consider running them on same native thread, but the jobs will still need to have appropriate contexts (the SgenThreadPool they belong to).
Stealing a gray section is relatively expensive since it requires locking. We want to avoid repeated stealing, since only one gray queue section is stolen at a time (in the case of the parallel collector).
AM_CONDITIONAL(DISABLE_LIBRARIES, test x$enable_libraries = xno)
case $host in
*nacl* ) with_shared_mono=yes;;
esac
if test "x$host_win32" = "xyes"; then
# Boehm GC requires the runtime to be in its own dll
with_static_mono=no
@@ -804,7 +850,10 @@ if test $csc_compiler = default; then
if test $endian = big; then
csc_compiler=mcs
elif test $endian = little; then
csc_compiler=roslyn
case "$host" in
powerpc*) csc_compiler=mcs ;;
*) csc_compiler=roslyn ;;
esac
else
csc_compiler=mcs
fi
@@ -825,10 +874,11 @@ AC_ARG_WITH(xammac, [ --with-xammac=yes,no If you w
AC_ARG_WITH(testing_aot_hybrid, [ --with-testing_aot_hybrid=yes,no If you want to build the testing_aot_hybrid assemblies (defaults to no)], [], [with_testing_aot_hybrid=default])
AC_ARG_WITH(testing_aot_full, [ --with-testing_aot_full=yes,no If you want to build the testing_aot_full assemblies (defaults to no)], [], [with_testing_aot_full=default])
AC_ARG_WITH(winaot, [ --with-winaot=yes,no If you want to build the Windows friendly AOT assemblies (defaults to no)], [], [with_winaot=default])
AC_ARG_WITH(orbis, [ --with-orbis=yes,no If you want to build the Orbis assemblies (defaults to no)], [], [with_orbis=default])
AC_ARG_WITH(orbis, [ --with-orbis=yes,no If you want to build the Orbis assemblies (defaults to no)], [], [with_orbis=default])
AC_ARG_WITH(unreal, [ --with-unreal=yes,no If you want to build the Unreal assemblies (defaults to no)], [], [with_unreal=default])
AC_ARG_WITH(runtime_preset, [ --with-runtime_preset=net_4_x,all,aot,hybridaot,fullaot,bitcode Which default profile to build (defaults to net_4_x)], [], [with_runtime_preset=net_4_x])
AC_ARG_WITH(runtime_preset, [ --with-runtime_preset=net_4_x,all,aot,hybridaot,fullaot,bitcode,unreal Which default profile to build (defaults to net_4_x)], [], [with_runtime_preset=net_4_x])
if test $ac_cv_sizeof_void_p = $ac_cv_sizeof_int; then
GPOINTER_TO_INT="((gint) (ptr))"
GPOINTER_TO_UINT="((guint) (ptr))"
GINT_TO_POINTER="((gpointer) (v))"
GUINT_TO_POINTER="((gpointer) (v))"
GSIZE="int"
GSIZE_FORMAT='"u"'
elif test $ac_cv_sizeof_void_p = $ac_cv_sizeof_long; then
GPOINTER_TO_INT="((gint)(long) (ptr))"
GPOINTER_TO_UINT="((guint)(long) (ptr))"
GINT_TO_POINTER="((gpointer)(glong) (v))"
GUINT_TO_POINTER="((gpointer)(gulong) (v))"
GSIZE="long"
GSIZE_FORMAT='"lu"'
elif test $ac_cv_sizeof_void_p = $ac_cv_sizeof_long_long; then
GPOINTER_TO_INT="((gint)(long long) (ptr))"
GPOINTER_TO_UINT="((guint)(unsigned long long) (ptr))"
GINT_TO_POINTER="((gpointer)(long long) (v))"
GUINT_TO_POINTER="((gpointer)(unsigned long long) (v))"
GSIZE="long long"
GSIZE_FORMAT='"I64u"'
else
AC_MSG_ERROR([unsupported pointer size])
fi
AC_SUBST(GPOINTER_TO_INT)
AC_SUBST(GPOINTER_TO_UINT)
AC_SUBST(GINT_TO_POINTER)
AC_SUBST(GUINT_TO_POINTER)
AC_SUBST(GSIZE)
AC_SUBST(GSIZE_FORMAT)
AC_SUBST(G_GUINT64_FORMAT)
AC_SUBST(G_GINT64_FORMAT)
AC_SUBST(G_GUINT32_FORMAT)
AC_SUBST(G_GINT32_FORMAT)
#
# END OF EGLIB CHECKS
#
AC_ARG_WITH([libgdiplus],
[ --with-libgdiplus=installed|sibling|<path> Override the libgdiplus used for System.Drawing tests (defaults to installed)],
[], [with_libgdiplus=installed])
@@ -4094,8 +4254,6 @@ dnl **************
AC_ARG_ENABLE(btls, [ --disable-btls Disable the BoringTls provider], enable_btls=$enableval, enable_btls=$BTLS_SUPPORTED)
AC_ARG_WITH(btls_android_ndk, [ --with-btls-android-ndk Android NDK for BoringTls])
AC_ARG_ENABLE(dynamic-btls, [ --enable-dynamic-btls Place the BTLS provider into a separate shared library/archive.], enable_dynamic_btls=$enableval, enable_dynamic_btls=no)
AM_CONDITIONAL(BTLS, test x$enable_btls = xyes)
btls_android=no
@@ -4124,6 +4282,16 @@ if test "x$enable_btls" = "xyes"; then
$(Q)cd$(MONO_API_SNAPSHOT_PATH); git diff --no-renames HEAD > $(abspath $(topdir))/temp.patch
@mkdir -p apidiff
$(Q) sed -e "/@diffdata@/r temp.patch" -e "/@diffdata@/d" -e "s/@title@/Public API Diff/g" -e "s/@description@/If the changes are intentional, run <code>make -C mcs mono-api-diff<\/code> locally and commit changes in external\/api-snapshot./g" diff.html.in > apidiff/index.html
$(Q)if[ -s temp.patch ];thenecho"Error: Found public API differences, see mcs/apidiff/index.html. If the changes are intentional, please go to external/api-snapshot and commit them."; rm -f temp.patch;exit 1;elseecho"No differences found."; rm -f temp.patch;fi
Loaded 100 of 5480 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.