[2018-08] [arm64] Correct exception filter stack frame size
Fixes: https://github.com/mono/mono/issues/14170
Fixes: https://github.com/xamarin/xamarin-android/issues/3112
The `call_filter()` function generated by `mono_arch_get_call_filter()`
was overwriting a part of the previous stack frame because it was not
creating a large enough new stack frame for itself. This had been
working by luck before the switch to building the runtime with Android
NDK r19.
I used a local build of [xamarin/xamarin-android/d16-1@87a80b][0] to
verify that this change resolved both of the issues linked above. I
also confirmed that I was able to reintroduce the issues in my local
environment by removing the change and rebuilding.
After this change, the generated `call_filter()` function now starts by
reserving sufficient space on the stack to hold a 64-bit value at the
`ctx_offset` location. The 64-bit `ctx` value is saved to the
`ctx_offset` location (offset 344 in the new stack frame) shortly
afterwards:
stp x29, x30, [sp,#-352]!
mov x29, sp
str x0, [x29,#344]
As expected, the invocation of `call_filter()` now no longer modifies
the top of the previous stack frame.
Top of the stack before `call_filter()`:
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0x2724b700 0x00000000
0x7fcd2774b0: 0x1ee19ff0 0x00000071 0x0cc00300 0x00000071
Top of the stack after `call_filter()` (unchanged):
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0x2724b700 0x00000000
0x7fcd2774b0: 0x1ee19ff0 0x00000071 0x0cc00300 0x00000071
<details>
<summary>Additional background information</summary>
### The original lucky, "good" behavior for [mono/mono/2018-08@725ba2a built with Android NDK r14][1]
1. The `resume` parameter for `mono_handle_exception_internal()` is
held in register `w23`.
2. That register is saved into the current stack frame at offset 20 by
a `str` instruction:
0x00000000000bc1bc <+3012>: str w23, [sp,#20]
3. `handle_exception_first_pass()` invokes `call_filter()` via a `blr`
instruction:
2279 filtered = call_filter (ctx, ei->data.filter);
0x00000000000bc60c <+4116>: add x9, x22, x24, lsl #6
0x00000000000bc610 <+4120>: ldr x8, [x8,#3120]
0x00000000000bc614 <+4124>: ldr x1, [x9,#112]
0x00000000000bc618 <+4128>: add x0, sp, #0x110
0x00000000000bc61c <+4132>: blr x8
Before the `blr` instruction, the top of the stack looks like this:
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0x0a8b31d8 0x00000071
0x7fcd2774b0: 0x00000000 0x00000000 0x1ef51980 0x00000071
4. `call_filter()` runs. This function is generated by
`mono_arch_get_call_filter()` and starts with:
stp x29, x30, [sp,#-336]!
mov x29, sp
str x0, [x29,#344]
Note in particular how the first line subtracts 336 from `sp` to
start a new stack frame, but the third line writes to a position
that is 344 bytes beyond that, back within the *previous* stack
frame.
5. After the invocations of `call_filter()` and
`handle_exception_first_pass()`, `w23` is restored from the stack:
0x00000000000bc820 <+4648>: ldr w23, [sp,#20]
At this step, the top of the stack looks like this:
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0xcd2775b0 0x0000007f
0x7fcd2774b0: 0x00000000 0x00000000 0x1ef51980 0x00000071
Notice how `call_filter()` has overwritten bytes 8 through 15 of the
stack frame. In this original lucky scenario, this does not affect
the value restored into register `w23` because that value starts at
byte 20.
6. `mono_handle_exception_internal()` tests the value of `w23` to
decide how to set the `ji` local variable:
2574 if (resume) {
0x00000000000bb960 <+872>: cbz w23, 0xbb9c4 <mono_handle_exception_internal+972>
Since `w23` is `0`, `mono_handle_exception_internal()` correctly
continues into the `else` branch.
### The bad behavior for [mono/mono/2018-08@725ba2a built with Android NDK r19][2] works slightly differently
1. As before, the local `resume` parameter starts in register `w23`.
2. This time, the register is saved into the stack frame at offset 12
(instead of 20):
0x00000000000bed7c <+3200>: str w23, [sp,#12]
3. As before, `handle_exception_first_pass()` invokes `call_filter()`
via a `blr` instruction.
At this step, the top of the stack looks like this:
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0x2724b700 0x00000000
0x7fcd2774b0: 0x1ee19ff0 0x00000071 0x0cc00300 0x00000071
4. `call_filter()` runs as before. And the first few instructions of
that function are the same as before:
stp x29, x30, [sp,#-336]!
mov x29, sp
str x0, [x29,#344]
5. `w23` is again restored from the stack, but this time from offset 12:
0x00000000000bf2c0 <+4548>: ldr w23, [sp,#12]
The problem is that `call_filter()` has again overwritten bytes 8
through 15 of the stack frame:
(gdb) x/8x $sp
0x7fcd2774a0: 0x7144d250 0x00000000 0xcd2775b0 0x0000007f
0x7fcd2774b0: 0x1ee19ff0 0x00000071 0x0cc00300 0x00000071
So after the `ldr` instruction, `w23` has an incorrect value of
`0x7f` rather than the correct value `0`.
6. As before, `mono_handle_exception_internal()` tests the value of `w23` to
decide how to set the `ji` local variable:
2574 if (resume) {
0x00000000000be430 <+820>: cbz w23, 0xbe834 <mono_handle_exception_internal+1848>
But this time because `w23` is *not* zero, execution continues
*incorrectly* into the `if` branch rather than the `else` branch.
The result is that `ji` is set to `0` rather than a valid
`(MonoJitInfo *)` value. This incorrect value for `ji` leads to a
null pointer dereference, as seen in the bug reports.
[0]: https://github.com/xamarin/xamarin-android/tree/xamarin-android-9.3.0.22
[1]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1432/Azure/
[2]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1436/Azure/
</details>
Backport of #14757.
/cc @lambdageek @brendanzagaeski
TypeTest.MakeGenericType_BadUserType
TypeTest.MakeGenericType_NestedUserDefinedType
TypeTest.MakeGenericType_UserDefinedType
TypeTest.TestMakeGenericType_UserDefinedType_DotNet20SP1
These tests pass when run with the interpreter on xamarin-macios.
They were disabled on master with https://github.com/mono/mono/commit/dc9bc9e0e6462739872117afc59e46fbc780d981
Bump to get the msbuild 16 update
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set. In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.
Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace
```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
* frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
I'm not sure about the correctness of this workaround but it works on macOS for the repro from https://github.com/mono/mono/issues/9664#issuecomment-472260968
Transition bounds used by time zone info represent some local date-time but actually they are of Unspecified date-time kind.
When we try to get transition offset we compare some current date-time converted to UTC with appropriate transition bounds.
The fix is to use UTC for both current date-time and transition bounds during comparing.
Fixes https://github.com/mono/mono/issues/9664
-> Doing stepping into in VSM in some situation the frame saved in TLS is not synchronised with what is really been executed in the main thread. This makes the debugger-agent crashes because it tries to get variable info in some memory that is not available anymore.
-> To fix it I forced the update of stack when CMD_THREAD_GET_FRAME_INFO is called.
-> Doing step over in Visual Studio for Windows, if you have the threads debugger window enabled, VSW calls frame_commands for each thread that is showed, and if the thread is not really_suspended it tries to get variable info from a memory that is not available anymore because the thread is not suspended yet.
-> To fix it I don't send variable info of a frame if the thread is not really_suspended and doesn't have an async_state valid.
#Fixes 12843
Concurrent request completions could cause following exception:
FATAL UNHANDLED EXCEPTION: System.ArgumentOutOfRangeException:
Index was out of range. Must be non-negative and less than
the size of the collection.
Parameter name: index
at System.Collections.Generic.List`1[T].RemoveAt (System.Int32 index) [0x00009] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Collections.Generic.List`1[T].Remove (T item) [0x0000c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.Cleanup () [0x00011] in <28c4695205ef47378f5370502b066592>:0
at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.set_IsCompleted (System.Boolean value) [0x00034] in <28c4695205ef47378f5370502b066592>:0
at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.Complete (System.Exception ex) [0x0001a] in <28c4695205ef47378f5370502b066592>:0
at System.ServiceModel.Channels.HttpRequestChannel+<>c__DisplayClass7_1.<BeginProcessRequest>b__0 (System.IAsyncResult r) [0x000bd] in <28c4695205ef47378f5370502b066592>:0
This stack was captured after some changes in AwaitTaskContinuation.RunOrScheduleAction.
Unfortunately the current implementation of AwaitTaskContinuation.RunOrScheduleAction re-throws the above
exception with a wrong stack:
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.FinishStageTwo () [0x000a1] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.Finish (System.Boolean bUserDelegateExecuted) [0x00003] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.TrySetException (System.Object exceptionObject) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetException (System.Exception exception) [0x00029] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Net.HttpWebRequest+<RunWithTimeoutWorker>d__241`1[T].MoveNext () [0x000e0] in <3e9b3e26c4694baab3f689687ad40612>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.InvokeMoveNext (System.Object stateMachine) [0x00000] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00024] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x0004f] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x0003d] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
at System.Net.ServicePointScheduler+<WaitAsync>d__45.MoveNext () [0x000f7] in <3e9b3e26c4694baab3f689687ad40612>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.InvokeMoveNext (System.Object stateMachine) [0x00000] in <4a8a0
On an Android Xamarin installation this causes an app crash. Since no user code is active the exception cannot be caught.
* [crash] Add more detailed reporting for exceptions
This bumps the crash reporting file format to version 0.0.3.
* [crash] Fix exception stack walker to consider captured_traces
Previously we were missing these frames from the captured_traces fields
of MonoException and MonoStackTrace.
* [interp] Add method to seq_points table after we finish registering it
When adding a breakpoint (mono_de_set_breakpoint) we search in this table for seq_point information for the method. If we find the information we expect that the method is already compiled and has the metadata initialized when actually adding the breakpoint (set_bp_in_method). We were crashing if the jinfo field of the interp method was not yet initialized.
Fixes https://github.com/xamarin/xamarin-macios/issues/5381
* [interp] Move profiler callback when we are actually done
Just in case.
What was happening was that this was building with a Mono.Cecil from
the build profile, but was trying to load the system Mono.Cecil.
Those versions mismatch when building 2018-08 with a nightly mono that is 5.23.0.
Rather than messing with the MONO_PATH and risking shadowing something
important there, I chose to copy the Mono.Cecil files into this
"lib/build/tmp" directory that we are using for the emitted executable.
It is loaded by the system Mono as the correct Mono.Cecil to use.
This fixes the build.
* [crash] Support extra merp params
* [runtime] Make infrastructure for merp tests
* [runtime] Fix dumping when crash happens without sigctx
* [runtime] Disable crashy (886/1000 runs) stacktrace walker
* [crash] Remove usage of allocating build/os info functions
* [crash] Remove often-crashing g_free on native crash path
* [crash] Add crash_reporter checked build
We add a checked build mode that asserts when mono mallocs inside of
the crash reporter. It makes risky allocations into assertions. It's
useful for automated testing because the double-abort often represents
itself as an indefinite hang. If it happens before the thread dumping
supervisor process is started, or after it ends, the crash reporter
hangs.
* [crash] Remove reliance on nested SIGABRT/double-fault (broken on OSX)
* [crash] Fix top-level handling of double faults/assertions
* [runtime] Make fatal unwinding errors return into handled error paths
* [crash] Change dumper logging for better info
* [runtime] Fix handling of segfault on sgen thread
Threads without domains that get segfaults will end up in
this handler. It's not safe to call this function with a NULL domain.
See crash below:
```
* thread #1, name = 'tid_307', queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x10eff40f8)
* frame #0: 0x000000010e1510d9 mono-sgen`mono_threads_summarize_execute(ctx=0x0000000000000000, out=0x0000001000000000, hashes=0x0000100000100000, silent=4096, mem="", provided_size=2199023296512) at threads.c:6414
frame #1: 0x000000010e152092 mono-sgen`mono_threads_summarize(ctx=0x000000010effda00, out=0x000000010effdba0, hashes=0x000000010effdb90, silent=0, signal_handler_controller=1, mem=0x0000000000000000, provided_size=0) at threads.c:6508
frame #2: 0x000000010df7c69f mono-sgen`dump_native_stacktrace(signal="SIGSEGV", ctx=0x000000010effef48) at mini-posix.c:1026
frame #3: 0x000000010df7c37f mono-sgen`mono_dump_native_crash_info(signal="SIGSEGV", ctx=0x000000010effef48, info=0x000000010effeee0) at mini-posix.c:1147
frame #4: 0x000000010de720a9 mono-sgen`mono_handle_native_crash(signal="SIGSEGV", ctx=0x000000010effef48, info=0x000000010effeee0) at mini-exceptions.c:3227
frame #5: 0x000000010dd6ac0d mono-sgen`mono_sigsegv_signal_handler_debug(_dummy=11, _info=0x000000010effeee0, context=0x000000010effef48, debug_fault_addr=0xffffffffffffffff) at mini-runtime.c:3574
frame #6: 0x000000010dd6a8d3 mono-sgen`mono_sigsegv_signal_handler(_dummy=11, _info=0x000000010effeee0, context=0x000000010effef48) at mini-runtime.c:3612
frame #7: 0x00007fff73dbdf5a libsystem_platform.dylib`_sigtramp + 26
frame #8: 0x0000000110bb81c1
frame #9: 0x000000011085ffe1
frame #10: 0x000000010dd6d4f3 mono-sgen`mono_jit_runtime_invoke(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, exc=0x00007ffee1ea9f08, error=0x00007ffee1eaa250) at mini-runtime.c:3215
frame #11: 0x000000010e11509d mono-sgen`do_runtime_invoke(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, exc=0x0000000000000000, error=0x00007ffee1eaa250) at object.c:2977
frame #12: 0x000000010e10d961 mono-sgen`mono_runtime_invoke_checked(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, error=0x00007ffee1eaa250) at object.c:3145
frame #13: 0x000000010e11aa58 mono-sgen`do_exec_main_checked(method=0x00007faae4f01fe8, args=0x000000010f0003e8, error=0x00007ffee1eaa250) at object.c:5042
frame #14: 0x000000010e118803 mono-sgen`mono_runtime_exec_main_checked(method=0x00007faae4f01fe8, args=0x000000010f0003e8, error=0x00007ffee1eaa250) at object.c:5138
frame #15: 0x000000010e118856 mono-sgen`mono_runtime_run_main_checked(method=0x00007faae4f01fe8, argc=2, argv=0x00007ffee1eaa760, error=0x00007ffee1eaa250) at object.c:4599
frame #16: 0x000000010de1db2f mono-sgen`mono_jit_exec_internal(domain=0x00007faae4f00860, assembly=0x00007faae4c02ab0, argc=2, argv=0x00007ffee1eaa760) at driver.c:1298
frame #17: 0x000000010de1d95d mono-sgen`mono_jit_exec(domain=0x00007faae4f00860, assembly=0x00007faae4c02ab0, argc=2, argv=0x00007ffee1eaa760) at driver.c:1257
frame #18: 0x000000010de2257f mono-sgen`main_thread_handler(user_data=0x00007ffee1eaa6a0) at driver.c:1375
frame #19: 0x000000010de20852 mono-sgen`mono_main(argc=3, argv=0x00007ffee1eaa758) at driver.c:2551
frame #20: 0x000000010dd56d7e mono-sgen`mono_main_with_options(argc=3, argv=0x00007ffee1eaa758) at main.c:50
frame #21: 0x000000010dd5638d mono-sgen`main(argc=3, argv=0x00007ffee1eaa758) at main.c:406
frame #22: 0x00007fff73aaf015 libdyld.dylib`start + 1
frame #23: 0x00007fff73aaf015 libdyld.dylib`start + 1
thread #2, name = 'SGen worker'
frame #0: 0x000000010e2afd77 mono-sgen`mono_get_hazardous_pointer(pp=0x0000000000000178, hp=0x000000010ef87618, hazard_index=0) at hazard-pointer.c:208
frame #1: 0x000000010e0b28e1 mono-sgen`mono_jit_info_table_find_internal(domain=0x0000000000000000, addr=0x00007fff73bffa16, try_aot=1, allow_trampolines=1) at jit-info.c:304
frame #2: 0x000000010dd6aa5f mono-sgen`mono_sigsegv_signal_handler_debug(_dummy=11, _info=0x000070000fb81c58, context=0x000070000fb81cc0, debug_fault_addr=0x000000010e28fb20) at mini-runtime.c:3540
frame #3: 0x000000010dd6a8d3 mono-sgen`mono_sigsegv_signal_handler(_dummy=11, _info=0x000070000fb81c58, context=0x000070000fb81cc0) at mini-runtime.c:3612
frame #4: 0x00007fff73dbdf5a libsystem_platform.dylib`_sigtramp + 26
frame #5: 0x00007fff73bffa17 libsystem_kernel.dylib`__psynch_cvwait + 11
frame #6: 0x00007fff73dc8589 libsystem_pthread.dylib`_pthread_cond_wait + 732
frame #7: 0x000000010e28d76d mono-sgen`mono_os_cond_wait(cond=0x000000010e44c9d8, mutex=0x000000010e44c998) at mono-os-mutex.h:168
frame #8: 0x000000010e28df4f mono-sgen`get_work(worker_index=0, work_context=0x000070000fb81ee0, do_idle=0x000070000fb81ed4, job=0x000070000fb81ec8) at sgen-thread-pool.c:165
frame #9: 0x000000010e28d2cb mono-sgen`thread_func(data=0x0000000000000000) at sgen-thread-pool.c:196
frame #10: 0x00007fff73dc7661 libsystem_pthread.dylib`_pthread_body + 340
frame #11: 0x00007fff73dc750d libsystem_pthread.dylib`_pthread_start + 377
frame #12: 0x00007fff73dc6bf9 libsystem_pthread.dylib`thread_start + 13
thread #3, name = 'Finalizer'
frame #0: 0x00007fff73bf6246 libsystem_kernel.dylib`semaphore_wait_trap + 10
frame #1: 0x000000010e1d9c0a mono-sgen`mono_os_sem_wait(sem=0x000000010e43e400, flags=MONO_SEM_FLAGS_ALERTABLE) at mono-os-semaphore.h:84
frame #2: 0x000000010e1d832d mono-sgen`mono_coop_sem_wait(sem=0x000000010e43e400, flags=MONO_SEM_FLAGS_ALERTABLE) at mono-coop-semaphore.h:41
frame #3: 0x000000010e1da787 mono-sgen`finalizer_thread(unused=0x0000000000000000) at gc.c:920
frame #4: 0x000000010e152919 mono-sgen`start_wrapper_internal(start_info=0x0000000000000000, stack_ptr=0x000070000fd85000) at threads.c:1178
frame #5: 0x000000010e1525b6 mono-sgen`start_wrapper(data=0x00007faae4f31bd0) at threads.c:1238
frame #6: 0x00007fff73dc7661 libsystem_pthread.dylib`_pthread_body + 340
frame #7: 0x00007fff73dc750d libsystem_pthread.dylib`_pthread_start + 377
frame #8: 0x00007fff73dc6bf9 libsystem_pthread.dylib`thread_start + 13
thread #4
frame #0: 0x00007fff73c0028a libsystem_kernel.dylib`__workq_kernreturn + 10
frame #1: 0x00007fff73dc7009 libsystem_pthread.dylib`_pthread_wqthread + 1035
frame #2: 0x00007fff73dc6be9 libsystem_pthread.dylib`start_wqthread + 13
(lldb)
```
* [crash] Add signal-safe mmap/file "allocator"
* [crash] Remove use of static memory from dumper
* [runtime] Reduce print buffer size for lockless printer.
Each frame that prints ends up increased by the size of buff.
In practice, clang often fails to deduplicate some of these buffers,
leading to 30k-big stackframes.
It was noticed by a series of hard-to-diagnose segfaults on stacks that
looked otherwise fine during the crash reporting stress test.
This change fixes this, making stacks a 1/10th of the size. It doesn't
seem to break the crash reporter messages anywhere (may need to shrink
other "max name length" fields), and it's not mission-critical anywhere
else.
* [crash] Use async-safe file memory for dumper internals
* [crash] Add memory barriers around merp configuration
* [crash] Use signal-safe printers on all native crash paths
* [crash] Move gdb/lldb lookup to startup
* [runtime] Move MOSTLY_ASYNC_SAFE_FPRINTF to eglib
* [runtime] Fix all callsites of MOSTLY_ASYNC_SAFE_PRINTF
* [runtime] Fix all callsites of MOSTLY_ASYNC_SAFE_FPRINTF
* [crash] Switch to signal-safe exit function
* [crash] Make dumper enum in managed
* [runtime] Add more information to managed frame
* [crash] Make async_safe printers inlined
* [runtime] Move basic pe_file functionality into proclib
* [crash] Fix handling of thread attributes
* [crash] Place hashes into the json file for all threads
* [crash] Fix 2018-08 CI, disable test
After .NET 4.5 .NET Framework has changed the System.ComponentModel.Composition ReflectionModelServices.IsExportFactoryImportDefinition to not throw.
Updating this to match the .NET Framework behavior.
Backport of: https://github.com/mono/mono/commit/eebe4b52857140ba9dea872c6249960273d222f2 from https://github.com/mono/mono/pull/12454
The latest Android NDK releases no longer ship with per-platform C headers.
Instead they ship with a set of unified headers and require the presence of the
`__ANDROID_API__` macro in order to select the target platform.
The AOT offset dumper doesn't support this new layout and this commit changes
that by adding support for the unified headers (without removing support for the
older NDKs) thus fixing the following error when trying to dump the offsets with
newer NDK:
Processing triple: aarch64-v8a-linux-android
Error parsing '[...]/mono/mono/metadata/metadata-cross-helpers.c'
[...]/mono/mono/metadata/metadata-cross-helpers.c(5,10): fatal: 'stdio.h' file not found
Error parsing '[...]/mono/mono/mini/mini-cross-helpers.c'
[...]/mono/mono/mini/mini-cross-helpers.c(7,10): fatal: 'stdio.h' file not found
* [Debugger] Fix crash when there is a generic struct with a field that is an enumerator. (#12368)
* [Debugger] Debugger crashes when inside a class, there is an internal struct, with a field that is an enumerator.
files.myBucket.GetEnumerator().get_Current().Key
Fixes#10735
* [Debugger] Debugger crashes when there is a generic struct with a field that is an enumerator.
Example: files.get_Current().Key
A unit test that reproduces this crash was added too.
Fixes#10735
* Removing the extra space.
* Merging changes from dtest-app.cs
* Adding extra line.
On some platforms (e.g. AppleTV) SigContext is available, but isn't as
precise as we would like to have it. For example, it doesn't save
callee-saved registers and thus aren't restored properly.
Unfortunately, this also means we can't use the fastpath for
interp2native, so we disable it on those platforms.
Xamarin.Android is looking to switch to using the latest Android NDK, r18.
However, there are a few build issues in Mono and BTLS which need to be
addressed before that happens.
* BTLS fails to build some ARM assembly sources with NDK r18's clang.
https://github.com/mono/boringssl/pull/12 fixes this issue
* BTLS uses obsolete Android cmake toolchain file.
Add a configure option to set alternative toolchain file.
* NDK r18 minimu supported API level is 16.
This commit makes the API level configurable and sets the minimum platform
to API 16
* NDK r18 makes changes to declaration of `mmap` which break Mono build.
Detect NDK version in `mono-compiler.h` and act accordingly (do not
redeclare `mmap`)
* NDK r18 doesn't have `cpp` and SDKs build assumes it's present.
Detect whether `cpp` is present and, if not, use `compiler -E` instead.
* NDK r18 uses unified headers exclusively and requires __ANDROID_API__ to be
set in order to present the correct set of native APIs.
SDKs now define the macro when the `ANDROID_NEW_NDK` Make variable is set to
anything else than `no`.
* [Network] Fix CVE 2018-8292 on Mac OS X.
This commit ensures that CVE 2018-8292 is fixed by not setting
auto-redirect in the first request done by CFNetwork. CFNetwork has the
same bug, but we can work around it by performing the first request,
checking the response and, if it is a rediret, follow it without the
Authorization headers.
* No need to close the obc, it can be reused.
* Small typo.
* Fix error while chaging the request :/
* Minor style changes.
There are two identical sounding functions:
mono_string_handle_to_utf8:
What we usually use. Sounds reasonable. Usually is reasonable.
Allocates with malloc.
mono_string_to_utf8str_handle:
What should be used here. Allocates with CoTaskMemAlloc.
So it can be freed with marshal_free => CoTaskMemFree.
i.e. The allocator and the freeer need to agree.
[2018-08] [mini] use AOT trampolines in interp mixed mode
Fixes this on mscorlib/mixed mode on Xamarin.iOS:
```
[FAIL] NonExceptional.Serialization_Deserialization : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt.Â
[FAIL] NonExceptional.Serialize_Deserialize_FixedDateRule : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt.Â
[FAIL] NonExceptional.Serialize_Deserialize_FloatingDateRule : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt.Â
[FAIL] SerializationTests.Serialization_Deserialization : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt.Â
[FAIL] TimeZoneTest.CurrentTimeZone_SerializationRoundtrip : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt.Â
```
Backport of #11767.
/cc @lewurm
[2018-08] [interp] attempt to intrinsify again after a method is resolved regarding generics
discovered when running https://github.com/Clancey/gMusic
Backport of #11712.
/cc @luhenry @lewurm
This pull request fixes starting a process with an empty environment variable collection in `ProcessStartInfo`. As per the issue it fixes a regression that it used to work but currently an empty environment variable collection works like it was never touched.
I also made a minor fix to remove the `StringBuilder` because a simple `string.Concat` does the job without the `StringBuilder` allocation and overhead.
I've included unit tests. Please let me know if you'd like any changes.
Fixes#8766
Backport of #11630.
Otherwise it might have left-over files from previous builds since it
uses our custom submodule system which isn't catched by Jenkins cleaning.
(cherry picked from commit 49a5af74f7)
(cherry picked from commit b429e290e8)
[2018-08] [ios] remove DISABLE_POLICY_EVIDENCE
Originally it was introduced with db3591b
It seems like only the icall definitions were disabled for device
runtimes (which aren't shipped there anyway), however the icalls
themselves are _included_ in the device runtime. Not sure what the
rationale was behind this. Since the original change is very old, I
think it just happened to be like that today. When running with the
interpreter we need the definitions at runtime too (via the optional
icall table that is linked in, so no additional cost for the fullAOT
runtime due to this change).
Backport of https://github.com/mono/mono/pull/11565
/cc @lewurm
[2018-08] [bcl] avoid false pinning in GCTest
Backport of #11521.
/cc @luhenry @lewurm
Description:
needed for mscorlibtests on Xamarin.iOS with interpreter
Backport of #11435.
/cc @alexanderkyte
Description:
The change which made the handler return (rather than crashing the
runtime) did not consider this control flow path that causes the sigterm
signal handler to trigger the merp dumper if we return.
We now always return.
This was previously so that a random sigterm could be used to trigger a
dump. This isn't really used, and won't work right now anyways
(we don't register it until right before we dump). My fix is to remove
it, and to make the signal handler only work as part of the dumper
machinery.
@monojenkins skip
Earlier in the 2018-08 branch, Mono on desktop Linux, OSX and Windows would
default to hybrid suspend. Turn that off for now (revert to preemptive suspend
by default).
Continue hybrid suspend work on 2018-10 and master.
[2018-08] [interp] enable more BCL tests
Backport of #11381.
/cc @lewurm
Description:
probably won't pass on all configurations, I wanna find out on which.
Mono doesn't use libxml2 and neither do the LLVM bits used by Mono when building
the AOT cross-compilers. Presence of `-lxml2` on the command line breaks 32-bit
builds on 64-bit Linux hosts:
libtool: link: cc -shared -fPIC -DPIC -Wl,--whole-archive ./.libs/libmini.a ./.libs/libmono-ee-interp.a ./.libs/libmono-dbg.a ../../mono/metadata/.libs/libmonoruntimesgen.a ../../mono/sgen/.libs/libmonosgen.a ../../mono/utils/.libs/libmonoutils.a ../../mono/eglib/.libs/libeglib.a -Wl,--no-whole-archive -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMBitReader -lLLVMCore -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -L$HOME/xamarin-android/external/mono/sdks/out/llvm-llvm32/lib -lz -ltinfo -lxml2 -lstdc++ -lm -lrt -ldl -lpthread -O0 -ggdb3 -m32 -g -mno-tls-direct-seg-refs -Wl,-export-dynamic -Wl,-soname -Wl,libmonosgen-2.0.so.1 -o .libs/libmonosgen-2.0.so.1.0.0
mv -f .deps/libmini_la-aot-runtime-wasm.Tpo .deps/libmini_la-aot-runtime-wasm.Plo
/usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libxml2.so when searching for -lxml2
/usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libxml2.a when searching for -lxml2
/usr/bin/ld: cannot find -lxml2
collect2: error: ld returned 1 exit status
make[7]: *** [Makefile:1495: libmonosgen-2.0.la] Error 1
Installation of `libxml2-dev` (on Ubuntu/Debian + friends) for the x86
architecture (or x86_64 architecture on 32-bit host) would be possible but then
it would break the 64-bit build because the 64-bit version of the library would
be absent. It would be a problem if Mono+LLVM actually used xml2, but since it
doesn't we can just disable the feature completely.
(cherry picked from commit 0d68ffd1ce)
[2018-08] [tests] lower recursion depth of PerformNoPinAction
Backport of #11065.
/cc @lewurm
Description:
so it doesn't overflow the stack when running on macOS with `clang -O0`.
* [threads] Make mono_add_internal_call external only
The runtime should use mono_add_internal_call_internal or
mono_add_internal_call_with_flags.
The thread suspend mechanism needs to know if an icall is added via the legacy
mono_add_internal_call API (which is not coop-aware, and so the registered
icalls must run in GC Safe mode), or if it is added by the runtime or by a
cooperative client (currently either the profiler or the System.Native PAL)
which knows to add GC transitions and safepoints and not to block indefinitely.
* [marshal] Factor out GCSafeTransitionBuilder
Make a local builder for creating the GC Safe transition calls for a method wraper.
* [sgen] Add coop GC transitions in mono_gc_toggleref_add
Also mark it external only. It's not used inside the runtime.
* [marshal] Emit GC Safe transitions around foreign icalls.
A foreign icall is added using mono_add_internal_call and that is not coop GC
aware. (The runtime uses mono_add_internal_call_with_flags or
mono_add_internal_call_internal)
Under hybrid suspend, foreign icalls will run in GC Safe mode and transition to
GC Unsafe only when the call back into the runtime or invoke managed code.
Fixes https://github.com/mono/mono/issues/11138
* [runtime] Add new API call mono_dangerous_add_raw_internal_call
Under hybrid suspend, this adds an icall that is assumed to run in GC Unsafe
mode. As such it has additional requirements for correct operation: it must
not run loops without periodically polling the runtime, and it must not perform
blocking operations such as blocking I/O or taking locks without manually
switching to GC Safe mode.
[2018-08] [aot] Reenable recursion checking when initing shared got entries
Backport of #11258.
/cc @BrzVlad
Description:
Broken in https://github.com/mono/mono/pull/11190
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
* [arm64] Remove the limitation on the number of nullable arguments for dyncalls.
Fixes https://github.com/xamarin/xamarin-macios/issues/4984.
* [amd64] Remove the limitation on the number of nullable arguments for dyncalls.
* [jit] Add tests for dyncalls with lots of nullable arguments.
* fix merge conflict
* [llvm] Make sure OP_LSHL_IMM is transformed to a 64 bit shift even if the input is 32 bit, which can happen because the JIT IR is not regular. Fixes bug-17590.exe on arm64+fullaot+llvm.
* [llvm] Fix a crash when trying to pass a vtype of size 0 on arm64. Fixes generic-constrained.2.exe.
Flags to DrawText never reached MeasureText therefore strings with '&' were measured incorrect when drawn with flag TextFormatFlags.NoPrefix.
Fixes#6352
They require the additional files:
- mono/eglib/eglib-config.h: that's to build `libmonodroid`
- mcs/build/Consts.cs: that's to build `Mono.Posix` and `Mono.Data.Sqlite`
* [interp] Avoid unnecessary allocations when pinvoke void functions
* [amd64] Refactor interp pinvoke context computation
* [amd64] Interp trampoline for interpreter entry
* [aot] Don't include interp_in_wrapper when we use entry trampoline
These wrappers take a lot of space in the image.
* [runtime] Add icall to format crashes to disk in uniform way
* [crash] More gentle error handling around stack walking
* [crash] Make self-summarizier not need managed thread state
* [crash] Remove reliance on sgen/gchandles
* [crash] Refactor dumper mechanism
* [crash] Make threads not die during thread dump
* [crash] Add support for registering/dumping native libraries
* [crash] Add silent option to dumper
* [crash] Print error when unable to dump thread
* [crash] Add icalls to directly trigger telemetry-style walks
* [crash] Increase maximum frame depth
* [crash] Add managed method names when privacy disabled
* [crash] Add tests for concurrent dumping
* [crash] Add support for concurrent global dump requests
* [crash] Switch from CAS+Sleep to semaphore for dumper
Locks and GC-Safe regions are both unsafe in a signal handler
context. CAS primitives and the sleep syscall was used before.
Semaphores appear to be safe, enabling the following cleanup.
* [runtime] Make find_method use stack memory for dumper-facing entry point
* [crash] Enable install_state_summarizer on all platforms
* [crash] Dump managed stacks from controlling thread
* [crash] Don't use allocating method name formatter
* [crash] Copy safer strings with crash privacy disabled
* [crash] Fix string races with DumpTotal icall
* [crash] Fix space in crash file name
* [crash] Fix lifetime and size checks on mono-state memory
* [crash] Fix icall declarations for new style
* [runtime] Make sequence point lookup lockless
* [crash] Fix managed exception dumper icall format
* [crash] Print half-completed dump when space runs out
* [crash] Bump static memory for crash string to 500k
Note:
- This memory isn't accessed until we're crashing
- On any modern machine, unused static memory is lazily allocated. When
we access it, the fault will trigger the MMU to map memory for us.
- This is guaranteed to be reentrant and safe. mmap (the alternative) is
surely fairly stable on many systems, but this is not guaranteed. The
documentation pages for mmap specifically claim it as not safe in these async
contexts.
* [crash] Make whitelist lookup lockless
* [crash] Split unmanaged ptrs into method ip + offset
* [System]: Reenable native Apple Certificate support on WatchOS. (#11173)
During the code refactoring this summer, support for native apple certificates
has been dropped on watchOS.
* add `apple-certificates.sources` and move all certificate-related code from
`appletls.sources`.
* move non-network related code from `MacProxy` into a new `AppleTypes`.
* add the `MONO_FEATURE_APPLE_X509` magic to `System/Makefile` where it was missing from.
(cherry picked from commit b61a3ffb02)
* [csproj] Update project files
* [sdks] Remove redundant --enable-cxx
* [sdks] Shorten `-m32`/`-m64` parameters detection for runtimes
* [sdks] Build BCL profiles without configure flags
* [sdks] Add BclTemplate to build BCL profiles
This is to avoid building all the profiles in a single `sdks/out/bcl` directory which will be shared between android, ios and wasm.
* [sdks] Add download/pack capabilities for LLVM
* [sdks] Create archive targets for pre-building on CI
* [sdks] Use archive-{android,ios,wasm} on CI
* [sdks] Run archive-llvm-llvm{,win}{32,64} on CI
* [sdks] Ensure LLVM is cloned before trying to download
* [sdks] Simplify call to BclTemplate
* [mcs] Make directory dependency as order-only
We would otherwise always rebuild the target since the directory is updated whenever something is added to the directory. This is a common Makefile technique.
* [sdks] Build tests for BCL targets
* [jit] Fix a constant propagation problem on 32 bit with iconst+conv.i8, we would be setting inst_imm even for long opcodes.
* [llvm] Fix the special casing of OP_LCOMPARE_IMM, its not needed.
* Use TARGET_SIZEOF_VOID_P.
`HttpWebRequest.RunWithTimeoutWorker()` needs to observe the worker task's exception
on timeout to prevent it from being thrown via the `TaskScheduler.UnobservedTaskException`
event. Fixes#10488.
* [interp] fix op_implicit/op_explicit conversions for native types
Conversion operators have been mostly broken in the interpreter, for
example conversion from nint to float didn't work. While some
conversions are easy to intrinsify (like nint -> I8), there are harder
cases like nint -> decimal. The JIT actually falls back to the managed
implementation, so let's do this in the interpreter as well.
However the managed implementation expects a pointer to the value type,
i.e. the container type System.nfloat instead of float/double primitives.
In such cases the value is going to be stored in a local and then the
address of its storage is pushed on the stack before invoking the
managed implementation of the conversion.
Added test cases for scenarios observed in Xamarin.iOS.
* [interp] support ntype.CompareTo ()
* [interp] support ntype.Equals ()
* [interp] support nfloat.*Infinity
* [interp] support ntype.ToString ()
* [interp] introduce float R4 stack type
Before we only did this for PR builds but it turns out it's needed for
normal master/branch builds too.
In situations where there are two Mac builds at roughly the same time
the downstream Windows builds would otherwise both refer to the same/latest
commit since they just checkout the latest ref on the branch.
Now we reset the Windows build to the sha1 which is passed to it.
(cherry picked from commit b13145bb12)
Right now if anything goes wrong in the security stack you get a message like this:
```
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
```
This is confusing since SSPI is a Windows-only thing and is not used on Mono.
We inherited this error message string from referencesource.
Change the string to something more generic which also aligns us with CoreFX:
https://github.com/dotnet/corefx/blob/a10890f4ffe0fadf090c922578ba0e606ebdd16c/src/System.Net.Security/src/Resources/Strings.resx#L210-L212
(cherry picked from commit 42e347ee47)
xamarin-macios currently builds zlib-helper.o using mono's source code.
It would be preferrable to use a version built by mono, and since
zlib-helper.o is a byproduct of building libMonoPosixHelper, we need to build
libMonoPosixHelper (we already build it for simulator architectures), even
though we don't need/use libMonoPosixHelper itself.
So they are next to their NUnit counterparts.
Move xunit remote executor app to tests dir too and compile it when
compiling the test assembly instead of doing it before running the test.
We need to use an absolute path for the xunit remote executor and it
needs to be in Windows format on Windows since we pass it as an env var
to the xunit tests.
[2018-08] [runtime] Disable stack guard for main thread on osx
Backport of #10899.
/cc @luhenry @BrzVlad
Description of #10899:
On macOS Mojave, it seems that changing the mapping of stack pages for main thread can lead to corruption bugs.
https://github.com/mono/mono/issues/10802
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
If another thread locked at least one of the other handles, and there is a
cooperative GC STW happening, this function will loop forever.
Add GC Safe transitions around the call to sleep to give this thread a chance
to suspend.
Fixes https://github.com/mono/mono/issues/10863
The `X509Certificate(byte[])` constructor should call `CertificateProvider.Import(byte[])`
to use the native apple implementation if possible.
(cherry picked from commit 8b3ca0dbbe0af1436d33d74bb2ca00b3cbfc28b8)
It turns out some Debian versions identify themselves as `Debian GNU/Linux`, add
this string to the list of distributions which have mingw. This *should* fix the
PR builds (fingers crossed)
Debian and its derivatives include full mingw chain (both 64 and 32-bit) and so
they don't need to build the toolchain at all. However, the current detection of
Linux flavor takes into account only Ubuntu and all other distros are configured
to *require* MXE but they do *not* build them.
This commit fixes detection of distros which provide the mingw toolchain in a
way that makes it easy to extend the set of compatible distributions as needed.
In order to mark a distribution as one with MinGW one has to add the `:Distro:`
string to the `LINUX_WITH_MINGW` make variable in `mxe.mk`. The `Distro` string
is whatever the `determine-linux-flavor.sh` script returns.
When building 32-bit binaries on a 64-bit system and vice versa, it is necessary
to pass the `-m32` or `-m64` compiler flags, respectively. Failure to do so will
result not only in linking problems (different binary architecture) but also in
problems when using some system headers which are sensitive to the architecture.
For instance, `ucontext.h` contains register set definitions for different
architectures and building Mono runtime with a 64-bit compiler targetting 32-bit
architecture results in this error:
.../mono/mono/utils/mono-sigcontext.h:173:74: error: ‘REG_EIP’ undeclared (first use in this function); did you mean ‘REG_RIP’?
#define UCONTEXT_REG_EIP(ctx) (((ucontext_t*)(ctx))->uc_mcontext.gregs [REG_EIP])
The issue here is that the compiler defines the `__x86_64__` macro which causes
`ucontext.h` to choose declarations for amd64 (which does not use `REG_EIP` but
has `REG_RIP` instead) while Mono runtime assumes its building for a 32-bit host
and uses 32-bit register names. Passing `-m32` to the compiler will fix the
issue as now the compiler will define the `__i386__` macro and all parties will
agree where they are and what they want.
This commit uses a rather baroque statement when constructing compiler flags to
build the Mono runtime to check whether the host triple targets 32 or 64-bit
host and adds the appropriate flag.
mono_aot_get_unbox_trampoline: amd64 hardcode unbox trampoline size to 5 at compile-time.
This fixes: MonoTests.System.Linq.ParallelEnumerableTests.TestGroupBy frequently crashing on FullAOT/LLVM/Linux/amd64 CI.
https://github.com/mono/mono/issues/10441https://github.com/mono/mono/issues/9554https://github.com/mono/mono/issues/9046
The actual size varies between 6 and 9 but we can just pretend the last instruction has size 1 (it is 2 or 5; calling it 5 can be fatal, calling it 2 is no better than 1).
Otherwise, a trampoline can overlap the next function, jit info lookup of the next function can fail, then gsharedvt wrapper omited, and crash.
The afflicted function in this case was
System_Linq_Parallel_WrapperEqualityComparer_1_T_GSHAREDVT_GetHashCode_System_Linq_Parallel_Wrapper_1_T_GSHAREDVT.
You can see it with debugprint: https://github.com/mono/mono/pull/10582.
There are many other possible fixes.
Such as runtime disassembly, hardcode in runtime instead of compiler, produce static trampinfo, omit trampinfo entirely, compute minimum size.
See https://github.com/mono/mono/commit/d589f3b64fca77c5c8bb5a8582eea5c0670312d7.
We want to attach a thread before any TLS operation is done by the
interpreter. In the JIT this happens inside the native-to-managed
wrapper via `mono_threads_attach_coop ()`. Unfortunately, this is too
late for the interpreter as it already requires a working TLS before
even calling into the wrapper.
Also, fixup naming of locals in the native-to-managed wrapper.
* [metadata] Hack - get mono_use_interpreter from metadata
Don't use this, it's a mistake. The granularity is all wrong, for one: in the
future the interpreter may not be an all-or-nothing deal.
* [marshal] Don't emit CEE_MONO_JIT_ATTACH except for interpreter
Always use the mono_threads_coop_attach/mono_threads_coop_detach pair.
* [aot] Drop MONO_PATCH_INFO_JIT_THREAD_ATTACH
We always emit the mono_threads_attach_coop icalls now.
Bump MONO_AOT_FILE_VERSION
* [mini] mark mono_jit_thread_attach external only
It's used by Xamarin.Mac and possibly other embedders.
Interpreter still used mono_jit_thread_attach_interp until it's fixed to work
with coop.
There shouldn't be any other callers.
* [interp] Add a thread attach EE callback
Use it to set the interpreter thread context on the JIT tls value
* [marshal] Always emit mono_threads_attach_coop instead of CEE_MONO_JIT_ATTACH
Use the coop icall for the interpreter, too.
* [interp] Drop MINT_MONO_JIT_ATTACH and detach opcodes
* [metadata] Drop CEE_MONO_JIT_ATTACH and detach opcodes
* Revert "[metadata] Hack - get mono_use_interpreter from metadata"
This reverts commit c3d2f488b824dcdfa261f8292013c805740f353f.
* [mini] Drop unused mini_jit_thread_attach_interp
It was a temporary workaround for the interpreter; no longer needed.
* [coop] Switch to GC Safe in mono_jit_thread_attach.
It's an external-only function that's not called by the runtime on any
threads. So the only calls are explicit uses by Mono API clients. Leave the
thread in GC Safe mode and leave it to the n2m invoke wrappers or MONO_API
entry points to switch to GC Unsafe.
* [interp] Add MINT_MONO_THREADS_ATTACH_COOP opcode
Because this icall can be called from detached threads, don't go through
do_icall () which needs initialized tls keys
* fixup for c++ build
* fixup indentation and comments
* fixup C++ a bit more
* [mini] Bump MONO_EE_API_VERSION
* Revert "[interp] Add a thread attach EE callback"
This reverts commit 8a343d6ce15f0475a62936f28264f638ebf0344f.
* [interp] sync jittls context before entering EH machinery
* [interp] get context refactor (#10561)
* [interp] Add declaration for set_context (#10575)
* Fix Mono Windows cross compiler using mono LLVM release_60 branch.
For scenario where llvm-config.exe can't be run (none WSL/CygWin build target)
Build will fallback to hard coded libraries not working with mono LLVM release_60
branch.
Added an exact mirror of what llvm-config.exe is returning for different components
on mono LLVM master and mono LLVM release_60 branch.
The selected codegen libraries (passed in as extra_libs) was not handled in any cases
so added them into the build script as well.
* Split parameter between lvm codegen libs and llvm extra libs.
Need to explicit pass -lz in case where configure have detected zlib availability
and llvm-config.exe can't be executed. In cases where zlib is not available, default
in mingw installs, this check make sure link step won't fail due to missing library.
* Fix inversion of aarch64codegen libs.
* Enable package-android-cross-{arm,arm64,x86,x86_64}-win builds on CI.
We can't decrement the size on removal as we use tombstones.
When using tombstones, the invariant to keep is on the number of null entries as those
are used as search stop-marks.
Without this change, it's possible that a long series of add/remove operations could leave the array with no null entries,
just tombstones and filled entries. In that case, search would degenerate into a linear search that would look into all array elements.
This change fixes worst-case perf of not-found searches to be bound to the load factor.
Additionally, clear the array to null entries, otherwise we fall into the above trap even more easily.
0dbcdd5 added CFLAGS etc in front of the command but that's not something the TESTCMD can execute.
We shouldn't pass CFLAGS etc at all for this since that is handled by bockbuild.
* Ensure the finalization of SafeHandle and GCHandle in DeflateStreamNative is performed in the correct order.
* Do not access BaseStream from finalizer.
* [exceptions] ftnptr_eh_callback test that stresses n2m transitions
on Xamarin.iOS it happens that the ftnptr_eh_callback calls into managed
code, that is yet another native-to-managed transition.
* [coop] Use unbalanced transition to GC Unsafe in mono_raise_exception
Since `mono_raise_exception_deprecated` never returns, the transition won't be
balanced. This fixes a cookie-counting problem (assert on mono shutdown) in
coop and hybrid checked builds using the new test case:
MONO_THREADS_SUSPEND=coop MONO_CHECK_MODE=thread,gc mono mono/tests/install_eh_callback.exe
Backport of #9843.
* Truncate MachineName (and UserDomainName) at first period
Unix machines often put their FQDN as the host name, and domain
name is used for things such as yp. As such, truncate the FQDN to
just the hostname, as other Unix utilities seem to do.
Matches the .NET Core and .NET Framework behaviour, and MS seems to
require this for a future project. Fixes#9839.
* Add a unit test for bug
* Formatting tweaks
* [sdks] Add CrossRuntimeTemplate
* [sdks] Add cross-{arm,arm64,x86,x86_64}{,-win} runtimes for XA
* [sdks] Factor various variables for RuntimeTemplate
* [sdks] Use CrossRuntimeTemplate for iOS
* [sdks] Unify provisioning of LLVM for both Android and iOS.
A global CFLAGS would be set in `run-jenkins.sh`, and that would screw up with LLVM. The global CFLAGS makes sense as long as we are only building Mono, but that's not the case with LLVM anymore.
* [sdks] Use MonoRuntime template for desktop-x86_64
* [sdks] Remove android-armeabi, it's been removed from XA already and the target has been deprecated in the NDK
The llvm/llvm_config.mk makefile fragment just includes make (not automake)
variable definitions. So there is no need to regenerate the mini/Makefile.am
file if the fragment updates - the include will pick it up.
In order to allow managed crash reporting using the existing telemetry
format, I have added initial support for converting a MonoException into
a json blob for telemetry.
When we execute the certificate verification callback we just don't test for success and set status11. Now we get the error (if any) and map the error code to those defined by ChainStatusFlags and add it to a list of X509ChainStatus objects for that chain. Now when get_ChainStatus is called we convert that list to an array of X509ChainStatus and return it.
* [threads] Add enums for suspend phases and for begin-suspend results
* [coop] Two phase suspend
If we're using hybrid suspend, initially request suspension for threads, but
don't preemptively suspend blocking threads. In the mop-up phase, peek at the
state and if they are still blocking_suspend_requested, preemptively suspend
them.
In full coop and in full preemptive, just use a single phase.
This should fix hybrid suspension deadlocks due to preemptively-suspending
threads taking locks that prevent cooperatively-suspending threads from
reaching a safepoint by leaving all threads running until all
cooperatively-suspending threads come to a safepoint. At that point we can
preemptively suspend the remaining threads.
Fixes https://github.com/mono/mono/issues/9921
* [tests] renable flaky hybrid suspend tests on PRs
* [coop] Thread state machine helper peek_blocking_suspend_requested
* [coop] Check async suspend status in ReqSuspendInitSuspendBlocking case
So the issue is that calling begin_suspend_for_blocking_thread under hybrid
suspend will perform a preemptive suspension of the victim thread.
The preemptive suspend doesn't always succeed.
In particular, thread_state_init_from_sigctx or thread_state_init_from_handle
could return FALSE and into->suspend_can_continue will be set to FALSE.
When the thread_state_init_* functions return FALSE, they also set the
MonoThreadUnwindState:valid to FALSE.
So it is important to check the value of MonoThreadInfo:suspend_can_continue
before proceeding with a suspension.
As a result in suspend_sync in the ReqSuspendInitSuspendRunning case we call
check_async_suspend which checks (for preemptively suspended threads) whethere
suspend_can_continue is true. If it isn't, we resume the victim thread and
loop once more in suspend_sync_nolock and try again.
We didn't have a check_async_suspend call in the ReqSuspendInitSuspendBlocking
case, so a thread could be considered suspended even if we couldn't capture a
valid MonoThreadUnwindState for it.
That breaks the assertion the call to mono_thread_info_get_last_managed in
async_suspend_critical which needs a valid MonoThreadUnwindState.
Fixes https://github.com/mono/mono/issues/10031
* fixup - remove coop assertion in check_async_suspend
result for coop could be iether BeginSuspendOkCooperative or
BeginSuspendOkNoWait depending on whether the thread was running (Ok) or
blocking (OkNoWait)
* [coop] suspend_can_continue if async raced with self-suspend (osx + win32)
Companion to 006d6ce097 from earlier this year
which applied this fix for POSIX signal based suspend. This is the
corresponding OSX and Win32 fix.
In hybrid suspend, in the case where a suspend initiator needs to preemptively
suspend a thread, but the thread self-suspended while after the suspend
initiator put it into the blocking_suspend_requested state, we want to allow
the suspend to continue. We do have to call the syscall to undo the native
async suspend, but after that, we can leave the victim to continue waiting for
a resume.
This should not have any effect on full cooperative or full preemptive suspend
- in the full coop case we never call async suspend, in the full preemptive
case the thread never self-suspends in blocking.
This https://github.com/mono/mono/issues/10143 and fix is from there.
I considered nulling the array elements as I go, and then the tail,
to help the GC, but does it help actually? I suspect not.
macOS 10.14 introduces a Network framework, which means Xamarin.Mac now has a Network namespace.
This conflicts with 'using Network = System.Net':
> mono/mcs/class/System.Xml.Linq/Test/System.Xml.Schema/ExtensionsTest.cs(97,27): error CS0576: Namespace '<global namespace>' contains a definition conflicting with alias 'Network'
so just remove the using and use the full 'System.Net' namespace instead.
* [reflection] Assembly.LoadFile should throw ArgumentException for relative paths
Fixes https://github.com/mono/mono/issues/9943
* [mcs/tests] Use full path in the compiler-tester
Otherwise the call to LoadFile will fail
* [mono/tests] Use full path in LoadFile in test-multi-netmodule-4-exe
* [mono/tests] Use the test runner for a couple of tests
* [fullaot] Disable test-multi-netmodule-4-exe and custom-attr-errors.exe
* [tests] Use Path.GetFullPath and Path.Combine in AssemblyTest
Fix windows test failures
If an AOT library is loaded that was compiled with safepoints, but currently
we're running with preemptive suspend (e.g. MONO_THREADS_SUSPEND=preemptive is
set), don't assert. `mono_polling_required` will always be false, so the
safepoint will be ignored.
Fixes https://github.com/mono/mono/issues/9947
* [threads] switch to GC Safe around pthread calls in mono_thread_internal_set_priority
Observed pthread_getschedparam taking a lock on OSX
* [win32] Capture GetLastError before MONO_EXIT_GC_SAFE
The LoadFrom assembly loading context and the individual assembly loading
context (aka LoadFile aka "no context") behave differently when requesting a
referenced assembly - for LoadFile we expect the runtime to probe the appdomain
base, the GAC + MONO_PATH but not the basedir of the requesting assembly. For
LoadFrom the basedir of the requesting assembly _is_ included. So the tests
here try all the possible variations.
1. If an assembly filename is in the GAC, always open the assembly in default
context.
2. If an assembly filename is in the appdomain search path (base directory or
private bin path) always open the assembly in default context.
3. If an assembly filename is in the base dir of the requesting assembly and
the requesting assembly was in LoadFrom context, always open the assembly in
LoadFrom context.
Fixes https://github.com/mono/mono/issues/9753
and Fixes https://github.com/mono/mono/issues/9542
Add a hook that can be used to check whether a given path belongs to some
assembly load context.
For example if a given path is in the application base of the current
appdomain, an appdomain hook would set out_asmctx to MONO_ASMCTX_DEFAULT and
return TRUE.
If an assembly was loaded with LoadFile, it can reference assemblies from the
application base, the GAC or MONO_PATH.
Fixes https://github.com/mono/mono/issues/8575
if an assembly in LoadFrom context calls Assembly.Load, the given assembly name
should be tried in the basedir of the executing assembly, and the resulting
loaded assembly should be in LoadFrom context.
Fixes https://github.com/mono/mono/issues/9869.
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
* [coop] Turn hybrid suspend on by default on desktop platforms
Use hybrid suspend by default on x86 and amd64 OSX, Windows and Linux
Pass `--disable-hybrid-suspend` to configure to disable at build time, or set the environment variable `MONO_THREADS_SUSPEND=preemptive` when running Mono to use preemptive suspend (the previous default).
Pass `--disable-hybrid-suspend --enable-cooperative-suspend` to configure cooperative suspend at build time.
* [sdks] Disable coop and hybrid suspend for iOS and Android
Only the devices and simulator builds, not the host ones.
* [coop] Add mono_threads_suspend_override_policy ()
Do not use it. It provides a last resort method to ignore configure and environment settings and use a hardcoded threads suspend policy.
* [interp] Override suspend policy - warn and use preemptive
Until the interpreter supports safepoints and performs threads suspend states switches in its trampolines, always use preemptive suspend if the interpreter is used.
* [coop] Enter GC Safe around mono_threads_join_lock ()
Like other uses of mono_threads_join_lock, enter GC Safe before trying to take the lock.
* [interp] Use preemptive suspend with --full-aot-interp
Move safepoint suspend check to mono_runtime_set_execution_mode
* [test] fullaotmixed - force preemptive suspend[test] fullaotmixed - force preemptive suspend
Force preemptive suspend in fullaotmixedcheck. We must compile the AOT images without safepoints.
* [sdks] Run the offset-tool in 64 bit mode.
* [sgen] SgenDescriptor should use the target wordsize.
* [sdks] Fix 'Run the offset-tool in 64 bit mode.'
* Manually add a csproj file for cil-stringreplacer
Adjust trace level for genproj
* Manually add cil-stringreplacer to bcl.sln and set it as a dependency of corlib
* Fix typo
* Add post-build event to corlib that runs cil-stringreplacer on it in mscorlib mode
* [csproj] Update project files
Right now, the set of mono-api-tools only works as console apps. This PR will make them work as normal dlls so they can be consumed by other apps and libraries. They will still work as .exes, so nothing visibly changes for existing users.
The motivation for this is for some build process or third party tool to take an assembly and generate an .xml doc from it. Since the writer is in-memory, the tool can then parse the .xml and do whatever it needs to. The same goes for the diff tools - if the diff happens in-memory, then the tool can process it without having to save and then load from the file system.
## Libraries
- [x] mono-api-info -> `Mono.ApiTools.ApiInfo`
- [x] mono-api-diff -> `Mono.ApiTools.ApiDiff`
- [x] mono-api-html -> `Mono.ApiTools.ApiDiffFormatted`
## Major changes
- the existing `Driver` classes are now just thin wrappers to parse the args and pass it on to the generators
- the "options" are now abstracted into config classes to avoid methods with numerous parameters
- the APIs have new support for reading assembly and info `Stream`s instead of just file paths
- added compiler flags to control the namespace and type visibility to make the library APIs nicer
- added compiler flag to remove the `Driver` class and `Mono.Options` dependencies
- the mono-api-info tool has some more options to allow it to run on libraries, and ignore dependencies that it cannot find - this makes it easier to use so that the entire dependency chain does not have to be loaded just to output the current assembly API
## Compiler defines
- `EXCLUDE_DRIVER` will remove the console entry point and remove the dependency of Mono.Options
- `USE_MONO_API_TOOLS_NAMESPACE` will switch the namespace to `Mono.ApiTools` so that libraries can have a nice API
- `USE_INTERNAL_VISIBILITY` will make any non-essential, `public` APIs turn `internal`
## Review points
- are there any variables/fields that are still static?
- are there any exposed types/members that need to be hidden?
- are the new names for the APIs and config classes OK?
- are my whitespaces according to the guidelines?
- did I go horribly wrong?
- Public headers should be wrapped in it (all but one are).
- MONO_API should include it.
But it is not needed "everywhere".
Placement of "extern" therefore tweaked.
- Remove from functions where it is meaningless and gets in the way.
- Mark data with MONO_API_DATA instead of MONO_API.
[not yet]Sprinkle in a few G_BEGIN_DECLS for monodis and where THREAD_INFO_TYPE makes prototypes incorrect.
include/add prototypes for all icalls. [undone per PR, possibly a misunderstanding]
A problem would be, if embedders manage to use functions that are neither MONO_API nor
in public headers.
Some parts of the larger solution are omited here in the hopes of finding better options.
In particular, compiling all of monodis as C++ so we don't need some extern "C" for it,
and THREAD_INFO_TYPE at least incoming being gpointer so declaration and definition match.
This is based on https://github.com/mono/mono/pull/9812/commits/40315ee475bdf89c40189707a8cca86b3f6d91c8 and is an alternative to https://github.com/mono/mono/pull/9782.
We also can/should prune the preexisting G_BEGIN_DECLS / G_END_DECLS uses.
* [runtime] Add a MONO_ABI_SIZEOF(MonoObject) define, use it instead of sizeof(MonoObject). Add a few helper functions to avoid sizeof(MonoObject) in many places.
* [aot] Add a 'deterministic' testing argument to have the compiler produce deterministic output by turning off the generation of the AOT id.
* [jit] Use target sizes in many places.
* [runtime] Avoid corlib checks in cross compile mode, its not possible to read the field value from corlib.
* [runtime] Fix the arm build.
Right now we have a race condition where parallel make on CI will barge ahead and start building things before monolite+basic-profile-check have completed, which results in some weird behavior - gensources silently fails to run, etc. Testing some possible fixes for this.
* [offsets-tool] Add i386-apple-darwin11.2.0 as an abi target, used only for development.
* [offset-tool] Emit sizes for structures.
* [runtime] Cleanup the cross targets in configure.ac and add more target defines.
Redefine SIZEOF_REGISTER to refer to the target size.
Add TARGET_SIZEOF_VOID_P define and target_mgreg_t type.
* [runtime] Add missing object offsets for x86.
* [runtime] Avoid trying to execute assembly load hooks in aot mode.
* [runtime] Fix the windows build.
* [offset-tool] Fix indentation.
* [runtime] Add a mono_gc_get_target_card_table () function which is the same as mono_gc_get_card_table () but returns information for the target. Use it in the JIT and in places where we generate IL code.
* [aot] Save the card table parameters in the AOT image and check them at load time.
* Fix llvm support.
Using solution dependencies in ```bcl.sln``` seems flaky and seems like it might not establish the full ordering we need to ensure that ```Consts.cs``` exists before we build things that require it. Let's try using project references (where ```corlib.dll``` 'depends' on ```genconsts.exe```) instead. This should also insert the dependency for any project that includes Consts.cs instead of just corlib.
This PR also makes update-solution-files actually fail if ```genconsts.exe``` fails to build because it was driving me mad.
Part of #6886
* Add check for a generic method when invoking in debugger
VSTU will try to invoke a generic (uninflated) method in certain
situations, which was causing the Mono debugger to assert and crash
Unity. Added a check for this situation before the invoke so the
debugger can return an error to the client. Previously, this case
wasn't caught until the method was in JIT compilation, which will
assert when this situation is detected.
* Fixing typo
The version imported and attempted to patch didn't work fully. This
preserves the .NET representation while converting if directives to
runtime checks, which CoreCLR inlines.
Testing was done poking some values in the REPL and running some
test suites (System.Json, System.Decimal) that broke as a result.
mcs should also properly handle constants; again, the "bits" inside
are consistent with amd64, and just converted as needed.
Unless I fucked up, should fix#9772.
Carbon implementation now throws exception instead of crashing in the mismatched p/invoke calls.
Cocoa implementation now support 64-bit mode and uses the handle of `NSView` that is passed in instead of relying on `NSGraphicsContext.CurrentContext`. It also fixes handling of flipped views.
Requires up to date libgdiplus.
* [appdomain] MONO_API mono_domain_set needs GC Unsafe
* [runtime] MONO_API mono_domain_set_config needs GC Unsafe
* [runtime] MONO_API mono_dllmap_insert sometimes needs GC Unsafe
When adding a global mapping, don't do the state transition because the thread may not be attached yet. When adding a per-image mapping, do the transition.
* [runtime] MONO_API mono_g_hash_table_insert needs GC Unsafe
* [runtime] MONO_API mono_g_hash_table_new_type needs GC Unsafe
* [runtime] MONO_API mono_gc_reference_queue_new needs GC Unsafe
* [runtime] mono_thread_detach_if_exiting thread state change
1. Mark it external only, it's unused in the runtime.
2. If detaching, switch to GC Unsafe before detaching
* [metadata] Make mono_class_get_fields external only
- Runtime should use mono_class_get_fields_internal
- MONO_API mono_class_get_fields needs GC Unsafe
* [runtime] MONO_API mono_gc_reference_queue_add needs GC Unsafe
* [metadata] Make mono_class_get_name and namespace external only
- Runtime should use m_class_get_name and m_class_get_name_space
- MONO_API mono_class_get_name and mono_class_get_namespace need GC Unsafe
* [metadata] MONO_API mono_class_get_parent needs GC Unsafe
- mark it external only; runtime should use m_class_get_parent
* [metadata] Make mono_class_is_delegate external only
- runtime should use m_class_is_delegate
- MONO_API mono_class_is_delegate needs GC Unsafe
* [metadata] Make mono_class_is_valuetype external only
- runtime should use m_class_is_valuetype
- MONO_API mono_class_is_valuetype needs GC Unsafe
* [metadata] Make mono_class_is_enum external only
- runtime should use m_class_is_enum_type
- MONO_API mono_class_is_enum needs GC Unsafe
* [metadata] Mark mono_class_get_element_class external only
- runtime should use m_class_get_element_class
- MONO_API mono_class_get_element_class needs GC Unsafe
* Add support for Blazor custom interop support.
This adds additional ability to perform unmarshalled calls for advanced but high-performance shared-memory interop.
* Note: * This is only safe in Blazor's case because of its exact usage patterns.
* Add dotnet_support.js backing file.
Instead of accessing them using the inst_ls_word/inst_ms_word macros which use the union in MonoInst.data, access them normally using
MonoInst.Data.i8const.
It wasn't possible to debug any F# app that contained PInvokes.
The F# compiler omitted MethodDebuginformation for PInvokes which in
turn caused Mono to fail on an assertion.
```
* Assertion at metadata.c:1117, condition `idx < t->rows' not met
```
https://github.com/mono/mono/commit/0f0e31842f1f394c4e95d3726c20af547e706278 added a new pipes_pns.sources file
but we forgot to add it to EXTRA_DISTFILES so it gets included in the tarball.
Given that this happens almost all the time when we add a new .sources file
we should just always include all *.sources
* [cxx] Build/CI support for C++.
configure -enable-cplusplus and CI_TAGS=cplusplus
https://jenkins.mono-project.com//job/test-mono-mainline-linux-cplusplus/
* [cxx] Rename CPLUSPLUS/cplusplus to CXX/cxx.
* [cxx] Add C++ build support to mini.
* [cxx] Jenkins/CI support for cxx not just cplusplus.
* [interp] Fix some conv ovf opcodes
* [interp] Correctly hande a few conv ovf cases
For conversion from unsigned value to i1 and i2.
* [interp] Fix multiplication overflow case
* [interp] Remove some constants from code
* Use System.Security.Policy.Publisher from corefx (they essentially stubbed it out).
Fixes the XUnit 2.4 problem.
* Use `PublisherMembershipCondition` from CoreFX as well (stubbed out in there) and remove the tests.
* Bump API snapshot submodule
* [Coop] System.IO mark all remaining NOHANDLES().
* Functions take no parameters, not any parameters -- () vs. (void).
* [Coop] Mark System.Threading.Interlocked as mostly NOHANDLES().
* [Coop] ves_icall_System_Threading_Thread_MemoryBarrier is NOHANDLES().
* [Coop] System.Threading volatile reads/writes are NOHANDLES().
* Make YieldInternal NOHANDLES and remove two potential layers.
* [Coop] Make ves_icall_System_Threading_Thread_current_lcid NOHANDLES().
* [Coop] mono_100ns_ticks nohandles.
* [Coop] More nohandles volatile reads/writes.
* [Coop] NOHANDLES() System.Security.SecurityManager.
Return objects from load_cattr_value () using an additional out parameter. Add a variant of
mono_reflection_create_custom_attr_data_args () which can only decode a subset of cattr types, but
it doesn't allocate any objects. Use this in marshal.c/marshal-ilgen.c to avoid allocating objects
while decoding FixedBufferAttribute and UnmanagedFunctionPointerAttribute.
* [interp] Refactor MINT_ICALL opcodes
Remove integer versions, since they add unnecessary complexity, and with some signatures we were already implicitly casting int to gpointer.
* [interp] Use fast versions for icalls
* [interp] Include more fast pinvoke/icall signatures
These cover almost all icalls
* [interp] Fix fast icall return value
The returned value needs to be converted to the stackval representation
* [corlib]: Cleanup `X509Certificate` implementation and sync it with CoreFX.
Completely refactored the `X509Certificate` implementation to make it based very closely on CoreFX.
We cannot fully switch to their implementation yet because we are missing things and they also have
a few issues in their implementation. However, this will make it a lot easier to do this at a later time.
The internal `X509CertificateImpl` has been cleaned up as well, using properties instead of methods in
most places and the `ToString()` implementation is now shared between all platforms.
* `X509Certificate.CreateFromSignedFile()`: in CoreFX, this is the same as `CreateFromCertFile()`.
Our old implementation was incomplete and did not verify the signature - we even had a test where
loading a certificate with an invalid signature was supposed to work; this test has been disabled
because we're now throwing an exception.
* `X509CertificateImpl.KeyAlgorithmParameters`: this is supposed to throw `CryptographicException`
on error, see `Certificate13` in `X509CertificateTest.cs`.
* We now match CoreFX's behavior regarding serialization - while the `X509Certificate` type is still
marked as `[Serializable]`, we now throw `PlatformNotSupportedException` when attempting to serialize
or deserialize it.
* Bump API snapshot submodule
* [csproj] Update project files
* Make `corlib` internals visible to `System.Security` on all profiles, not just monotouch watch.
* Moves all generated `SR` files and their generation logic from `System` and `System.Security` into `corlib`. Since `corlib` internals are visible to these assemblies, we don't need to duplicate the resource strings.
* Cleanup the `RESX_RESOURCE_STRING` is `corlib` and sort them alphabetically.
* Add the internal `Internal.Cryptography.Helpers` helper class from CoreFX to `corlib` (currently unused, but "Blue" will need it).
* Fix up behaviour of verbose method with arguments
Given a method spec which looks like:
`MainClass:Do(int,int,int)`
Via f024e820a025aacc1cf80eb82a8aee4552c3ac11 the code would match 3 methods:
`MainClass:Do(int`, `int`, `int)`, crashing the runtime.
Accept the multiple method matches via `;`, rather than `,`.
Now, the behaviour is changed so:
`MainClass:Do(int,int,int);MainClass:Do(int,int)`
Get parsed as:
`MainClass:Do(int,int,int)`
`MainClass:Do(int,int)`
* Add tests
* add test file and make tests actually output something
* Smarter unit test around disassembling methods
* Try single quoting environment variable value
* Try using non-throwing setter
Temporarily switch back to ref-source impl for `CharUnicodeInfo` which contains a few special BE patches while I am trying to upstream them.
Fixes#9684
Both `Certificate_11_Properties` and `Certificate_14_Properties` use an invalid
certificate that violates X.690 11.8.2 (UTCTime under a DER encoding).
This will no longer be supported once we switch to CoreFX's PAL code.
See also https://github.com/dotnet/corefx/pull/31293.
Instead of digging around in the environment for the coop and hybrid suspend
settings, define in terms of mono_threads_suspend_policy (or the
MONO_ENABLE_BLOCKING_TRANSITION env var as before).
Set to one of "preemptive", "coop" or "hybrid".
Unlike the old MONO_ENABLE_COOP, MONO_ENABLE_COOP_SUSPEND,
MONO_ENABLE_HYBRID_SUSPEND, the value of MONO_THREADS_SUSPEND will take
priority over the configure flags default.
For compatability the old environment variables will be respected if neither
MONO_THREADS_SUSPEND nor a configure flag was set. A warning will be emitted
if MONO_THREADS_SUSPEND isn't set. (Rationale: maybe the environment set both
because they're not sure which version of Mono they will be running - so don't
nag them).
Rework of #5448, contributes to #7143.
The anonymous pipes were tested on macOS, build was tested only on few configurations (hoping to solve some issue through Jenkins builds). Named pipe tests get stuck on macOS, but it may be unrelated Xunit issue and needs some proper debugging.
@Therzok, do you know of anything else that is missing for parallel MSBuild? Do you have some test environment for it by chance?
[MacOS] On Mac, use the copyfile API to copy files
Rebase of #9038 with added fix for the failing unit tests. Note that there's a race condition in the check between `source file == destination file` and the `copyfile` call, but a similar race condition already exists there in the current code except the outcome is different.
* `X509Helper`, `X509CertificateImpl`, `X509Certificate2Impl` and `ISystemCertificateProvider` now use `SafePasswordHandle` instead of strings.
* `X509Certificate`: actually implement `SecurePassword` based APIs using `SafePasswordHandle`.
* `mono_btls_pkcs12_import()`: when passed an empty password, try both the empty string and null.
After the dependency injector changes, the `X509Certificate` code in corlib now uses the implementation from `System.dll`, including the platform-specific switch via `MONO_TLS_PROVIDER` / `MonoTlsProviderFactory`.
This adds a new `X509Certificates2` category to the corlib tests and runs them with `MONO_TLS_PROVIDER=btls` on OS X (where the default is applets).
* [interp] fix stack type warning around ldlen instruction
removes this warning in e.g. mcs-tests:
> Object.CallSite.Target: Store local stack type mismatch 0 1
> Object.CallSite.Target: Store local stack type mismatch 0 1
> Object.CallSite.Target: Store local stack type mismatch 0 1
> Object.CallSite.Target: Store local stack type mismatch 0 1
> Object.CallSite.Target: Store local stack type mismatch 0 1
> Object.CallSite.Target: Store local stack type mismatch 0 1
* [interp] fix compiler warning around MINT_INTRINS_BYREFERENCE_CTOR
fixes those warnings:
> interp/interp.c:3924:13: warning: unused variable 'valuetype_this' [-Wunused-variable]
> stackval valuetype_this;
> ^
> interp/interp.c:3926:13: warning: unused variable 'retval' [-Wunused-variable]
> stackval retval;
> ^
> 2 warnings generated.
Adds 1192 xunit tests for System.XML from CoreFX
Also, moves common lines from `mobile_System.Xml.sources` and `System.Xml.sources` to `common.sources`
Part of #8132
* Use __declspec(thread) with Visual C++, and __thread with Win32/gcc,
instead of much slower TlsGetValue / TlsSetValue
* Avoid affecting Boehm.
* Replace HAVE_KW_THREAD and USE_KW_THREAD with MONO_KEYWORD_THREAD.
* Refactor the AppleTls Certificate code into a PAL-based approach.
* Added the internal `SafeHandle` based classes from CoreFX's PAL:
`SafeSecCertificateHandle`, `SafeSecIdentityHandle`, `SafeKeychainItemHandle`
and `SafeSecKeyRefHandle`.
* Added a new static internal class `MonoCertificatePal`:
- `Mono.AppleTls/MonoCertificatePal.cs` is shared between OS X and Mobile.
- `Mono.AppleTls/MonoCertificatePal.OSX.cs` contains the OS X specific parts.
- `Mono.AppleTls/MonoCertificatePal.Mobile.cs` contains the Mobile specific parts.
This split allows us to avoid all the `MONOTOUCH` conditionals in the code.
* In the `Mono.AppleTls` directory, all code from `Certificate.cs`, `Items.cs` and
`ImportExport.cs` have all been merged into the new `MonoCertificatePal` class.
* The entire AppleTls code is now using these safe handles instead of the old
Mono-specific types. This allows us to swap out the implementation of the certificate
code with CoreFX's PAL code.
* Cleaned up and removed some ancient dead code.
* `build/profiles/build.make`: Don't set `MONO_FEATURE_APPLETLS` here.
* Add `appletls.sources` and cleanup `MONO_FEATURE_APPLETLS` conditionals.
* [csproj] Update project files
These programs work if you just compile them with g++ and most of the automake/autoconf/libtool
flags, but not if you use all of them. Something to do with search paths perhaps.
* [linker] Add BCL size regression testing
We're now tracking the size of the BCL after it was processed by the linker
across various profiles we care about (currently net_4_x, monotouch and monodroid).
The way it works is that we keep a file with size measurements in the repo
and during CI we run the linker against the current state and compare.
Right now we're simply running the linker tests as they are a good approximation
of various usages (this PR additionally adds more comprehensive linker tests we had in XI).
If changes are found there's a report to check where the increase/decrease came from.
The changes can be applied via `@monojenkins commit sizediff` similar to
existing csproj and apidiff usages.
Fixes https://github.com/mono/mono/issues/6992
* Don't run new linker tests yet, just use for size checking
The mono_load_runtime_and_bcl allows upstream to remap requests and report download progress.
This cb semantics might change in the future when we implement more agressive loading.
The `mono_threads_attach_coop` / `mono_threads_detach_coop` icalls are called
from the wrapper around managed methods that will be called from native code
`emit_managed_wrapper_ilgen`.
If the managed method throws an exception, the mono_threads_detach_coop will
not be called, and the checked build
`mono_threads_enter_gc_unsafe_region_internal`/`mono_threads_exit_gc_unsafe_region_internal`
accounting will be mis-matched which will break later checked mode checks.
(The unbalanced thread state transition is not a problem - the next outermost
exit_gc_unsafe will operate just fine - it's just the checked mode check that's
wrong)
Once we lost the try/finally in b20d5ce1e5 these
should have become unbalanced.
* [Coop] Convert 3 more functions and strengthen some types to be not just gpointer.
i.e. to see in debugger and for static type checking
* ves_* probably shall not call ves_*.
non-ves_* (mono_*) really shall not call ves_*.
Only C#/icall-def.h shall call ves_*, and only ves_*.
Where there is a contradiction, rename ves_* to mono_*,
have ves_* and mono_* call mono_*, and possibly cleanup.
For example, removing MonoError* parameter, and moving asserts
around to preserve or improve semantics.
* Remove incorrect comment.
This adds a simple script for generating Consts.cs that can run on Windows, and makes it a dependency of corlib.csproj in the BCL solution file. From my initial testing this will autogenerate Consts.cs if it's missing and then corlib will successfully build.
This (mostly?) addresses issue https://github.com/mono/mono/issues/6886.
This currently has an awkward dependency on ```winsetup.bat``` and I modified it slightly to make it clearer whether the batch file succeeded.
Rather than always writing to the std out, provide an option to write to a file. This makes things easier for tooling by not requiring `RedirectStandardOutput = true`
* [interp] Check delegates for null this
* [interp] Resolve target method for delegates created with abstract methods
* [interp] Fix delegate invocation
Previously, when first calling the Invoke method on a delegate we would silently transform the delegate invoke wrapper of the delegate on which Invoke is called, after which all calls would go through this method. This is problematic because the same Invoke method can be called on a different delegate afterwards (the delegate_invoke wrapper needs to be tied to the delegate and it's not global for all invokes of same type),
We solve this by storing the invoke method in the delegate, similarly to the JIT.
* [interp] Remove unused argument
[llvm] Avoid using the preserveall calling convention in another plac…
…e on watchos.
Fixes https://github.com/mono/mono/issues/9318.
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
Generate similar array-coop*.cs files from pieces.
System.Array icall logging/testing tweaks.
Add const, but only in comment.
[Coop] Convert ves_icall_System_Array_CreateInstanceImpl.
PR feedback: Don't have icall call icall, refactor into common helper instead.
Add more disabled printing.
test_get_generic_value fails even CI JIT due to AOT runtime,
but does work with full JIT configuration.
Generic ref/out parameters are not supported by HANDLES(), so
NOHANDLES(SetGenericValueImpl)
NOHANDLES(GetGenericValueImpl)
Also add MONO_REQ_GC_UNSAFE_MODE because of unsafe gpointer value (which reinforces NOHANDLES?)
Remove extra evaluation of arg.
Enable 3 more tailcall tests on x86.
Enable more tailcall AOT tests.
Program to split fsharp-deeptail.il into separate tests.
Split up F# test into individual tests.
Simplify test IL and make it stack overflow upon failure.
Cleanup test code before splitting.
Program to split tailcall-interface-conservestack into separate tests.
Enable split up versions of tailcall-interface-conservestack and F#.
There is a problem in that we use gsharedvt unexpectedly, and gsharedvt
has almost-fundament contradiction with tailcall.
Need to figure out why it used, if that is correct.
Disable some FullAOT tests that are broken by gsharedvt.
Lower apparent memory use.
Move tests into directories and reset disabled lists.
Centralize tailcall debug helpers.
Add comments why test_virt.exe fails on 32bit x86.
Remove tail. prefixes that are besides the point, and add noise to the logging.
Print more in failure.
Better align the stack for x86, so tests pass.
There were already tailcalling and conserving stack, but still using a little stack.
Real world code will use zero or a little. The test self-check is very strict
and wants zero, and prints how much.
Rename duplicate tests, and sort them.
Otherwise we were missing split tests.
Add 96 ARM failures and 30 FullAOT failures, mostly all understood.
ARM cannot pass extra parameter in a tailcall (except bitcode that we do not fix).
FullAOT uses gsharedvt a lot even for statically discovered types created, due to
interface and virtual.
Convert to C++98 and fix warnings about /* in //
Split the tests up in the build instead of ad hoc by developer.
Disable tests on wasm and win32 (and cross) due to build problem.
* [Coop] Convert ves_icall_System_Runtime_InteropServices_Marshal_IsComObject.
* [Coop] cominterop and marshalling partial conversion and cleanup.
* Some refinement/correction/commenting to mono_string_to_utf8str_handle.
* Fix index out of bounds.
* Fix error handling.
* Remove some mono_error_set_pending_exception.
* Fix icall interface -- string vs. ptr/len.
* tweaks
* Restore the corlib interface, and maybe address the apidiff gate.
* cleanup
* Fix PtrToStructure GC assertion failure tests/marshal2 interpreter.
* Fix merge.
* Revert "Restore the corlib interface, and maybe address the apidiff gate."
This reverts commit ca8456a05f75f0fc05f6559e2f075e18a455a362.
* Try harder to support fixed in array copying.
* Allow for overlap.
* Put mono_ in front of type and fix Windows.
* Fix small problems on Windows build.
* Fix Windows and support larger data.
* [arm64] Make OP_CHECK_THIS read only a byte, to prevent unaligned access errors.
* Bump corlib higher.
* Merge with https://github.com/mono/mono/pull/9430/files.
* Bump API snapshot submodule
* cominterop_get_interface(gboolean throw_exception) throw_exception is always true.
* Remove unused cominterop_get_hresult_for_exception.
* Guids are constant.
* Replace g_assert (mono_error_ok (error)) with mono_error_assert_ok (error).
* Move function name to start of line for grep ^foo.
* PR feedback: g_assert(ex)
* Cleanup cominterop_get_interface -- single point of exit, more canonical non-coop form,
assert error indicator equivalence.
* Cleanup cominterop_get_interface_checked: early return and assert equivalent error indicators.
* Combine two versions of mono_cominterop_init.
* g_new0 instead of g_new+memset
* Remove obviously redundantly error_init.
* Inline cominterop_get_idispatch_for_object into its only caller ves_icall_System_Runtime_InteropServices_Marshal_GetIDispatchForObjectInternal,
and make it a more coop-friendly.
* add const
* Combine forks of some functions and ifdef out more on Win32.
* Initialize with declaration.
* Use integers for sizes, not pointers.
* More asserts though redundant.
* Combine two forks of mono_marshal_safearray_set_value.
Dependency Injector
===================
* `corlib/Mono/DependencyInjector.cs`: this is corlib's entry point.
* `corlib/Mono/ISystemDependencyProvider.cs`: this interface is implemented in `System.dll`.
* `System/Mono/SystemDependencyProvider.cs`: the `System.dll` side of it.
To use this, we need to call `SystemDependencyProvider.Initialize()` during system startup on Mobile
(this is already done for Android, but still needs to be done for XI/XM).
For Desktop, this will be triggered by a few `System.dll` based APIs such as for instance `SslStream`
or `X509Certificate2` - and the corlib side can use reflection if needed.
The corlib side should use `DependencyInjector.GetSystemDependency()`.
X509Certificate - corlib side
=============================
We previously had a mechanic called `INativeCertificateHelper` (which unfortunately was not really
working as intended) to use the BTLS code (which lives in System) in corlib. This has now been
replaced to use the new dependency injector.
The actual implementation as well as all the platform conditionals have now been moved into `System.dll`
where they can take advantage of existing PAL code.
This also removes all `Mono.Security.X509` usage from the `System.Security.Cryptography.X509Certificates`
namespace (it is still used by some of the other crypto classes, so we can't remove it yet).
* `Mono/ISystemCertificateProvider.cs`: this interface is now hooked up via the new
dependency injector and replaces `INativeCertificateHelper`.
* `System.Security.Cryptography.X509Certificates/X509Helper.cs`: this file has been completely
reworked and is now just a tiny stub around `ISystemCertificateProvider` and will eventually go
away completely. The entire implementation has been moved into `System.dll`.
* `System.Security.Cryptography.X509Certificates/X509CertificateImplMono.cs`: removed, we now use
`X509Certificate2ImplMono` from System via `ISystemCertificateProvider`.
* `System.Security.Cryptography.X509Certificates/X509CertificateImplApple.cs`: moved into `System.dll`
and slightly reworked there.
* `System.Security.Cryptography.X509Certificates/X509Helper.Apple.cs`: removed, we do not have any
platform-specific code here anymore; the new `ISystemCertificateProvider` provides this for us.
X509Certificate - System side
=============================
The actual implementation as well as all the platform-specific code now lives in `System.dll`.
* `Mono/SystemCertificateProvider.cs`: the `System.dll` implementation of `ISystemCertificateProvider`.
We do the platform / TLS Provider checks in here, then construct a custom PAL class.
* `Mono/X509Pal.cs`: New internal static class.
* `Mono/X509PalImpl.cs`: New abstract internal class. It contains some common code, including the
fallback implementation.
* `Mono/X509PalImpl.Mono.cs`: Fallback PAL, used by platforms without SslStream / MonoTlsProvider.
* `Mono.Btls/X509PalImpl.Btls.cs`: BTLS implementation.
* `Mono.AppleTls/X509PalImpl.Apple.cs`: We are using the `MONO_FEATURE_APPLE_X509` conditional in
here, so it can be used without the actual Apple TLS code.
* `Mono.AppleTls/X509CertificateImplApple.cs`: moved here from corlib.
* `Mono.AppleTls/*.cs`: Use the `X509CertificateImplApple(IntPtr)` constructor everywhere instead of
`X509Certificate(IntPtr)` to make sure we are using the "correct" PAL.
* [System]: Fix `mono_btls_x509_get_public_key()` implementation.
* Copy corlib's `X509CertificateTest.Certificate8` into System, call it
`Corlib_Test_Certificate8` and use `X509Certificate2`.
The test in corlib does not expose the problem because it would use the
fallback implementation.
* Use `[SetCulture ("")]` as suggested by Marek.
[coop] Fix MONO_HANDLE_TRACK_OWNER build
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
Originated from PR #1283, they now pass because we're using referencesource
code for the hash algorithm classes now.
In RSAPKCS1SignatureDeformatterTest.cs we weren't testing against the
actual `badSignature` (it doesn't make a difference since the referencesource
code or .NET Framework throws NullReferenceException anyway, but is more correct).
Closes#1283
* `HashCode` is now enabled in all profiles.
* Added `RNGCryptoServiceProvider` based fallback implementation.
* Added internal `RNGCryptoServiceProvider.GetRandom(byte*,IntPtr)` overload.
* The fallback implementation is enabled by default in `corlib.dll.sources`,
platforms which provide a native implementation need to add it to their
`.exclude.sources`.
[sdb] Avoid calling vm.GetThread () in the event callback, it could c…
…all into the debuggee leading to a deadlock if the thread mirror object has not been created yet.
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
[interp] fullaotmixedcheck fixes
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
It is too easily trashed, e.g. by mono_thread_info_uninstall_interrupt.
See https://github.com/mono/mono/pull/9104.
See https://github.com/Unity-Technologies/mono/commit/b6f1ba89f7ab285a0990e696e78798f7cf18dafb
Which this should be an alterate fix for, as discussed in 9104.
This is a small change, but ultimately we should be able to thread
gint32 *win32error around enough, such as to eliminate the thread local on Unix.
It doesn't appear to take much. Thread locals, it turns out, are not a good mechanism.
They are sensitive to loss as addressed here. They are slower to access than regular globals
or locals. They do have the advantage of "automatic threading through a callgraph", as long
as you are very careful with their use (this PR demonstrates that problem).
As well we could/should move the GetLastError calls down one layer on Win32,
from w32file.c to w32file-win32.c.
But this change addresses only ReadFile/WriteFile.
Also remove unnecessary void** vs. void* cast.
Also combine Unix mono_w32file_read and mono_w32file_write.
[mini] make interp work with AOT image generated with aot=interp
[aot-compiler] use mono_aot_mode_is_interp instead of directly using the flag
---------
[interp] add FULL_AOT flags to AOT image when `--aot=interp` is used
fixes regression from 67e995c6e2
---------
partial forward port of https://github.com/mono/mono/pull/9266
needs backporting to `2018-06`
* Fix unit tests to use SERIALNUMBER instead of OID.2.5.4.5 in X.500 names. Modify the code to pass the tests.
* Add SERIALNUMBER for BTLS implementation.
* Fix one more place where SERIALNUMBER was not handled.
[interp] Optimise allocation path
Speeds up allocation by 30-40%
<!--
Thank you for your Pull Request!
If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.
Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
As part of #7589
- CoreFX import only for Windows platform (no support for non-Windows platforms, see https://github.com/dotnet/corefx/issues/22510)
- left Mono managed implementation for non-Windows platforms
- removed DataProtectionScope parameter value check (no such a check in CoreFX and NET Framework, see https://github.com/dotnet/corefx/pull/30726) and couple of related unit tests
- added CoreFX xunit tests
When implementing Windows x64 valuetype ABI mono’s SIMD optimization didn’t
handle the case when passed arguments uses ArgValuetypeAddrInIReg/
ArgValuetypeAddrOnStack so optimization was disabled on Windows x64.
This commit fixes mono SIMD support for ABI’s passing value types using
ArgValuetypeAddrInIReg/ArgValuetypeAddrOnStack on Windows x64. It does this
as part of regular value type decomposition, including a pass decomposing SIMD
intrinsic ending up as arguments of type OP_ARG into correct load/store of the
value type reference before argument can be used by SIMD instructions.
All JIT and full AOT regression tests pass with enabled SIMD optimization
on Windows x64. Running this optimization on some vector intensive applications
showed some great performance increase. Just running this on basic-simd.exe
regression shows:
Elapsed time: 0.040471 secs (no optimizations at all)
Elapsed time: 0.015756 secs (only simd and intrins optimization)
So in this small, isolated test, the optimization indicates a 2.5x performance increase.
* [runtime] Map memory used by code managers with MAP_JIT on osx.
* [runtime] Use MAP_JIT only on osx mojave and later, it seems to cause crashes on earlier versions.
* [runtime] Add a constant for darwin 18.
* [runtime] Fix the race in get_darwin_version ().
* Fix indentation.
When doing full AOT of System.dll on Windows and other platforms building
runtime without HOST_DARWIN or HOST_BSD we will get an ugly error,
but compilation will still succeed leaving the user confused around the result.
The following error is printed:
--------------------------------------
cant resolve internal call to
"System.Net.NetworkInformation.MacOsIPInterfaceProperties::ParseRouteInfo_internal"
(tested without signature also)
Your mono runtime and class libraries are out of sync.
The out of sync library is: C:\github\mono\msvc\build\fullaot\winaot\system.dll
When you update one from git you need to update, compile and install
the other too.
Do not report this as a bug unless you're sure you have updated correctly:
you probably have a broken mono install.
If you see other errors or faults after this message they are probably related
and you need to fix your mono install first.
cant resolve internal call to
"System.Net.NetworkInformation.MacOsIPInterfaceProperties::ParseRouteInfo_internal"
(tested without signature also)
Your mono runtime and class libraries are out of sync.
The out of sync library is: C:\github\mono\msvc\build\fullaot\winaot\system.dll
When you update one from git you need to update, compile and install
the other too.
Do not report this as a bug unless you're sure you have updated correctly:
you probably have a broken mono install.
If you see other errors or faults after this message they are probably related
and you need to fix your mono install first.
--------------------------------------
The problem is that MacOs/BSD specific NetworkInformation classes are build into
System.dll on all platforms. One way to solve this is to start splitting the different
platforms specific implementations into different source files so we could start to
include/exclude sources per platform and profile in a nicer way instead of just
ifdef:ing inside current source files. This strategy also matches how corefx have
structured their NetworkInformation sources making it easier in the future to bring
mono’s NetworkInformation implementation closer to corefx. This PR implements this strategy.
Due to the fact that we don’t have platform specific configs for all profiles
(like basic and build) we can’t exclude all platform specific sources from
common.sources in System and move them to platform specific profiles. Ones we
have that full capability we could drop the #if WIN_PLATFORM in all the Win32*
files and exclude them from common.sources and only include them in the corresponding
win32 specific profiles. The same strategy could then be applied for other platform
specific profiles as well.
For Windows specific profiles, like win32_net_4_x_System.dll and winaot_System.dll we
use the .exclude.sources to remove the unix/linux/macos NetworkInformation sources,
so with this commit we will get a System.dll that only includes the Win32 specific
NetworkInformation classes and none of the unix/linux/macos, this will make sure the
compile error during full AOT won’t happen since there is no classes left including
the icall not available on a Windows mono runtime.
Other platforms will have the same classes in System.dll as before for now
but the source code is now structured making it possible to include/exclude platform
specific source files for other platforms as well, when needed.
The previous system was a nice prototype, but it ran into the problem of
taking a global lock on every thread doing logging. A fully lockless
implementation of a data structure with so much state would have been
madness to debug, and I really didn't want to spend much time in the future
debugging this.
What I did is to make a thread that reads from a lockless queue and logs the
message from the queue.
Most of the rest of this code is to address the fact that hazard
pointers depend on state that is initialized after logging is expected
to come up.
[checked-build] don't collect state transition backtraces on abort_blocking
for the same reason as the other cases in this function - if we're in `RUNNING`
or `ASYNC_SUSPEND_REQUESTED` under hybrid suspend we're supposed to cooperatively
suspend. If we call `backtrace()`, we might block waiting for the low-level
dynamic linker mutex on Linux which may be held by a thread that was
preemptively-suspended already.
Returns a text description of the suspend policy currently in effect.
Also combine logic that probes for suspend policy defines and environment
variables into a single function that returns an enum value; use that to drive
the individual boolean predicate functions.
The policy is subject to configure flags from when mono was built, and the
assorted environment variables at runtime:
MONO_ENABLE_COOPERATIVE_SUSPEND
MONO_ENABLE_COOP
MONO_ENABLE_HYBRID_SUSPEND
(the second one is a backward-compatible alias for the first)
The logic is:
If coop xor hybrid is enabled by configure flags, always return that policy.
Otherwise if coop and hybrid env vars are set simultaneously, it's an error.
Otherwise if MONO_ENABLE_COOP or MONO_ENABLE_COOPERATIVE_SUSPEND is set use full coop.
Otherwise if MONO_ENABLE_HYBRID_SUSPEND is set use hybrid.
Otherwise use full preemptive suspend.
[mini] respect additional flags when caching runtime invoke wrappers
depending on `virtual_` and `need_direct_wrapper` we generate slightly
different IL. The interpreter always set `need_direct_wrapper`, but the
JIT doesn't, so this gets problemantic with mixed mode. We need to be
careful when we cache those results.
This bug has been lurking for four years, being introduced in commit
5b2bb8c901. BYTE_ORDER != G_BYTE_ORDER.
Not throughly tested, but System.Decimal from csharp seems to return
sane results again. (related: is z or AIX CI available for PR checks?
if not, what test suite is System.Decimal under?)
Also remove my shoddy workaround from a few months ago, it got the
BCL building, but was incorrect.
* [sdb] Add a AssemblyMirror.GetMetadata () method to read the metadata from the debuggee, make AssemblyMirror.Metadata return it, or null if its not set/read yet.
Fixes a regression introduced by b26a41cc3e. Parts of Mono.Debugger.Soft would check
AssemblyMirror.Metadata against null, and that would throw an exception when the debuggee was using an older protocol version.
* Bump API snapshot submodule
This PR prepares the upcoming removal of the Legacy TLS Provider.
- The internal (and soon-to-be-removed) class `MD5SHA1` has been moved and changed namespace.
- `Novell.Directory.Ldap.Connection` now uses `SslStream` instead of Legacy TLS, this ports the Novell Ldap library to use TLS 1.2, see #9363.
- Removed obsolete Ssl/Spnego authentication from `System.ServiceModel`.
- Removed duplicate copy of the Legacy TLS Provider from `System.ServiceModel`.
- Ported `System.Web.Mail.SmtpClient` to use `SslStream`.
- Removed the `Mono.Security.Protocol.Tls` documentation.
* Functions at start of lines for grep ^foo.
* [Coop] Convert System.Security.Cryptography.RNGCryptoServiceProvider.
* Remove 'count' in more places in favor of inc/dec buffer/size.
Guard all loops with while (size > 0).
Don't pretend that sizeof int32/16/8 is other than 4/2/1.
assert instead of loop forever of degenerate systems with small RAND_MAX and
without better random.
* [Coop] Some conversion of custom-attrs.c. (#9309)
* Add comment explaining mono_array_addr_with_size_fast vs. mono_array_addr_with_size.
* Have mono_array_addr_with_size reuse mono_array_addr_with_size_fast.
* split create_custom_attr_data into self and create_custom_attr_data_into_array, to avoid handle creation in loop (like alloca in loop)
Note there is a subtle semantic change here, in that previously
when invoke failed, we'd still store into the array.
The caller checks the error and throws out the data so it should be ok,
the same in the end.
* Combine mono_cominterop_get_com_interface and ves_icall_System_Runtime_InteropServices_Marshal_GetIUnknownForObjectInternal which are nearly identical.
* Fix disabled COM.
* Replace unformative and possibly dangerous g_assert_not_reached with safer more informative g_assertf.
Fix many warnings like: icall.c:666:2: warning: format string is not a string literal (potentially insecure) [-Wformat-security]
Most uses of mono_error_set_argument do not take a format string.
So rename mono_error_set_argument to mono_error_set_argument_format
and provide mono_error_set_argument that just takes one string, not to be formated.
Visit every call to mono_error_set_argument and change just a few to be mono_error_set_argument_format.
While there, notice that method_encode_code forms up extra data and does not use it.
* [debug] mono_pmip shouldn't assume it's called from an attached thread
`mono_pmip` calls `get_method_from_ip` which needs to make a string with the
name of a method. It used to call `mono_method_full_name` which is a `MONO_API`
function and so tries to switch to GC Unsafe mode (which requires the thread to
be attached).
However `mono_pmip` is often used from GDB/LLDB and it might be invoked by the
developer from a stopped thread that isn't currently attached which would assert.
Instead call `mono_method_get_name_full` in `get_method_from_ip` which doesn't
switch the thread state.
* [coop] mono_coop_cond_signal and mono_coop_cond_broadcast should switch to GC Safe
Under Linux/glibc `pthread_cond_signal` and `pthread_cond_broadcast` can block
on a futex in `__cond_quiesce_and_switch_g1`, so switch the thread to GC Safe
mode around the blocking operation.
* [coop] ves_icall_System_Threading_WaitHandle_SignalAndWait_Internal shouldn't swith to GC Safe
On Win32, still do the thread state change here; on other platforms
its handled in the implementation of mono_w32handle_signal_and_wait
Add support for platform customization of native crash info output.
Add support to replace implementation of a post native crash handler behavior
if needed (used on none desktop Windows platforms).
NOTE, currently alternative implementations are in separate source file only
for Windows (since there was a need to implement in sources keept in external
closed repository). All other platforms (none Windows) still uses the exact same
logic still located in the same source file as before.
[interp] fix scope issue regarding `klass` var
`klass` exists in the main loop, but we defined `klass` again inside the
block, so the outer `klass` never had the right value.
Fixes https://github.com/mono/mono/issues/9023
* Update genproj makefile to include gensources
Update genproj argument parser to be more generous about displaying help
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint
* Fix rebase issue
* Checkpoint
* Checkpoint
* Fix built sources only being added to one profile
* Fix typo
* Checkpoint
* Fix indentation
* Use csc instead of mcs
* Checkpoint
* Fix BUILT_SOURCES only being handled for the first profile processed
* Checkpoint
* Checkpoint
* Strip double slashes from paths to fix spurious csproj change
* Checkpoint
* Checkpoint
* Checkpoint
* Checkpoint: Fix genproj compilation
* Checkpoint
* Checkpoint
* Checkpoint
* Fix crash when no targets were loaded (due to an error)
* Checkpoint
* Checkpoint
* Checkpoint
* Fix TryParseTargetInto bug
* Checkpoint
* Shuffle exclude logic around so that it works correctly during genproj diffing
* Remove gensources tracing
* Checkpoint
* Fix handling of oddball sources paths from executable.make
* Fix jay not being set to build
* Fix wrong slashes being used for embedded resource paths
* [csproj] Update project files
We had a problem with order of operations before. Makefile.am.in is
interpolated twice in the standard build. The first time happens when
autoconf runs. The second time happens when we enter the mono/mini directory
and execute our build. In the refresh, we lose our AM_CONDITIONAL conditionals
defined that aren't also AC_DEFINE variables on linux.
That's what it's vital to define all variables in all namespaces to have maintainable
automake code.
Furthermore, some of the CFLAGS logic we defined is designed for older LLVM copies, as
it includes some flags like O3 and Wall that we don't want.
In order to not get hacky with this, I moved the logic for using llvm-config into
a bash file in /llvm that creates a makefile fragment. The logic says that this fragment
should be null on calling autoconf, and then will be instantiated after we build llvm.
This fixes the build on Linux. We can expect bash behavior to be more consistent between
OSX and Linux than we can with automake too. I think this is a reorg with good long-time
maintainability.
* Use System.Security.Cryptography.Oid* from CoreFX.
* [csproj] Update project files
* Add CoreFX unit tests.
* Add obsolete S/MIME OID to make the Oid CoreFX tests pass on non-Mac systems.
* Enable Windows specific implementation of OidLookup.
* [sdks] Only rebuild MXE when it doesn't exist, not when we just cloned the sources
* [sdks] Default to ~/android-toolchain and ~/android-archives to better share with XA
Current DYNCALL on AMD64 didn’t handle scenario when spilling parameters from
registers to stack and that parameter was of type float/double. It then incorrectly
updated the floating-point register slot, but should have written it into the general
stack slot, making sure it will be correctly placed on callers stack in correct slot.
Current DYNCALL implementation didn’t handle the Windows x64 ABI when hitting a scenario
when spilling a valuetype into stack, it currently only supported the regular ArgOnStack
storage type, but Windows x64 will use ArgValuetypeAddrOnStack storage type.
There was also an additional issue with the number of stack slots allocated since it used
the stack argument offset instead of number of slots represented by the offset
(as used in later calculations). This caused to much stack space to be allocated and copied
(but things still worked as expected).
Commit also include a number of new test in aot-tests.exe mini regression test hitting above
issues.
* Modernize Windows random number generation.
- Less code.
- Less chance for failure.
- Not seedable.
- Requires Windows 7 or newer -- er, require Vista or newer until/unless Unity drops it, under ifdef.
This might get much further pruned with CoreFX import, which is on hold here, and doesn't really change much, except to indeed drop Vista support. (https://github.com/mono/mono/pull/9249)
After fixes issues in DYNCALL Windows x64 ABI all aot-tests.exe previously
excluded using !FULLAOT-AMD64 can now be reenabled since outstanding issues
have been fixed. There is one new test still failing on Windows x64,
test_42_arm64_dyncall_vtypebyval, that is now excluded until investigated
and fixed.
Correctly handle use cases when parameter storage is ArgValuetypeAddInIReg
or ArgValuetypeAddrOnStack. This is used for value types not fitting into
registers on Windows x64.
The Winx64 calling convention handles parameters in a way that was not
correctly implemented in some DYNCALL use cases. When using a method signature
like this (double, int, double), the Winx64 ABI dictates that the first parameter
goes into first floating point parameter register (XMM0), second parameter goes into
second general register (RDX) and third floating point goes into third floating point
parameter register (XMM2). So above call should use registers XMM0, RDX and XMM2,
but current DYNCALL implementation will just use next floating-point register,
like XMM0, RDX and XMM1 and this will load wrong parameter registers as expected by callee.
For an example https://msdn.microsoft.com/en-us/library/zthk2dkh.aspx#Anchor_2
Fix makes sure mono_arch_start_dyn_call uses the already evaluated registry information
for floating point parameters. Currently it assumes floating point registers are assigned
in a linear fashion and that works for System V ABI, but not for Winx64’s ABI.
Commit also adds a couple of tests into ao-test.cs that works with mixed signatures
together with DYNCALL.
* [wasm] Add helper target to update the runtimes.
* [wasm] Reenable tests that previously crashed due to JS engine issues.
* [wasm] Update jsvu 1.3.1 so we get jsc on linux. Fix uname magic on OSX.
Omitting positional parameters works for gdb linked to python 2.7 / 3.1+. On older systems (e.g. redhat 6, centos 6) gdb might be linked to python 2.4-2.6 or 3.0, adding parameter ordinals enables proper backtraces on those systems.
Fixes#9292.
* [System]: `HttpWebRequest` now throws `WebExceptionStatus.RequestCanceled` on abort. #9031.
Cleanup the internal `HttpWebRequest.RunWithTimeout()` function and use
`GetWebException()` everywhere to throw `WebException` with
`WebExceptionStatus.RequestCanceled` on abort.
New tests for this will be added to the web-tests by extending the existing
`Xamarin.WebTests.HttpInstrumentationTests.AbortDuringHandshake` to check the
synchronous and begin/end async APIs as well, using both `GET` and `POST`.
See https://github.com/xamarin/web-tests/blob/master/Xamarin.WebTests.Tests/Xamarin.WebTests.HttpInstrumentationTests/AbortDuringHandshake.cs.
Fixes#9031.
* Fix exception checks, setting `TransferEncoding` without `SendChunked` is supposed to throw `InvalidOperationException`.
* [System]: Renegotiation in AppleTls requires OS X 10.12+.
* Make renegotiation version check conditional to `!MONOTOUCH`.
The rest of the renegotiation code is currently conditional to `!MONOTOUCH`
as well, we might want to eventually revisit this at some point.
Fixes#9234
Had additional restructure not been ok, at least typedefing the function pointer
would have helped.
Add const for efficiency -- make the copy-on-write faults count for more and
not take extra readonly data with them.
Remove unused and actually confusing parameter -- it is just zeroed memory
and almost cannot be used for anything (granted, could hash it).
* [sgen] Use longs with nursery array fill
Objects are 8 byte aligned. This enables us to support nurseries of up to 32GB.
* [sgen] Add explicit limit to nursery size
* g_string_insert does not update string->len, and copies the wrong number of characters around,
and does not allow for overlapping parameters,
and is not used -- remove it.
g_string_erase does not always update string->len,
and is not used -- remove it.
g_string_prepend does not update string->len.
Switch the only use to append in the reverse order, and fix a leak.
[runtime] Fix debugger dumping when unused
Somewhere in the daily rebasing to edit the PR, I lost the correct version of the early-exit. This fixes our native crash when we did not use the debugger. It's important to merge, or we'll crash before MERP/gdb dumping happens.
Using dynamic symbol resolve (LoadLibrary/GetProcAddress) for all unwind info related registration functions enables
building the same code on all Windows platforms where JIT is build and included.
There are Windows platforms (like UWP) that doesn't support getenv. The implementation
of g_getenv makes it possible to replace the implementation at build time on
none desktop platforms.
The throwing of the pending exception after the finally blocks are executed was in unreachable code.
Refactor the code a bit and exit the abort protected block directly in endfinally. We throw the abort after all guarding finally blocks were invoked, to avoid throwing from inside a finally block. Guard also fault handler, which are called from EH.
* [runtime] Implement tombstone scrubbing for our concurrent hashtables.
Tombstones must be scrubbed as they can end up taking up all space of a hashtable and make lookup fail.
* [runtime] Added regression stress test for #9176
This test triggers insertion and removal from runtime_invoke_hash to the point
that if there's no scrubbing it will cause a deadlock.
* [runtime] Disable new test under full-aot as it uses SRE.
The callback consists in `mono_thread_info_abort_socket_syscall_for_close` which itself calls `mono_threads_suspend_abort_syscall`. The problem with the previously existing interrupt callback is that, even if `mono_threads_suspend_abort_syscall` is called by the thread suspend mechanism, it would still call `mono_thread_info_abort_socket_syscall_for_close` which would duplicate the call to `mono_threads_suspend_abort_syscall`
* Helper scripts running mini regression tests in JIT/Full AOT on Windows.
A set of helper scripts running all mini regression tests, supporting both
JIT and full AOT (including building full AOT assemblies) using local build
Visual Studio mono runtime (can be switched in setp.env.bat). Includes logic
to pick correct Visual Studio toolchain and support to setup command prompt
targeting x64 full AOT build and running tests. NOTE, the scripts are not limited
to the mini regression tests, only includes a set of predefined assemblies making
it easier to execute regression tests.
Full AOT scripts includes capabilities to build dynamic library, static linked objects
or assembler output, it also supports switches to use LLVM when building full AOT
assemblies if supported by local build mono runtime.
By default, scripts follow default build configurations used by mono BCL builds and
Visual Studio x64 Release build mono runtime.
Examples to run all JIT regression tests:
run-jit-regression.bat all
To run individual programs using JIT (doesn’t need to be a regression test):
run-jit.bat [full path to assembly]
NOTE, for regression tests there is a shortcut making them easier to locate,
so then just the assembly name can be used, not the full path is needed.
run-jit.bat basic.exe
Examples to build all regression tests using full AOT:
build-full-aot-regression-tests.bat all
Builds all regression tests in full AOT including all needed support and
BCL assemblies. All build assemblies will end up in msvc\build\fullaot\winaot
folder by default. It is possible to build individual assemblies as well using:
build-full-aot-regression-tests.bat [full path to assembly]
NOTE, for regression tests there is a shortcut making them easier to locate,
so then just the assembly name can be used, not the full path is needed.
build-full-aot-regression-tests.bat basic.exe
Examples to run all regression tests using full AOT:
run-full-aot-regression-tests.bat all
Runs all regression tests in full AOT. It is possible to run individual assemblies
as well using:
run-full-aot-regression-tests.bat [full path to assembly]
NOTE, for regression tests there is a shortcut making them easier to locate,
so then just the assembly name can be used, not the full path is needed.
run-full-aot-regression-tests.bat basic.exe
There are some more features included in the scripts as well, but above shows
basic usage. It could also be possible to reuse the scripts in Visual Studio
build targets going forward to at least run the JIT regression tests
as part of Visual Studio build.
* Fixed some review feedback.
* Switching to 1 instead of -1 in case of errors.
* Used case insensetiv compare for some if's.
* Dropped use of ECHO OFF/ON in nested scripts.
* Removed clear of PATH (was not intended).
* [mini] Implement delegate tramp info using gsharing context.
* [mini] Remove duplicate patch type.
* expand a FIXME to include what needs fixing.
* [mini] Copy MonoDelegate::method from the delegate tramp info instead of emiting a rgctx fetch. This is 6% faster on a delegate microbenchmark.
* Fix Process.ProcessName on AIX
AIX has a procfs with raw structures congruent with Solaris. For i,
which lacks procfs, fall back to a libc call.
This doesn't solve the Process weirdness that bungs up the test
suite, but I hope it's a start.
* Bye bye, annoying AIX setschedparam warning
AIX (but weirdly, not i) complains every time you do this. Squelch
the warning on AIX and avoid line noise.
* Remove more assumptions of /proc on AIX
* should probably always use getprocs64 for w32process,
but looks like we could just use proclib on the fallback?
* there's some more cases relating to resource usage stat,
will have to investigate those cases further
* Basic process modules support on AIX (but not i)
Not aware of a PASE compatible API, but this will work on AIX. Not
sure about .ModuleName versus .FileName, but it seems to fill it in...
The gensources.cs currently checked in to master doesn't fail on missing files by default because it didn't work at the time. This turns that behavior on. It is currently only in my other in-progress branch so I'm moving it here.
Fixes#9232
* Enable failing on missing files in gensources
* Copy over the missing file checks from genproj2
* Trim whitespace off file names from sources files
* Strip double slashes from paths to fix spurious csproj change
Previously we called it only if gdb was enabled, and we called it after
we invoked handlers asynchronously, in ways that could eat our session
state. By moving it higher up, we can increase the numbder environments in which
debugger crashes are logged, and log them even in the case when
telemetry dumping kills the runtime quickly.
* [runtime] MONO_API mono_assembly_get_object needs GC Unsafe
* [runtime] MONO_API mono_array_length needs GC Unsafe
* [runtime] Make MONO_API mono_reflection_load_from_full external only
Also mono_reflection_load_from ()
Also make them both transition to GC Unsafe
* [runtime] Make MONO_API mono_assembly_load_full external only
And make it transition to GC Unsafe
* [runtime] Make MONO_API mono_assembly_loaded external only
Runtime should use mono_assembly_loaded_full.
Make mono_assembly_load transition to GC Unsafe
* [runtime] MONO_API mono_assembly_open needs GC Unsafe
Also mono_assembly_open_full
* [runtime] MONO_API mono_check_corlib_version needs GC Unsafe
* [runtime] MONO_API mono_class_enum_basetype needs GC Unsafe
* [runtime] MONO_API mono_value_box needs GC Unsafe
* [runtime] Make mono_type_get_object external only
Runtime should use mono_type_get_object_checked or mono_type_get_object_handle
* [runtime] MONO_API mono_string_to_utf8 needs GC Unsafe
* [runtime] MONO_API mono_raise_exception needs GC Unsafe
* [runtime] Make mono_reflection_type_get_type external only
Also make it use GC Unsafe.
Runtime should use mono_reflection_type_get_handle
* [runtime] MONO_API mono_object_new needs GC Unsafe
* [runtime] MONO_API mono_domain_assembly_open needs GC Unsafe
* [runtime] MONO_API mono_method_get_object needs GC Unsafe
* [runtime] MONO_API mono_method_full_name needs GC Unsafe
* [runtime] MONO_API mono_method_signature needs GC Unsafe
* [runtime] MONO_API mono_thread_create needs GC Unsafe
* [runtime] MONO_API mono_field_get_type needs GC Unsafe
* [runtime] MONO_API mono_field_get_value needs GC Unsafe
* [runtime] MONO_API mono_field_set_value needs GC Unsafe
* [runtime] Mark mono_image_open_from_data_with_name external only
Runtime should use mono_image_open_from_data_internal
* [runtime] MONO_API mono_image_open_from_data_with_name needs GC Unsafe
* [runtime] MONO_API mono_field_static_set_value needs GC Unsafe
* [runtime] MONO_API mono_property_set_value needs GC Unsafe
* [runtime] Make MONO_API mono_assembly_load_with_partial_name external only
Runtime should use mono_assembly_load_with_partial_name_internal.
Make mono_assembly_load_with_partial_name transition to GC Unsafe
* [profiler] Log whenever the profiler library fails to load
* [test] Fixup return type of mono_gc_toggleref_add
* [sgen] Don't pin obj in test_toggleref_callback under coop
Don't call mono_field_get_value which under coop will perform a thread state transition to GC Unsafe and that may include copying the stack data which will pin the toggle object and subsequently make the toggleref test fail. Call mono_field_get_value_internal which assumes we're already in GC Unsafe mode.
Uses mono LLVM master branch, build using cmake and VisualStudio 2015/2017.
Commit adds support for AOT/Full AOT LLVM codegen for Windows x64 Visual Studio
build mono runtime. Normal configure/make using cygwin won’t support LLVM and will
trigger an error if configured.
In order to build using LLVM first mono LLVM master branch must be build using
static linked LLVM configuration (dynamic loading can be added later),
NOTE mono LLVM build needs to include the following commit,
https://github.com/mono/llvm/commit/9b92b4b87607e137266f84dc307181b8842fe54a
in order to successfully build on Windows x64.
Build mono LLVM branch using the following cmake configuration:
-DCMAKE_BUILD_TYPE=Release
-DLLVM_ENABLE_ZLIB=OFF
-DLLVM_TARGETS_TO_BUILD="X86"
-DCMAKE_CROSSCOMPILING=False
-DCMAKE_SYSTEM_NAME=Windows
Use cmake with -G "Visual Studio 14 2015 Win64" to generate VS 2015 x64
targets. NOTE, if mono runtime is build using VS 2017, then LLVM should
be build using the same VS version using -G "Visual Studio 15 2017 Win64"
To enable LLVM build in mono.sln, set MONO_ENABLE_LLVM to “true” and make
sure MONO_LLVM_INSTALL_DIR_PREFIX points to the install directory used in
LLVM build above, default msvc/dist/llvm. Above LLVM properties can also
be set if build using msbuild.
A sample LLVM configure script has been included in msvc/llvm-cmake-config.bat.
* [typesafe coop] Go through generated token-pasted helpers
instead of casting function pointers. This should be safer and
not trigger quashed gcc warning and I had hoped it would
work with WebAssembly, but seemingly not.
Add MONO_TYPEOF_CAST to avoid repitition.
Here we have a structured log and a richer "model" of states.
This serves to be a 10-foot view of the debugger. The debugger
has a lot of layers of abstractions, and lots of intermingled
bookkeeping.
This debugger log serves to be a summary of the state of the debugger
as a whole, and our ringbuffer of recent events helps put the debugger
state into context.
It also allows us to easily identify, for instance when we resume
a new thread versus a suspended thread. We can see single-stepping
through the breakpoints added, removed, and hit. Lastly, we have a view
of how commands and events are totally ordered over the TCP socket,
helping the server to understand what the client has seen and in which
order.
Previously one had to include the proper types for this header in
the file including this header, before this header is included. Rather
than put irrelevant includes in other files, I'm adding includes here so
this file brings in its own dependencies.
* Make coop handles typesafe.
Type-safe handles are a struct with a pointer to pointer.
The only operations allowed on them are the functions/macros in handle.h, and assignment
from same handle type to same handle type (which cannot be prohibited in C, but is ok here anyway.)
Type-unsafe handles are a pointer to a struct with a pointer.
This is a leakier abstraction and besides the type-safe operations, also allows:
1. compared to NULL, instead of only MONO_HANDLE_IS_NULL
2. assigned from NULL, instead of only a handle
3. MONO_HANDLE_NEW (T) from anything, instead of only a T*
4. MONO_HANDLE_CAST from anything, instead of only another handle type
5. assigned from any void*, at least in C
6. Cast from any handle type to any handle type, without using MONO_HANDLE_CAST.
7. Cast from any handle type to any pointer type and vice versa, such as incorrect unboxing.
8. mono_object_class (handle), instead of mono_handle_class.
None of those operations were likely intended and some are crashing sort of bugs, esp.
the last two. See https://github.com/mono/mono/pull/9188 for the last two. The rest all occur also but are not necessarily harmful. Raw(NULL_HANDLE) and IsNull(NULL_HANDLE) were both access violations, so NULL just as well. MONO_HANDLE_CAST is still pretty liberal in what it allows.
NULL_HANDLE has been brought into question and this will allow grepping for it instead of the plainer NULL (i.e. returning them as handles).
The interaction of these handles with JITed code depends on the ABI to pass/return a struct
with a pointer the same as a pointer. Marshalling assumes this.
There might be some OS/CPU ABIs that do not support this.
CI suggests it is broadly but not 100% viable (disabled for non-Mac non-Windows x86, but enabled for arm, arm64, amd64, Mac/x86, Windows/x86).
* Typesafe handles enforcement brings with it a little more pointer enforcement, mostly around const and volatile. Repair a bit, and disable temporary for Windows.
* Fix MONO_HANDLE_IS_NULL (NULL_HANDLE) to not null dereference.
* Fix MONO_HANDLE_RAW(NULL_HANDLE) to not null dereference.
This depends on a gcc/clang extension, and a Visual C++ (C) quirk.
Otherwise we have to chose double evaluation, or C++, or assignment or cast from void* to temporary (possibly passing more types to more macros).
Another choice is a runtime assert, and double evaluation or not.
Asserts seen can be covered up by first checking MONO_HANDLE_IS_NULL, which will not null dereference nor assert.
Prior to making RAW(NULL) not null derefence, fixing code of the form:
```gboolean is_null_handle = handle && MONO_HANDLE_IS_NULL(handle)```
to be
```gboolean is_null_handle = MONO_HANDLE_IS_NULL(handle)```
while working on this change led to failures.
* Add gcc -Werror=incompatible-pointer-types to catch some handle type mismatches.
This has slight collateral damage on other code, mostly around const and volatile, mostly in Android-specific code. Disabled for Windows/gcc at first.
* Change mono_icall_handle_new_interior to return gpointer instead of MonoObjectHandle. It isn't returning a handle and making the non-handle typesafe is unnecessary/incorrect.
Likewise change mono_handle_new_interior to return gpointer instead of MonoRawHandle (same thing).
gpointer* is more correct, gpointer is more opaque; unclear which is preferred here.
gpointer makes this a smaller semantic change, as MonoRawHandle was already equivalent, just a rename.
* Move NULL_HANDLE, NULL_HANDLE_STRING, NULL_HANDLE_ARRAY so they are near the comment that refers to all of them.
* Remove mono_handle_new_full prototype, the function does not exist.
* Remove unnecessary casts.
* Remove unused MONO_HANDLE_INIT.
* Add an assert to handle_assign.
* A refinement of this change should enable WebAssembly and skirt underlying safety less. WebAssembly does not allow the function pointer casts used here.
Try to catch situations leading to a segv in mono_domain_assembly_search by
checking that we never put NULL into domain_assemblies and that callers of
mono_assembly_names_equal_flags never pass NULL args to it.
* [coop] Use coop mutex
ldstr_section can be taken during jit compilation.
* [runtime] Switch to non recursive mutex
ldstr_section only guards hashtable accesses.
* Import System.Security.Cryptography.Pkcs from CoreFX.
* Change 'as' operator to a cast.
* Add missing EnvelopedCms overloads.
* Include CMS signing classes in all builds.
* [csproj] Update project files
* Add missing DSACertificateExtensions class.
* Update references.
* Rename EnvelopedCms.Overloads.cs to EnvelopedCms.cs.
* Enable some of the System.Security.Cryptography.Pkcs xunit tests.
* Update references.
* Bump API snapshot submodule
* Disable xunit tests that may depend on Windows PAL.
Contributes to https://github.com/mono/mono/issues/7589
One unit test is removed as part of the commit because it is not consistent with documentation and CoreFX behaviour on non-Windows platforms (https://github.com/dotnet/corefx/issues/30416).
The main goal of this patch is to actually get meaningful error messages when constructing X509Certificate2 from a byte array. Previously PKCS#12/PFX certificate import could fail due to unimplemented features (such as ECDsa support) or bugs. The certificate import would then be retried as PKCS#7 and the original error message was lost.
* Fix some handle type violations.
* MONO_HANDLE_CAST (MonoArray, NULL_HANDLE)
is fragile and deprecated except in optimized hot paths,
prefer NEW instead. NULL_HANDLE is const and cannot be assigned to,
NEW is not const and can be assigned to. It takes extra validation
to insure NULL_HANDLE is correct, and it is usually not worthwhile.
Brings in https://github.com/mono/bockbuild/pull/67.
This fixes the msbuild build problem where it'd crash with `error MSB4242: The SDK resolver "NuGetSdkResolver" failed to run. Unexpected failure reading NuGet.Config. Path: '@/external/bockbuild/builds/msbuild-15/artifacts/toolset/restore.proj/$HOME/.config/NuGet/NuGet.Config'.`.
The reason is that bockbuild exported the `XDG_CONFIG_HOME` environment variable with the value `$HOME/.config`. The problem is that the `$HOME` in there is never expanded so it just stays as the literal value.
`XDG_CONFIG_HOME` is later used by e.g. `Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)` which is what nuget uses to construct the path to NuGet.config - hence why we end up with the weird path.
We now just expand the variables in bockbuild before we pass them on.
Fixes https://github.com/mono/mono/issues/9179
(cherry-picked from cb54c26f6c)
Threads which do things with signals and locks may end up unresponsive
to our signals. I cannot get threads that crash in a lock to respond to
SIGTERM on Darwin even if I explicitly unmask SIGTERM on those threads.
In order to defensively program against platform bugs around signals and
stubborn threads, we need to be able to abandon these threads and
continue, dumping other threads.
[tests] reduce IO load in process-stress-3
The spawned process by the test iterates through the whole mono repo,
including submodules. On a loaded machine with a already slow I/O
device, 10 seconds could be a tight time budget. Instead of increasing
the timeout, let's try to reduce the generated output.
Fixes https://github.com/mono/mono/issues/6947 hopefully.
Ubuntu Linux (possibly others too, but only Ubuntu is tested thus the
limitation) don't need to build or use `mxe`, they instead use mingw packaged
for the distribution, thus making the build faster.
This commit detects the Linux flavor in use and, if it's Ubuntu, configures the
build so that system mingw is used instead of mxe.
Followup for 9ad5255338
Note that on win32 we already did the right thing because of nested
conditionals in mono_image_open_a_lot, but on non-win32 we need the extra
check.
[mono/mini/helpers.c:195]: (warning) %d in format string (no. 1)
requires 'int' but the argument type is 'unsigned int'.
[mono/mini/helpers.c:197]: (warning) %d in format string (no. 1)
requires 'int' but the argument type is 'unsigned int'.
* [mcs] Rename darwin paltform to macos
Darwin is common to both macOS and iOS/tvOS/watchOS, while we use the darwin platform only for macOS.
* [mcs] Add Unix platform for other Unixes than Linux
* [mcs] Remove PLATFORMS from xammac_net_4_5 since it only make sense on macos
* [bcl] Bump corlib version
* [csproj] Update project files
Android NDK supports cmake in addition to the older make-based ndk-build as the
build system and Xamarin.Android is in the process of switching the build of its
native bits to cmake. The reason for this switch is that certain parts of
Xamarin.Android (libmonodroid) are built for both the host system and
the Android target. The host system build currently uses hand-crafted code
directly in .csproj to build the native bits while the Android build uses
ndk-build. This results in duplication of code, compiler parameters are
specified in two locations - it is easy to introduce incompatibilities, bug
fixes may be applied only in one scenario etc.
Using cmake allows us to use a single build "script" for all the targets, thus
making the build more reliable. Android SDK has a cmake package which contains
full support for NDK builds and allows us to easily use cmake across all the
supported platforms without having to install cmake on the host system.
This commit adds the cmake package to the Android SDK provisiong process and,
alas, at the same time adds a workaround to properly unpack the package. Unlike
other Android SDK packages, cmake doesn't unpack to a single subdirectory of the
zip root, instead all its files and directories are placed directly in the zip
root. This requires yet another workaround to install the package properly.
[segfault] hard crash if it is not a NRE
that is, if a `ji` is available, but the offset is too large to be a
plausible implicit null pointer.
The "upper bound" must be higher, but I don't know a good value. Any good suggestions other than "increase the value until all tests are passing"?
* [mini] Change region formation pivot from BB to EH clause.
The original BB region computation would loop all BBs and search which EH clause each of them.
EH search is implemented as a linear search, meaning performance was count(BB) * count(clauses).
Since each clause generates at least one BB, it's quadratic on the number of clauses.
Degenerate methods, with tens of thousands of EH clauses, would lead to minutes of JIT time.
New code loops over the clauses and mark all BBs covered by each of them. Given EH ranges are a much
smaller subset of all BBs, this performs significantly better. It remains possible to construct methods
that leads to degenerate compilation times, but it's less likely now.
* [mini] Bound the size of a linked list during liveness.
The `visited` linked list can grown to tens of thousands of entries with some degenerate methods.
This change uses a stack allocated array and switches to a hashtable on overflow.
* [mini] Change printf to g_print
* [mini][amd64] Limit more precise branch target optimization based on number of basic blocks.
On amd64 we update branch target data before each BB is emitted. This has quadratic behavior on number of basic blocks
and can lead to excessive compilation times in degenerate cases.
Limit the increased precision to methods with less than 800 basic blocks.
Reduces compilation time of #7809 by 88% (from 44s to 5s).
When creating dynamic methods, we put those in a queue so they can be collected later via `mono_jit_free_method ()`. the interpreter part was assuming, as soon as we allocate a dynamic method, it will be executed eventually and therefore an InterpMethod must exist in `info.interp_code_hash`, but this isn't necessarily the case.
Fixes https://github.com/mono/mono/issues/9109
We can't raise event before that as the profiler depends on too much runtime
infrastructure at this point for it to reliably work.
For example, we need the threading infrastructure to be init'd so we can access the
thread-local profiler state to store events there.
issue detected by cppcheck
[mono/metadata/threads.c:3993] -> [mono/metadata/threads.c:3991]:
(warning) Either the condition 'if(location&&location->source_file)'
is redundant or there is possible null pointer dereference: location.
condition is redundant indeed, it is nested in "if (location) {"
* [jit] Add some rules to the local propagation pass to transform iconst+sext+mul into mul_imm.
* [jit] Make OP_LDLEN more general by storing the offset of the array length field in it.
* [jit] Add span intrinsics.
* [jit] Avoid converting Span.get_Length () to OP_LDLEN, it doesn't help abcrem. Avoid asserts when running on portable spans.
* [jit] Convert Span.get_Length () to OP_LDLEN, only mark the loading of array lengths invariant.
* [uwp] UWP supports: ReplaceFile CopyFile LockFile UnlockFile GetDriveType GetComputerNameW
Some of these were already supported and identical code copied.
* Remove pre-Windows 2000 support -- use GetFileSizeEx instead of GetFileSize.
UWP does not support GetFileSize.
* Introduce fine-grained ifdefs.
This should make maintenance of Win32 subsets easier (see w32subset.h).
* Generally the code should be/stay in "classic.c" and the "uwp.c" files can/should go away. The uwp.c files were a medium-sized granularlity of reuse/replace, but the fine-grained ifdefs are better/smaller -- more code reuse.
Which involves:
`serialize_object`
`deserialize_object`
`mono_object_xdomain_representation`
and then the other users of `mono_object_xdomain_representation`:
`create_unhandled_exception_eventargs`
`call_unhandled_exception_delegate`
* [runtime] Add an 'interp_method' field to MonoDelegate.
* [runtime] Add a MONO_LD_DELEGATE_PTR jit opcode which is used to load the method ptr from a delegate, so the JIT and interpreter could have a different implementation.
* [runtime] Store the interpreter method invoked by the delegate in MonoDelegate.interp_method, make MonoDelegate.method_ptr always point to a JIT callable entry point.
* [interp] Avoid sharing pinvoke and non-pinvoke signatures in mini_get_interp_in_wrapper ().
* [interp] Fix interp_get_remoting_invoke (), it receives a interp entry trampoline as well.
* [interp] Store remoting invokes into interp_method in delegates.
* [interp] Handle compilation failures in do_jit_icall (). Handle calls to delegates created in JITted code.
* [jit] Add a 'aot-skip=' MONO_DEBUG option to skip loading the Nth aot loadable method, useful for forcing mixed mode transitions during testing.
* [jit] Add tests for fullaot+interp mode
The tests work by injecting interp-jit transitions in varying locations controlled by the MONO_DEBUG=aot-skip= option.
* [interp] Avoid races when caching method pointers. Fix wasm support.
* Revert "[interp] Avoid races when caching method pointers. Fix wasm support."
This reverts commit 0f78c230b5102d625c300a4e1c6eaf97ad055dd5.
* [interp] stub native-to-managed on platforms where it is not working
* [ci] add fullaotmixed-regression step
Currently if we change for example `corlib.dll.sources` and then execute `make` - the `make` won't rebuild the lib, because it depends only on `net_4_x_corlib.dll.sources` and doesn't depend on all `#include` inside it.
As a temporarily solution we can depend on all *.sources files in the directory so a change in any of them will trigger rebuild. Unfortunately changes in un-related profiles will also trigger the rebuild but it's better than nothing for now I guess. For better experience we should fetch all #includes recursively for the current profile
The only reason for the restriction was that handle_alloc previously would not handle generic instances.
This speeded up the following microbenchmark by 19x (3.38s x 0.17s):
```
class Driver {
static T Identity<T> (T t) {
return t;
}
[MethodImpl(MethodImplOptions.NoInlining)]
static Func<T,T> MakeDelegate<T> () {
return new Func<T, T> (Identity);
}
static void Main () {
object o = new object ();
for (int i = 0; i < 400 * 1000; ++i) {
MakeDelegate<object>()(o);
}
}
}
```
* [Coop] Cleanup ahead of converting ves_icall_System_Threading_Thread_GetAbortExceptionState.
- serialize_object and deserialize_object are almost identical and
can be implemented on top of the same function.
- Error propagation was a little wierd and had FIXME comments.
It involved MonoError, MonoException, and gboolean *failed, and
some conversions. Converge on mostly MonoError, with fewer conversions,
albeit still sometimes holding a MonoException.
* Move error parameter to last.
Move static locals to global -- more easily seen in debugger.
Remove errant exclamation point.
* [jit] Make the interp_in wrappers work better with byref arguments.
* [runtime] Move some EH tests to exceptions.cs.
* [interp] Fix some problems in the interp entry/exit code in mixed mode.
* [jit] Avoid code patching from gshared code when the callee is an interp entry trampoline.
* [aot] Map byref return values to int&.
* [interp] Handle r4 return values in do_jit_call ().
This lets the linker catch any missing symbols, and fixes a problem
I had building AIX with CoreFX PAL. Fixes#8979, though there may
be more cases like this.
dotnet/corefx#30012 is blocking on this.
Since GitHub pages rolled out HTTPS support, links to `www.mono-project.com` can now be `https://` :) This also removes a `www.` 301 users currently hits.
In the case of the FSEvent FileSystemWatcher, the CFRunLoop event loop thread has already an interrupt handler installed in the icall, and mono_monitor_try_enter may try to install one as well which is not supported. So we create separate event tasks before we use the Operation() method.
We'd get the following warning during the build:
```
'/Users/alexander/dev/mono/mcs/class/System.Data.Services.Client/ONO_PATH' in MONO_PATH doesn't exist or has wrong permissions.
```
The reason is that we didn't escape the $ symbol in the resgen template
so the embedded $$MONO_PATH in the RESGEN variable ended up being mangled.
I'm kinda surprised this worked at all for so many years ¯\_(ツ)_/¯
* [loader] Make mono_assembly_names_equal_flags visible
Previously was static assembly_names_equal_flags in assembly.c
Add "mono" prefix to all the names
* [loader] Ignore version if assembly name isn't a strong name
When checking if a loaded assembly satisfies a request for a given assembly
name, don't compare versions if requested name doesn't include a public key
token.
Fixes https://github.com/mono/mono/issues/8820
* [loader] Add MONO_ANAME_EQ_IGNORE_CASE flag
In some circumstances we may need to compare one MonoAssemblyName to another
ignore case differences in simple names (ie "Asm1" vs "asm1" should be
considered equal)
* [tests] Regression test for simple name assembly preloading
Regression test for https://github.com/mono/mono/issues/8820
If one assembly references another that has a simple name (no public key), then
the requested version and the actual version don't need to match.
Also - the simple names are compared case-insensitively.
* [loader] When looking for already loaded assemblies, ignore case differences
When we're looking for a loaded assembly by simple name, ignore case differences.
* [debugger tests] use runtime flags for debugee
* [debugger] handle managed-to-native frames
fixes DebuggerTests.InspectThreadSuspenedOnWaitOne on the interpreter
* [interp] set SP in monoctx that is passed to exception handling
fixes DebuggerTests.ExceptionFilter on the interpreter
* [debugger] fix debugger_invoke frame in interp case
fixes DebuggerTests.BreakpointDuringInvoke on the interpreter
Currently the list elements are allocated in the root domain, while the data can be allocated in other domains. We need to clear the reference when removing the list element so we are not left with hidden cross domain refs.
Ideally we should have per domain lists, so we also would just let the list and elements die, without having to iterate through them at domain unload time.
* [mono-threads] Introduce mono_thread_info_get/set_tools_data so they can store data sanely.
The tools_data is a helper to bind tls data lifecycle to MonoThreadInfo.
This is needed to make it sane to deal with posix TLS destructors.
* [profiler] Switch tls storage to use mono_thread_info_get/set_tools_data. Fixes#7683
External threads are detached using TLS destructors and this leads to non-determinism around to cleanup ordering of keys.
This causes the profiler to see the profiler tls key null'd even though it was not detached yet.
* [profiler] Since we depend on thread_info to be attached before tls works, do so for the helper threads.
We can't just move the mono_thread_attach call earlier as it leads to all sorts of weird crashes (as it generates events and we won't be attached).
This is the second and final part to bring Client Certificate support.
It needs to be landed on top of #8753 and #8756.
* `Mono.Security.Interface.IMonoSslStream`: Add `CanRenegotiate` and `RenegotiateAsync()`.
* `Mono.Security.Interface.MonoTlsSettings`: Add `DisallowUnauthenticatedCertificateRequest`.
* `AppleTlsContext`: fully support renegotiation.
- we may now receive `SslStatus.PeerAuthCompleted` and `SslStatus.PeerClientCertRequested`
during `Read()`. It should in theory not happen during `Write()`, but I added it there
as well just to be on the safe side.
- `SetSessionOption()` may only be called before the initial handshake.
* `MobileAuthenticatedStream`: this is the major part of the work and the most complex one.
- added a new `Operation` enum to keep track of what is going on and detect invalid state.
- a renegotion may only be triggered while we're idle - that is no handshake, read or write
operation is currently active.
- `InternalWrite()` may now be called from `SSLRead()`, the new `Operation` tells us what
is currently happening.
- `ProcessHandshake()` now takes a `bool renegotiate` argument.
- added sanity checks to `ProcessRead()` and `ProcessWrite()`.
* `MobileTlsContext.SelectClientCertificate()`: check for
`MonoTlsSettings.DisallowUnauthenticatedCertificateRequest`
* `MonoTlsProviderFactory.InternalVersion`: bump the internal version number.
Tests have already been added to `web-tests/master`, they will auto-enable themselves when
using a Mono runtime that contains this code.
* Remove two methods with incorrect arguments
These two methods on `IPersonalizable` don't match the .NET 4.7.1 API.
In both cases the methods should take one `PersonalizableDictionary`
argument, but _that_ type does not exist.
Since nothing in the current class library code is using these methods,
just remove them.
* Bump API snapshot submodule
This is the first of two Pull Requests to implement Client Certificates :-)
Part One binds the new native APIs that will be used internally, finishes
the certificate selection callbacks, but without the more riskly changes
to the underlying handshake and I/O layer.
Part Two will bring support for TLS Renegotiation - and due to the required
changes in the underlying handshake, it is the more risky one.
* `Mono.Security.Interface.MonoTlsSettings`: Add `ClientCertificateIssuers`.
* `MobileTlsContext`:
- fully implement `SelectClientCertificate()`; the `acceptableIssuers` parameter
is now actually set and we also have a reasonable default selection.
- add `CanRenegotiate` and `RenegotiateAsync()` - these are not hooked up yet.
* `AppleTlsContext`:
- we will only ever call `RequirePeerTrust()` once per session, so we can
also remove it alltogether and just use `EvaluatePeerTrust()` instead.
- use proper exceptions for `SslStatus.PeerNoRenegotiation` and `PeerUnexpectedMsg`.
- don't call `SetClientSideAuthenticate()` on the client side.
- bind and hook up `SSLAddDistinguishedName()` and `SSLCopyDistinguishedNames()`.
- bind `SSLReHandshake()`.
* `MobileAuthenticatedStream`: minor cleanups; there will be more uses of the new
`GetInvalidNestedCallException()` helper class once Part Two lands.
* Enable some more constants in `SecureTransport.cs`.
* Add new `MonoBtlsError.GetErrorReason()` and `mono_btls_error_get_reason()`
implementation, only supporting `SSL_R_NO_RENEGOTIATION` at the moment.
* Add new native `mono_btls_ssl_ctx_set_client_ca_list()` function and managed
`MonoBtlsSslCtx.SetClientCertificateIssuers()`; hooked up via
`MonoTlsSettings.ClientCertificateIssuers`.
* According to a comment in the header file, `SSL_get_client_CA_list()` may only
be called during the selection callback or while the handshake is paused.
To respect this restriction, we now call it during the client certificate
selection callback and pass the list from native to managed.
- changed signature of `MonoBtlsSelectFunc` from
`int (* MonoBtlsSelectFunc) (void *instance)` to
`int (* MonoBtlsSelectFunc) (void *instance, int countIssuers, const int *sizes, void **issuerData)`.
- the managed counter-part is in `MonoBtlsSslCtx.NativeSelectFunc` / `NativeSelectCallback`.
* MonoBtlsContext:
- use the new `MonoBtlsError.GetErrorReason()` to throw a `TlsException` with
`AlertDescription.NoRenegotiation` that can be checked for by user code.
- `SelectCallback()` now has a `string[] acceptableIssuers` argument; pass it
to `SelectClientCertificate()`.
- the native backend does not support TLS Renegotiation, so `CanRenegotiate`
always returns false.
Implements #7075
Rather than reaching into the library internals and potentially
violating its invariants, assemble a cipher string and call the
supported public API. (There's no need to touch the version-specific
cipher lists because mono/btls never configures them.) The existing code
also had a bug; it incorrectly assumed OPENSSL_malloc returned a
zero-initialized buffer for in_group_flags. (Though Mono doesn't set
SSL_OP_CIPHER_SERVER_PREFERENCE, so I suspect this was a no-op.)
Additionally, this should hopefully ease updating to a less outdated
version of BoringSSL in the future.
Note I've changed the return value of this function from returning the
number of ciphers to merely returning 0/1 for whether the operation
succeeded. The only caller of this function, MonoBtlsSslCtx.SetCiphers,
only cares about success vs. failure.
* [jit] Move the handling of special methods like pinvoke etc. into mini-runtime.c, since they are not handled by the JIT.
* [jit] Add a --full-aot-interp command line argument which is the same as --full-aot with a fallback to the interpreter.
* [jit] Implement fallback to the interpreter in full-aot+interp mode.
* [jit] Add a 'methods using the interpreter' stat.
The old thunk machinery was sort of working on 64bit (before
f4bd084438 ), but it relies on having on
the memory layout of allocated chunks for code. Either the property got
lost due to some refactoring, or it was never true on ppc64. Anyway,
this patch switches the implementation to something we do on arm32 too:
If we potentially need some thunks for the emitted code, we reserve some
space for it at the end of the code buffer. That wastes a bit more
memory, because we often don't need it, but at least it works reliable.
Untested on 32bit systems.
The "problematic images" are Windows-specific assemblies that are often
included with applications that do not work on Mono, but for which Mono has its
own implementation.
Previously (9ec8ec5389) we changed mono to deny
loading problematic images using Assembly.Load or when one assembly references
another and we find a problematic image using the usual assembly search.
For Assembly.LoadFrom, we used to emit a warning and then load the assembly anyway.
In this commit, we change the behavior: If Assembly.LoadFrom tries to open a
problematic image, we will instead probe for an assembly of that name in the
default context. That should eventually end up opening Mono's version of that
assembly.
Example:
Suppose a problematic System.Runtime.InteropServices.RuntimeInformation.dll
is in the application bin path: /home/user/myapp/bin
1. User code does (for whatever reason)
Assembly.LoadFrom ("/home/user/myapp/bin/System.Runtime.InteropServices.RuntimeInformation.dll")
2. We find the image and try to open it; add it to the images absfpath to
image hash; note that there are no binding redirects; but that it is a
problematic image. We extract the assembly
name ("System.Runtime.InteropServices.RuntimeInformation", version x.y.z.w
and public key token 123456789a) from the problematic image.
3. We probe for "System.Runtime.InteropServices.RuntimeInformation version
x.y.z.w public key token 123456789a" in the default context.
4. We find
/home/user/myapp/bin/System.Runtime.InteropServices.RuntimeInformation.dll in
the application bin path; we try to open it, we see it's already in the
images hash, but it's a problematic image, and we're not in a loadfrom
context (we re-probed with default!) so we keep probing
5. We find
<prefix>/lib/mono/4.5/Facaded/System.Runtime.InteropServices.RuntimeInformation.dll,
it's not a problematic image, so we open that and return it to (3)
6. We return to (2) and close the problematic image and return the good one
instead.
7. Assembly.LoadFrom returns with the good image.
Fixes https://github.com/mono/mono/issues/8726
When we switched to the new version of Monitor.Enter, we didn't expand the try block to include the call. This led to unreleased locks when we acquired the lock but were aborted shortly afterwards.
This brings the internal `MailAddressParser` class and switches `MailAddress` and `MailAddressCollection` to the CoreFX version.
The tests have been updated to reflect the CoreFX behavior, which is more strict about rejecting invalid addresses.
When PropertyGrid contains more than one propertyTab it crashes whith NullReferenceException. And when clicking on propertyTab toolstripbutton it didn't switch to this tab.
* [jit] Unify the way the results of inlined calls are pushed on the stack.
* [jit] Allow class inits during inlining if the class has no cctor.
* [bcl] Mark EqualityComparer<T>.get_Default () with MethodImplOptions.AggressiveInlining, its used in inner loops in the corefx code.
* [jit] Implement devirtualization of some EqualityComparer<T>.Equals calls, the corefx code assumes this.
* [jit] Rename has_array_access to needs_decompose to better reflect its meaning.
To make sure that System.Reflection.Emit types in the class libraries
are properly removed by the linker, mark them with the "sre" feature in
the embedded linker descriptor file for mscorlib.
It contains this fragment:
```
<TargetFrameworkFallbackSearchPaths Condition="$([MSBuild]::IsOSPlatform('osx'))">$(TargetFrameworkFallbackSearchPaths);/Library/Frameworks/Mono.framework/External/xbuild-frameworks</TargetFrameworkFallbackSearchPaths>
```
This PR introduces a new tool, gensources.cs, which replaces gensources.sh. It is able to parse the whole set of .sources files for a library in one go so that genproj can use that information to encode all our platform and profile specific files into one csproj file.
For now this PR just introduces it and switches libraries to using it instead of gensources.sh.
* Fix assert to print the error, and not sometimes itself crash. It was evaluating varargs twice.
* Fix all the warnings in the eglib tests. Leave them not building for now.
* [jit] Decompose some CEE_BOX opcodes later to allow more optimizations.
Optimize a code sequence emitted for Enum.HasFlag () by newer csc versions.
Also mark the temporary bblocks used by decompose_vtypes () as out-of-line if
the bblock under processing is out-of-line. Mark the domainvar as volatile to avoid
optimizing it away since references to it can now be generated later from the
decomposition of OP_BOX.
* [jit] Check that the enum types are the same.
* [jit] Fix support for generic enums.
* [jit] Disable the optimization with 64 bit enums on 32 bit platforms.
* Use size_t for gsize if it matches, else C99 uintptr_t.
Using size_t provides for an exact match with functions like malloc and pthread_attr_getstack, no warnings.
* GINT_TO_POINTER and co. need no autoconf, beyond chosing gsize, gssize.
* Use autoconf to find size_t among unsigned int, long, long long, __int64 for printf purposes.
Avoid 18+ year old C99 print("zu") dependency but still less likely to warn than the previous approach which only looked for matching size, not matching type.
* mini-ppc-os.h doesn't exist. If the OS lacks a define,
give an error instead of a broken include.
* Remove redundant Mac OS case.
* Move AIX define for sigaction above cross-compile case,
so it won't get lost in the noise there.
* [aot] Fix encoding/decoding of types with custom modifiers.
Fixes https://github.com/mono/mono/issues/8701.
* [aot] Add a test, disabling for now because it depends on c# 7.2.
* [aot] Bump AOT file format version.
* [aot] Only disable the test with mcs.
* [aot] Fix compilation of test.
* [sre] ParameterBuilder.SetConstant should work with Nullable<T>
Rewrite ParameterBuilder.SetConstant like CoreCLR
Same logic as TypeBuidler.SetConstantValue from coreclr at
src/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.cs
The details are different because we don't have CorElementType in corlib (so
use TypeCode), and we want to write to a var instead of directly invoking an icall.
Among other things, this enables ParameterBuilder to work with Nullable<T>.
If a parameter is of type Nullable<T> then pb.SetConstant (null) should work.
Also pb.SetConstant (value) should work when T is one of the supported
types (primitives, enum, datetime, etc).
Fixes https://github.com/mono/mono/issues/8597 and fixes https://github.com/mono/mono/issues/8504
* [test] ParameterBuilder.SetConstant with a nullable type
mkbundle works by generating a small C source file which, in addition to
embedded assemblies/configs/etc, contains a certain amount of code to initialize
the bundle, uncompress the assemblies and set runtime options (e.g. AOT mode).
This source code is generated based on a number of templates shipped as
resources inside `mkbundle.exe` and is not part of any public, maintained API.
The generated code contains calls to Mono runtime which work just fine when
mkbundle is used to generate a desktop application bundle with the default
template code. However, it breaks when a third party (like Xamarin.Android)
embeds Mono runtime and loads it dynamically **after** the application is
initialized. Such third parties must ensure that the bundle's startup code is
able to access the Mono runtime functions it needs by providing pointers to
them, loaded using e.g. the `dlsym` function.
However, as the Mono calls are hard-coded into the generated source code, the
third party needs to resort to search-and-replace methods (e.g. Xamarin.Android
https://github.com/xamarin/xamarin-android/blob/62e7792ca52949f2fa213330d292865ad78d2bfa/src/Xamarin.Android.Build.Tasks/Tasks/MakeBundleNativeCodeExternal.cs#L159-L166)
which are very fragile and prone to undetected breakage.
One example of such breakage which slipped under the radar is
https://github.com/xamarin/xamarin-android/issues/1651 which was caused by
addition of the `mono_jit_set_aot_mode` call to the generated code. The change
slipped under the radar and was discovered only after release.
This commit attempts to make the situation better by establishing a sort of a
code contract between "upstream" mkbundle and the third party. Since the
template code is not "public" (nor should it be), the third party cannot simply
include a header file with the API definitions. Instead, the idea here is to
ensure that all and any Mono runtime calls needed by the template code are
placed in a dispatch structure as pointers to functions and that the structure
needs to be initialized by the embedding party before calling the
`mono_mkbundle_init()` function. The third party is responsible for maintenance
of its own version of the dispatch structure as well as for its initialization
via the `initialize_mono_api` function in the template code. The generated code
must include the third party's version of the structure, in place of the default
one as shipped with `mkbundle`, and to make it possible this commit adds the
`--mono-api-struct-path` mkbundle parameter which points to a file that's
injected into the generated code, replacing the default structure definition.
Following that, the `mono_mkbundle_init()` code calls the `validate_api_struct`
function which checks each and every pointer in the structure for `null`,
exiting the application with error if any `null` pointer is encountered.
This (along with the `initialize_mono_api` function in the template) has the
additional effect that if the third party's structure is incompatible with the
default one a build error will happen, thus allowing early detection of obvious
incompatibilities/omissions.
As long as all the new Mono runtime calls are added to the structure, the above
should make it possible for issues like the one linked to above to not happen in
the future.
* `SslStream`: Reject attempts of setting conflicting callbacks using both the
Mono-specific `MonoTlsSettings` and the new `SslClientAuthenticationOptions` /
`SslServerAuthenticationOptions`.
This makes it consistent with CoreFx behavior where those callbacks may only be
specified in one of the possible places.
* `ChainValidationHelper` - this internal class has received a major overhaul and
lots of old and unused code removed.
All callbacks are not invoked with the correct `sender` parameter to make it
match the .NET / CoreFx behavior.
* `Mono.Security.Interface.CertificateValidationHelper`: remove unused internal code.
* Volatile static field accesses lack barriers.
This is a contributing factor to https://github.com/mono/mono/issues/8321.
finally_block_ending_in_dead_bb.exe does not poll the volatile field.
This is not the entire problem.
* [llvm] Notice the MONO_INST_VOLATILE flag on load/store, not just MONO_INST_FAULT.
This is related to https://github.com/mono/mono/issues/8321 finally_block_ending_in_dead_bb.exe
however emitting a barrier suffices to get LLVM to do the read.
* method_to_ir: Factor out operand decoding, including of constants.
* Cleanup unused file.
* Use MonoOpcodeEnum_Invalid instead of (MonoOpcodeEnum)-1.
* Move CHECK_STACK and CHECK_STACK_OVF mostly to central place, except variable
ones like call/ret/newobj.
* No _Static_assert on msvc.
* Remove some FIXMEs.
* Remove an inline.
* Less use of `tailcall_calli`, more use of `tailcall`, by assigning first to second.
And then eliminate `tailcall_testvalue`, as it just becomes `tailcall`.
* Merge the end, almost just the end, of CALLI with CALL/CALLVIRT.
Also merge the start, and with JMP, i.e. initializing shared variables.
This eliminates a bunch of shared code, without being too complicated (subjective)
and gives CALLI its missing sequence point.
goto here is disliked, maybe later a function.
* Change virtual_ from int to gboolean, assign it TRUE and FALSE instead of 0 and 1.
* Change guint32 call_opcode to MonoOpcodeEnum il_op.
* Replace *ip with il_op.
il_op is chosen over op, to disambiguate IL from IR.
And this is why C++ enums are not just ints:
enum Animal {dog, cat};
enum Color {red, blue};
Color color = dog; // valid C, invalid C++ (sometimes a warning in C)
Animal animal;
if (color == animal) ... // valid C, invalid C++ (sometimes a warning in C)
* add a FIXME
* Bring `SslClientAuthenticationOptions`, `SslServerAuthenticationOptions`
and `SslApplicationProtocol` from CoreFX.
* SslStream: Added new public overloads from CoreFX:
- All authentication methods have a new overload without the `SslProtocols` argument.
- New `AuthenticateAsClientAsync(SslClientAuthenticationOptions,CancellationToken)`.
- New `AuthenticateAsServerAsync(SslServerAuthenticationOptions,CancellationToken)`.
- We now use `SecurityProtocol.SystemDefaultSecurityProtocols` (which is zero) as
default value everywhere.
Everything below is internal:
* `Mono.Security.Interface`: Add internal `MonoServerCertificateSelectionCallback`,
`IMonoAuthenticationOptions`, `IMonoSslClientAuthenticationOptions` and
`IMonoSslServerAuthenticationOptions`.
* `Mono.Security.Interface.IMonoSslStream`: Add new overloads without the
`SslProtocols` argument.
* `Mono.Security.Interface.IMonoSslStream2`: New internal interface, extending
`IMonoSslStream` with the new internal APIs.
* `Mono.Net.Security`: New internal `MonoSslAuthenticationOptions`,
`MonoSslClientAuthenticationOptions` and `MonoSslServerAuthenticationOptions`
classes; these are just proxies for `SslClientAuthenticationOptions` and
`SslServerAuthenticationOptions` (which unfortunately don't share a common
base class).
* `Mono.Net.Security.MobileAuthenticatedStream`:
- Implement `IMonoSslStream2`.
- `ProcessAuthentication()` now takes `MonoSslAuthenticationOptions` instead
of the old argument list, added `CancellationToken`.
- All `IMonoSslStream` facing APIs now construct `MonoSslAuthenticationOptions`
to pass it to `ProcessAuthentication()`.
* `Mono.Net.Security`: `MobileTlsContext` and `MobileTlsStream` now use
`MonoSslAuthenticationOptions` to store all options.
Tests for the new APIs are in https://github.com/xamarin/web-tests/commit/fe347589326206f36d3cd42b59d305ff3c291857.
* Use MonoOpcodeEnum instead of int. Provide MonoOpcodeEnum_Invalid = -1.
* Fix and cleanup bounds checks.
Some were off by one, some could be clearer.
This pattern will go away anyway and is partly gone below.
* Prepare to fix incorrect read behind.
* Add test to hit assert in incorrect readbehind in ldvirtftn.
* Confuse the JIT (or runtime) into crashing.
* Fix parens.
* Move some macros from verify.c to verify-internals.h.
* Attempt to make IL readahead easier to understand, by having
less spread out instruction sizing and decoding.
This converts several readaheads but not all.
It is possible we should just convert to a large array of pointers,
and then do these optimizations.
This also fixes the incorrect readbehind.
* Disable new test. Fixing the readbehind fixes the assertion failure but still sigsegvs later in trampoline.
* More use of common code. More to do here.
Old gcc does not allow pragma diagnostic within function.
Fixes#8572
#if is a deliberate shorter form than #ifdef
- shorter
- allows for #define foo 0
- standardly undefined silently evaluates to 0
- potential for if instead of #if, though that does require #define foo 0, no
silent undefined => 0
* [jit] Fix a warning about disabling an unknown warning: trace.c:147:32: warning: unknown warning group '-Wframe-address', ignored [-Wunknown-warning-option]
* [jit] Fix a warning.
* [jit] Fix a warning.
Fix OSX x86 warnings.
In particular:
1) Be sure to pass a function all of its parameters, no more, no less.
You can't have a stack imbalance because parameters usually fit in registers,
and because calling conventions except for __stdcall are explicitly designed
to accomodate common programming mistakes, e.g. printf ("%d") and printf ("%d", 1, 2),
are well defined on all modern ABIs to never crash (except for the extreme
case of printf ("%d") where stack is near the top and it accesses invalid next page,
most unlikely).
2) size_t and gsize are not always the same type.
For example, one can be unsigned int and the other unsigned long (32bit).
Or unsigned long and unsigned long long (64bit).
3) gsize is not necessarily unsigned long, nor even the same size as it.
It could be any of:
unsigned int -- viable on all 32bit (Win32 and Unix32)
unsigned long -- viable on Win32 and all Unix (Unix32 and Unix64)
unsigned long long -- viable on Win64 and Unix64
unsigned __int64 -- viable on Win64, possibly same as unsigned long long
To printf a gsize in the "u" style (not the "x" nor "d" styled), G_GSIZE_FORMAT is provided.
(Fix for Win32 and Win64 forthcoming.)
We should strongly consider typedef size_t gsize but this a larger change.
4) Pop without push.
```
mono-publib.c:27:30: warning: incompatible function pointer types initializing 'gpointer (*)(gsize)' (aka 'void *(*)(unsigned int)') with an expression of type
'void *(*)(size_t)' (aka 'void *(*)(unsigned long)') [-Wincompatible-function-pointer-types]
GMemVTable g_mem_vtable = { vtable->malloc, vtable->realloc, vtable->free, vtable->calloc};
^~~~~~~~~~~~~~
mono-publib.c:27:46: warning: incompatible function pointer types initializing 'gpointer (*)(gpointer, gsize)' (aka 'void *(*)(void *, unsigned int)') with an
expression of type 'void *(*)(void *, size_t)' (aka 'void *(*)(void *, unsigned long)') [-Wincompatible-function-pointer-types]
GMemVTable g_mem_vtable = { vtable->malloc, vtable->realloc, vtable->free, vtable->calloc};
^~~~~~~~~~~~~~~
mono-publib.c:27:77: warning: incompsatible function pointer types initializing 'gpointer (*)(gsize, gsize)' (aka 'void *(*)(unsigned int, unsigned int)') with an
expression of type 'void *(*)(size_t, size_t)' (aka 'void *(*)(unsigned long, unsigned long)') [-Wincompatible-function-pointer-types]
GMemVTable g_mem_vtable = { vtable->malloc, vtable->realloc, vtable->free, vtable->calloc};
^~~~~~~~~~~~~~
mono-time.c:204:1: warning: unused label 'exit' [-Wunused-label]
exit:
^~~~~
mono-threads-posix.c:92:43: warning: incompatible pointer types passing 'gsize *const' (aka 'unsigned int *const') to parameter of type 'size_t *'
(aka 'unsigned long *') [-Wincompatible-pointer-types]
res = pthread_attr_getstacksize (&attr, stack_size);
^~~~~~~~~~
/usr/include/pthread.h:250:85: note: passing argument to parameter here
int pthread_attr_getstacksize(const pthread_attr_t * __restrict, size_t * __restrict);
^
profiler.c:1062:57: warning: incompatible pointer types passing 'void (MonoProfiler *, MonoProfilerGCEvent, uint32_t)' (aka 'void (struct _MonoProfiler *,
MonoProfilerGCEvent, unsigned int)') to parameter of type 'MonoProfilerGCEvent2Callback' (aka 'void (*)(struct _MonoProfiler *, MonoProfilerGCEvent,
unsigned int, int)') [-Wincompatible-pointer-types]
mono_profiler_set_gc_event_callback (current->handle, gc_event_cb);
^~~~~~~~~~~
../../mono/metadata/profiler-events.h:85:1: note: passing argument to parameter 'cb' here
MONO_PROFILER_EVENT_3(gc_event, GCEvent2, MonoProfilerGCEvent, event, uint32_t, generation, mono_bool, is_serial)
^
profiler.c:908:2: note: expanded from macro 'MONO_PROFILER_EVENT_3'
_MONO_PROFILER_EVENT(name, type)
^
profiler.c:897:103: note: expanded from macro '_MONO_PROFILER_EVENT'
mono_profiler_set_ ## name ## _callback (MonoProfilerHandle handle, MonoProfiler ## type ## Callback cb) \
^
marshal.c:4770:72: warning: format specifies type 'unsigned long' but the argument has type 'gsize' (aka 'unsigned int') [-Wformat]
mono_error_set_out_of_memory (error, "Could not allocate %lu bytes", size);
~~~ ^~~~
%u
trace.c:146:32: warning: unknown warning group '-Wframe-address', ignored [-Wunknown-warning-option]
^
trace.c:148:24: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
^
```
* Cleanup IL operation size checks -- data driven.
* Removed duplicate declaration.
* Prepare to cleanup IL IP advancement. Do a self-checking run first, which found incorrect sizing of switch, dyncall, icall.
* Cleanup -- looks like macro debris.
* Cleanup IL IP advancement -- mostly data driven.
Occasionally due to https://github.com/Microsoft/visualfsharp/issues/4637
Mono will assert if it tries to load a PPDB file that has an incorrect number
of MethodDebugInformation rows:
* Assertion at metadata.c:1117, condition `idx < t->rows' not met
Instead of asserting, check that the method index is in bounds, and print a more informative error.
* Fix mono_icall and mono_dyn_call operands in opcode.def.
This might require cpan install XML::Parser, or disable
the generation of opcode.def in Makefile.am.
* Fix sizing of switch instructions: 5 + 4 * n, not 4 + 4 * n.
* Hi Mono Team,
This commit fixes an interesting bug that relates to the System.Net.Http nuget packages.
So the short story is that System.Net.Http can view the internals of this assembly because of InternalsVisibleTo and the HttpClient references System.Net.Logging's "On" _property_. The problem is that mono has implemented "On" as a _field_ . When the latest version of System.Net.Http.HttpClient tries to invoke a SyncAsync using Mono, you get a MissingMethodException because:
bool System.Net.Logging.get_On()
does not exist.
This commit is simple and only changes the signature of On to match that of the reference source (property instead of field). This will cause the Missing Method Exception not to be thrown.
* Fix indentation
* [System]: The big WebConnectionStream cleanup.
`WebResponseStream` has been broken down into smaller pieces and now
fully supports GZip / Deflate decompression.
The entire `contentLength`, `readBuffer` and `totalRead` logic has been
replaced by a nested layer of `WebReadStream` classes.
* `WebReadStream` is a new abstract base class.
* `BufferedReadStream` handles the situation where we have some buffered
data and an (optional) inner stream to read from.
* `ContentDecodeStream` takes care of GZip / Deflate decompression.
* `FixedSizeReadStream` handles the `contentLength` logic - ie. making sure
we are only reading up to `contentLength` bytes from the stream.
* The old `MonoChunkStream` has been renamed into `MonoChunkParser` (as it was
really just a parser and not a stream) and a new `MonoChunkStream` has been
added that's based on `WebReadStream`.
* Remove HttpWebRequestTest.TestLargeDataReading().
This test was using Reflection to poke around implementation internals.
* [csproj] Update project files
* Cosmetic cleanups.
* [btls] Do not give gcc flags to msvc.
* [btls] #include "btls-foo.h" instead of #include <btls-foo.h> to work
with existing build system, on Windows, with out-of-tree build.
Not sure which factor does not like <> but "" should be ok.
* [blts] MONO_API is not used consistently and therefore errors on Windows.
1. Make it be nothing.
1b. Use a .def file if there are actual exports.
Really, .def files are a good idea.
or
2. Use it consistently.
This does #1.
* Fix, classify, enable some CoreCLR tests.
* [Tailcall] Add partial calli support, so a test passes.
* Disable r4/r8 mismatch for all architectures, and one due to interface on ARM.
* [runtime] Transition to GC Unsafe in MONO_API functions
The idea is that on any external->runtime transition we want to ensure that we
switch to GC Unsafe mode (and back on return). This allows Mono embedders to
be oblivous about hybrid suspend - the runtime will run native embedder code in
GC Safe mode (and use preemptive suspend when necessary) and when they switch
back to the runtime we will switch to GC Unsafe mode (and use cooperative suspend).
Since this needs to be backported, we're going to make this osx-specific
for now, and then I'll backport this. This is fully working on linux,
including the android SDK at the time of this commit
The debug log fnptr is used with MonoStringHandle arguments since
cc7406b226
The corresponding functions and prototypes in mini/ have already been updated.
The renderer would stop at the first XML comment `-->` which means
the diff was cut off.
We can fix this by switching from storing the diff in an HTML div to
a script tag instead.
Fixes https://github.com/mono/mono/issues/8469
* [Tailcall] Implement x86 OP_TAILCALL_REG (https://github.com/mono/mono/issues/8582)
This should enable calli, which isn't just for cee_calli but also internal purposes.
* [Tailcall] Split x86_align_for_patch out of emit_call.
* [Tailcall] Align x86 tailcall patches, using x86_align_for_patch.
* [Tailcall] Allow x86 tailcall to be AOTed.
* [Tailcall] Make all the 'missed tailcall' cases more similar looking.
* [Tailcall] Fill in a little more tailcall support.
* [arm] emit_big_add: Handle degenerate imm == 0 and dreg == sreg.
* [arm] Split emit_call_reg into emit_call_reg and emit_jmp_reg (for reuse by tailcall).
* [arm] handle hardfloat in tailcall
* [arm] Fill in TAILCALL_MEMBASE and TAILCALL_REG (reuse emit_jmp_reg)
virtual calls should work now (but not interface)
* [Tailcall] Comment about tail.calli -- no this-type in signature, to check for byref valuetype.
(data is available on compile-time staci, and a proposal to check for ldloca/ldarg pending)
* Add some asserts.
* More detangling of vtable_arg/imt_arg (extra_arg) from tailcall_supported -- they must be set, or nullness predicted, earlier.
* [Tailcall] Capture inputs to is_supported_tailcall and assert they don't change.
* [Tailcall] ARM32 tailcall not supported if mono_class_is_interface (cmethod->klass).
* [System]: Free ServicePoints when they're no longer used.
* Cleanup.
* Don't crash when ServicePoint.CloseConnectionGroup() is called after the ServicePoint has been freed.
* Remove the CloseConnectionGroupConcurrency() test.
This test was for a regression in a piece of code that no longer exist
and the test is poorly designed as it depends on precise timings.
It also attempts to make a web request to a non-exisiting server and
just ignores the error - it is unclear whether that is intentional or not.
In CoreFX, `ServicePointManager.SecurityProtocol` defaults to
`SecurityProtocolType.SystemDefault` (which is zero).
We now allow this value to mean "whatever the system default is"
in both AppleTls and BTLS.
The default value of `ServicePointManager.SecurityProtocol` has also
been changed to be consistent with CoreFX.
* [jit x86] Move microoptimization down a level: x86_mov_reg_reg do nothing
if source == desta and size == 4 (size is always 4, except in amd64 reuse,
which was only used once, unnecessarily, remove it).
[jit x86] x86_mov_reg_reg size is always 4,
split into unused. x86_mov_reg_reg_size and specialized x86_mov_reg_reg
* Remove last parameter from x86_mov_reg_reg that is always 4.
It is not guaranteed to work if the valuetype was allocated in the same stackframe.
This was initially leading to crashes because when storing in **this**, we were storing as a valuetype instead of the pointer (**this** valuetype is passed as pointer). Even if we fixed this issue, the other problem still exists.
* [interp] Fix stobj
When writing to a stackval, we write at least 4 bytes. Because we were wrongly casting destination to stackval*, we could be clearing other fields for 1/2 byte stores.
* [interp] Use memmove also with stackval_to_data
As we do with stackval_from_data to prevent unaligned faults.
* [sdks] Use cmake to build llvm.
* [sdks] Build all targets for llvm32 as well, since android might not build llvm64.
* [sdks] Build the llvm windows builds using cmake as well.
* Address review comments.
* Switch to the release60 branch of LLVM as an experiment.
* [sdks] Bump LLVM.
* Bump LLVM.
* [sdks] Fix the build.
* [llvm] Emit default debugging info for all llvm builders otherwise methods without debug info like iterators will fail to compile.
* Fix a crash if ctx->minfo is not set.
* [sdks] Add an LLVM_BRANCH variable.
* [llvm] Emit default debugging info for all llvm builders otherwise methods without debug info like iterators will fail to compile.
* Fix a crash if ctx->minfo is not set.
* [Tailcall] Add tailcall-return-valuetype.il test.
This can be fixed easily -- when F1 calls F2 tailcalls F3.
F2 can pass F3 the temporary it got from F1 instead of having another temporary.
* [Tailcall] Add a test for passing large valuetypes by value.
This has problems on some ABIs but not others -- arm64 and Windows/amd64 fail.
Presumably the temporary is either within the paramete area
or is a local. The first works, the second does not.
* [Tailcall ppc/s390x] Update ppc/s390x to be more like x86/amd64/arm/arm64
i.e. matching return storage, and s390x matching struct-ness/size.
* [Tailcall] Let valuetypes be returned through tailcalls.
Specifically there was already this handling them:
mono_emit_call_args:
{
if (tailcall) {
if (mini_type_is_vtype (sig_ret)) {
call->vret_var = cfg->vret_addr;
//g_assert_not_reached ();
}
so when F1 calls F2 and F2 tailcalls F3, F3 will return the value
where F1 was expecting it. There is no extra code needed after F3
returns to copy it from F2's temporary.
* CallInst.tailcall field is read before written in amd64, and is also wrong for llvmonly.
Replace CallInst.tailcall with MONO_IS_TAILCALL_OPCODE when accuracy is desirable and works.
Add FIXME when inaccurate CallInst.tailcall is presently required.
Tailcall field should probably be replaced by FALSE, except CEE_JMP needs attention first.
(see https://github.com/mono/mono/pull/8571)
* [Tailcall arm64 amd64 s390x] Do not allow tailcall when passing large valuetypes.
For s390x this is being conservative and might not be needed.
For ppc it is already done. Needed?
arm32/x86 seem to pass struct inline so it works, nice.
Small structs are sometimes passed in registers as well.
* Disable tailcall/coreclr/JIT/opt/ETW/TailCallCases.exe.
It passes a struct by value and fails.
Typo: is_adressable_valuetype_load => is_addressable_valuetype_load
Don't mark large multi-called function inline.
Remove lines that are all whitespace (leave whitespace at ends of lines, there is a lot).
One case of spaces to tabs.
* [System] When doing process redirect, use Console.OutputEncoding instead of Console.Out.Encoding. Fixes#8417.
This change makes mono behave in the same way as dotnet, which uses Console.OutputEncoding instead of whatever Console.Out is.
* [System] Add regression test for #8417.
https://jenkins.mono-project.com/job/test-mono-pull-request-coop/3429/testReport/junit/(root)/BuildSubmissionTest/EndBuildWaitsForSubmissionCompletion/
```
BuildSubmissionTest.EndBuildWaitsForSubmissionCompletion
Failing for the past 1 build (Since Unstable#3429 )
Took 36 sec.
add description
Stacktrace
MESSAGE:
#5
Expected: less than or equal to 00:00:02.5000000
But was: 00:00:09.1408213
+++++++++++++++++++
STACK TRACE:
at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0003b] in /mnt/jenkins/workspace/test-mono-pull-request-coop/mcs/class/corlib/System.Reflection/MonoMethod.cs:305
```
* [x86] Add support for tracing native-to-managed thiscall wrappers.
* [x86] Add support for native-to-managed thiscall wrappers.
* Add test for managed thiscall methods.
interp: If one of the operands is NaN, we have a default result: It is false (0)
when doing a compare (`ceq_r8`), but must be true (1) on `cne_r8`.
JIT: match behaviour what C# compiler would generate for float/double.
If you load Foo.dll and it happens to be version 2.0.0.0 and an app.config says
to redirect all versions of Foo.dll to version 2.0.0.0, we're there already -
don't close the image and probe again, that's silly.
When running with cookieless sessions, the file_path does not necessarily contain a leading '/' character which can lead to a Uri being constructed from something that looks like, "http://hostmyPage.aspx" which immediately fails and throw an exception stating that the hostname could not be parsed.
This is not trivial to fix -- either a range check
in the backend as to if the parameter is in the frame, or
value tracking in the frontend to see if it came from the frame.
mcs/tools/xbuild/data/deniedAssembliesList.txt:
- sorted by name and assembly version
- This file had one duplicate line
System.Text.Encoding.CodePages.dll,FD178CD4-EF4F-44D5-9C3F-812B1E25126B,4,0,2,0
mono/metadata/image.c:
- sorted by hashcode
- This had one duplicate line
IGNORED_ASSEMBLY (0xFA686A38, SYS_TEXT_ENC_CODEPAGES, "FD178CD4-EF4F-44D5-9C3F-812B1E25126B", "4.3.0 net46"),
commit 0b3cb8ac12cd839f8110775d4085e822e8af4d7b (HEAD -> master, origin/master, origin/HEAD)
Author: Zoltan Varga <[email protected]>
Date: Tue May 1 14:52:22 2018 -0400
Emit an LSDA for methods without landing pads too since we need the information about the 'this' register.
to fix generic-stack-traces2.2 -- so it gets a generic type definition
instead of a generid type instance, consistent with other execution modes.
See https://github.com/mono/mono/issues/8356
Backtrace can deadlock with threads that have been suspended while holding a
dynamic linker lock on Linux.
In checked build mode we collect backtraces on state machine transitions.
When we transition from BLOCKING->RUNNING under hybrid suspend the thread must
not block without polling. So we must not collect a backtrace on this
transition.
* Make autogen.sh output accurate -- print actual commands run.
* Have automake copy instead of symlink.
- This sounds analogous to libtool --copy that we do.
- It enables sharing a source tree more broadly.
For real example, Cygwin symlinks are not portable to Windows Subsystem For Linux, and probably vice versa.
For real example, Homebrew symlinks are not portable to Linux in Docker.
Downside I notice is that ls -l is less informative as to what is from git vs. a symlink to outside.
git clean -xxddff works just as well either way.
Hypothetical downside is that updating automake/autoconf doesn't upgrade the copies.
This has never mattered to me, but it might matter. It might be viewed as an advantage.
* [interp] Don't throw exceptions if we are in the middle of EH
Fixes delegate2.exe failures
* [interp] Don't set parent for invoke frame
This leads to unwinding same frame twice (once when going through the parent frame and, after we have no more parent frames, again through the pushed LMF). We should unwind past invoke only through LMF structures.
Fixes https://github.com/mono/mono/issues/8372
* [Coop] Introduce mono_gchandle_new_weakref_from_handle.
* Convert some old code to `mono_gchandle_new_weakref_from_handle`.
* [Coop] Convert ves_icall_Mono_Interop_ComInteropProxy_AddProxy to coop.
* [Coop] Convert ves_icall_Mono_Interop_ComInteropProxy_FindProxy to coop.
* Remove whitespace from whitespace-only line.
* [Coop] Remove 140+ uncooperative pointers due to calls to mono_set_pending_exception ().
Instead typically use:
ERROR_DECL (error); // sometimes already present
mono_error_set_xxx (error, ...);
mono_error_set_exception_pending ();
* Improve error in not supported.
* Allow null in set_error_messagev.
* () and (void) are not the same in C, use (void)
When/if these functions are revisited for further coop conversion, the first and last line might be deleted but the second is likely to survive asis.
Context: https://unix.stackexchange.com/questions/30091/fix-or-alternative-for-mktemp-in-os-x
When making the bumps to use the `mono-sdks` upstream in Xamarin.Android,
we were getting build errors such as:
usage: mktemp [-d] [-q] [-t prefix] [-u] template ... (TaskId:404)
The build bot is running Mavericks (10.9), so it must be an issue with older
versions of macOS.
When reading the Unix StackExchange post, it appears that the `-t` option
without a template should work more reliably on older macOS versions.
Then I tested it locally via `make -C sdks/builds provision-android`.
The command:
mktemp -d unzip_android_archive_XXXXXXXX
Resulted in paths such as:
creating: unzip_android_archive_Nt1XzgS5/android-P/
And sure enough, looking into `sdks/builds` it had created a directory in
the working directory, so it isn't using a path inside of `$TMPDIR`.
Changing the command to:
mktemp -d -t unzip_android_archive_XXXXXXXX
Resulted in paths such as:
creating: /var/folders/nx/rzr6lcqj6j3_vp7vtp28v2cw0000gn/T/unzip_android_archive_W247QkyW/android-P/
The StackExchange post mentions `-t` is a "prefix" that gets expanded to
`$TMPDIR/{prefix}.XXXXXXXX`, so I think it is better to use this option as
long as it works on all platforms.
[Coop] Convert Microsoft.Win32.NativeMethod to coop.
These are all pretty trivial. There are no pointers or handles.
The only small wrinkle is internal calls -- split the function into w/ and w/o MonoError*.
CEE_LDFLD replaces at the top of the stack the object/valuetype one of its fields. MINT_LDFLD_VT loads a valuetype field from the object/valuetype at the top of the stack. When loading a field from a valuetype, at the end we also did a MINT_VTRESULT to pop the vtspace. If previously we were loading also a valuetype, then, when popping the original valuetype, we discard memory from the loaded field. With this change, we use 2 MINT_VTRESULT, one to pop the loaded field and second to copy it in place of the original valuetype.
* [threads] (windows,mach) If a thread self-suspended, initiator shouldn't wait
Under hybrid suspend if the suspend initiator raced with
done_blocking/abort_blocking, then the thread self-suspended and is already
waiting for the resume semaphor - there's nothing else for the suspend initiator
to do, so don't add victim to the pending operation set.
* [profiler] Do GC state transitions in profiler helper threads
After the calls to `profiler_thread_check_detach`, the GC doesn't skip
the profiler helper threads anymore (because of the call to
`mono_thread_info_set_flags (MONO_THREAD_INFO_FLAGS_NONE)`.
Under hybrid or full cooperative GC, the helper threads in GC Unsafe
mode should not block without periodically polling. Otherwise we get
into a deadlock on shutdown when finalizing the root domain: The profiler
threads have been told to detach and we run a GC in preparation for
running the finalizers. The GC STW ends up waiting for the dumper and
writer threads which never self-suspend.
* style - (void) vs. (), favor (void) in C; function names at start of line for grep ^foo
* [Coop] Convert ves_icall_System_Threading_Thread_Resume to coop.
Cleanup three instances of "Thread has not been started, or is dead.".
Move raising exception (on current thread) outside of locking of resuming thread.
Keep lock hold times shorter.
* [Coop] Convert ves_icall_System_Threading_Thread_Interrupt_internal.
Cleanup uses of ThreadStateException.
I added an option to allow storing a "thread dump" of the running program into a file in the specified directory. It is useful when you need to understand what is going on in your application and you cannot capture stdout or attach a debugger.
Replace bounds check assertions of mono_assembly_get_assemblyref (and
everything that it calls) by Bad Image MonoErrors.
Call from System.Reflection.Assembly.InternalGetReferencedAssemblies to cope
with malformed images
* [runtime] If a MonoError was set, display its message.
* [verifier] Cleanup the metadata verifier interface to use MonoError instead of the old GSList hack.
Runs of bumping `code` should be followed by calling set_code_cursor, every so often, or least when done.
Checks for enough room and realloc should be instead realloc_code.
Reads and esp. write of code_len, code_size, native_code should be relatively rare.
* [mini] Don't verify types if you store to a pointer byref. Workarounds #8403.
* [mini] Add regression test for punting a type system check.
* Fix variable names in test to match comments
The only place where the handler thread should block is in the
kqueue_notimeout() call; interrupting the thread should cause
that call to exit with EINTR, and allow the thread to exit.
* [Tailcall] Fix the patching of GOT sites under FullAOT.
This fixes bug https://github.com/mono/mono/issues/8273 where FullAOT generic-shared (valuetype?) runs hundreds of times slower than regular AOT due to hitting the trampoline for every call.
[Coop] Fork test 8273.il from 3.il and increase iteration count
so it takes a long time previously.
Also possibly fix a JIT path where gshared but not gsharedvt handled.
Tailcall:
Remove arm32 limit of 64 bytes of stack usage. Limit it to 4K instead (12bit immediate displacement).
Remove arm64 limit of 0 bytes of stack usage. Limit it to 1G instead (32bit limit).
i.e. copy arm64 stacked parameters
Limit x86 and amd64 stack usage to 1G (32bit limit).
Even as small as 100 is reasonable.
Add OP_TAILCALL_PARAMETER for more accurate earlier tailcall sizing,
including of unbounded parameter lists.
i.e. a certain number of bytes per parameter (including enregistered,
which makes it an overestimate)
Allow arm64 to return r4 -- caller and callee must agree, which means
caller's caller gets what it expects.
Add test coverage for large parameter list (256+).
Cleanup:
Introduce ins_get_size in place of ((guint8 *)ins_get_spec (ins->opcode))[MONO_INST_LEN];
Use it in a few places. Replace PowerPC's ins_native_length with it.
Cleanup usage of EXTRA_CODE_SPACE vs. 16 -- EXTRA_CODE_SPACE everywhere.
This will get further cleaned up shortly.
Replace if (need to realloc code) with while (need to realloc code.
It is almost the same, but protects against degenerate very small code
where doubling is not enough (except for zero).
Use fsig we already have instead of another call to sig = mono_method_signature.
(for OP_TAILCALL and nearby existing code)
There are 4 different ways that an assembly can be loaded in .NET:
1. default context - `Assembly.Load(string)` does this.
2. reflection only context - `Assembly.ReflectionOnlyLoad` and `Assembly.ReflectionOnlyLoadFrom` do this.
2. load from context - `Assembly.LoadFrom(string)` does this. The interesting bit is that the assembly at the specified path is probed for its assembly name, and then binding redirects are applied to figure out if we need to load a different version of the assembly (from elsewhere in the search path, presumably).
3. "no context" (.NET Core's [AssemblyLoadContext](https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/assemblyloadcontext.md) design doc calls it "custom context", and internally they implement it as `IndividualAssemblyLoadContext`) - in Mono I'm going to call it the "individual context". This is what `Assembly.LoadFile(string)` and `Assembly.Load(byte[])` do: you get exactly the thing you asked for even if something with the same assembly name is already loaded and you get to have problems with type resolution.
So this PR adds:
1. `MonoAssemblyLoadContextKind` enum - default, refonly, loadfrom and individual.
2. Store the `MonoAssemblyLoadContext` in every `MonoAssembly` instead of a boolean `refonly` flag.
3. When loading references from assemblies, consider the load context of the requesting assembly.
4. Do the wacky `LoadFrom` probe and rebind dance.
5. Don't check for duplicated for the individual assembly context.
Not in this PR:
The managed assembly load context stuff in .NET Core. If we eventually want to add it, it will likely build on this work, but it's not here now.
Details:
* [loader] Replace 'refonly' by an assembly load context (NFC)
Replace MonoAssembly:ref_only field by a new context field of type MonoAssemblyContext (which right now just contains a single 'ref_only' boolean).
The .NET Framework documentation describes for contexts for assembly loading:
1. default context - Load(string) and referened assembly loading use this context.
2. LoadFrom context - LoadFrom(string) and references from LoadFrom assemblies.
3. reflection only context - ReflectionOnlyLoad and ReflectionOnlyLoadFrom use this.
4. individual assembly context[footnote:1] - Used by Load(byte[]) and LoadFile(string)
This commit only implements default context and reflection only context. As such, there should be no functional change.
[footnote:1]: Individual assembly context is called "no context" in the .NET
Framework docs, but "individual" is what the .NET Core assembly load context
implementation calls it.
https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/assemblyloadcontext.md
* [loader] propagate MonoAssemblyContextKind instead of refonly, loader_from_context flags
Instead of passing a pair of booleans around, pass a MonoAssemblyContextKind value.
In System.Reflection.Assembly.LoadFrom, pass MONO_ASMCTX_LOADFROM, instead of default.
In System.Reflection.Assembly.Load(byte[]), pass MONO_ASMCTX_INDIVIDUAL instead of default.
In SRE, create new assemblies with MONO_ASMCTX_INDIVIDUAL or refonly
Finally in mono_assembly_load_reference:
For a LOADFROM context always include the requesting assembly's basedir when probing for references.
For INDIVIDUAL context, don't probe for their references - they must be already loaded or else resolved by an AssemblyResolve event.
In a REFONLY context, only look at loaded assemblies or fire the
ReferenceOnlyAssemblyResolve event.
In a DEFAULT context, follow Mono's existing strategy: first probe without
including the basedir (and let the managed AssemblyResolve event fire), then
probe again with the basedir set (potentially firing the event again).
* [reflection] Implement System.Reflection.Assembly.LoadFile as a separate icall
The difference between LoadFrom and LoadFile is that LoadFile should not look in the loaded assembly's basedir when resolving references. (It is also not be subject to remapping).
Also, for LoadFile never return a previously loaded assembly even if one with the same name is already cached by a domain.
Fixes https://github.com/mono/mono/issues/8149
The changes to mono_assembly_load_from_predicate (in conjunction with using MONO_ASMCTX_INDIVIDUAL) for the LoadAssemblyRaw icall also fix https://bugzilla.xamarin.com/show_bug.cgi?id=11199 and https://bugzilla.xamarin.com/show_bug.cgi?id=33728
* [loader] Apply binding redirects to LoadFrom assemblies
Fixes https://github.com/mono/mono/issues/8152
* [loader] Include class failure cause in find_method_in_class TypeLoadException
* [loader] apply binding redirects to LoadFile() and Load(byte[])
Same as LoadFrom: If the identity of the assembly matches a binding redirection, drop the currently opened image and go probing by name instead.
* [loader] Print assembly load context in debug traces
* [loader] Split up mono_assembly_load_full_nosearch
There are two phases: first try the preload hook, second look in the gac, the given base directory, and in the default_path.
Split out the second phase into its own function.
* [loader] Apply remapping and binding when looking for INDIVIDUAL asmctx references
* [loader] Probe the GAC and assemblies path for INDIVIDUAL asmctx
Checked on .NET Framework 4.7.1
Despite the documentation insisting that `LoadFile` does not do any probing for referenced assemblies, it seems that actually, assemblies from the GAC (or at least the framework Facade assemblies such as `System.Runtime`) are nonetheless loaded if no context assembly depends on it.
This commit does the same thing for Mono (except we will additionally look in MONO_PATH).
This makes the xunit console runner work:
At startup they call LoadFile() on all the assemblies in the main executable's dir without doing anything to resolve the references. Those LoadFile assemblies then have broken references to things like System.Runtime that should be in the GAC but that haven't been loaded yet.
* [tests] Add tests for Assembly.Load(byte[]), LoadFile and LoadFrom
Test that loading two assemblies from two different paths (or byte arrays) with the same AssemblyName will produce different results if using Load(byte[]) or LoadFile() but will reuse the first assembly if using LoadFrom().
Regression tests for:
- https://bugzilla.xamarin.com/show_bug.cgi?id=11199
- https://bugzilla.xamarin.com/show_bug.cgi?id=33728
- https://github.com/mono/mono/issues/8149
* [tests] Check that binding redirects work for Assembly.Load(byte[]), LoadFile and LoadFrom
In old version of .NET it was only supposed to work for LoadFrom, but now it should work for all three.
Regression test for https://github.com/mono/mono/issues/8152
* [tests] Don't run Assembly.Load(byte[]) tests with AOT
* [aot] Make load_image return the aot module assembly for LoadFile
LoadFile allows loading more than one assembly with the same name. That means that just calling mono_assembly_load is unlikely to find the correct assembly (it'll just return the first one loaded). But we shouldn't hardcode the full assembly filepath into the AOT image, so it's not obvious that we can call mono_assembly_open_predicate.
Fortunately, an assembly opened with LoadFile is supposed to only refer to already-loaded assemblies (or to GAC & MONO_PATH) assemblies - so nothing new should be loading. And for the LoadFile'd assembly itself, we can check if the name and guid of the current AOT module matches the wanted name and guid and just return the AOT module's assembly.
* [mini] Fixup one MonoClass member access to use getter
* If check_call_signature fails and we break on it, run it again, i.e. so you can
step in a debugger.
* Change check_call_signature to returning a boolean, albeit in the same perhaps backwards way.
* [arm] Create trampoline for passing fixed arguments
It is the same as static rgctx trampoline, but it clobbers ARMREG_IP
* [arm] Create trampoline for native to interp transition
* [arm] Add interp entry from trampoline
Tweak CallContext methods to work in opposite direction
* [interp] Create wrapper that saves lmf for m2i transitions
If we want to unwind from interpreter directly into managed code we need to have a wrapper whose purpose is to save the lmf, so we can start unwinding frame by frame.
* [arm] Add unwind info for interp entry trampoline
We need to unwind through this trampoline since it sits between jitted code and the interp lmf wrapper.
* [interp] Let the unwinder unwind through trampolines
These frames are still ignored (by EH and stack walking), they are only used for unwiding further to the caller.
[Coop] Abort_internal and ResetAbortNative to coop/handle.
* [Coop] Add some comments.
* [Coop] Convert thread abort and interruption to coop/handle.
* [Coop] Convert ves_icall_System_Threading_Thread_ResetAbort to coop/handle.
* Make mono_set_pending_exception_handle static.
* [Coop] mono_join_uninterrupted: Avoid creating handles in loops.
If we have
`catch (Exception e) when (SomeMethod (e))`
.. then the exception object received in `SomeMethod` has an empty
trace. This is because in `handle_exception_first_pass` we populate the
trace in `MonoException` object only at the "end":
- filter returns TRUE
- exception caught
- unhandled exception
But if we have a filter then the trace at that point needs to be
accessible. And if the filter fails then as we unwind more frames, the
earlier frames (`trace_ips`) need to be retained to get the correct full
trace.
`dynamic_methods` needs to be handled similarly and in
`setup_stack_trace` we need to add to the existing `dynamic_methods` in
the MonoException object.
Fixes https://github.com/mono/mono/issues/7649 .
* [Tailcall] Print more. Clarify printing. Dial back a fix by one line
until a test demonstrates it is required.
Move debug_tail up for easier editing.
* [Tailcall arm64] Add tailcall_reg support.
* Fix new warning about maybe-used-uninitialized -- moving variables down is incompatible slightly with goto around them.
* [runtime] Handle failed classes in mono_type_get_checked (). Fixes https://github.com/mono/mono/issues/8311.
* [runtime] Use m_class_has_failure ().
* [runtime] Set the error if the class has a failure.
* [runtime] Add a signal handler for sigterm
* [runtime] Register sigtramp when merp handler enabled
* [runtime] Limit sigterm handler symbol definition to OSX
* [Tailcall] Use tailcall_reg to tailcall internal `calli` to improve F# FullAOT tests.
Rework printing a bit and add more.
Split mini_emit_calli into mini_emit_calli and mini_emit_calli_full.
mini_emit_calli_full takes an extra tailcall parameter.
This enables fixing some F# tests.
`check_sp` inhibits tailcall -- share logic and assert -- in order
to only call `is_supported_tailcall` once.
mono_emit_call_args is big and probably shouldn't be inlined; let compiler decide
anyway, which it always does.
Significant reworking of is_supported_tailcall.
Have it compute "regular" and "calli".
Replace is_interface with imt_arg which should be a more accurate
representation of the important part.
Capture complex conditionals `virtual_generic` and `virtual_generic_imt`
into local variables, so they can be evaluated a second time,
so that `is_supported_tailcall()` can be called just once with its
final inputs.
Move `tailcall` and `tailcall_calli` `tailcall` down
to initialization and make them const (with a temporary/internal non-const).
Note that tailcall still gets changed/reevaluated in multiple places,
and the new value is not returned. One such duplication is the form
of shared logic and an assert (check_sp), the other (calli + llvmonly) is not.
The correct value is needed, sometimes, for e.g. remove_ret.
There also remains a path that appears to change it to false but true in `ins`.
* [interp] fix warning
```
interp/interp.c: In function ‘ves_real_abort’:
interp/interp.c:251:17: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘int’ [-Wformat=]
printf ("\t[%ld] 0x%08x %0.5f\n", sp-stack, sp[-1].data.i, sp[-1].data.f);
~~^ ~~~~~~~~
%d
```
* [mini] ignore GCC warning in trace.c
```
trace.c: In function ‘mono_trace_enter_method’:
trace.c:145:3: warning: calling ‘__builtin_return_address’ with a nonzero argument is unsafe [-Wframe-address]
printf (") ip: %p\n", MONO_RETURN_ADDRESS_N (1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
* [tramp-x86] remove unused function
```
tramp-x86.c:171:1: warning: ‘get_vcall_slot_addr’ defined but not used [-Wunused-function]
get_vcall_slot_addr (guint8* code, mgreg_t *regs)
^~~~~~~~~~~~~~~~~~~
```
* [mini-x86] remove unused function
```
mini-x86.c:6327:1: warning: ‘emit_load_aotconst’ defined but not used [-Wunused-function]
emit_load_aotconst (guint8 *start, guint8 *code, MonoCompile *cfg, MonoJumpInfo **ji, int dreg, int tramp_type, gconstpointer target)
^~~~~~~~~~~~~~~~~~
```
* [tramp-x86] remove unused function
```
tramp-x86.c:146:1: warning: ‘get_vcall_slot’ defined but not used [-Wunused-function]
get_vcall_slot (guint8 *code, mgreg_t *regs, int *displacement)
^~~~~~~~~~~~~~
```
* [mini-x86] change previous_lmf to pointer type (same as amd64)
```
mini-runtime.c: In function ‘mono_push_lmf’:
mini-runtime.c:806:24: warning: assignment makes integer from pointer without a cast [-Wint-conversion]
ext->lmf.previous_lmf = *lmf_addr;
^
mini-runtime.c:808:24: warning: assignment makes integer from pointer without a cast [-Wint-conversion]
ext->lmf.previous_lmf = (gpointer)(((gssize)ext->lmf.previous_lmf) | 2);
^
mini-exceptions.c: In function ‘arch_unwind_frame’:
mini-exceptions.c:420:11: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
*lmf = (MonoLMF *)(((guint64)(*lmf)->previous_lmf) & ~3);
^
mini-exceptions.c: In function ‘unwinder_unwind_frame’:
mini-exceptions.c:715:13: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
*lmf = (MonoLMF *)(((guint64)(*lmf)->previous_lmf) & ~7);
^
```
* [monolmf/x86] make int cast dependend one machine word size
```
In file included from debugger-agent.c:36:0:
debugger-agent.c: In function ‘get_top_method_ji’:
debugger-agent.c:2501:14: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
g_assert (((guint64)lmf->previous_lmf) & 2);
^
../../mono/eglib/glib.h:710:46: note: in definition of macro ‘G_UNLIKELY’
#define G_UNLIKELY(expr) (__builtin_expect ((expr) != 0, 0))
^~~~
debugger-agent.c:2501:3: note: in expansion of macro ‘g_assert’
g_assert (((guint64)lmf->previous_lmf) & 2);
^~~~~~~~
```
* [interp] remove unnecessary code
* [mini] fix ptr with int cmp
this was wrongly changed.
```
mini-exceptions.c: In function ‘unwinder_unwind_frame’:
mini-exceptions.c:702:33: warning: comparison between pointer and integer
if (unwinder->last_frame_addr > (gsize)(*lmf)) {
^
```
This include breaks compiling the interpreter on platforms which don't have a
signal.h. The code in interp.c doesn't seem to need it to be included anyway.
* [wasm] Fix CI target.
* [wasm] Switch to do a local install of npm packages.
* [wasm] Switch to fine-grained execution steps to avoid VM bugs.
* [wasm] Disable System.Core under Chakra as the JS engine crashes.
* [Tailcall] Remove OP_JMP support. Require OP_TAILCALL.
This only hurts mips and sparc -- it breaks them.
Remove front end support for OP_JMP and conditionalality of OP_TAILCALL.
* And remove MONO_ARCH_HAVE_OP_TAILCALL and require mono_arch_tailcall_supported.
* And tailcall_remove_ret becomes temporarily irrelevant.
* [Tailcall] Keep mips and sparc working, albeit never tailcalling.
* Remove jmp opcode too.
for F# FullAOT.
Add some FIXMEs about alignbment and possible
improvements (fixed size rep mov to move tailcall parameters).
Grow static unknown sizes to 255.
* [sdks] Wrap configure to provide better output.
Wrap configure in a script that:
* Captures stdout/stderr and only shows it in case of errors.
* If configure fails, and a cache file was used, deletes the cache file and
tries again.
* If configure really fails, then print config.log in addition to
stdout/stderr, since a lot of valuable diagnostic output is in that file
(and it's annoying to track down configure failures on bots when the file
with the important output has already been deleted).
* Only show config.log when running on bots.
* [System.ServiceMode] Create unit test for #46917
* [System.ServiceModel.Discovery] Fix messages validation
The current validations are not correct and throws an exception calling
GetVideoSources API in an ONVIF media client. This API
is a combination of a first message with an input parameter and
a second message with the reply that is not void.
Removing this checks makes the client work correctly.
Fixes#46917
* [threads] Check that thread is in a valid suspend state before notifying the initiator.
* [threads] Suspend running threads cooperatively in coop or hybrid suspend
Reverts an inadvertant change from 282ee4bfd7
* [checked-build] Add a mutex in collect_backtrace
GNU libc backtrace() calls _Unwind_Backtrace in libgcc which internally may
take a lock.
That allows for a deadlock situation where two threads try to call collect_backtrace and
one of them takes the lock and is async suspended with the lock held, while the
other is supposed to cooperatively suspend but it is blocked waiting for the
backtrace mutex.
The suspend initiator ends up waiting indefinitely for the threads to suspend,
and the runtime hangs.
As a workaround we add a mutex around the call to backtrace() and only collect
a backtrace if we can immediately grab the lock without contention.
* [threads] Drop DoneBlockingNotifyAndWait & AbortBlockingNotifyAndWait
These two actions were causing a race in hybrid suspend.
Suppose thread T is in blocking and thread S (suspend initiator) wants it to
suspend:
S sets T to BLOCKING_SUSPEND_REQUESTED and sends a suspend signal and waits for
pending operations.
T continues running and comes to exit the GC Safe region.
T executes done_blocking and transitions to BLOCKING_SELF_SUSPENDED and returns
DoneBlockingNotifyAndWait.
T notifies S; T waits for a resume.
S wakes up, does whatever it needs to do and resumes T - T goes from
BLOCKING_SELF_SUSPENDED to RUNNING.
T wakes up and finishes the done blocking transition.
Suspend signal finally arrives in T. T attempts finish_async_transition from
RUNNING. Assertion. Crash.
What we do in this commit is change done blocking not to notify the initiator.
Instead done blocking immediately waits for a resume.
In that case T is waiting for a resume in BLOCKING_SELF_SUSPENDED, and S is
waiting for pending operations. So when T's suspend signal handler runs, it
will return FALSE (raced with self suspend), send a notification to the
suspend initator, and then immediately return. T then will continue waiting for
the resume. S will wake up, do what it needs to do and switch T to RUNNING and
post the resume to wake it.
class-inlines.h:13:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
mono_get_void_type ()
In C: () means accept any parameters -- K&R unprototyped
(void) is new-standardized-in-1989 form meaning accept no parameters
In C++ this is fixed and they mean the same thing -- accept no parameters.
To accept any parameters in C++, use (...), which is not valid in C, C requires
at least one fixed-type parameter when you use "...".
() is very rarely what you want in C, or in C++ headers consumable from C.
If you do need () for highly unusual code, for a function that accepts anything,
and has no fixed parameters, then use ifdef __cplusplus and ... for C++.
printf for example does not fit in this bucket because it always has the const char *format.
In fact, it is so rare because a variable-argument ("varargs") function would need *some* way to decide what is in "...".
* Teach sn.exe about PE32+, and while there ROM images and MS-DOS-less.
See https://github.com/mono/mono/issues/8218.
* Make Error static.
* Remove .Close () redundant with using.
* Use helper function ReadMore and System.Array.Resize.
* Fix typos in comment.
* Move hash.Hash to fix 'Hash must be finalized before the hash value is retrieved.'.
* Cleanup.
* Fix Automatic Proxy Configuration protocol for OSX 10.13
Reference comparison of kCFProxyTypeAutoConfigurationURL doesn't work in 10.13. Added fallback to do value/string comparison to get proxy type
* Change string comparison from C# string to CFString for possible memory safety and efficiency
* Code style changes
The previous code would produce
```diff
-public const string Version = "11.11.0";
+public const string Version = "11.9.1";
```
instead of
```diff
-public const string Version = "11.9.1";
+public const string Version = "11.11.0";
```
when diff are inlined. The addition and removal sections are fine.
* [wasm] Fix build scripts to work on linux.
* [wasm] Force emscripten version to 1.37.36 to fix OSX breakage.
* [wasm] Fix failure in new FloatF test.
* [wasm] Fix detection of test failure.
commit message said
> [w32file-unix] convert nano seconds to microseconds properly, by using
> TICKS_PER_MILLISECOND (which has L suffix in its definition)
first it talks about microseconds, but then uses constant for _milli_
seconds
* [runtime] Add public API to set the pending exception.
Xamarin.iOS and Xamarin.Mac need a public API to set the pending exception
when converting Objective-C exceptions to managed exceptions.
This adds a public API to support this scenario.
Partially fixes https://bugzilla.xamarin.com/show_bug.cgi?id=60065.
* [runtime] Document that pending exceptions should not be overwritten when the runtime is handling a thread abort request.
* [threads] Have Hybrid Suspend not depend on Coop Suspend
* [threads] Enable CI
* [hybrid suspend] Wait for pending operations when suspending a blocking thread
In hybrid suspend we should wait for the pending suspend to finish when we're
suspending a blocking thread.
In full cooperative suspend, we treat blocking threads as already suspended, so
in that case there is nothing to wait for.
* [checked-build] don't use malloc for thread state history
Use a ring buffer, so we're signal safe.
* [checked-build] checked_build_thread_transition - async signal safe variant
Can't call backtrace on Linux from a signal handler.
Change mono_threads_transition_finish_async_suspend not to collect a
backtrace.
* [class] Rename MonoClass byval_arg field to _byval_arg
Update all use sites to use the getter method m_class_get_byval_arg instead
when possible.
* [runtime] add mono_get_*_type accessors for common MonoTypes
Just do void, object, int32 and int (aka IntPtr) for now
Refactor g_print on top of new g_printv. g_log / g_logv provided precedent.
The intent is like:
void foo_print (const char *format, ...)
{
if (debug_foo || MONO_IS_TRACED (foo, log)) {
va_list args;
va_args (format, args);
g_printv (format args)
}
}
so I can either set the environment variable, or edit the global,
besides removing the two parameters from every mono_trace call.
* [jit] Enable float32 by default.
* [arm64] Fix some 32 bit float compares so they handle unordered operands correctly.
* [runtime] Disable a coreclr test which fails with float32.
* [FileTest] regression for https://github.com/xamarin/xamarin-macios/issues/3929
* [w32file-unix] fix File.Copy timestamps
4a8d840d6f introduced that the destination
file of a copy operation has the same modification and access timestamps
as the source file.
With e0e185d70a this behaviour regresses,
since getting timestamps provides more precise information. Fixing this
by being more accurate when applying timestamps to destination file.
* [w32file-unix] convert nano seconds to microseconds properly, by using TICKS_PER_MILLISECOND (which has L suffix in its definition)
* [w32file-unix] introduce constant for ns->qs conversion with proper suffix
fixes problem on 32bit.
* [jit] Add a r8->r4 conversion in stind.r4 when the item on the stack is a r8 because of stack merging.
* [jit] Disable a SIMD test which can fail with -O=float32 due to precision problems.
* [jit] Use m_class_get_byval_arg ().
* [jit] Fix OP_RCONV_TO_I on amd64.
* [jit] Revert '[jit] Use r8 for r4 stack vars to avoid problems with stack merges when one side is an r4 and the other an r8. Add conversions for compares. (#8196)'. It causes r4-r4 merges to produce an r8 stack value, leading to invalid il errors.
* [Mono.Options] Allow CommandSet nesting, spaces in command names.
`git` (largely) only provides a single level of sub-commands, and
this is the use case that `Mono.Options.CommandSet` originally
supported.
However, this isn't the case for all command-line applications.
Take for example the Azure [`azure`][azure] command, which allows near
arbitrary command name depths:
[azure]: https://docs.microsoft.com/en-us/azure/virtual-machines/azure-cli-arm-commands
$ azure config mode arm
$ azure vm create
$ azure vm list
$ azure vm disk attach
Try to integrate "arbitrary" command nesting through two new
mechanisms:
1. `Command` names may now contain spaces, and
2. `CommandSet`s may be nested within `CommandSet`s.
`Command` names containing spaces allows for the same *style* of
"nested" sub-commands, but means that C# nesting isn't as deep:
var azure = new CommandSet ("azure") {
new Command ("config mode") {...},
new Command ("vm create") {...},
new Command ("vm list") {...},
new Command ("vm disk attach") {...},
};
When `CommandSet.Run()` is invoked, separate string indexes will be
"merged" while attempting to find a matching command-name, allowing
the "natural" command-line invocation to be used:
mono example.exe vm disk attach
# Invokes `vm disk attach` command
Alternatively, a `CommandSet` may be added to a `CommandSet`:
// C#
var azure = new CommandSet ("azure") {
new CommandSet ("vm") {
new Command ("create") {...},
new Command ("list") {...},
},
};
When this is done, `CommandSet.Suite` is required on the command-line:
mono example.exe vm list
If a "`Command` with spaces" has the same command-name as a nested
`CommandSet`, the `Command` "wins" and is invoked.
* [Mono.Options] Fix unsupported MCS compiler feature
* Bump API snapshot submodule
This brings the patch: https://github.com/dotnet/coreclr/pull/6898
Fail faster for `Wait(0)` when no counts available.
Don't allocate in the `Wait(0)` and `WaitAsync(0)` fail paths.
1M iters Server GC
method | pre (ms) | post (ms) | improvement
-------- |------- |------- |------ |
Wait(0) | 33,788.25 | 12.28 | x 2751.5
WaitAsync(0) | 360.79 | 24.38 | x 14.8
Memory allocations, saves 360MB over the 1M ops
* [jit] Use r8 for r4 stack vars to avoid problems with stack merges when one side is an r4 and the other an r8. Add conversions for compares.
* [jit] Use m_class_get_byval_arg in one place.
* Add a FIXME.
* [runtime] Get rid of the GLIB_LIBS autoconf define, it prevents automake from detecting the dependency on libeglib.la. Fixes#8175.
* [runtime] Link eglib against -lm to avoid duplication.
* [runtime] Remove a few extra -lm options.
* [runtime] More GLIB_LIBS fixes.
In interp we implement the leave statement by enqueuing finally ips (and the destination ip that we are leaving to) that we call as we exit finally blocks. When throwing an exception from a finally block we need to flush these ips, so we don't jump back into clauses that were already skipped by EH.
* [runtime] Make GetFrames consider nested exceptions
Fixes#7260
* [runtime] Add test from repro for #7260
* [runtime] FIx test using absolute line numbers
* [ci] Add {,hybrid,full}aot_llvm presets to make sure we compile with LLVM
* [configure] Rename runtime_preset to runtime-preset
* [bcl] Make sure we build the testing_aot_{hybrid,full} profiles when building all profiles
* [test-runner] Factor test-runner parameters for picking up and passing arguments to the runtime
* [hybridaot] Fix testing_aot_hybrid build
* [mini-llvm-aot] Fix common race condition in parallel builds due to common defaults.
When a connection fails, the socket error was being incorrectly reported
as an address family not supported error, instead of connection refused.
Why did this happen? The order of addresses returned from
`Dns.GetHostAddresses` changed. The newer code returns the IPv6 address
first, and the IPv4 address second. While `GetCheckedIPs` mentioned in a
comment that it skips addresses that don't match the address family, it
did not. The code using the addresses returned by `GetCheckedIPs` will
use the first address in the array. If this address happens to match the
address family of the socket, things will work "properly" (the correct
socket error will occur).
If, however the first entry has the wrong address family, then the
socket error will be an address family not supported error, since the
socket was created with different address family.
This change modifies `GetCheckedIPs` to filter the addresses, removing
those which do not match the address family.
[coop] Hybrid suspend
## Summary
This is a new suspend mode where threads that are running managed or native runtime code (what we call "GC Unsafe") are suspended cooperatively using safepoints, but threads running either embedder code or P/Invoke code or blocking syscalls (what we call "GC Safe") are suspended preemptively using signals.
The motivation is to make an embedder-friendly coop suspend mechanism where the runtime will do GC Safe -> GC Unsafe transitions on calls to `MONO_API` functions, but the embedders themselves don't have to have any special knowledge of coop safepoints or state transitions.
To try this out, set **both** `MONO_ENABLE_COOP` and `MONO_ENABLE_HYBRID_COOP` environment variables.
## Implementation
Here's a picture of the new state machine. There's two new states `Blocking_Suspend_Requested` and `Blocking_Async_Suspended` and one updated transition `req_s` (request suspension).

### State machine states and transitions
The hybrid cooperative suspend mechanism works like this:
- when a thread is executing in GC Unsafe mode, we use cooperative suspend and expect
the thread to periodically checkpoint its execution (as in the ordinary full
coop mode).
- when a thread is executing in GC Safe mode, we use a preemptive signal-based
suspend. This is new - previously in full coop we would allow `BLOCKING`
threads to continue executing and only suspend them when they wanted to go
from GC Safe to GC Unsafe (via the `BLOCKING_SELF_SUSPENDED` state).
There are two new states and one updated transition - the transition will
service both running and blocking threads: The idea is that suspend mechanism
is determined by the suspend initiator, and is not embodied in the the state
machine. The state machine just has to return distinctive enough values from
the transitions for the initiator to select a policy.
Resume mechanism is determined by the state that a thread finds itself in when
it is resumed, and the original suspend policy.
The primary differences between suspend mechanisms are: the preemptive
suspend is two phase - the `finish_async_suspension` does not
apply for cooperative suspend; the cooperative policy uses the `poll` transition when the thread is running.
We renamed `mono_threads_transition_request_async_suspension` to `mono_threads_transition_request_suspension`.
- It must be called by a suspend initiator on a victim thread that is not
itself.
- There can only be one suspend initiator for a victim at a time - if the
suspend initiator initiates a suspension it must follow through with the
whole protocol.
- If a victim is `RUNNING` we transition it to `ASYNC_SUSPEND_REQUSTED` and return
`ReqSuspendInitSuspendRunning` which signals that the caller must initiate
suspension for a running thread. (Same as the old `AsyncSuspendInitSuspend`).
- If a victim is `BLOCKING` we transition it to `BLOCKING_SUSPEND_REQUESTED`.
Note that this is different from the old `request_async_suspension` which just
incremented the suspend count and returned `AsyncSuspendBlocking`. Now we
return `ReqSuspendInitSuspendBlocking`. The return of
`ReqSuspendInitSuspendBlocking` means the initiator may signal the victim and
must wait to be notified of the suspension (in preemptive suspend it will be
notified by the signal handler, in cooperative when the blocking thread
attempts to exit from blocking mode).
- In `BLOCKING_SUSPEND_REQUESTED` we
may increment the suspend count. This is slightly too lax if we're going to
be using preemptive suspend on blocking threads: in that case we're in the
middle of a two-phase suspend and since there is only one suspend initiator,
we wouldn't expect another suspend to be initiated until we have a chance to
finish suspending. We leave it to the suspend initiator to rule out by returning
`ReqSuspendAlreadySuspendedBlocking` from this state.
- In all the not executing states (`SELF_SUSPENDED`, `ASYNC_SUSPENDED`,
`BLOCKING_SELF_SUSPENDED`, `BLOCKING_ASYNC_SUSPENDED`) we just
increment the suspend count.
- All other transitions are illegal.
- Note that `BLOCKING` must always have a suspend count of 0 and
`BLOCKING_SUSPEND_REQUESTED` must have suspend_count > 0.
We add two new states:
- `STATE_BLOCKING_SUSPEND_REQUESTED` - this is a new state that indicates that a
suspend initiator wants to suspend this victim thread.
- The thread is still executing in this state.
- We only transition into this state with
`mono_threads_transition_request_suspension`.
- The only legal transitions out from this state are:
- `resume` - decrements the `suspend_count`. If the `suspend_count` becomes 0 we
go back to `BLOCKING`, otherwise we stay in `BLOCKING_SUSPEND_REQUESTED`.
- `finish_async_suspension` (executed by the suspend signal handler to
finish the two phase preemptive suspend request and notify the suspend initiator),
- `done_blocking` and `abort_blocking`. It means that the a
suspend initiator requested a suspend but we got to done (or abort)
blocking before the signal was delivered. We return `NotifyAndWait` and the
caller is supposed to send a notification to the suspend initiator and
wait for a resume(with suspend count == 0) which will return to
`RUNNING` (the victim thread goes to `STATE_BLOCKING_SELF_SUSPENDED`).
- `STATE_BLOCKING_ASYNC_SUSPENDED`
- The thread is not executing in this state (it is waiting for a `resume`).
- a thread transitions into this state from `STATE_BLOCKING_SUSPEND_REQUESTED`
on a `finish_async_suspension` transition (see above).
- on a `resume` transition we decrement the suspend count. if it's still
positive we stay in this state. if it's 0 the thread transitions out of
this state. Unlike other resume transitions, in this case the thread goes
back to `BLOCKING` and resumes executing in GC Safe mode. The resume caller
must notify the resume initiator (ie this is a `ResumeInitAsyncResume`).
- on a `request_suspension` we stay in this state and just increment the
suspend count.
- all other transitions from this state are illegal.
We add two new return values for `abort_blocking` and `done_blocking`:
- `DoneBlockingNotifyAndWait` and `AbortBlockingNotifyAndWait`. Similar to
`SelfSuspendNotifyAndWait` - there was a race between the second phase of a
preemptive suspend and another operation (in that case a poll, in this case a
done or abort blocking) and the other operation won, so its caller should
notify the suspend initiator. Since the thread is now suspended, the caller
should wait for the resume signal.
We add new results for `request_initiate_suspension` `MonoRequestSuspendResult`
(formerly `MonoRequestAsyncSuspendResult`):
- `ReqSuspendAlreadySuspended` means the thread is not executing and we just
incremented its suspend count and there's nothing else for the suspend
initiator to do. (The old `AsyncSuspendAlreadySuspended`)
- `ReqSuspendAlreadySuspendedBlocking` means the thread is in
`BLOCKING_SUSPEND_REQUESTED` and we initiated another suspend request. This
should only happen with full cooperative suspend - with hybrid suspend we
will use a two phase preemptive suspend on a blocking thread and since only
a single suspend initiator is active, this state should not be visible to
another suspend initiator.
- `ReqSuspendInitSuspendRunning` - Thread is executing in GC Unsafe mode and the
caller should suspend it. This is the old `AsyncSuspendInitSuspend`. In
full preemptive suspend this is the only suspend initiation action and the caller
should begin the preemptive suspend procedure. In full coop we expect the
victim thread to reach a safepoint and poll to finish suspending.
- `ReqSuspendInitSuspendBlocking` - Thread is executing in GC Safe mode and
the caller should initiate a suspend. In full coop the caller has nothing
to do - a thread executing in blocking mode is assumed to be suspended. In
hybrid suspend, the caller has to initiate a preemptive suspend.
### Suspend initiator, signal handler, self suspender
Basically we call `mono_threads_transition_request_suspension` instead of `mono_threads_transition_request_async_suspension`.
Then if it says to suspend a running thread we call `begin_suspend_for_running_thread`, if it says initiate a suspension of a blocking thread, we call `begin_suspend_for_blocking_thread`, and if it says do nothing we do nothing.
**Questions**
1. We have an assumption in `safe_interrupt_thread` that we can return from a self interrupt if we're using coop suspend. I'm not sure if that makes sense for hybrid suspend, and not clear what to do about it.
2. In `mono_thread_info_safe_suspend_and_run` we expect that the callback won't return `KeepSuspended` if coop is enabled. I'm not sure what this means for hybrid.
3. A bunch of tests like `monitor-abort` and `appdomain-threadpool-unload` are failing if I enable hybrid suspend. Don't know why yet.
* fix ThreadPool.SetMinThreads fails if ThreadPool.GetMaxThreads was not called before #7988
* check mono_lazy_initialize in ves_icall_System_Threading_ThreadPool_SetMaxThreadsNative first
[console-unix] retry `tcsetattr` if it was interrupted by signal
if configuring the terminal failed, we still try to continue (see
a3ce2f008b ) and therefore succedings
`read`s can fail.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=26983
Change begin_cooperative_suspend and begin_preemptive_suspend to return one of
three results: suspend fail, suspend succeeded using cooperative suspend,
suspend succeeded using preemptive suspend.
This is used by check_async_suspend to decide whether to check the
MonoThreadInfo for the result of a suspend. (If a thread is suspended
cooperatively, it doesn't make sense to check).
Fixes sporadic failures in mono/tests/monitor-abort.exe
When the environment variable is set, Mono will use cooperative safepoint
suspend for threads running in GC Unsafe mode (running managed code or native
runtime code), and preemptive suspend for threads running in GC Safe
mode (running native embedder or P/Invoke code, or blocking system calls)
The environment variable is visible through
mono_threads_is_hybrid_suspension_enabled ().
This commit just adds the env var, it doesn't turn on hybrid coop suspend.
The hybrid cooperative suspend mechanism works like this:
- when a thread is executing in GC Unsafe mode, we use cooperative suspend and expect
the thread to periodically checkpoint its execution (as in the ordinary full
coop mode).
- when a thread is executing in GC Safe mode, we use a preemptive signal-based
suspend. This is new - previously in full coop we would allow BLOCKING
threads to continue executing and only suspend them when they wanted to go
from GC Safe to GC Unsafe (via the BLOCKING_SELF_SUSPENDED state).
There are two new states and one updated transition - the transition will
service both running and blocking threads: The idea is that suspend mechanism
is determined by the suspend initiator, and is not embodied in the the state
machine. The state machine just has to return distinctive enough values from
the transitions for the initiator to select a policy.
Resume mechanism is determined by the state that a thread finds itself in when
it is resumed, and the original suspend policy.
The primary differences between suspend mechanisms are: the preemptive
suspend is two phase - the finish_async_suspension transition only does not
apply for cooperative suspend; the cooperative policy uses the poll transition
when the thread is running.
We renamed mono_threads_transition_request_async_suspension to mono_threads_transition_request_suspension.
- It must be called by a suspend initiator on a victim thread that is not
itself.
- There can only be one suspend initiator for a victim at a time - if the
suspend initiator initiates a suspension it must follow through with the
whole protocol.
- If a victim is RUNNING we transition it to ASYNC_SUSPEND_REQUSTED and return
ReqSuspendInitSuspendRunning which signals that the caller must initiate
suspension for a running thread. (Same as the old AsyncSuspendInitSuspend).
- If a victim is BLOCKING we transition it to BLOCKING_SUSPEND_REQUESTED.
Note that this is different from the old request_async_suspension which just
incremented the suspend count and returned AsyncSuspendBlocking. Now we
return ReqSuspendInitSuspendBlocking. The return of
ReqSuspendInitSuspendBlocking means the initiator may signal the victim and
must wait to be notified of the suspension (in preemptive suspend it will be
notified by the signal handler, in cooperative when the blocking thread
attempts to exit from blocking mode).
- In BLOCKING_SUSPEND_REQUESTED we may increment the suspend count. This is
slightly too lax if we're going to be using preemptive suspend on blocking
threads: in that case we're in the middle of a two-phase suspend and since
there is only one suspend initiator, we wouldn't expect another suspend to
be initiated until we have a chance to finish suspending. We leave it to
the suspend initiator to rule out by returning
ReqSuspendAlreadySuspendedBlocking from this state.
- In all the not executing states (SELF_SUSPENDED, ASYNC_SUSPENDED,
BLOCKING_SELF_SUSPENDED, BLOCKING_ASYNC_SUSPENDED) we just increment the
suspend count.
- All other transitions are illegal.
- Note that BLOCKING must always have a suspend count of 0 and
BLOCKING_SUSPEND_REQUESTED must have suspend_count > 0.
We add two new states:
- STATE_BLOCKING_SUSPEND_REQUESTED - this is a new state that indicates that a
suspend initiator wants to suspend this victim thread.
- The thread is still executing in this state.
- We only transition into this state with
mono_threads_transition_request_suspension.
- The only legal transitions out from this state are:
- resume - decrements the suspend_count. If the suspend_count becomes 0 we
go back to BLOCKING, otherwise we stay in BLOCKING_SUSPEND_REQUESTED.
- finish_async_suspension (executed by the suspend signal handler to
finish the two phase preemptive suspend request and notify the suspend initiator),
- done_blocking and abort_blocking. It means that the a
suspend initiator requested a suspend but we got to done (or abort)
blocking before the signal was delivered. We return NotifyAndWait and the
caller is supposed to send a notification to the suspend initiator and
wait for a resume(with suspend count == 0) which will return to
RUNNING (the victim thread goes to STATE_BLOCKING_SELF_SUSPENDED).
- STATE_BLOCKING_ASYNC_SUSPENDED
- The thread is not executing in this state (it is waiting for a resume).
- a thread transitions into this state from STATE_BLOCKING_SUSPEND_REQUESTED
on a finish_async_suspension transition (see above).
- on a resume transition we decrement the suspend count. if it's still
positive we stay in this state. if it's 0 the thread transitions out of
this state. Unlike other resume transitions, in this case the thread goes
back to BLOCKING and resumes executing in GC Safe mode. The resume caller
must notify the resume initiator (ie this is a ResumeInitAsyncResume).
- on a request_suspension we stay in this state and just increment the
suspend count.
- all other transitions from this state are illegal.
We add two new return values for abort_blocking and done_blocking:
- DoneBlockingNotifyAndWait and AbortBlockingNotifyAndWait. Similar to
SelfSuspendNotifyAndWait - there was a race between the second phase of a
preemptive suspend and another operation (in that case a poll, in this case a
done or abort blocking) and the other operation won, so its caller should
notify the suspend initiator. Since the thread is now suspended, the caller
should wait for the resume signal.
We add new results for request_suspension MonoRequestSuspendResult
(formerly MonoRequestAsyncSuspendResult):
- ReqSuspendAlreadySuspended means the thread is not executing and we just
incremented its suspend count and there's nothing else for the suspend
initiator to do. (The old AsyncSuspendAlreadySuspended)
- ReqSuspendAlreadySuspendedBlocking means the thread is in
BLOCKING_SUSPEND_REQUESTED and we initiated another suspend request. This
should only happen with full cooperative suspend - with hybrid suspend we
will use a two phase preemptive suspend on a blocking thread and since only
a single suspend initiator is active, this state should not be visible to
another suspend initiator.
- ReqSuspendInitSuspendRunning - Thread is executing in GC Unsafe mode and the
caller should suspend it. This is the old AsyncSuspendInitSuspend. In
full preemptive suspend this is the only suspend initiation action and the caller
should begin the preemptive suspend procedure. In full coop we expect the
victim thread to reach a safepoint and poll to finish suspending.
- ReqSuspendInitSuspendBlocking - Thread is executing in GC Safe mode and
the caller should initiate a suspend. In full coop the caller has nothing
to do - a thread executing in blocking mode is assumed to be suspended. In
hybrid suspend, the caller has to initiate a preemptive suspend.
* [sdks] New SDK target: soft debugger + non-nunit exe
This adds support for building an arbitrary .exe (dtest-app is hardcoded
now, but changing that and setting the arguments is another line or two,
to bu customized to the app being bound)
When an environment variable is set, the runtime will wait for the
managed debugger. This enables the desktop to debug the deployed .exe.
This will be used for the Mono.Debugger.Soft tests, but is useful for
isolated reproductions with android bugs.
An unmanaged debugger versions of the managed debugger is coming in
another PR.
* [runtime] Provide argument to dtest-app when running remotely
* [runtime] Fix case of VMDisconnectionException when getting Process of remote host that has disconnected
* [runtime] Script monodroid desktop debugger agent
* [runtime] Use g_warning in debugger agent
* [sdks] Ship PDBs on android
* [sdks] Add Android target to run debugger tests
* [sdks] Add desktop profile target for debugger tests
* [sdks] Disable debugger tests on mobile that use unsupported apis
* [sdks] Add monodroid test profile to android sdks
* [runtime] Allow dtest-app to be built on non-debugger-client profiles
* [runtime] Allow dtest-app building on mobile profiles
* [bcl] Mark some some code with DISABLE_COM which depends on icalls which are disabled in the runtime when --enable-minimal=com is used.
* [bcl] Fix compilation of System.Runtime.InteropServices.WindowsRuntime.dll.
* .
When we tried switching mono-project.com to enforce HTTPS these tests
started failing on systems without working SSL configuration.
Use a different domain which is guaranteed to be available over HTTP instead.
Note that eventually, the length will depend on stack_usage
difference between caller and callee, though for arm64 it does not currently.
This should fix https://github.com/mono/mono/issues/8112.
* [jit] Add a MONO_ARCH_FLOAT32_SUPPORTED arch-specific define, use it to disable float32 on platforms which don't support it.
* [amd64] Fix r4fp support for the ATOMIC_LOAD/STORE_R4 opcodes.
* [jit] Disable a unused test method with invalid IL.
* [jit] Handle r4->u conversion correctly in float32 mode.
* [amd64] Fix some OP_R<cmp> opcodes.
If the runtime is configured using --enable-minimal=soft_debug, the debugger code will be compiled into a separate
libmono-dbg.a library which needs to be linked into the app. The debugger needs to be registered with the runtime by
calling mono_debugger_agent_init () before runtime initialization.
* [llvm] Add intrinsics support for some of the new MathF icalls when using llvm+float32.
* [jit] Use m_class_get_name ().
* [llvm] Fix the signature of the llvm instrinsic used by OP_ABSF.
* [llvm] Use llvm.fabs only on amd64, it doesn't seem to be supported on some platforms.
* [ci] Update csproj files during PRs
Similar to how API diff is done.
Replace EXTERNAL_DRAWING_FACADE in csproj in genproj
It'd embed the absolute path in the csproj, but we want the variable instead.
* [csproj] Update project files
Custom modifiers are an odd feature of .net which allow types to
be tagged. These tags are semantically relevant. A method with a
modifier should not override a method without a modifier, for
instance (github bug #6936).
We don't put enough information into the modifier to uniquely find the
Image associated with the token we have in the type. Modifiers from
different referring assemblies will have different tokens to refer to
the same modifier class, making it important to resolve the tokens to
MonoClass pointers at runtime. Pointer equality of these classes is the
way to determine two custom modifiers the same, or unique.
We cannot resolve the image just from the MonoType because it is
possible to put modifiers on primitive, builtin types. These builtin
types lack any filled-out pointers in the data block, meaning we have
zero metadata associated with our floating token.
Other places in the codebase fixed this by passing around MonoImages,
but adding a MonoImage argument to our type comparison function for
this single bug would be a very poor decision with much code churn all
throughout the runtime.
The bug above resolves to two I4 types, one of which has a modifier, one
of which does not.
In order to pass this MonoImage around, we need to put a reference to it
into something reachable from the MonoType. We can't put it in the
custom module type because that is exported. It is part of the public
API.
MonoTypes do not adhere to pointer equality, and custom modules are structs
baked directly into the MonoType. We need therefore to make MonoType
point to a MonoCustomModInternal, a new type. This type embeds the
public type as a struct and carries around the image as another element
in the struct.
All public interfaces still see the same metadata endpoints for parsing the
modules, including the icalls. It is only our type comparison code that
accesses our new field.
This fixes https://github.com/mono/mono/issues/8074.
I see this 100%. Why others do not? I usually skip aot so usually I don't see it. Linux Ubuntu 1604 and
1710 tested. mono-devel package is installed, which is probably not relevant.
git clean -xddf (or new git clone)
./configure --with-runtime_preset=fullaot likely also with others (hybridaot)
make
Many errors like:
MONO_PATH="./../../class/lib/build:$MONO_PATH" /s/mono/runtime/mono-wrapper ./../../class/lib/build/sn.exe
-q -R ../../class/lib/testing_aot_full/mscorlib.dll ./../../class/mono.snk Couldn't sign the assembly
../../class/lib/testing_aot_full/mscorlib.dll with this key pair. Public key of assembly did not match
signing public key. ../../build/library.make:342: recipe for target
'../../class/lib/testing_aot_full/mscorlib.dll' failed
If you run make V=1 you see the sn command.
You can see the installed sn works ok. Remove the -Q for quiet and you see a warning about missing
machine.config. Run with ptrace and you see it looking for 4.5/machine.config. Look in the tree and you
see 1.1 and 2.0. Copy the 2.0 to 4.5 and problem goes away. And can be edited down a bunch. Actual fix is
to move symlink of the file to outside a conditional.
Maybe people aren't using fullaot or git clean? Repro is 100% for me.
I was going to bisect, but it was occuring as far back as September 2017 so I did not.
.
* [sdks] Add support for building llvm unless the USE_PREBUILT_LLVM Make.config var is set.
* [sdks] Use prebuilt llvm binaries on ios CI.
* [sdks] Use the IGNORE_PACKAGE_LLVM variable to enable the building of llvm.
* [runtime] Handle interpreter-related frames in async stack walks.
* [profiler] Fix exception clause reporting.
This fixes three separate issues:
* Finally clauses in the non-exceptional case were not reported because we
implement these by simply calling them from managed code, as opposed to the
exceptional case where we call them from runtime C code.
* The catch portion of a try-filter-catch clause was being incorrectly reported
as a filter.
* Actual exception filters were not being reported at all.
Because we have to instrument JIT'd code to report non-exceptional finally
clauses, a new mono_profiler_enable_clauses () function has been introduced
which profiler API users must call in order to get exception clause events, in
a similar fashion to mono_profiler_enable_allocations (). Since this is a
breaking change, the profiler API version has been bumped.
Fixes#7525.
* [profiler] Remove the iomap_report event and associated code.
The IOMap profiler no longer exists, so we no longer need to expose this
somewhat hacky event.
* [profiler] Replace the old gc_event with gc_event2.
* [coop] Squash the thread state machine state numbering
In
https://github.com/mono/mono/commit/9767b8a55ec8be3fc46055d7a1e21b6894fd0fc5
we removed STATE_SELF_SUSPEND_REQUESTED
but left a hole in the state numbering, as a result, `state_name()` was off by
one (and falling off the end of the array).
* [coop] Renamed STATE_BLOCKING_AND_SUSPENDED -> STATE_BLOCKING_SELF_SUSPENDED
Noticed this while looking at Jenkins logs today:
```
15:00:56 /mnt/jenkins/workspace/test-mono-pull-request-amd64/mcs/class/lib/net_4_x/mscorlib.dll
15:00:56 cant resolve internal call to "System.MathF::ModF(single,single*)" (tested without signature also)
```
We need to use `single` in the icall def instead of `float`.
It was introduced by https://github.com/mono/mono/pull/7941
Using `Task.Delay (timeout)` has the potential of possibly allocating and holding large
chunks of memory throughout the timeout period. While this is not strictly speaking a
memory leak because those objects will eventually be disposed and garbage collected, it
can cause memory usage of apps to grow quite significantly in the short run.
The default timeout values are as follows:
* `HttpWebRequest.ReadWriteTimeout`: 5 minutes.
* `HttpWebRequest.Timeout`: 100 seconds.
* `ServicePointManager.MaxServicePointIdleTime`: 100 seconds.
And these values can be increased by the user. This means that we're potentially holding
hundreds, maybe even thousands of timeout objects in memory for 5+ minutes.
This has now been fixed by using the `Task.Delay(int,CancellationToken)` constructor.
Note that disposing a `CancellationTokenSource` will not cancel it, so the following code
will still leak:
using (var cts = new CancellationTokenSource ()) {
Task.Delay (TimeSpan.FromDays (3), cts.Token);
// Work
}
To actually make the `Task.Delay` instance go away when we're done with it, we need to
use this pattern:
var cts = new CancellationTokenSource ();
try {
Task.Delay (TimeSpan.FromDays (3), cts.Token);
// Work
} finally {
cts.Cancel ();
cts.Dispose ();
}
We also cannot use `CancellationToken.CancelAfter()` for timeouts because unfortunately
some of the underlying framework code that we're using does not honor cancellation
requests - like for instance `NetworkStream`.
This is a partial fix for #7356. While this commit significantly improves the problem,
there may be other unrelated issues left that needs to be investigated.
Output formatting is now delegate to a `Formatter` subclass. The default
one remains HTML with `HtmlFormatter` so any script should not see any
difference.
The output file is largely identical, except a few whitespace changes
that could not easily be avoided and how modified base types were
reported, since that was different from the rest.
The big change is the `--md` option which can output markdown, which
is the format to use from docs.microsoft.com. There are less options
available, e.g. colors, so additions are rendered using the `csharp`
blocks, while modification are rendered as a `diff` block.
Each assembly output is made to be easy to re-assemble into a single
document if needed, e.g. https://github.com/xamarin/xamarin-macios/pull/3814
* [coop] Include function names in coop state change assertions
i.e. better reporting here:
https://jenkins.mono-project.com/job/test-mono-pull-request-coop/2298/parsed_console/log.html
***** MonoTests.System.Runtime.Remoting.RemotingServicesTest
***** MonoTests.System.Runtime.Remoting.RemotingServicesTest.Bug46473
Assertion failure in thread 0x7f21a504b740 due to: Cannot transition thread 0x7f21a504b740 from STATE_BLOCKING with DO_BLOCKING
Last 3 state transitions: (most recent first)
[DO_BLOCKING] RUNNING -> STATE_BLOCKING (0) 0 at:
at /mnt/jenkins/workspace/test-mono-pull-request-coop/mono/mini/mono(+0x2ef31a) [0x560336a6f31a]
at /mnt/jenkins/workspace/test-mono-pull-request-coop/mono/mini/mono(+0x2eb477) [0x560336a6b477]
at /mnt/jenkins/workspace/test-mono-pull-request-coop/mono/mini/mono(+0x2ec7a2) [0x560336a6c7a2]
at /mnt/jenkins/workspace/test-mono-pull-request-coop/mono/mini/mono(+0x2ec7e6) [0x560336a6c7e6]
This will provide one function instead of 0 or n, and could be expanded to include more.
This bloats shipping binary some (under condfigure only) which can be addressed with
an #if that conditionally turns the strings to null or empty. And slight code bloat.
Note that JIT and products and test use all this stuff and are left compatible and unchanged.
Previously:
foo(gpointer *stackdata)
{ ...
}
stackdata pointing to uninitialized, and not even size-specified
is now:
foo_internal(MonoStackData *stackdata)
{
...
}
foo(gpointer* stackpointer)
{
MonoStackData stackData = { &stackdata, __func__ }; // define MONO_STACKDATA
stackdata.stackpointer = stackpointer; // replacing &stackdata
foo_internal(&stackdata);
}
with __func__ being a known-inaccurate standin for the caller.
If products/JIT/test evolve, flags to indicate which fields are present are likely in order.
Currently only the top function is printed, but we can likely store them longer.
As well running the debugger will help.
backtrace library could also be expanded. Currently it finds symbols on Mac but not Linux.
* [Tailcall] Disable tailcall-virt.exe tailcall/fsharp.exe per-arch instead of !AMD64
so we can whittle away at it.
* [Tailcall, arm64, jit] Make mono_arm_emit_destroy_frame flexible
as to if ip0 or ip1 (or both) is temporary. This isn't strictly necessary as caller
can carefully chose the correct one, i.e. ip0.
* [Tailcall jit arm64] Add virtual tailcall support to ARM64 (OP_TAILCALL_MEMBASE).
It could be either "interleaved" with regular tailcall support as done here,
or split up into a separate case. The level of code sharing with interleaving
is not huge.
* [Tailcall] Spell out 'membase' as 'tailcall_membase' for clarity.
Remove extra "shadowed local" MonoCallInst *call and use
the other one. It achieves the same thing in this case
and helps debugging experience. Alternative would be
to rename the second one to "tailcall".
* [Tailcall] x86 tailcall_membase support (i.e. for virtual now, interface later)
* [Tailcall] Be sure not to tailcall interface calls on ARM.
* [Tailcall] More debugprinting.
* [Tailcall] arm64: allow floating point through (but only full registers)
arm, x86, amd64: require caller and callee return storage to match (arm64 already did)
Cleanup and debugprint.
* [Tailcall test] Always get hugeint (recursion depth) via function call,
so we can edit it one place.)
* [Tailcall] Add a FIXME for returning structs that can be made to work.
* [Tailcall test] Turn the F# test into three variations:
deeptail same as original -- failure mode is stack overflow
shallowtail which tailcalls but not to great depth -- it won't stack overflow
and so it should have no other failure, but make sure -- for example
this might still struggle on arm and we might split it up further
shallownotail which is really should be nothing special
In this commit, all three are the same -- copy w/o edit.
* [Tailcall test] Make shallow in two tests and remove tail in one.
* [Tailcall test] Disable deeptail TailCallLoopGenericClass<DateTime> for all architectures.
It fails on x86 and arm64. It passes on amd64. arm tbd.
* [Tailcall test] Enable F# tests for x86 and arm64 (three internally are disabled for all architectures, one of which passes for amd64).
* [Tailcall test] Every configuration should be able to run the shallow/notail F#/tailcall tests.
They don't depend on tailcall, but either correctness w/ or w/o tailcall,
and if they do fail, it is not due to stack overflow.
* [Tailcall test] Fix .assembly directive to make some tailcall tests AOT compatible.
* [Tailcall] Dial back debugprint bloat after merge.
it won't be scanned by the GC, defer string allocation to execution-time instead.
this fixes a GC crash when running mini regression suite on iOS/arm64.
OS: Ubuntu 18.03
CMake: 3.10
For some reason cmake fails to detect any make program (be it ninja or make)
when cross-compiling for Android in XA. It happens despite the programs being in
$PATH and working correctly. The effect is that CMAKE_MAKE_PROGRAM is set to
"not found" in CMakeCache.txt and configuration fails as follows:
Call Stack (most recent call first):
external/mono/external/boringssl/util/android-cmake/android.toolchain.cmake:1140 (CMAKE_FORCE_CXX_COMPILER)
external/mono/sdks/builds/android-armeabi-v7a-debug/mono/btls/build-shared/CMakeFiles/3.10.2/CMakeSystem.cmake:6 (include)
CMakeLists.txt:3 (project)
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
This commit does not fix the issue, per se, but it works around it by finding
the first ninja/make (as configured) executable in $PATH and setting the
CMAKE_MAKE_PROGRAM to full path of the executable. I haven't spent too much time
investigating *why* cmake fails to find the executable (I merely ran it with
--trace-expand --debug-output which revealed nothing of consequence) but this
workaround should be safe for any version of CMake and any OS we use makefiles
on.
* [gc] Silently ignore mono_gchandle_free(0) so callers do not have to check.
* [threads] Take advantage of mono_gchandle_free(0) being nop (a few extra instructions
in dynamic path for when it is 0, same path length if it is not 0).
* [threads] Restore one if, it isn't clearly correct otherwise.
This pull request brings support to Mono's class libraries for the new `System.MathF` type which includes operations for single-floating point.
This is needed to run some new applications like:
https://github.com/aras-p/ToyPathTracer
With this change, it is possible to run this unmodified. With this support, and the `-O=float32` option, it is possible to improve the performance from 6.3Mrays/sec to 7.8Mrays/sec on a Mac.
Oddly, this seems to be faster than .NET Core 2.1.4, which is at 4.2Mrays/sec.
* Bump API snapshot submodule
* [wasm][corlib] Disable some tests that don't work with WASM.
* [wasm][corlib] Disable IO test that requires threading.
* [wasm] Disable parallelism when building the wasm runtimes. Fixes CI build.
* [wasm] The incremental test runner can now skip tests in bulk.
* [wasm] Address reviewer feedback.
Current build uses `tar` to extract ZIP archives. This works with BSD
tar but not with GNU tar, alas. Tar was used in order to strip the root
directory of Android archive while moving files into place. This commit
replaces `tar` with a call to a shell script (because doing what it does
using Make defines is an exercise in pain) which works both on macOS/BSD
and on Linux.
* [Tailcall] Dial back single -v output, and enable under MONO_DEBUG=tailcall, and -v -v.
* [Tailcall] MONO_LOG_LEVEL=debug MONO_LOG_MASK=tailcall and mono_trace instead of mono -v -v and g_print
This fixes `test_0_simple_double_casts` on iOS/arm64.
According to the C standard (section [conv.fpint]), the behaviour of
such a cast is undefined if the truncated value doesn't fit into the
destination type.
```c
unsigned char __attribute__ ((noinline)) convr8u1 (double i)
{
return (unsigned char) i;
}
```
for arm64, `clang -O0` produces:
```
_convr8u1:
100006848: ff 43 00 d1 sub sp, sp, #16
10000684c: e0 07 00 fd str d0, [sp, #8]
100006850: e0 07 40 fd ldr d0, [sp, #8]
100006854: 08 00 78 1e fcvtzs w8, d0
100006858: 00 1d 00 53 uxtb w0, w8
10000685c: ff 43 00 91 add sp, sp, #16
100006860: c0 03 5f d6 ret
```
while `clang -O2` produces:
```
_convr8u1:
100006ae8: 00 00 78 1e fcvtzs w0, d0
100006aec: c0 03 5f d6 ret
```
which is fine according the standard: Any value < 256 gets casted just
fine, but for example `convr8u1 ((double) 0xffff)` would end up with
`0xffff` in `w0`.
We workaround this behaviour by casting to a 32bit int first, the
resulting code looks like this then:
```
_convr8u1:
100006ae4: 08 00 78 1e fcvtzs w8, d0
100006ae8: 00 1d 00 12 and w0, w8, #0xff
100006aec: c0 03 5f d6 ret
```
* [Tailcall] Require an exact match on return type. Sometimes this makes sense (float<=>double), sometimes
it is a little overly restrictive (integer truncation, I <=> I[48])
* [Tailcall] Disable CoreCLR tests that have mismatched return types.
[mini] Introduce MonoEEFeatures with a single feature use_aot_trampolines.
MonoEEFeatures is meant to replace the mode flag soup with have.
Each feature is supposed to be checked alone and descriptive enough of the underlying behavior.
use_aot_trampolines is a good example because the inter_aot_llvmonly mode could set it to false and
skip all the trampoline engine nonsense as trampolines are C functions.
Contributes to #7885
* [wasm] Switch wasm interp to not set use_aot_trampolines.
This allows the regular "JIT" tramp path be used instead of hacking our way around the AOT runtime.
* [mini] Introduce MonoEEMode to let us enumerate non-aot execution modes.
This lets us have all code setting up mono_ee_features in a single place.
* [Coop] Opportunistically remove some raw pointers.
Replace some uses of mono_thread_interruption_checkpoint with
mono_thread_interruption_checkpoint_void or mono_thread_interruption_checkpoint_bool.
* [Coop] Remove some more raw pointers opportunistically -- mono_thread_execute_interruption to mono_thread_execute_interruption_void.
* Fix warnings from old bad merge.
* Add another static.
* [coop] Remove more raw pointers.
* [sdks] Add provisioning for Android SDK and NDK
This will also be used by XA
* Add missing backslash to mxe.mk
* Only grab bin/autopoint
* [sdks] Use provisioning for MXE
We want to treat MXE the same way we do Android SDK and NDK because it only rarely changes, and it's very long to build it.
* WCF: ChannelDispatcher/ListenerLoopManager.Setup validate InstanceProvider, not InstanceContextProvider
- the creation of endpoints requires an instance provider or a default constructor
- InstanceProvider vs InstanceContextProvider was obviously a typo
* include InstanceContextProvider for validation as before
- two unittests failed due InstanceProvider is null and ed.DispatchRuntime has not type set
- comment for the validation is not useful anyway (It is tested at Open(), but strangely it is not instantiated at this point.)
* [Facades] Add System.Memory facade to mobile profiles
Contains only the types we have right now.
Fixes https://github.com/mono/mono/issues/7610
* Bump API snapshot submodule
In 38a348c991 we got rid of
mono_threads_transition_reqest_self_suspension. So there is now no way to
enter into STATE_SELF_SUSPEND_REQUESTED, so we no longer need to have cases for
transitioning out of it.
2018-04-02 16:54:50 -04:00
2570 changed files with 129132 additions and 141121 deletions
# make reset-llvm will checkout a version of llvm which is suitable for this version of mono
# into $top_srcdir/llvm/llvm.
#
top_srcdir?=$(abspath $(CURDIR)/..)
LLVM_PATH?=$(abspath $(top_srcdir)/external/llvm)
LLVM_BUILD?=$(abspath $(top_srcdir)/llvm/build)
LLVM_PREFIX?=$(abspath $(top_srcdir)/llvm/usr)
# FIXME: URL should be http://xamjenkinsartifact.blob.core.windows.net/build-package-osx-llvm-$(NEEDED_LLVM_BRANCH)/llvm-osx64-$(NEEDED_LLVM_VERSION).tar.gz
$(Q) sed -e "/@diffdata@/r temp.patch" -e "/@diffdata@/d" -e "s/@title@/Public API Diff/g" -e "s/@description@/If the changes are intentional, run <code>make -C mcs mono-api-diff<\/code> locally and commit changes in external\/api-snapshot./g" diff.html.in > apidiff/index.html
$(Q)if[ -s temp.patch ];thenecho"Error: Found public API differences, see mcs/apidiff/index.html. If the changes are intentional, please go to external/api-snapshot and commit them."; rm -f temp.patch;exit 1;elseecho"No differences found."; rm -f temp.patch;fi
mono-csproj-diff:
@echo "Generating csproj diff..."
$(Q) git add -A "*.csproj"$(topdir)../bcl.sln
$(Q) git diff --no-renames HEAD "*.csproj"$(topdir)../bcl.sln > $(abspath $(topdir))/temp.patch
The new makefiles try to abstract building on Windows and Linux. They
try to provide a consistent set of ways to express the things that our
build system needs to let us do, specifically:
* Build recursively
* Build libraries and executables easily
* Let developers use different runtimes and class libaries
* Make distributions easily
* Provide a framework for testing
* Build platform-independently whenever possible
* Generate, update, and build monodoc documentation.
* Build recursively
* Build libraries and executables easily
* Let developers use different runtimes and class libaries
* Make distributions easily
* Provide a framework for testing
* Build platform-independently whenever possible
* Generate, update, and build monodoc documentation.
** Makefile structure
Makefile structure
==================
A general makefile looks like this:
========================================
```
thisdir = class/Mono.My.Library
SUBDIRS =
include ../../build/rules.make
@@ -50,59 +50,60 @@ doc-update-local:
my_test_program.exe: my_test_source.cs
$(CSCOMPILE) /target:exe /out:$@ $<
========================================
```
Each makefile follows the same pattern: it does some setup, includes
the standard make rules, and provides rules for eight standard targets:
all, install, test, run-test, clean, dist, and doc-update.
`all`, `install`, `test`, `run-test`, `clean`, `dist`, and `doc-update`.
"Some setup" is defining two variables: $(thisdir) and
$(SUBDIRS). $(thisdir) is the directory that the makefile lives in,
relative to the top directory (ie, class/corlib) and $(SUBDIRS)
"Some setup" is defining two variables: `$(thisdir)` and
`$(SUBDIRS)`. `$(thisdir)` is the directory that the makefile lives in,
relative to the top directory (ie, `class/corlib`) and `$(SUBDIRS)`
defines the subdirectories that should be built in.
The eight targets do the following:
* all-local builds whatever someone would expect to be built
when they just type 'make'. Most likely Foo.dll or Foo.exe
*`all-local` builds whatever someone would expect to be built
when they just type `make'` Most likely `Foo.dll` or `Foo.exe`
* install-local installs whatever got built by all-local.
*`install-local` installs whatever got built by `all-local`.
* test-local _builds_ the test programs or libraries but does
_not_ run them.
*`test-local`_builds_ the test programs or libraries but does
_not_ run them.
* run-test-local actually runs the tests. It shouldn't
necessarily exit in an error if the test fails, but should make that
situation obvious. It should only run tests that take care of
themselves automatically; interactive tests should have an individual
target. The idea is that 'make run-test' from the toplevel should be
able to proceed unsupervised and test everything that can be tested in
such a manner.
*`run-test-local` actually runs the tests. It shouldn't
necessarily exit in an error if the test fails, but should make that
situation obvious. It should only run tests that take care of
themselves automatically; interactive tests should have an individual
target. The idea is that `make run-test` from the toplevel should be
able to proceed unsupervised and test everything that can be tested in
such a manner.
* run-test-ondotnet-local is a variant of run-test-local. It is used only to validate if our tests themselves works fine under Microsoft runtime (on Windows). Basically, in this target, we should not use $(TEST_RUNTIME) to test our libraries.
*`run-xunit-test-local` this is a variation of the above, but it runs the
tests that were built with Xunit instead of NUnit.
*clean-local removes built files; 'make clean' should leave
only files that go into a distribution tarball. (But it is not necessarily
true that all files that go into a tarball need to be left after a make clean.)
*`run-test-ondotnet-local` is a variant of `run-test-local`. It is used only to validate if our tests themselves works fine under Microsoft runtime (on Windows). Basically, in this target, we should not use $(TEST_RUNTIME) to test our libraries.
*dist-local copies files into the distribution tree, which is
given by the variable $(distdir). dist-local always depends on the
target 'dist-default'. See ** 'make dist' below.
*`clean-local` removes built files; `make clean` should leave
only files that go into a distribution tarball. (But it is not
necessarily true that all files that go into a tarball need to be left
after a make clean.)
*doc-update-local should generate or update monodoc documentation,
if appropriate. This is usually only appropriate for libraries. It's
defined as a standard target so that it can easily be run recursively
across all libraries within the module.
*`dist-local` copies files into the distribution tree, which is
given by the variable `$(distdir)`. `dist-local` always depends on the
target `dist-default`. See **`make dist` below.
*`doc-update-local` should generate or update monodoc documentation,
if appropriate. This is usually only appropriate for libraries. It's
defined as a standard target so that it can easily be run recursively
across all libraries within the module.
** Build configuration
Build configuration
===================
In general, MCS needs to be able to build relying only on the
existence of a runtime and core libraries (corlib, System,
System.Xml). So there shouldn't be any checking for libraries or
existence of a runtime and core libraries (`mscorlib`, `System`,
`System.Xml`). So there shouldn't be any checking for libraries or
whatnot; MCS should be able to build out of the box. We try to keep
platform detection and feature testing (ie, for HP/UX echo) inside
the makefiles; right now, there's no configuration script, and it'd
@@ -115,41 +116,37 @@ customize their builds to suit their needs. To allow this, the
Makefile rules are set up to allow people to override pretty much any
important variable.
Configuration variables are given defaults in `config-default.make';
`rules.make' optionally includes `$(topdir)/build/config.make', so you
Configuration variables are given defaults in `config-default.make`;
`rules.make` optionally includes `$(topdir)/build/config.make`, so you
can customize your build without CVS trying to commit your modified
`config-default.make' all the time. Platform-specific variables are
defined in `$(topdir)/build/platforms/$(BUILD_PLATFORM).make', where
$(BUILD_PLATFORM) is detected in config-default.make. (Currently, the only
choices are linux.make and win32.make.)
`config-default.make` all the time. Platform-specific variables are
defined in `$(topdir)/build/platforms/$(BUILD_PLATFORM).make`, where
`$(BUILD_PLATFORM)` is detected in `config-default.make`. (Currently, the only
choices are `linux.make` and `win32.make`)
The best way to learn what the configuration variables are is to read
`config.make' and `platform.make'. There aren't too many and hopefully
`config.make` and `platform.make`. There aren't too many and hopefully
they should be self-explanatory; see the numerous examples below for
more information if you're confused.
** Recommendations for platform specifics
Recommendations for platform specifics
--------------------------------------
If you find yourself needing a platform-specific customization, try
and express it in terms of a feature, rather than a platform test. In
other words, this is good:
========================================
```
run-test-local: my-test.exe
ifdef PLATFORM_NEEDS_CRAZY_CRAP
crazy-crap
endif
$(RUNTIME) my-test.exe
========================================
```
and this is bad:
========================================
```
run-test-local: my-test.exe
ifdef WINDOWS
crazy-crap
@@ -159,7 +156,7 @@ ifdef AMIGA
endif
endif
$(RUNTIME) my-test.exe
========================================
```
The latter accumulates and gets unpleasant and it sucks. Granted,
right now we only have two platforms, so it's not a big deal, but it's
@@ -168,88 +165,79 @@ do the various corlib building hacks for examples of how we've done
platform-specificity. It certainly isn't pretty, but at least it's a
little structured.
Saving effort
=============
** Saving effort
The point of the build system is to abstract things and take
The point of the build system is to abstract things and take
care of all the easy stuff. So if you find yourself writing a
Makefile, know that there's probably already infrastructure to do what
you want. Here are all the common cases I can think of ...
you want. Here are all the common cases I can think of.
Compiling C# code? use:
-----------------------
* Compiling C# code? use:
========================================
```
my-program.exe: my-source.cs
$(CSCOMPILE) /target:exe /out:$@ $^
========================================
```
or
or
========================================
```
my-lib.dll: my-source.cs
$(CSCOMPILE) /target:library /out:$@ $^
========================================
```
Note the '$@' and '$^' variables. The former means "the name of the
Note the `$@` and `$^` variables. The former means "the name of the
file that I am trying to make" and the latter means "all the
dependencies of the file I am trying to make." USE THESE VARIABLES
AGGRESSIVELY. Say that you add a new source to your program:
========================================
```
my-program.exe: my-source.cs my-new-source.cs
$(CSCOMPILE) /target:exe /out:$@ $^
========================================
```
Because of the $^ variable, you don't need to remember to add another
file to the command line. Similarly, if you rename your program, you
won't need to remember to change the rule:
========================================
```
MonoVaporizer.exe: my-source.cs my-new-source.cs
$(CSCOMPILE) /target:exe /out:$@ $^
========================================
```
will still work. Another useful variable is $<, which means "the first
dependency of whatever I'm building." If you order your dependencies
carefully it can be extremely useful.
Just building an executable?
----------------------------
Then use:
* Just building an executable? use:
========================================
```
PROGRAM = myprogram.exe
LOCAL_MCS_FLAGS = /r:System.Xml.dll
include ../build/executable.make
========================================
```
executable.make builds a program in the current directory. Its name is
held in $(PROGRAM), and its sources are listed in the file
$(PROGRAM).sources. It might seem to make more sense to just list the
held in `$(PROGRAM)`, and its sources are listed in the file
`$(PROGRAM).sources`. It might seem to make more sense to just list the
program's sources in the Makefile, but when we build on Windows we
need to change slashes around, which is much easier to do if the
sources are listed in a file. The variable $(LOCAL_MCS_FLAGS) changes
the flags given to the compiler; it is included in $(CSCOMPILE) so you
sources are listed in a file. The variable `$(LOCAL_MCS_FLAGS)` changes
the flags given to the compiler; it is included in `$(CSCOMPILE)` so you
don't need to worry about it.
executable.make does a lot for you: it builds the program in 'make
all-local', installs the program in $(prefix)/bin, distributes the
`executable.make` does a lot for you: it builds the program in 'make
all-local', installs the program in `$(prefix)/bin`, distributes the
sources, and defines empty test targets. Now, if your program has a
Console.Error.WriteLine("You can specify * for platformName and profileName to read all sources files");
Console.Error.WriteLine("Available options:");
Console.Error.WriteLine("--help -h -?");
Console.Error.WriteLine(" Show command line info");
Console.Error.WriteLine("--trace:n");
Console.Error.WriteLine(" Enable diagnostic output, at tracing level n (1-4)");
Console.Error.WriteLine("--stdout");
Console.Error.WriteLine(" Writes results to standard output (omit outputFileName if you use this)");
Console.Error.WriteLine("--strict");
Console.Error.WriteLine(" Produces an error exit code if files or directories are invalid/missing");
Console.Error.WriteLine("--basedir:<dir>");
Console.Error.WriteLine(" Sets the base directory when reading a single sources/exclusions pair (default is the directory containing the sources file)");
if [ ! -f "TestResult-$(PROFILE).xml" ]; then echo "<?xml version='1.0' encoding='utf-8'?><test-results failures='1' total='1' not-run='0' name='bcl-tests' date='$$(date +%F)' time='$$(date +%T)'><test-suite name='$(strip $(test_assemblies))' success='False' time='0'><results><test-case name='crash' executed='True' success='False' time='0'><failure><message>The test runner didn't produce a test result XML, probably due to a crash of the runtime. Check the log for more details.</message><stack-trace></stack-trace></failure></test-case></results></test-suite></test-results>" > TestResult-$(PROFILE).xml; fi; \
if [ ! -f "TestResult-$(PROFILE).xml" ]; then echo "<?xml version='1.0' encoding='utf-8'?><test-results failures='1' total='1' not-run='0' name='bcl-tests' date='$$(date +%F)' time='$$(date +%T)'><test-suite name='$(strip $(test_assemblies))' success='False' time='0'><results><test-case name='$(notdir $(strip $(test_assemblies))).crash' executed='True' success='False' time='0'><failure><message>The test runner didn't produce a test result XML, probably due to a crash of the runtime. Check the log for more details.</message><stack-trace></stack-trace></failure></test-case></results></test-suite></test-results>" > TestResult-$(PROFILE).xml; fi; \
$$ok
## Instructs compiler to compile to target .net execution, it can be usefull in rare cases when runtime detection is not possible
<!-- WARNING: this file is autogenerated, don't modify it. Edit the .sources file of the corresponding assembly instead if you want to add/remove C# source files. -->
<!-- WARNING: this file is autogenerated, don't modify it. Edit the .sources file of the corresponding assembly instead if you want to add/remove C# source files. -->
Loaded 100 of 2570 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.