.. item.
If we have a project, that sets some properties or items at runtime,
like in a target:
```
<Target Name="Build">
<PropertyGroup>
<Bar>Bar01</Bar>
</PropertyGroup>
```
.. such a property can be accessed as `$(Bar)` subsequently.
Then at later point, a target tries to update metadata on some item, like:
```
<Target Name="Foo">
<ItemGroup>
<FooItem>
<SomeMetadata>MetadataValue</SomeMetadata>
</FooItem>
</ItemGroup>
```
.. then it is seen that the value for the earlier created `$(Bar)` disappears!
So, if we try to print the value of `$(Bar)` right after that item
group, it would appear as `''`!
The issue is that the metadata update in target `Foo` caused the project
to get re-evaluated, which meant that items/properites created after the
project load were lost! We should not be reevaluating the project when
setting metadata on the basis of dynamic items, like in the target
`Foo`.
These dynamic item groups in a target are represented as
`BuildItemTask`, and these create BuildItems which have their
`IsDynamic` property set. And this is used to avoid re-evaluating the
project in `BuildItem.SetMetadata`.
But when updating metadata (`BuildItem.UpdateMetadata`), we need to
update metadata on the *existing* items, which might not have been
created from such dynamic item groups! So, their `IsDynamic==false`.
Hence, trying to `SetMetadata` on such items would cause the project to
be reevaluated, and thus properties/items like `$(Bar)` would be lost!
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=45137
In this particular case, `Bar` was the `DesignTimeBuild` property, which
was losing it's value when `ProjectReference`'s `AdditionalProperties`
metadata was set in:
```
<Target Name="BclBuildAddProjectReferenceProperties"
<ItemGroup>
<ProjectReference>
<AdditionalProperties>$(_BclBuildProjectReferenceProperties);%(ProjectReference.AdditionalProperties)</AdditionalProperties>
</ProjectReference>
</ItemGroup>
```
The generic setter receives the element by reference which means we will never emit the write barrier since a reference to a reference is not a reference (mini_type_is_reference). Check the actual value instead. Fixes#42903.
It is possible for SSLHandshake() to return SslStatus.Success while we're still
having a pending write (AsyncOperationStatus.WantWrite).
This is because our managed write callback must never return 'WouldBlock', so
SSLHandshake() things that all data have been sent while we still have them in
our write queue.
When this happens, we currently flush the write queue then call SSLHandshake()
again via AsyncOperationStatus.WantWrite -> AsyncOperationStatus.Continue.
This returns SslStatus.Protocol on iOS 10.
(cherry picked from commit 2cc1b887c1)
It is possible for SSLHandshake() to return SslStatus.Success while we're still
having a pending write (AsyncOperationStatus.WantWrite).
This is because our managed write callback must never return 'WouldBlock', so
SSLHandshake() things that all data have been sent while we still have them in
our write queue.
When this happens, we currently flush the write queue then call SSLHandshake()
again via AsyncOperationStatus.WantWrite -> AsyncOperationStatus.Continue.
This returns SslStatus.Protocol on iOS 10.
(cherry picked from commit 2cc1b887c1)
This is a workaround. It's needed for the same reason that we need the fix for
Clang's atomic intrinsics in mono/utils/atomic.h.
Conflicts:
mono/mini/mini-llvm.c
All ia64 compilers have intrinsics that we use. Also, this ia64 code was very
out of date (missing many functions we use now) so it wouldn't compile anyway.
Previously, we would try to satisfy the heapshot request every single time
process_requests () was invoked, but since heap_walk () would never unset
heapshot_requested, we'd end up doing GCs over and over, causing massive
overhead.
All events now come with a time field right after the event byte, and it is
encoded in the same way for all events (based on LogBuffer.time_base).
This allows readers to queue up events until a sync point event, then sort all
of those events by time, and process them in the order they were actually
written in.
Maintaining it for every format version is getting unwieldy and most readers
likely won't care about ancient format versions. Further, we haven't really
stabilized the format yet, so documenting every iteration of it doesn't make
a whole lot of sense. Finally, if somebody does care about older versions,
they can simply look in decode.c to see how mprof-report deals with them.
* Methods in backtraces are now properly encoded based on the method_base
rather than the ptr_base.
* The unused IL and native offset fields in TYPE_SAMPLE_HIT's backtrace frame
format have been removed.
* The unused flags fields in the backtrace format and in metadata events have
been removed.
* Changed a number of fields to be encoded as a single byte instead of LEB128.
These events basically establish points in the event stream where all previous
events can be processed and a complete view of the profiling state can be
formed. For instance, if an allocation event for object A of type T was seen
before a sync point, then it's guaranteed that the class load event for T also
came before that sync point (although it may have arrived after the allocation
event for A).
Other than timely arrival, there was never really a need to force buffers to be
flushed at these arbitrary points. Now that we flush all buffers periodically,
that reason is no longer relevant.
Heapshots have nothing to do with call depth, so they should be processed in
any case.
Also flush the local buffer in all enter/leave callbacks if needed.
This ensures that even for low-activity threads, it won't take ages before the
events from it arrive. We do this flush every 1sec (as with sampling of perf
counters).
The problem is this: Given an allocation of object O on address A1 in thread T1
and a subsequent GC move of O from A1 to A2 in T2, if T2's event buffer is
flushed before T1's buffer, the reader of the log file will a seemingly
nonexistent object being moved, and then see the actual allocation of that
object later. This makes it extremely difficult to keep track of the managed
heap.
The way we solve this is by flushing all thread-local buffers when the GC stops
the world. This way, we know that all prior allocations in the program have
been committed to the log file before any moves that are about to follow as
part of the collection. Similarly, we flush the GC thread's buffer before
restarting the world to ensure that all moves are committed before allocations
that may happen later (which could use some of the addresses freed up for use
due to the moves).
This is reasonably easy now that we have a global LLS of threads which are
attached to the profiler. However, we have to use some kind of lock to ensure
that buffers aren't being written to while we're trying to flush them, or that
a suspended thread was in the middle of modifying its buffer. For this, we use
a reader/writer spin lock of sorts, somewhat similar to the lock used by the
SGen binary protocol. When threads are modifying their own buffer (whether it
be writing to it or flushing it), they take the reader lock. This means that
many different threads can be operating on their own buffers without any lock
contention (which would kinda defeat the purpose of the thread-local buffers).
Now, when a thread -- such as the thread that initiated a GC -- wants to access
the buffers of other threads, it takes the writer lock. In doing so, all other
threads are prevented from modifying their own buffers. This gives the thread
holding the writer lock an opportunity to flush the buffers of other threads.
* Actual thread-local state is now stored in a MonoProfilerThread struct which
is stored in a single TLS slot per thread. This avoids storing things like
call_depth in LogBuffer. The current LogBuffer of a thread is also stored in
the MonoProfilerThread.
* All MonoProfilerThread structs are stored in a global linked list set. This
LLS is used to manually clean up the thread-local state of threads that, for
whatever reason, don't have the thread_end () callback invoked.
* Lifetime management of thread-local buffers is now much more straightforward.
* The thread-local list of referenced methods is now only allocated if needed.
* Fixed a MethodInfo memory leak in handle_writer_queue_entry ().
* Fixed some class load events that could be lost in the writer thread when
fetching the full name of a method.
* TLS slots are now actually freed on shutdown.
* Get rid of the arch-specific headers.
* Do verbose printing in one place.
* Simplify the cross compilation case to avoid repetition.
* Define all variables with a macro in a single header.
* Print detected hwcap info via g_print () so it's visible on mobile.
* Expose plain bool variables in the s390x implementation.
(cherry picked from commit 76a11d5b2d)
We need this in C8 because the old implementation was missing some netstandard required
members such as the InvalidHandle property.
When encoding a UTF-16 C# string into bytes using "latin-1"/"iso-8859-1" encoding we were
not properly taking surrogate pairs into account. MSDN[1] provides a good introduction about the topic.
For example, this string would fail to encode and throw an IndexOutOfRangeException: "\ud83d\ude0a "
However, this string would not: "\ud83d\ude0a" (note missing space at the end). More on that later.
In https://bugzilla.xamarin.com/show_bug.cgi?id=42887 it was noted that encoding the string
didn't cause the exception on Mono 2.10.8.1 and 3.2.8. Indeed, it doesn't happen until Mono 4.0
where we switched most of the string code to referencesources.
While looking at the output of `Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")` on the
"working" Mono verions I noticed that they essentially returned wrong results, e.g.:
```
csharp> Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")
{ 63, 32, 0 }
```
This is wrong, because a) there's only one 63 (i.e. the '?' question mark char which is used as
a replacement char whenever the target encoding doesn't support a specific char,
and b) there is a trailing '\0' which should not be there.
Checking on .NET confirms the expected result: { 63, 63, 32 } i.e. '?? '
Back to the IndexOutOfRangeException in Mono 4.0+: digging into Latin1Encoding.InternalGetBytes() it
turns out that when we detect a surrogate pair we were incrementing charIndex, but we didn't decrement count.
This means that after the fallback characters (the two '?') were added we would still try to encode one more
char in the loop even though we're already at the end of the string, resulting in the exception.
For the "\ud83d\ude0a" example we would not hit the `count > 1` code path which made it work in this case.
Adding the decrement fixes the bug and makes the code consistent with MonoEncoding.cs[2].
The reason why it "worked" in earlier Mono versions was that we had a very simple EncoderReplacementFallbackBuffer
implementation[3] which only output a single '?' for the surrogate pair:
```
csharp> var buf = Encoding.GetEncoding("latin1").EncoderFallback.CreateFallbackBuffer()
csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0
csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0'
```
This means that for the "\ud83d\ude0a " example from above we would only add one '?' character
for the surrogate pair to the result buffer, then the space and then we'd be done, leaving the
uninitialized array value 0 in the buffer on index 2.
I added tests for Latin1Encoding, mostly copied from ASCIIEncoding with additions for these
specific cases.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42887
[1] https://msdn.microsoft.com/en-us/library/ms404377(v=vs.110).aspx
[2] https://github.com/mono/mono/blob/5b69e839e12b3d3512f310aa5a0e873346d7e50e/mcs/class/I18N/Common/MonoEncoding.cs#L62-L66
[3] https://github.com/mono/mono/blob/mono-3.12.0-branch/mcs/class/corlib/System.Text/EncoderReplacementFallbackBuffer.cs
It doesn't work on nunitlite e.g. on XA:
```
src/corlib/Test/System.Runtime.InteropServices/MarshalTest.cs(268,11): error CS0117: `NUnit.Framework.Assert' does not contain a definition for `Less'
```
(cherry picked from commit 448da87d44)
Even though we register emulations for these opcodes, the native function doesn't not map to the opcode logic, being used instead as part of the opcode decomposition along with other instructions. Make sure we don't try to emulate these opcodes through the general emulation pass.
When having a float variable in memory, we should always access its contents through the fload_r4/fstore_r4 funcs, which do the conversion between float and double and different memsize vs var size. It is common case, to have a temp float, have its address loaded with ldaddr and later do a load_r4_membase on that address to access the contents. If alias analyis lowers this load_r4_membase to an op_fmove directly on the register pair, this would in turn be decomposed into 2 simple int op_moves on soft float, which is wrong if these op_moves are converted to memory accesses.
The names were fixed only for the subset of methods that are included in netstandard,
and in cases where netstandard differs from Desktop .NET we match with .NET.
(cherry picked from commit c4692001a9)
The targets depended on a feature that doesn't exist in
xbuild (https://bugzilla.xamarin.com/show_bug.cgi?id=43645)
and as a consequence would fail to add the generated file to
the build when the GenerateTargetFrameworkMonikerAttribute
target was up to date.
Previously we were wrongly trying to achieve this by doing a load-acq followed by a store-rel. Assuming we have memory access on address A (before) and C (after) while the atomic operation happens on address B, the order of the instructions would look like this :
[A], [B]load, load-acq, store-rel, [B]store, [C]
In this case a reordering of memory accesses like the following is possible, which clearly breaks the desired constraints :
[B]load, load-acq, [C], [A], store-rel, [B]store
In order to provide the sequential memory constraint we could emit the barriers like this instead :
[A], membar, [B]load, [B]store, membar, [C]
We can additionally save an instruction and emit the barriers like this instead (this is also what gcc does) :
[A], [B]load, store-rel, [B]store, membar, [C]
In this case we only need to worry about the relation between memory accesses on A and the loading of B. We need to consider what happens if B is loaded before the access on A is fulfilled. This only matters if we load an older value for B (rather than the one visible by other threads) before A. In that case the erroneous store will fail due to using exclusive stores. At the retry the value read from B will be the right one.
Loads with acquire semantics guarantee that following memory accesses are not reordered with the load. Add full membar before it to ensure full sequencing.
Stores with release semantics guarantee that previous memory accesses are not reordered with the store. Add a full membar after it to ensure full sequencing.
Simplify code by always accessing tlab information and inside_critial_region through the SgenThreadInfo pointer. This also fixes potential failure to detect that we are in a critical region within the managed allocator.
Conflicts:
mono/mini/mini.h
MONO_THREAD_VAR_OFFSET definitions don't work when loading mono as a dynamic library. This means that we need to be conservative and don't aot code that contains these tls chunks.
In the future, we should remove HAVE_KW_THREAD altogether and use only pthread since it simplifies the code alot.
Otherwise the NotImplementedException would prevent instantiating the class even though
we just use it as a dummy object on mobile.
(cherry picked from commit 70602566de)
* [corlib] Regression test for non-visible custom attributes.
See https://bugzilla.xamarin.com/show_bug.cgi?id=43291
* [reflection] Don't crash when building dynamic custom attributes on dynamic types (Fixes#43291)
1. Don't dereference a NULL custom attribute ctor, throw a type load
exception.
2. When building MonoCustomAttrInfo from an array of
MonoReflectinoCustomAttr*, compute the number of
non-visible (non-public) attributes correctly, and make sure to
iterate over all the attributes when populating the result array.
Fixes [#43291](https://bugzilla.xamarin.com/show_bug.cgi?id=43291)
* [reflection] Marginally better TLE for custom attrs
At least include a message about what went wrong when trying to
construct a custom attribute from a type that isn't finished yet.
I accidentally added this in 3c3ef17376 because I missed that on
full .NET this class inherits from NameValueCollection which provides `this[string name]` already
(the diff tool showed it as missing because it didn't take this into account).
We need a valid MonoThreadInfo in the TLS key while we run
unregister_thread because we may need to block the thread when it tries
to take the sgen GC mutex.
Fixes [#43099](https://bugzilla.xamarin.com/show_bug.cgi?id=43099)
MVID and AOTID metadata is no longer displayed at the end of the stack
trace.
Now, every stack frame contains a MVID and AOTID inlined.
An IL offset stackframe looks like:
at StackTraceDumper.Main () [0x00000] in <a3ec873a838c42c29f4eba7cab7d1dfb>:0
A native offset stack frame looks like:
at StackTraceDumper.<Main>m__0 () <0x100e15d50 + 0x00029> in <a3ec873a838c42c29f4eba7cab7d1dfb#b37a9d11247b811ffffdaff8090d8a59>:0
This is the recent behavior in .NET/MSBuild as well [1]:
> Starting in Visual Studio 2013, the MSBuild Toolset version is the same
> as the Visual Studio version number. MSBuild defaults to this Toolset
> within Visual Studio and on the command line, regardless of the
> toolset version specified in project file.
With this change xbuild will always use its own version instead
of the one specified in the project, unlessa ToolsVersion is forced
via the /tv switch on the command line.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42938
[1] https://msdn.microsoft.com/en-us/library/bb383796.aspx#Anchor_1
(cherry picked from commit a1a54b5f14)
Some Android devices for some reason have a 'model name' line instead of a
'Processor' line. On such devices, we would fail to detect the architecture
version at all, and end up thinking that it's ARMv4 (even if the runtime
was compiled for ARMv7).
This is necessary on weird devices where the hwcap code might fail to detect
the ARM version it's running on (usually caused by differences in /proc/cpuinfo
output).
The store/store fence incurs quite a high cost, so we only want it to
run when absolutely necessary. The most efficient way is to
separately compile nursery object operations for when the concurrent
collector is running, so there has to be no dynamic check.
In the future we might want to try batching updates of references to
moved objects so we have to do only one fence per batch.
Conflicts:
mono/sgen/sgen-gc.c
When we shutdown we set suspend_finalizers flag in metadata so we no longer run any finalizers. After recent changes, we avoid traversing gc hashes for finalizer enqueuing when this flag is set. The counterpart, popping the queues empty (even without running the finalizers) can still take some time, especially since we lock for each entry. Exit early from this loop if we no longer need to run finalizers.
Achieve this by setting a new flag inside sgen, instead of passing the address of suspend_finalizers all the time to the sgen api.
After the latest code changes, we may land here with `errors == SslPolicyErrors.None`,
so we need to make sure that we actually set it to `RemoteCertificateChainErrors` if
the system validator returns an error.
(cherry picked from commit b4304354b1)
Since acceptance-tests/Makefile (and llvm/Makefile) include scripts/submodules/versions.mk which evals versions.py which looks
for a SUBMODULES.json file in the current working directory we got confusing error messages if the build directory is not the src dir:
```
Traceback (most recent call last):
File "../../mono/scripts/submodules/versions.py", line 22, in <module>
submodules = json.load(open(CONFIG_FILE))
IOError: [Errno 2] No such file or directory: 'SUBMODULES.json'
```
Instead we now pass the path to SUBMODULES.json as a variable to versions.py and set it based on $(top_srcdir) so it works
even for out-of-tree builds.
(cherry picked from commit d8d1fe64c1)
When doing the first phase of sweeping we tag the block pointer within the block list with a CAS, after which we own the block and we are free to update the state and do sweep computations on the block. When we finish this phase we directly write the untagged block pointer back in the list, making the block available to inspect by other threads. We need a write barrier before writting the untagged pointer back because, when another thread might acquire the block, it might encounter an invalid block state.
A common pattern that we use across the test suite is to create objects on a new thread, join the thread, collect then expect certain objects to be dead/finalized since we assumed that thread to be no longer a source of pinning. This is not always the case since, after joining, the thread might still be alive and running unmanaged code related to the thread shutdown, which can reuse the stack that previously contained the managed references.
The workaround is to still allocate objects far down the stack, even if we are on a separate thread.
At the moment the helper can be used in tests which run under the TestDriver (since it's included in TestDriver.dll) and can also be included as a helper source for test bcl classes that need it.
For objects that die at the same time, normal finalizers should be run before critical finalizers. When we were unloading a domain we were running finalizers for all objects in that domain without checking whether the finalizer is critical or not. Now we enqueue the objects first in the finalization queues so we can run all the normal finalizers before the critical ones.
Fixes issues where ResponseStream.Close would throw an exception when
peer had already closed the connection and ResponseStream.Close would
still try to communicating to peer that it was closing.
The issue was brought to our attention by failures on:
- MonoTests.System.Net.WebRequestTest.TestReceiveCancelation
- MonoTests.System.Net.HttpWebRequestTest.TestLargeDataReading
This reverts commit 7adea6681b.
It seems this causes something that prevents the chroot from unmounting in the Linux builds on Jenkins,
resulting in an endless "can't unmount" loop and stuck builds. Reverting to see if this fixes it.
Wrapper stack frames don't have MVID or IL associated.
ex: (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
- Microsoft.WebApplication.targets was inadvertently removed.
- The new Microsoft.NuGet.Build.Tasks assembly sources were not getting dist'ed. Adding to DIST_ONLY_SUBDIRS fixes this for now,
though we can maybe add it to SUBDIRS instead (but neither me nor Marek know for what exactly that's used)
(cherry picked from commit 18de6419a1)
This fixes an inconsistency where xbuild would add System.Core
references to all builds, and MSBuild would only add them for
.NETFramework.
This change could break user XA/XI/XM projects on Mac if the user has
manually removed the explicit System.Core reference from the project,
however such projects would already be broken on Windows.
This is required for interoperating with project.json-based NuGet package configuration files
(not to be confused with DNX or AspNet Core which use it as a project file too) from MSBuild/xbuild.
The task hooks into the build process, resolves assembly references from the nuget package cache
and injects them into the compilation project so the csproj doesn't need to be modified when
adding new references/packages.
It is the first step on our road to PCL5/NuGet3.
The mutex on pending_bridge_callbacks was overkill because the init
model for the mono runtime already enforces the callbacks being set on
the init thread. Added a couple comments to formalize this.
Enable compact seq points by default so that the stack trace have IL
offsets instead os native ones.
This changes will allow any stacktrace generated from now on to be
symbolicated using mono-symbolicate.
Deprecates options gen-compact-seq-points.
This is cleanup to the previous emergency fix for bug 42469. The goal of that fix was to defer bridge processor initialization until after sgen init, but it introduced a requirement mono_gc_register_bridge_callback be called before sgen init. This breaks Xamarin Designer, which sets up callbacks late.
This patch introduces an sgen_init_bridge () which is called both at startup and after callbacks are set, and performs the initialization once all pieces are in place. Additional concurrency and style issues are also addressed.
mono_gc_register_bridge_callbacks is a function which the android implementation calls before initializing the runtime. One of the things this function previously did was initialize the default bridge processor if none has been initialized yet. However, because the function gets called before initializing the runtime, a bridge processor was never initialized. This lead to two bad effects:
- If a bridge processor was ever manually specified, the bridge processor would get initialized twice
- Tarjan crashes if initialized before sgen, so when Tarjan became the default bridge processor Android always crashed on launch
The default bridge processor is now initialized at the end of sgen_gc_init.
The reason why mkbundle is not working is that for some reason
this constant expression is producing different code than the
-24 that it is supposed to.
What is odd is that the return value from the lseek system
call is the same, but yet, read fails with an end-of-file with
this value, and otherwise it passes.
With this call:
lseek (fd, -(16+sizeof(uint64_t)), SEEK_END)
strace displays:
lseek(0x3, 0xFFFFFFE8, 0x2) = 19375512 0
With this call:
lseek (fd, -24, SEEK_END)
strace displays:
lseek(0x3, 0xFFFFFFFFFFFFFFE8, 0x2) = 19375512 0
Now, while both results from lseek are the same, a subsequent read
with 24 bytes to read returns zero on the former case, and 24 on
the latter case.
* Making sure ownerThread is not nulled unless lockCount is zero.
* Fixed so Locked property checks lockCount when determining if locked
instead of _locked field which was never set.
* Added test for setting Locked property.
* Added test capturing the original issue.
I saw this failing occasionally on iOS on Wrench:
```
[FAIL] TimerTest.AutoResetEventFalseStopsFiringElapsed : #2 wrong elapsedCount
Expected: 1
But was: 0
at MonoTests.System.Timers.TimerTest.AutoResetEventFalseStopsFiringElapsed () [0x00091] in /Users/builder/data/lanes/1411/b07703ed/source/xamarin-macios/external/mono/mcs/class/System/Test/System.Timers/TimerTest.cs:327
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /Users/builder/data/lanes/1411/b07703ed/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:309
```
MVID and AOTID stacktrace metadata no longer displays guids with '-'.
Metadata is now also using lowercase instead of uppercase.
Changed mono-symbolicate and mono --aot to use the minimalistic GUIDs.
Replaced static constructor with delayed initialization.
Fixed possible undeterministic order of dumped MVIDs.
Used AppendFormat when possible.
Added comment to StackTrace.AddMetadataHandler explaining that it can be
used with reflection by outside code, to avoid removing it or changing
its signature.
Added StringComparer.Ordinal to dictionary.
Extracted Symbolicate.Process stacktrace dump logic into DumpStackTrace
method. Which can also be used to dump the last stack trace after the
process loop.
mono-symbolicate can now be called:
symbolicate store-symbols <msym dir> [<dir>]+
This will store in <msym dir> all the managed symbols found in the
provided directories.
mono-symbolicate now parses stack traces with metadata at their end.
All the stack trace and metada are read, so that symbolicate can use
MVID and AOTID to retrieve the symbols from the msym directory..
The MVID metadata handler displays multiple MVID(s) followed by the
comma separated list of StackFrame lines whose method belongs to the
module with the displayed MVID.
Added a reflectable static method that can be used to customize data
displayed after a stack trace.
Added metadata handler that display the AOTID of the executing assembly.
AOTID is a GUID generated by the AOT compiler and stored along with the
native code.
The AOTID uniquely identifies the generated native assembly.
The AOTID is required when we want to map native offsets to IL offsets
and the sequence points mapping those values need to be determined.
libmonosgen-2.0.dll and breaks the build of monograph.exe which links against
it.
Also adds -O2 to CFLAGS on all platforms to align with how releases are built.
(cherry picked from commit ba79e78de5)
.NET supports this, from MSDN: "If an entry with the specified path and name already exists in the archive, a second entry is created with the same path and name.".
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42219.
This should fix some problem with Nuget archives created by Mono.
No test case for this one because it would involve writing a mini-ZIP file parser to make sure the value is written correctly (or hardcore the value offset in the generated file, which could lead to a very flakey test).
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=41775.
(cherry picked from commit 90ee90bbb8)
This was committed to the C7SR0 branch but missed from the main C7 branch (i.e. mono-4.4.0-branch).
Covers #41564
When multiple methods shared soap header a duplicate key exception would
be thrown.
This issue only occurs on XAMMAC and MOBILE where the non reference
source serializer is still used.
Fixes#41616
ConnectAsyncCallback now throws an exception when called a second time.
ConnectAsync reuses SocketAsyncEventArgs in BeginConnect which
already updates its state and calls ConnectAsyncCallback.
The changes make sure that ConnectAsyncCallback no longer called on
ConnectAsync catch block, while the state is still updated with the
exception.
Covers #41616
Our console runner is currently swallowing this test unhandled
exceptions.
To make it test show the failure I had to set legacyUnhandledExceptionPolicy
to 0 in mcs/class/lib/net_4_x/nunit-console.exe.config
Then running we can observe the crash by running:
make -C mcs/class/System check TESTNAME=System.Net.Sockets.SocketTest.ConnectAsyncUnhandledEx
Other tests appear to crash with legacyUEP set to 0, while the runner
shows no errors.
When we cleanup the gc at shutdown, we no longer care about running the remaining finalizers so we set the suspend_finalizers flag. This flag skipped running the finalizers but we would still traverse and modify the sgen finalizer hashes (on an arm box this could take even longer than 1s and lead to finalizer_thread_exited assertion failure since it's over the 100ms limit).
Avoid traversing sgen finalization hashes as soon as we see that we are shutting down.
Freeing a block descriptor (along with the block) can happen either when freeing a pointer in the block (we do this in order to prevent situations where we need to wait for the next alloc in order to free a block), either when we acquire an empty block while allocating. Either way, in order to free a block descriptor, we need to acquire it first.
When freeing a slot in a block, we are checking if the block is now empty in which case we acquire the block descriptor in order to free it (along with the block). While doing so, we were failing to account for the case where, by the time we make the block descriptor empty and the time we acquire it, another thread would try to allocate from the same descriptor, see that it's empty, free it, allocate a new descriptor (which would happen to have the same address as the retired one) to fulfill the allocation and then set it as the active descriptor for the allocator. When the first thread finally acquires the descriptor, we need to double check that it is still empty since it might be logically a different descriptor than the one we actually freed from. In the unlikely case it's not empty, we need to put it back in the allocator's data structures.
This would lead to a crash when unregistering the thread from the currnt MonoThreadInfo* destructor, as `mono_thread_info_current_unchecked` would return NULL. This would trigger an infinite loop in `mono_threads_enter_gc_safe_region_unbalanced_with_info`.
unregister_thread is the destructor for the current MonoThreadInfo* TLS key, meaning that when in this function, calling mono_thread_info_current_unchecked will always return NULL.
On Unix, errno value might change between the native call and the call to mono_marshal_set_last_error, if we exit the gc safe region. We need to ensure that it's done right after the native call.
As described in https://bugzilla.xamarin.com/show_bug.cgi?id=34883 at
least Samsung SG3 doesn't allow access to the local network
interface (in fact, even ifconfig doesn't work on the device). Therefore
the test needs to be disabled on those devices. This is the first part
of the fix, the second one will be implemented in Xamarin.Android test
suite which will provide the AndroidShouldPingWork method which will
check the device id against list of those that do not work for this
test.
Part #1 of fix for https://bugzilla.xamarin.com/show_bug.cgi?id=34883
MXR is not coming back (see https://bugzilla.mozilla.org/show_bug.cgi?id=1279952 and https://github.com/SuaveIO/mono-script-buildpack/issues/9#issuecomment-226749314)
and this broke everyone who still relied on mozroots. Fix by using the version from the hg repository instead.
Starting from Mono 3.12 we've been shipping cert-sync to import CA certificates from the local computer trust store to Mono's trust store,
we strongly encourage everyone to move to that instead. Added a deprecation warning to mozroots to further strengthen this message.
Note that when fetching the CA list over the new https URL we need to skip cert validation errors, as the common use case for mozroots is when
no certs are present on the system (bootstrapping). This enables MITM but we don't really have any other options. Another reason why cert-sync is superior.
(cherry picked from commit e8bd8e656d)
We're seeing it randomly fail on Wrench with durations of 1550 - 1800ms.
Talked to Zoltan, this might be normal so bumping the timeout to reduce the failure rate.
(cherry picked from commit 0fa5f5fb4f)
We're seeing the following exception on devices in CI:
```
[FAIL] HttpListenerRequestTest.HttpRequestIsLocal : System.Net.Sockets.SocketException : Could not resolve host 'XQA-AppleTV-1'
at System.Net.Dns.Error_11001 (System.String hostName) [0x00000] in /Users/builder/data/lanes/1381/d8287824/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/Dns.cs:292
at System.Net.Dns.GetHostByName (System.String hostName) [0x00024] in /Users/builder/data/lanes/1381/d8287824/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/Dns.cs:419
at System.Net.Dns.GetHostEntry (System.String hostNameOrAddress) [0x00061] in /Users/builder/data/lanes/1381/d8287824/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/Dns.cs:380
at System.Net.Dns.GetHostAddresses (System.String hostNameOrAddress) [0x00065] in /Users/builder/data/lanes/1381/d8287824/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/Dns.cs:406
at MonoTests.System.Net.HttpListenerRequestTest.HttpRequestIsLocal () [0x00006] in /Users/xamarinqa/QABot/data/lanes/2441/d8287824/source/xamarin-macios/external/mono/mcs/class/System/Test/System.Net/HttpListenerRequestTest.cs:197
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /Users/builder/data/lanes/1381/d8287824/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:309
```
From looking at the code, resolving the hostname to IP addresses via DNS fails so the test doesn't even run.
We don't care where the local IPs come from so we can use an alternative way of getting them - via querying the network interfaces.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=41671
PS: fixed a hardcoded port while I was touching the code
(cherry picked from commit 0178821e50)
We were not exiting the critical section on this code path. This was leading to hangs at stw since the native code would take locks when compiling the code.
We are still seeing quite rare failures of this test:
```
HostEntry should result in the same IPHostEntry object.
Expected: same as <System.Net.IPHostEntry>
But was: <System.Net.IPHostEntry>
+++++++++++++++++++
STACK TRACE:
at MonoTests.System.Net.ServicePointTest.DnsRefreshTimeout () [0x00045] in /Users/builder/jenkins/workspace/test-mono-mainline/label/osx-i386/mcs/class/System/Test/System.Net/ServicePointTest.cs:228
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /Users/builder/jenkins/workspace/test-mono-mainline/label/osx-i386/mcs/class/corlib/System.Reflection/MonoMethod.cs:309
```
Since the test is only checking if expiring of the cached DNS lookup works, we don't really need to use an external domain name for this
as "localhost" works just fine as well and typically doesn't trigger a DNS query which makes the test more reliable against network issues.
(cherry picked from commit ea4ad81185)
Without doing this, the check would consider a valid option name like
IpTimeToLive to be the same as ReuseAddress, as they share the same enum
value.
Fixes SocketTest.TtlChange and SocketTest.UdpMulticasTimeToLive on Android and
other platforms that don't support ReuseAddress.
Before closing a socket we try to interrupt and wait for all socket
pending system calls to return. For that we set the socket as non
blocking which makes the system calls to return with EAGAIN\EWOULDBLOCK.
This issue was caught by System.Net.Sockets.SocketTest.SendAsyncFile
that was intermittently failing, we were only able to reproduce it while
running multiple test suites in parallel. We now can say that it only failed when
the the socket was closed and the sendfile was waiting for the OS to
allocate enough resources to send the file.
This was fixed by changing wapi_sendfile to return on EAGAIN.
Stack traces of blocked threads are now displayed on the exception.
The threads stacktraces are only stored when MONO_TESTS_IN_PROGRESS env
var is set to yes.
We were seeing intermittent failures on devices like:
```
[FAIL] TimerTest.EnabledInElapsed : #1 loss of events
Expected: True
But was: False
at MonoTests.System.Timers.TimerTest.EnabledInElapsed () <0x33b168 + 0x0014f> in <filename unknown>:0
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object , BindingFlags , System.Reflection.Binder , System.Object[] , System.Globalization.CultureInfo ) <0x66d400 + 0x000b7> in <filename unknown>:0
```
Since OS scheduling and GCs can influence the exact timing of the timer intervals sleeping for 200ms is likely just too short.
Instead of bumping the timeout, I decided to rewrite the test to not rely on Thread.Sleep ().
I also added another test which verifies setting AutoResetEvent=false stops the timer event from firing after the first one,
which was implicitly tested before by the other test.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=41530
(cherry picked from commit bab393e095)
When we allocate the complex descriptor for a new type we might have the concurrent marker scan objects. While the concurrent marker scans the bitmap for a complex object, which resides in the complex_descriptor array, another thread might allocate a new complex descriptor, thus reallocate the complex_descriptor array, which would lead to corruption of the bitmap used by the concurrent marker.
We reuse the SgenArrayList which never reallocates, using instead a list of exponentially increasing arrays.
This appears to serve no purpose as tlab_next is a thread-local variable. It's
only ever accessed by other threads in order to write to it, not read it. We
also don't do this memory barrier in the native allocation functions.
Previously, if we stopped a thread and it had a stack looking like this:
...
profiler_signal_handler
<signal handler called>
managed_allocator
...
We would fail to identify the fact that we've just interrupted a thread that's
in a managed allocator (uninterruptible code) because we only look at the very
latest instruction pointer of the thread, which will be pointing into
profiler_signal_handler or some other function called by it. So we would
happily continue along with the STW process and proceed to doing the actual GC,
where we would see a broken heap.
We could solve this by unwinding the stack and checking all frames, but that's
complicated and error-prone. Instead, register a critical region while the
managed allocator. This way, if we don't identify it by instruction pointer, we
will identify it by the fact that the thread is in a critical region.
This fixes a regression introduced in 123aaa5db0.
We would sometimes see an ObjectDisposedException when calling AcceptAsync ()
because that patch removed the try-catch around it.
(cherry picked from commit d9fc8f866a)
* Debugger stepping does not understand recursion (bug 38025)
Testing with the debugger showed multiple failures around recursive
functions:
1. Step-out breakpoints can trigger on inner frames.
Solution: Extend existing inner-frame test from step-over to step out
also.
2. "Same source line, continuing" test triggers even on wrong frame
(affects both step over and step out)
Solution: Check frame as part of this test
3. When step breakpoints pick next sequence points, they can register
multiple breakpoints at the same place if a recursive call happens to
exit there. (Possibly harmless but may have resulted in duplicate
debugger events being generated in one test.)
Solution: Check for duplicates in step-request breakpoint list before
adding new breakpoints
4. Mono.Debugging.Soft in the debugger-libs repo sometimes resumes
after breakpoints if it decides Mono.Debugger.Soft has not stepped
far enough, but the check to trigger this was not recursion-aware.
Solution: This one is not addressed in this commit, but a fix was
checked in to debugger-libs. Any code using this commit therefore
needs to be run in conjunction with debugger-libs dfbf6d8b13.
Tests were also added for issues 1-3 above.
* Fixes for peer feedback on recursive step patch
Made breakpoint uniqueness test switch to a hash after 8th item added,
consolidated the code that handles the uniqueness test and makes the
breakpoint in one function, fixed style issues.
* Halve the stack depth for both the new single step tests
Xamarin.Android internal test suite takes advantage of the BCL tests
in Mono, so far their inclusion was a manual process and this commit
fixes it. A set of source inclusion and exclusion lists is added
and a handful of code fixes to make them compile on Xamarin.Android
SendAsyncFile uses StartSocketServer to start a server and at the same
time begins to accept a client socket. The issue is that when
the accept callback is called SendAsyncFile already closed the socket.
Fixed race condition by moving StartSocketServer and ReceiveCallback
logic into SendAsyncFile, and by using a new MRE to ensure that the
client receives all the data before the server is closed.
Fixes#41264
(cherry picked from commit 0693884580)
The test assembly used *.dll.excludes before while the library uses *.dll.exclude.sources.
Make this consistent to reduce confusion.
Also remove corlib_cmp.dll.excludes which is from a profile that no longer exists.
(cherry picked from commit 87b1150070)
The date/time struct in zip.h defines the month value as
uInt tm_mon; /* months since January - [0,11] */
.NET's DateTime object in turn stores months from 1-12. We didn't
take this into account when passing the values from managed->native
so the month ended up being off by one.
We didn't use this codepath before https://github.com/mono/mono/pull/3036
that's why it didn't show up until now.
(cherry picked from commit 07baa1cce4)
We were seeing timeouts in the runtime step if two builds happen to be scheduled
at the exact same time on the ARM workers on Jenkins, causing increased load and
longer test times. They aborted the runtime step right before it'd finish.
Bump the timeout a bit to avoid this.
If we have binary protocol enabled and the debugged executable will spawn a new mono instance, both of the processes will write to the same debug file, because they share the environment variable, producing invalid results. Add a pid sufix if output file exists in order to separately obtain binary protocol outputs from all mono instances.
We had the problem that we were logging at sweep finish which can happen anytime. If we happen to suspend a thread, while doing a GC, in the middle of logging we might deadlock if we wait for sweep to finish (which can be blocked waiting for logging).
When logging multiple collections in the same stw pause, we would not log the real results for both collections since we would compute the major sizes at the very end.
We logged collections in sgen_perform_collection, outside the function calls for specific collections, which makes it cumbersome to add additional log entries.
This refactoring solves these issues by having GC code log info at any time. This log entry is enqueued in a list of pending entries. We flush the list of logs, as before, at the end of the stw, when the world is restarted.
A while ago, we switched our PR builders from using the Github-provided PR branch head (i.e. `pull/merge/XXXX`) to the exact sha1 of the latest commit on the PR (i.e. `abcdefabc`). This change was needed for the performance team, otherwise the changes they wanted to test would be rebased against a new master several times per day, breaking their ability to meaningfully track the changes in a PR.
As a side effect of this, `version.h` went from setting the version from `XXXX/abcdefabc` to `explicit/abcdefabc`, and the Jenkins PR packages use the branch name `explicit` instead of the actual PR number `XXXX`, which is pretty bad from a UX perspective.
This change checks for the `ghprbPullId` environment variable and uses it instead of the actual branch name or `explicit` - it is always set on PR builds.
[mini] small improvements around `MONO_IS_CALL` macro and `local-prop`
showed up in vtunes when running
```
./mono/mini/mono-sgen --stats -O=all --aot=threads=1,asmonly ~/work/benchmarker/tests/fsharp/lib/bootstrap/4.0/FSharp.Compiler.dll
```
also, I think when `OP_DYN_CALL` was introduced, it wasn't added to the `MONO_IS_CALL` macro. @vargaz, what do you think?
* Merge System/MonoType.cs into ReferenceSources/RuntimeType.cs, use the latter in the runtime code.
* [sdb] Fix a test which referenced System.MonoType.
MonoError must be explicitly reinitialized if it is used again after
mono_error_cleanup. But it is better to introduce an explicit inner
MonoError when possible.
I had a look in vtunes for
```
$ mono/mini/mono-sgen --stats -O=all --aot=threads=1,asmonly ~/work/benchmarker/tests/fsharp/lib/bootstrap/4.0/FSharp.Compiler.dll
```
and that method showed up as hot.
With this change I get an improvement from ~27.4s down to ~24.3s on my
machine.
This change makes it not need to throw exceptions.
In the runtime, use mono_thread_internal_stop instead of mono_thread_stop.
mono_thread_internal_stop only stops other threads, which is what runtime usage calls for.
mono_thread_stop will stop the current thread by raising and exception.
The first argument is whether it's a single or multiple wait. We can infer that from the number of handles.
The second argument is alertable. All waits are alertable, so I baked the constant directly.
They can only ever fail if allocation fails, which we can't recover
from, so let's just assert that we succeed.
This in turn simplifies the error handling in a lot of other thread routines.
Since referencesource is no longer a submodule under external/ after https://github.com/mono/mono/pull/2962
it didn't get included in the tarball anymore.
We need to add it again to fix the tarball build.
In https://github.com/mono/mono/commit/4164d9fbe19a1e6acb3a5ea122891020ff2868d7 the 14.0 target was updated to include
the DependsOnSystemRuntime property. However, it wasn't actually implemented in the code behind. This resulted in the
property not being set/true even if we depend on assemblies that reference System.Runtime, which in turn leads us to not
adding the Facades in ImplicitlyExpandDesignTimeFacades. This then breaks compilation because the System.Runtime lookup fails.
The fix is to actually implement the property.
This change makes it not need to throw exceptions.
In the runtime, use mono_thread_internal_stop instead of mono_thread_stop.
mono_thread_internal_stop only stops other threads, which is what runtime usage calls for.
mono_thread_stop will stop the current thread by raising and exception.
The first argument is whether it's a single or multiple wait. We can infer that from the number of handles.
The second argument is alertable. All waits are alertable, so I baked the constant directly.
When running mono/mini aot-tests.exe on full AOT:ed Windows build the following
warning message is incorrectly written to the console several times:
g_warning ("failed to get 100ns ticks");
This happens since the check triggering this warning is not taking into account
that startup time between first call to mono_ns100_ticks storing the start time
and the launch of the thread pool requesting 100ns ticks and convert it to
elapsed seconds can occur during the first second of the process lifetime.
This is more likely to happen in a full AOT:ed image since nothing needs to
be JIT:ed at startup, making it more likely to exploit the problem reaching this
code within the first second of process lifetime.
The purpose of the check is to validate that mono_ns100_ticks is returning “valid”
data, but currently it checks it after converting it to seconds, meaning that the result
of the division gets truncated to 0 during the first second. The only invalid, deterministic error
from mono_ns100_ticks that could be used to trigger this warning is either 0 or a negative time value,
but currently it just checks for 0 as an error. The fix is to make the check on the return from
mono_ns100_ticks instead of on the result of the division by 10 000 000 as it is today.
On the new Jenkins infrastructure with the new ARM64 bots we're hitting the 2h runtime test
limit right before they'd be finished. Bump the timeout a bit to fix this.
They fail on the new Jenkins ARM64 bots, disable them until we figure out the cause.
It looks like a libgdiplus issue to me which makes deploying a fixed version a bit more difficult.
Filed https://bugzilla.xamarin.com/show_bug.cgi?id=41171 to track.
Due to performance concerns, we don't account for the space wasted in blocks, since it would require dereferencing all vtables. We also don't account for allocation that might happen during sweep.
As with mono_runtime_invoke, we split up into three functions:
1. mono_runtime_invoke_array is external only. Runtime should use
one of the other two.
2. mono_runtime_invoke_array_checked threads exceptions and failures through
MonoError in coop mode. In non-coop managed exceptions unwind the
stack as before.
3. mono_runtime_try_invoke_array sends exceptions from method invocation
through a MonoException* outarg, and other errors through MonoError.
When one of the tests doesn't fail with the expected exception, we'd get just a useless message
like "TtlChangeoverflow #3" instead of the much more useful exception info + stacktrace that
NUnit catches when an unhandled exception is thrown.
During mono shutdown on windows there is, on rare occasions, an exception
in CloseHandle indicating the closure of an invalid handle. This exception
is normally a strong indication of a much more severe underlying problem,
a doubled close on a handle, that could lead to random data corruptions
and very hard to track bugs in random areas of the code. If the system reused
the handle for a different object that silently gets incorrectly closed by above
call, the above bug could manifest itself into some completely different issues,
so getting a consistent repro with the above exception with CloseHandle was a good
point to start tracking down the bug.
The exception throwed in CloseHandle boils down to the fact that threads
in thread pool's that have their finalizer run on shutdown can end up in
a scenario where the thread handle has already been closed elsewhere.
When the threads are finalized in mono_gc_finialize_threadpool_threads,
the handle on the internal thread object, stored in MonoInternalThread,
will be closed by the thread objects finializer in ves_icall_System_Threading_InternalThread_Thread_free_internal.
Under some rare occasions, one of the handles are not valid and will cause
the exception in CloseHandle or even worse, pointing to a complete different
kind of object, potentially closed by the threads finalizer, causing an instable
process with random behavior related to WIN32 objects (files, mutexes, threads, events etc).
The reason why it rarely manifests itself is due to a race between the finializer thread and the
main thread during shutdown. Main thread gives finalizer thread 2000 ms to complete its work and
if it’s time outs, the main thread won’t call the code that will run the finalizers,
mono_gc_finalize_threadpool_threads for the thread objects added to an internal list
by the finalizer thread, so if the finalizer takes more than 2000 ms to complete during shutdown
the bug won’t manifest itself. By increasing the wait time in mono_domain_finalize to for example INFINITE,
the repro of this bug will be 100% and not timing dependent anymore.
Turns out that the offending CloseHandle on the thread object happens at an
earlier point in time during shutdown, when mono_thread_manage tries to wait for
threads to complete manage thread it will do remove_and_abort_threads for background
threads still around. The way this works is that it will setup a list of thread handles
that it will wait for using WaitForMultipleObjects and then close the handles.
It turns out that the implementation of remove_and_abort_threads adds the wrong
handle to the list, not the duplicated handle but the original thread handle,
and that will trigger this bug, since the original thread handle will be closed
(and the duplicated handle will leak). The fix is to use the duplicated thread
handle instead of the original, similar to how it was already done in other similar methods, like build_wait_tids.
Sinces this bug seems to been around for quite some time and is a race related to managed threads and mono
shutdown sequence it could give other weird side effects on windows during shutdown
(since it could have closed a random WIN32 object instead of the intended thread).
Looking at Bugzilla, there is one bug that very well could be related to this specific problem, 10108:
https://bugzilla.xamarin.com/show_bug.cgi?id=10108
This bug should most likely be retested with this fix to see if it still reproducible once merged into master.
* Adds support for a handful of new command line arguments that we now use in the build.
* Always attempts to match to a project name, to get the proper project dependency.
* Update to support .exe and .dll in the generation.
* Remove warnings and some dead code
* Update the resulting csproj files based on running:
make update-csproj
make package-inputs
mono genproj.exe
bridge_processing_in_progress is set by sgen-tarjan-bridge.c at the
start and end of each bridge, although sgen-bridge sets this flag at
the same time.
There is some code that reports the length of a gc bridge operation in
sgen-stw. This is dead code dating from a period when gc bridges
occurred synchronously as part of a stop the world operation. The new
code globally tracks the end of the last stw and reports time since
then when the bridge thread completes.
This allow the removal of MONO_(PREPARE|FINISH)BLOCKING and MONO(PREPARE|FINISH)_RESET_BLOCKING, which were not explicit in their intent, leading to confusion. This also harmonize how the product and the runtime transition between thread states.
`Assertion.Fail` isn't supported by NUnitLite in monodroid, use `Assert` instead to fix the build:
```
[12:29:46] /Users/builder/data/lanes/1845/88dbce4e/source/monodroid/tests/bcl-test/BclTests.csproj (default targets) ->
[12:29:46] /Library/Frameworks/Mono.framework/Versions/4.2.1/lib/mono/4.5/Microsoft.CSharp.Targets (CoreCompile target) ->
[12:29:46]
[12:29:46] src/System/Test/System.Net/IPAddressTest.cs(282,4): error CS0103: The name `Assertion' does not exist in the current context
[12:29:46]
[12:29:46] 14 Warning(s)
[12:29:46] 1 Error(s)
```
This will be used by managed-to-native and native-to-managed wrappers, as well as trampolines, to switch between states. This is necessary as their calls might be unbalanced (a prepare blocking, without the corresponding finish blocking).
If we do not throw it from the finally clause, then in case of a Thread.Abort, we will not rethrow the catched exception, but the ThreadAbortException, and this without using the callback, leading to native stack unwinding.
The support will be broken when we will introduce MonoHandle, as we can't pass the exception naked object ref around, but need to wrap it in a MonoHandle, while still in RUNNING mode.
Fixes needed in order to get full AOT working for mono/mini regression tests on windows x64 builds (currently disabling the gsharedvt and dyncall tests).
Fixes are done in two main areas, exception and thread local storage support.
SEH exception handling enabled in order to catch CPU generated exceptions. All exception related regression tests pass with fix and runtime won't attempt to JIT any code since it is prevented when running with --full-aot startup argument.
Thread local storage (TLS) support was not complete on windows x64. When running full AOT the sgen allocator uses TLS for its allocator. AOT compiler will issues OP_TLS_GET_REG operations when compiling the “Alloc” methods. On win x64 this was not implemented, just the default OP_TLS_GET, meaning that methods compiled using OP_TLS_GET_REG was "silently" dropped when full AOT:ed compiled. At runtime, the sgen allocator would tried to locate the “Alloc” method but fails to find it and since this will cause an exception (that also needs the allocator to be used) and the program entered a loop that will end in a stack overflow crashing the program.
The fix implements OP_TLS_GET_REG and other needed support for win x64. It also fixes the offset calculation needed by full AOT builds (translating to correct runtime TLS key) for keys used by mono/mini regression tests. Since windows TLS support is a little more complex (two different conditions depending on where the TLS key gets allocated) the max size of tls_get_reg for amd64 has been increased from 32 to 64 byte as well.
OpenSSL and the MS RSA CSP (Win7 x64) generate RSA keys with a public exponent of 65537. Smaller public exponents are not allowed by some specifications and may be rejected by other software.
* Start of packager for mkbundle
* [mkbundle] Add fingerprint and directory at end of package
* [mkbundle] Update embedding file format, add support for --options, support for --runtime, chmod +x
* [driver] Add a way to parse arbitrary command line options
* [mkbundle] Align the assemblies on a page boundary, do not release directory buffer
* Work-around odd compiler error by using memcpy
* Update makefile
* Fix the windows build
* [Windows] Export the new mono_parse_options_from API
* Implement the various cross-compiler target fetching commands
* Clean
Marked it external only. Runtime should use mono_string_from_utf16_checked
Also mark mono_string_from_utf32 as external only.
(It's currently unused in the runtime, but in the future
mono_string_from_utf32_checked is available)
The actual meaning is about whether or not the name is allowed to change after
having been set by the mono_thread_set_name_internal () call; it has nothing to
do with whether the call came from managed per se.
If we did consider this, an internal runtime thread that had its name set by a
mono_thread_set_name_internal (..., FALSE /*managed*/, ...) call prior to
starting would not have its native name set nor its name reported to the
profiler.
Process.Exited event can get invoked before the process has really
exited. So, accessing process.ExitCode before the real exit can throw:
Error executing task Exec: System.InvalidOperationException: Process must exit before requested information can be determined.
at System.Diagnostics.Process.EnsureState (System.Diagnostics.Process+State state) [0x000b9] in /Users/ankit/dev/mono/mcs/class/referencesource/System/services/monitoring/system/diagnosticts/Process.cs:1439
at System.Diagnostics.Process.get_ExitCode () [0x00000] in /Users/ankit/dev/mono/mcs/class/referencesource/System/services/monitoring/system/diagnosticts/Process.cs:219
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:get_ExitCode ()
at Microsoft.Build.Utilities.ToolTask.ExecuteTool (System.String pathToTool, System.String responseFileCommands, System.String commandLineCommands) [0x00101] in /Users/ankit/dev/mono/mcs/class/Microsoft.Build.Utilities/Microsoft.Build.Utilities/ToolTask.cs:185
at Microsoft.Build.Tasks.Exec.ExecuteTool (System.String pathToTool, System.String responseFileCommands, System.String commandLineCommands) [0x00026] in /Users/ankit/dev/mono/mcs/class/Microsoft.Build.Tasks/Microsoft.Build.Tasks/Exec.cs:83
at Microsoft.Build.Utilities.ToolTask.Execute () [0x0001c] in /Users/ankit/dev/mono/mcs/class/Microsoft.Build.Utilities/Microsoft.Build.Utilities/ToolTask.cs:128
at Microsoft.Build.BuildEngine.TaskEngine.Execute () [0x00000] in /Users/ankit/dev/mono/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/TaskEngine.cs:134
at Microsoft.Build.BuildEngine.BuildTask.Execute () [0x0008f] in /Users/ankit/dev/mono/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/BuildTask.cs:101
ProcessWrapper.WaitForOutput depends on the
`endEventExit`(ManualResetEvent) to be set, which is done in the event
handler for Process.Exited . So, effectively, WaitForOutput can return
before the process has really exited and ToolTask ends up throwing an
exception when it accesses the ExitCode.
Fix: Add a WaitForExit in WaitForOutput, to be sure!
Some existing tests only works when IPv4 is available, which was fine as
it match how we historically ran those tests.
Apple has a new requirement [1] to test under an "IPv6 only network" and
the change facilitate this.
[1] https://developer.apple.com/news/?id=05042016a
We were seeing compilation errors like this:
```
System.Security.Cryptography.X509Certificates/X509EnhancedKeyUsageExtension.cs(71,20): error CS0135: `oid' conflicts with a declaration in a child block
System.Security.Cryptography.X509Certificates/X509EnhancedKeyUsageExtension.cs(74,17): (Location of the symbol related to previous error)
Compilation failed: 1 error(s), 38 warnings
make[9]: *** [../../class/lib/basic/secxml/System.dll] Error 1
```
The new foreach variables introduced in 4a6e5d24e1 with the same "oid" name as the global constant apparently confused the older mcs compiler.
Rename them to avoid this.
We were seeing random failures of the test in the iOS simulator and devices.
Looking at the test, it seems to me like the 1000ms timeout is just too short. In a single iteration we're
enqueuing 500 threadpool workitems and effectively give each of them just 2ms to run and signal the CountdownEvent.
Indeed I can easily repro this in the simulator on my box if I lower the timeout to 900ms.
This may be a side effect of the new threadpool which starts threads more conservatively afaik.
Bumping the timeout should fix this.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=39305
https://bugzilla.xamarin.com/show_bug.cgi?id=8554
Added unit tests to cover the Asynchronous API and the Task Based API.
Fixed an issue with the Task Based API that would previously throw a
`NullReferenceException` when started as the `CancellationTokenSource`
was not instantiated.
The `CancellationTokenSource` was also not disposed of after use.
It seems that older iOS6 devices don't allow access to /dev, as a DirectoryNotFoundException is
thrown when accessing /dev/zero in the FileStreamTest.OpenCharDeviceRepeatedly test there.
Ignore this exception and the test in those cases.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=40462
AppleTLS supports a lazily-initialized X509Certificate, but not X509Certificate2 so
we need to fall-back to using Mono.Security.X509 whenever we need an X509Certificate2.
To avoid unnecessary fallbacks, the private Mono.Net.Security APIs use X509Certificate
instead of X509Certificate2.
Since 'ExtraStore' returns X509Certificate2Collection, we need to convert these to
X509Certificate2.
(cherry picked from commit 05e2372339)
The sgen tests should run on Windows x86 too (or at least they will in the near future), so we can get rid of all the special casing.
This also fixes an error in the ARM64 test run on Jenkins because test-sgen wasn't defined there.
There would previously be a race between reset blocking and a STW, that would happen as follow:
- T1 goes into blocking - T1 state is BLOCKING(0)
- T2 suspends T1 - T1 state is BLOCKING(1)
- T1 does a reset blocking -> T1 state is SELF SUSPEND REQUESTED
- T2 calls `mono_thread_info_get_suspend_state` on T1 --- SIGABRT as we cannot get suspend state when state is SELF SUSPEND REQUESTED
- T1 does polling -> T1 state is SELF SUSPENDED
The fix is for reset blocking to work like done blocking: switch to BLOCKING AND SUSPENDED.
We were seeing issues on the new Jenkins Azure boxes which had 7GB of physical memory but only about 1GB free at the time of the test, so it'd fail.
Checking the available memory should fix this.
If there's a pending exception and an AbortRequested, the pending
excpetion should be thrown.
This fixes threadpool-exceptions5.cs in coop mode.
There are two situations where this could happen.
1. Like threadpool-exception5.cs we have a threadpool worker thread
request an abort and then its async callback runs and throws an
exception. In this case apparently the async callback's exception
should be allowed to unwind and hit the unhandled exception handler.
2. A thread catches ThreadAbortException and then posts a pending
exception of some other excpetion from the catch clause. In this case
we unwind that other exception, and when it's caught we rethrow the
abort.
When the system certificate validator is invoked with an empty hostname
(caller does not want us to validate the hostname), then we need to pass
null to SecPolicyCreateSSL() and not the empty string.
(cherry picked from commit f29804d149)
Only attempt the automatic PEM->DER conversion when no password was given
and make this version of X509Helper.Import() also take care of the fallback.
(cherry picked from commit 63a214c3f0)
Previously we had separate codepaths in mono_jit_thread_attach for coop
and non-coop modes.
Now there's a common codepath for attaching the thread, and separate
code paths for performing coop state machine transitions and computing
the return value cookie.
Fixes Mono.Debugger.Soft tests under coop.
By skipping thread which have a NULL domain or lmf, we would also skip threads which are in RUNNING mode, but which are still attaching to the runtime. They could have in practice called `mono_thread_info_attach`, without having executed any managed code or set a domain yet. Because we would simply discard the suspending for these threads, we could potentially not suspend them, meaning they would keep running, even potentially run some managed code, while the GC is still running.
To fix that, we need to slightly modify the expectation of sgen STW when using unified suspend. The non-unified expectation of sgen STW would be that even if a thread has a non-initialized domain or lmf, it should still be suspended, but not scanned. The unified expectation of sgen STW would be that if a thread has a non-initialized domain or lmf, then the thread shouldn't be suspended. That second behaviour works well if we only consider that the domain or lmf could be non-initialized when the thread is detaching, but that consideration is just wrong. The main example is the beginning of `start_wrapper_internal` (in threads.c:637), where we have non-initialized domain and lmf, but where the thread is in RUNNING state, and about to access some managed memory.
On coop, an OS thread can go
STARTING->BLOCKING->RUNNING->BLOCKING->DETACHED.
For ordinary managed threads BLOCKING->DETACHED is still a bug like
before, but hopefully the checked build machinery will help us to catch
those situations.
When checked build is running with GC checking, it uses a non-zero
integer cookie instead of the thread info pointer.
This fixes pinvoke3.cs test_43_thread_attach() in checked coop builds.
This is going to be used by Xamarin.iOS to convert a managed exception to an Obj-C exception. We need this as if we don't do that, the managed exception mechanism will unwind native stack, which can lead to undefined behavior. By converting to an Obj-C exception, we let the native exception mechanism do what it's supposed to do.
Because mono_msec_ticks would previously return the number of milliseconds since boot time, it would overflow after ~49 days. That would mean that anywhere you would use it, you would need to be careful of arithmetic overflow.
By simply returning a gint64 we will not overflow before ~30k years.
Fixes bug https://bugzilla.xamarin.com/show_bug.cgi?id=38666
We load these at runtime so they will naturally have undefined symbols. This is
not allowed under bitcode, though, and we use the static libraries in that case
anyway.
Operating systems that use linux kernels before 3.9 do not support
binding multiple socket to the same address and port.
In those systems even if it was possible to set
SocketOptionName.ReuseAddress an exception was thrown on the second
bind.
These changes introduce a SocketException that is thrown when
SocketOptionName.ReuseAddress is set on a system that will fail to bind
multiple times to the same address.
2016-02-17 09:56:14 +00:00
1623 changed files with 63242 additions and 45695 deletions
if test x$with_sgen_default_concurrent != xno; then
AC_DEFINE(HAVE_CONC_GC_AS_DEFAULT, 1, [Defaults to concurrent GC])
conc_gc_msg=" (concurrent by default)"
fi
if test "x$gc_msg" = "x"; then
gc_msg="sgen$conc_gc_msg"
else
gc_msg="sgen$conc_gc_msg and $gc_msg"
fi
fi
AC_SUBST(SGEN_DEFINES)
AM_CONDITIONAL(SUPPORT_SGEN, test x$buildsgen = xyes)
@@ -3447,6 +3459,10 @@ if test ${TARGET} = ARM; then
])
fi
if test x$arm_fpu != x; then
fpu=$arm_fpu
fi
AC_MSG_RESULT($fpu)
CPPFLAGS="$CPPFLAGS -DARM_FPU_$fpu=1"
unset fpu
@@ -3510,12 +3526,6 @@ if test ${TARGET} = ARM; then
fi
fi
if test ${TARGET} = ARM; then
if test "x${with_jumptables}" = "xyes"; then
AC_DEFINE(USE_JUMP_TABLES, 1, Use jump tables in JIT)
fi
fi
if test ${TARGET} = unknown; then
CPPFLAGS="$CPPFLAGS -DNO_PORT"
AC_MSG_WARN("mono has not been ported to $host: some things may not work.")
@@ -3558,20 +3568,6 @@ AC_ARG_WITH(bitcode, [ --with-bitcode=yes,no If bitcode is en
AC_ARG_WITH(xammac, [ --with-xammac=yes,no If you want to build the Xamarin.Mac assemblies (defaults to no)], [], [with_xammac=no])
AC_ARG_WITH(mobile_static, [ --with-mobile_static=yes,no If you want to build the mobile_static assemblies (defaults to no)], [], [with_mobile_static=no])
OPROFILE=no
AC_ARG_WITH(oprofile,[ --with-oprofile=no,<oprofile install dir> Enable oprofile support (defaults to no)],[
if test x$with_oprofile != xno; then
oprofile_include=$with_oprofile/include
if test ! -f $oprofile_include/opagent.h; then
AC_MSG_ERROR([oprofile include file not found at $oprofile_include/opagent.h])
AC_ARG_WITH(malloc_mempools,[ --with-malloc-mempools=yes,no Use malloc for each single mempool allocation (only for runtime debugging, defaults to no)],[
AC_ARG_ENABLE(checked_build, [ --enable-checked-build=LIST To enable checked build (expensive asserts), configure with a comma-separated LIST of checked build modules and then include that same list in the environment variable MONO_CHECK_MODE at runtime. Recognized checked build modules: all, gc, metadata, thread],[
if test x$enable_checked_build != x ; then
@@ -3681,10 +3679,6 @@ AC_SUBST(docs_dir)
## Maybe should also disable if mcsdir is invalid. Let's punt the issue for now.
AM_CONDITIONAL(BUILD_MCS, [test x$cross_compiling = xno && test x$enable_mcs_build != xno])
AM_CONDITIONAL(HAVE_OPROFILE, test x$OPROFILE = xyes)
Loaded 100 of 1623 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.