The code in X509Store only checked the legacy X509StoreManager.LocalMachinePath
when deciding whether CspProviderFlags.UseMachineKeyStore should be
set when dealing with private keys.
However, with BTLS we now also have X509StoreManager.NewLocalMachinePath.
This caused an issue/regression in the recent 4.8 builds that
started importing into the BTLS cert store when using cert-sync:
https://github.com/mono/mono/commit/968c60a0f32c7a4c92568c3feda7503f99b98c94
Because cert-sync is ran at package installation time as root
on Linux we would erroneously not pass UseMachineKeyStore (because
the paths between legacy and BTLS store are different), which in turn
causes KeyPairPersistence to look into the user directory, e.g.
~/.config/.mono/keypairs. Since on a clean machine that directory
hierarchy doesn't exist, it goes on and creates it -> we now have
the ~/.config directory owned by root which means a normal user
application can't write to it anymore (like NuGet) and crashes.
The fix is to check for the BTLS store path as well so KeyPairPersistence
uses the machine path and ~/.config is not created on package installation.
See e.g. https://github.com/travis-ci/travis-ci/issues/7271
(cherry picked from commit 12b44a0e94)
This removes the need to set core.autocrlf=input when compiling
Mono on Windows via cygwin.
Always use LF line endings for config.rpath/configure.host as well
Add .sh extension to update_submodules so it gets correct line ending
Cherry-picked from:
b112f67c45a16103f4b48aba97e42a7ef66cccd9
Older Cecil (0.9) were saving the scope of the actual type. This changed
with Cecil 0.10 where the ExportedType.Scope is saved, which could lead
to invalid assemblies being saved with the current linker code (which
could not update the scope without the cecil bump)
reference:
https://bugzilla.xamarin.com/show_bug.cgi?id=51805
should fix https://bugzilla.xamarin.com/show_bug.cgi?id=41133
The bug exposed itself as System.MethodAccessExceptionMethod which is
thrown when a method isn't allowed to call another method. Several
things are checked for this: Among other things, the runtime checks if
the public key in the friend list of the caller assembly matches the
public key in the callee assembly.
The issue here was that two threads computed the SHA1 at the same time,
using the same buffer and therefore computing garbage.
image.c (do_mono_image_load): Ignore refonly images when checking for ignorable assemblies.
icall.c (ves_icall_System_Reflection_Assembly_InternalGetAssemblyName): Load the image in ref only mode.
This tool takes a folder full of nupkg file and print macro invocations to add to image.c:ignored_assemblies.
Usage:
make download
make run
The download.sh script downloads all nugets we want to ignore. So update it before running if you have more you want to add.
Matching is done by assembly name and GUID. This should be enough to narrow
down the effect.
This approach is not very efficient as it waits for quite some of the loading
process to be on its way before detection. It works in the case that only a few
rare images are matched.
* [aot] Rewrite the AOT profiler. (#4176)
* [aot] Rewrite the AOT profiler.
The AOT profiler is used to collect the methods/instances used at runtime.
Usage:
* mono --profile=aot:output=out.aotprof <app>
* mono --aot=profile=out.aotprof <assembly>
The first command will run <app>, collecting the list of methods it uses into the out.aotprof file.
The second command passes the profile file to the AOT compiler to have it generate additional generic instances.
* [docs] Add a section for AOT profiling to the mono(1) man page.
* [build] Collect AOT profile info from building mscorlib in the build profile, and use it when aot-ing the csc assemblies.
* [runtime] Add missing mono-profiler-aot.h file.
* [runtime] Tidy up usage of MONO_PROFILER_API.
* [aot] Improve error handling in the aot profiler.
* [aot] Improve error handling, add more diagnostics.
* [aot] Add support for loading multiple profile files.
* [aot] Add a 'verbose' argument to the aot compiler, use it to print out more information on profile loading errors.
* [aot] Add 'profile-only' argument to only aot methods found in the profile file.
* [aot] Fix the build.
* Lldb support code for mono (#4225)
* Initial version of lldb support code.
* [lldb] Keep all the debug entries in a linked list so lldb can read it after attaching to the process.
* [lldb] Add an --enable-minimal=lldb option to disable the code. Add locking.
* [lldb] Add support for dynamic methods by adding a new protocol entry for unloading code regions.
* [lldb] Add beginnings of a test suite using the lldb python bindings.
* [lldb] Emit line number info for methods, not used yet.
* [lldb] Add dummy fields to structures to make sure 64 bit fields are 64 bit aligned.
* [lldb] Finish emission of line number info.
* [lldb] Use mono_method_full_name () for computing method names.
* [lldb] Emit info for specific trampolines as well.
* [lldb] Add tests for line number info.
* [lldb] Fix unwind info encoding on x86.
* [lldb] Skip seq points with no native offset.
* [lldb] Add tests for breakpoints.
* [lldb] Use mono_unwind_get_cie_program () instead of mono_arch_get_cie_program () which might not be available on all platforms.
* [lldb] Avoid a crash in mono_lldb_remove_method () if lldb is not enabled.
cert-sync already knew how to import into the BTLS cert store,
update it to always import into both stores so we don't need
the btls-cert-sync tool anymore.
(cherry picked from commit 968c60a0f3)
It only looked at the legacy format where certificates had the *.cer
extension, the new BTLS store uses *.0 extension.
(cherry picked from commit 3aa59f2e6b)
This was keeping around potentially many unused gray queue sections, or even constantly leaking in the case of many majors since every time we start a concurrent major we redirect some additional gray queue sections from main to worker thread gray queues, leading to unbounded growth. The gc thread always trims its gray queues at the end of the collection.
On graph8 this reduces memory usage by 5%. Fixes mem leak from #41995
This change implements retrieving a list of DNS server addresses from Android
instead of attempting to parse `/etc/resolv.conf` which doesn't exist on
Android.
The list is retrieved by calling native function in Xamarin.Android that gets
the data from the system.
Part 2 of fix for https://bugzilla.xamarin.com/show_bug.cgi?id=48016
When setting X509CertificateImplBtls.PrivateKey we'd previously just
update the fallback instance. This causes problems if other code
internal to System.dll like MonoBtlsContext.SetPrivateCertificate()
tries to fetch the NativePrivateKey since it'd be null at that point.
This scenario happens e.g. with HttpListener, it supports a special
directory in ~/.config/.mono/httplistener which contains <port>.cer
and <port>.pvk files (public and private key) that will be used
as the server certificate. In that code we're reading the private
key from the file and assigning it to X509CertificateImplBtls.PrivateKey
without ever having a native BTLS handle.
The fix is to create such a native BTLS handle from the managed
private key when needed.
Note that we can't simply always do this e.g. in the PrivateKey
setter because the setter will be used with instances that
don't actually contain a private key. We can't throw in those
cases so instead we do it only when NativePrivateKey is accessed.
To make the code clearer and easier to read, `privateKey` was
renamed to `nativePrivateKey` in X509CertificateImplBtls.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=46602
(cherry picked from commit a9d75b4b66
and commit e135936e2f)
Otherwise we'd not clear the fields like subjectName, issuerName, etc.
and the properties they're backing would continue to return old values.
(cherry picked from commit af85a09321)
From the header docs:
> The |get1| functions return a fresh reference to the underlying
> object or NULL if |pkey| is not of the correct type.
(cherry picked from commit 2cadd74de8)
* Initial version of lldb support code.
* [lldb] Keep all the debug entries in a linked list so lldb can read it after attaching to the process.
* [lldb] Add an --enable-minimal=lldb option to disable the code. Add locking.
* [lldb] Add support for dynamic methods by adding a new protocol entry for unloading code regions.
* [lldb] Add beginnings of a test suite using the lldb python bindings.
* [lldb] Emit line number info for methods, not used yet.
* [lldb] Add dummy fields to structures to make sure 64 bit fields are 64 bit aligned.
* [lldb] Finish emission of line number info.
* [lldb] Use mono_method_full_name () for computing method names.
* [lldb] Emit info for specific trampolines as well.
* [lldb] Add tests for line number info.
* [lldb] Fix unwind info encoding on x86.
* [lldb] Skip seq points with no native offset.
* [lldb] Add tests for breakpoints.
We won't import it into Mono anytime soon and the long paths
cause issues when checking out Mono on Windows.
(cherry picked from commit ca5104094c)
Marek requested this being backported to the 4.8 branch.
Previously we'd just throw the catched exception again
which overwrites its stacktrace, making it less useful
since all you can see is e.g. a NullReferenceException
(re)thrown at MobileAuthenticatedStream:218.
We now capture the original exception information so
the stacktrace is more complete.
(cherry picked from commit 0007e99a87)
The path ".mono/certs/NewTrust" doesn't actually exist,
it's called ".mono/new-certs/Trust". Look it up from
MonoBtlsX509StoreManager instead.
This API isn't used anywhere in the BTLS code, that's
probably why it wasn't noticed.
(cherry picked from commit 6d2420f49a)
CMake doesn't pick up the compiler from the PATH by default,
it only looks in default locations so without this we might
end up using a different compiler for BTLS than for the main
Mono build which we don't want.
BTLS also requires a fairly new GCC which includes C++11
features, added a quick check for stdalign.h which BTLS
requires and disable it if the header isn't found.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=46712
(cherry picked from commit 488b550ac0)
The machineXYZPath fields are initialized with machinePath instead of
userPath which caused the machine stores not to work at all
(cherry picked from commit 10e1dc5314)
This addresses a regression here:
https://bugzilla.xamarin.com/show_bug.cgi?id=45761
The commits
97e51abfcf and
d35b5d2188
changed control flow such that if the host can't be resolved
and host is set to null, then the later check will return null
for the duration that HasTimedOut is false.
Finding a free port on watchOS fails with a PlatformNotSupportedException, but
if the exception is thrown in the constructor, the ExpectedException attribute
isn't taken into account, causing the test to error out.
https://bugzilla.xamarin.com/show_bug.cgi?id=47167
`directory` is already a directory. It's in the name.
`Path.GetDirectoryName ("/path/to/a/thing")` returns `"/path/to/a"`, whether
`thing` is a directory or not. If there's a trailing `/` then it returns
the directory, not the parent - but we don't have a trailing `/`.
This fixes an issue where the path needs to be created by hand before an
SDK can be downloaded.
(cherry picked from commit 031d79c2c4)
First the bug fix.
If we hit a method with a nested type as argument, we would not include the outer types and fail to match.
Now for the big change.
The runtime now includes mvid and aotid in the stacktraces it generates. This makes them symboficatable but
with the side effect that the runtime adopts a more compact and slightly different format. Differences includes:
- short name for primitives (int, object, string, etc)
- <> for generic instances
- / for type nesting
- generics vars are appended with _REF
Commit 41272d634b removed mono/mini/mini-
exceptions-native-unwinder.c from the source tree but didn't update msvc
/libmono-static.vcxproj accordingly causing the VS build to fail.
we will rely on the output of debuggerd in order to get a native stack
trace. The arguments to support that:
(a) native stack trace by debuggerd is at least as good as the one we get via
libunwind (most of the time the trace provided by libunwind is useless?)
(b) we cannot screw up by accident (e.g. we have seen issues with a too small altstack)
(c) we don't need to maintain the libunwind integration into mono (which we
sort of had to do because of the upcoming dlopen limitation in Android 7.0
Nougat).
(d) the libmonosgen-2.0.so had about one megabyte more footprint (16mb ->
17mb). I'm talking about a debug build here though.
See mailing list discussion for more information:
http://lists.dot.net/pipermail/android-devel/2016-December/000026.html
partial cherry-pick of #4124
This changes mono_mach_arch_get_thread_fpstate_size to return a value,
rather than asserting. The actual float state isn't used yet. This also
assumes NEON, which may not be correct.
The existing code for marking custom attributes was only marking the type
and the lowest element type, i.e. marking (1) would also mark (3) but
would miss (2).
1. [System.ServiceModel.ServiceKnownTypeAttribute(typeof(System.Collections.Generic.Dictionary<string, object>[][]))]
2. [System.ServiceModel.ServiceKnownTypeAttribute(typeof(System.Collections.Generic.Dictionary<string, object>[]))]
3. [System.ServiceModel.ServiceKnownTypeAttribute(typeof(System.Collections.Generic.Dictionary<string, object>))]
That's generally not an issue since the type is marked (elsewhere).
However, in the case of custom attributes, we need to update the Scope
to what was resolved (and that was missed).
That lack of updated Scope means that a PCL assembly would still keep
references to System.Runtime, which is something that the linker is
eliminating (causing a TypeLoadException at runtime)
reference:
* https://bugzilla.xamarin.com/show_bug.cgi?id=47064
Note: The linker is not in mono/master and there's a PR in the new
linker repo for the same https://github.com/mono/linker/pull/7
Per .NET documentation, DynamicMethod does not support custom attributes
exception the MethodImplAttribute pseudo-custom attribute.
Also change GetMethodImplementationFlags to include NoInlining as on .NET
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=49686
mono_monitor_ensure_owned used to throw exception directly and not return. After switching the runtime to use pending exceptions, monitor code would continue running assuming we owned to monitor.
Fixes 46994
It needs special logic like the other monotouch profiles in the products
since we don't have a System.Drawing assembly/types for these profiles in Mono
and don't embed the types in the facade anymore after 1074fe91bd.
(cherry picked from commit 1562fc6fe5)
.. `Project.Build(..)`. For example, in:
```
<MSBuild Projects="bar.proj" Targets="foo;foo" />
```
The second invocation of `foo`, in the *same* `Project.Build(..)` call,
would get skipped, but BuildTarget assumed that no previous `outputs`
entry existed in the `targetOutputs` dictionary.
System.Exception: MSBuild operation failed ---> System.ArgumentException: Item has already been added. Key in dictionary: 'foo' Key being added: 'foo'
at System.Collections.Hashtable.Insert (System.Object key, System.Object nvalue, System.Boolean add) [0x00200] in <94fd79a3b7144c54b4cb162b50fc7761>:0
at System.Collections.Hashtable.Add (System.Object key, System.Object value) [0x00000] in <94fd79a3b7144c54b4cb162b50fc7761>:0
at Microsoft.Build.BuildEngine.Project.BuildTarget (System.String target_name, System.Collections.IDictionary targetOutputs) [0x0007e] in <b1524340ac7e4c06b22fac1b2db62c08>:0
at Microsoft.Build.BuildEngine.Project.BuildInternal (System.String[] targetNames, System.Collections.IDictionary targetOutputs, Microsoft.Build.BuildEngine.BuildSettings buildFlags) [0x00134] in <b1524340ac7e4c06b22fac1b2db62c08>:0
at Microsoft.Build.BuildEngine.Project.Build (System.String[] targetNames, System.Collections.IDictionary targetOutputs, Microsoft.Build.BuildEngine.BuildSettings buildFlags) [0x0004b] in <b1524340ac7e4c06b22fac1b2db62c08>:0
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=44549
While CMAKE_POSITION_INDEPENDENT_CODE will be automatically set if
BUILD_SHARED_LIBS is TRUE, it won't if BUILD_DYNAMIC_BTLS is TRUE.
This change sets CMAKE_POSITION_INDEPENDENT_CODE manually in such a case.
.. item.
If we have a project, that sets some properties or items at runtime,
like in a target:
```
<Target Name="Build">
<PropertyGroup>
<Bar>Bar01</Bar>
</PropertyGroup>
```
.. such a property can be accessed as `$(Bar)` subsequently.
Then at later point, a target tries to update metadata on an item group
with existing items, like:
```
<ItemGroup>
<FooItem Include="xyz" />
</ItemGroup>
<Target Name="Foo">
<ItemGroup>
<FooItem>
<SomeMetadata>MetadataValue</SomeMetadata>
</FooItem>
</ItemGroup>
```
.. then it is seen that the value for the earlier created `$(Bar)` disappears!
So, if we try to print the value of `$(Bar)` right after that item
group, it would appear as `''`!
The issue is that the metadata update in target `Foo` caused the project
to get re-evaluated, which meant that items/properites created after the
project load were lost! We should not be reevaluating the project when
setting metadata on the basis of dynamic items, like in the target
`Foo`.
These dynamic item groups in a target are represented as
`BuildItemTask`, and these create BuildItems which have their
`IsDynamic` property set. And this is used to avoid re-evaluating the
project in `BuildItem.SetMetadata`.
But when updating metadata (`BuildItem.UpdateMetadata`), we need to
update metadata on the *existing* items, which might not have been
created from such dynamic item groups! So, their `IsDynamic==false`.
Hence, trying to `SetMetadata` on such items would cause the project to
be reevaluated, and thus properties/items like `$(Bar)` would be lost!
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=45137
In this particular case, `Bar` was the `DesignTimeBuild` property, which
was losing it's value when `ProjectReference`'s `AdditionalProperties`
metadata was set in:
```
<Target Name="BclBuildAddProjectReferenceProperties"
<ItemGroup>
<ProjectReference>
<AdditionalProperties>$(_BclBuildProjectReferenceProperties);%(ProjectReference.AdditionalProperties)</AdditionalProperties>
</ProjectReference>
</ItemGroup>
```
Apparently there's no way to get CMake to just spit out its own uncached variables, so we can't do this programatically.
(cherry picked from commit 6843f6d61d)
Because we wouldn't return the correct ThreadPoolDomain for the requested domain, we would simply wait on the last inserted MonoDomain. This would lead to a hang when unloading the domain, as we wouldn't wait on the correct domain.
If we fail to send the signal, we should not add a pending operation, as the signal handler is never going to answer this operation (via mono_threads_notify_initiator_of_resume).
Because certain platform (such as WatchOS) do not support syscall abort, we need to ensure that it is supported before we call the `mono_threads_suspend_abort_syscall` function.
The SecureStringHelper from referencesource I added in #3887
relies on this to convert from SecureString to plaintext.
It failed on mobile because we always throw a NotImplementedException there,
but we actually have an implementation on the runtime side (even in the DISABLE_COM case).
(cherry-picked from df1475d95e)
It's an optional element according to [1] and we were simply
throwing a NotImplementedException when encountering it.
Fixed by reading the element value instead. I noticed that we
had a 'details' variable that was never assigned, so the check
for `details == null` was always true and can be removed.
This makes inlining the body of the CreateFault() call easier
since we no longer need to choose different overloads and makes
the code consistent with the CreateFault12 method.
Also added a better exception in case of an unrecognized element.
The CreateFault12 method (handles SOAP 1.2) does not need a similar
fix since actor is not an element in that version [2].
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42226
[1] https://www.w3.org/TR/soap11/#_Toc478383507
[2] https://www.w3.org/TR/soap12/#soapfault
(cherry picked from commit 69b3397c8e)
* [BTLS]: Use `int64_t` instead of `long` in the native API.
* Copy time64.c from Android's source code to get a 64-bit timegm64().
* Rename into btls_timegm64() and remove everything we don't need.
* [aot-compiler] Add generic class and gsharedvt for hybrid AOT mode
* [aot-compiler] Remove dead code
* [hybrid-aot] Add --hybrid-aot for future use
* [aot-compiler] Cleanup GSharedVT check
* [aot-compiler] Enable partial sharing for hybrid AOT
* [aot] Enable GSharedVT for Android
* [aot] Ensure GSharedVT is supported at runtime as well as compile time
* [aot] Factor aot mode setting
* [mkbundle] - Add support for bundling native libraries.
* [mkbundle] add support for bundling native libraries
* [glib] add g_mkdtemp
* mono: Load the various library payloads for bundled executables
* [msvc] Export mono_dl_open and mono_loader_register_module
* [scripts] Add script to package the basic runtime to reuse by mkbundle
* [mkbundle] Add diagnostics, fix crash, fix registered path to match runtime name on embedded MonoImages (#3669)
* Resolve
* Complete the work, it should be working now
* Remove stray code from cherry pick
* [mkbundle] Fix the argument order to put the entry point on argv[1]
What was happening is that we set the stream EOF flag to true when we
could not read any more input data, and that made us early out before
fully finishing the processing of the compressed data.
Now the only EOF flag is set to true is if zlib's inflate() has actually
finished processing the data.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=44994#c2.
[C9][custom attr] Add low-level metadata iterator for assembly custom attributes
And use it to check for the reference assembly attribute.
Using the low level APIs means we don't trigger load of referenced
assemblies and we don't need to take locks, so it is safe to call this
during assembly loading.
Follow-up work for https://bugzilla.xamarin.com/show_bug.cgi?id=42584
This is a followup to e9205142b2.
Monodroid is missing the Cng and CryptoServiceProvider variants of the SHA
algorithms as well so we need to map to the Managed variants there too,
like on monotouch.
(cherry picked from commit e796ded61a)
Now that we're using a metadata-only ReferenceAssemblyAttribute check,
we can treat corlib the same as any other assembly - if we see
ReferenceAssemblyAttribute when loading a candidate assembly, skip it.
And use it to check for the reference assembly attribute.
Using the low level APIs means we don't trigger load of referenced
assemblies and we don't need to take locks, so it is safe to call this
during assembly loading.
This backports the mkbundle support that allows mkbundle to use complete SDKs, as opposed to a single runtime to bundle packages as completing the native library loading support.
* [mkbundle] - Add support for bundling native libraries.
* [mkbundle] add support for bundling native libraries
* [glib] add g_mkdtemp
* mono: Load the various library payloads for bundled executables
* [msvc] Export mono_dl_open and mono_loader_register_module
* [scripts] Add script to package the basic runtime to reuse by mkbundle
* [mkbundle] Add diagnostics, fix crash, fix registered path to match runtime name on embedded MonoImages (#3669)
This race condition has been introduced with 348d793ebb. This is because we would wait on the cleanup_semaphore outside of the threadpool->domains_lock lock.
By using a cond variable instead of a semaphore, we avoid a potential deadlock.
After 2fb07d6c6d BTLS can be used as a shared lib,
but this caused an issue in some monotouch tools which grepped the P/Invokes for `__Internal`
since these icalls wouldn't resolve on monotouch since BTLS is disabled there.
Instead, we now completely leave out building the managed parts of BTLS when
BTLS is not enabled.
(cherry picked from commit db21455da9)
The MacProxy class uses CFNetwork, but since CFNetwork is not a public
framework on watchOS, we can't use it.
So remove MacProxy completely (it only contains internal classes), and throw
PlatformNotSupportedException in any API that used it (the managed networking
stack is not supported on watchOS anyway, so this should be safe).
https://bugzilla.xamarin.com/show_bug.cgi?id=45847
* [btls] Convert BTLS icalls to pinvokes by invoking them using [DllImport("__Internal")], which will make it easier to redirect them to a separate dylib in the future.
* [btls] Add a --enable-dynamic-btls configure flag to enable compiling btls into a separate shared library instead of embedding it into the runtime.
In mono_assembly_load_from_full, the call to
mono_assembly_has_reference_assembly_attribute has to come after
image->assembly is set. This is because image->assembly->base_dir is
used for probing for referenced assemblies, and the reference assembly
attribute check will probe for assemblies referenced by the custom
attributes on the assembly in image being loaded.
Add 'ignore_missing' argument to mono_custom_attrs_from_index_checked:
if it is TRUE, a single custom attribute constructor that can't be
resolved to a MonoMethod won't cause the whole method to return NULL.
if 'ignore_missing' is FALSE, use existing behavior.
Updated all the call sites to pass FALSE, except
mono_custom_attrs_from_assembly_checked which also gets a new
'ignore_missing' arg.
Updated 'mono_assembly_has_reference_assembly_attribute to pass TRUE for
ignore_missing.
SecPolicyCreateSSL() distinguishes between an empty string (validation will fail
if the certificate contains host name constraints) and a null pointer (no host name
checking will be performed) for the hostname.
We incorrectly used `string.IsNullOrEmpty (host)` in a few places.
(cherry picked from commit 00f252d01f)
An assembly with a System.Runtime.CompilerServices.ReferenceAssemblyAttribute
may be loaded in a reflection-only context, but not for execution.
When loading assembly refs, or when loading assemblies by name (as
opposed to file path), a reference assembly will be skipped over and
other assemblies from the GAC and the load path will be tried.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42584
information about network interfaces from the /sys filesystem. Google
placed restrictions on two files used by the Mono BCL to read
information about the interface:
/sys/class/net/[DEVICE]/flags
/sys/class/net/[DEVICE]/operstate
The information we used to read from `flags` let us determine whether
the interface supports multicast and the `operstate` file provided us
with detailed information on the current device state.
Additionally, it is now impossible to read the hardware MAC address used
by any interface. This bit of information is also unavailable when using
the Java APIs and therefore applications running on devices with API
24 (and possibly newer) will not be able to access this information.
More details about the issue are available from the upstream Android
bug:
https://code.google.com/p/android/issues/detail?id=205565
In order to obtain the required information, we now need to use the Java
APIs on devices with API 24+ and this commit implements the backend to
retrieve this information. It is done in C in order to avoid overhead of
creating mirror .NET objects when calling Java, thus saving time and
memory during an operation which may be performed by some applications
quite frequently.
This patch implements code that calls into the Xamarin.Android
runtime when runnin under API24 or newer to obtain the information.
Part of fix for https://bugzilla.xamarin.com/show_bug.cgi?id=44296
Enviroment::get_bundled_machine_config icall should be ves_icall_System_Environment_get_bundled_machine_config.
Mark all icalls doing get_bundled_machine_config as ICALL_EXPORT so maccore can link them in/out.
When connecting to an FTP server we weren't handling the RFC2428
extension to the FTP protocol which allows connections over IPv6
for the data channel.
We'd try to connect via IPv4 and the server would return an error
saying that this protocol is not supported.
The fix is to add handling for the protocol extension, namely
to use EPSV (for passive FTP) or EPRT (for active FTP) when opening
the data channel and send/parse the IPv6 addresses.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=44025
Note: we're now using the IP address of the the control channel we connected to for the
data channel instead of the address that is returned by the server in the FTP response message.
The reason is that for EPSV there's no IP returned by the server, and it should be
more reliable for normal PASV as well since the server might not know the
correct external address. This is also what MS does in referencesource.
(cherry picked from commit aebed51257)
* [System]: Move MonoBtlsProvider.IsSupported() -> MonoTlsProviderFactory.IsBtlsSupported().
Move the intern-call which is used to determine whether we're using BTLS out of MonoBtlsProvider
to allow the managed linker to eliminate the Mono.Btls.* classes when it's not supported.
* Fix Android build.
Without arguments, 'cert-sync' now operates on the old certificate store
again, like it was before. To test BTLS, you need to explicitly run it
with '--btls'.
We weren't checking whether a PE file is in PE32 or PE32+ format,
causing us to use the PE32 offsets on the newer format as well.
This means that using Process.Start() on a managed assembly didn't
always work when a PE32+ file was in play. Such a file is created
when targeting x64 e.g. via the -platform:x64 csc.exe/mcs option.
The reason why we probably didn't notice until now is that in an
assembly produced by mcs there happens to be some data at the PE32
CLR header offset even in a PE32+ file, causing the check to "work".
This doesn't apply to csc.exe/roslyn though and so we couldn't
execute those assemblies via Process.Start() even though they're
perfectly managed. Since the .NET Core project.json toolchain
produces x64-targeted assemblies by default more people ran into
the issue trying to run those on Mono: https://github.com/cake-build/cake/issues/1247
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=44937
PE/COFF spec at https://msdn.microsoft.com/en-us/library/windows/desktop/ms680547(v=vs.85).aspx
(cherry picked from commit 68bcbb4b77)
The epoch is specified as being in UTC, but the DateTime(int, int, int) constructor
creates a DateTime with an 'Unspecified' timezone, which could lead to problems later on.
(cherry picked from commit 8d1a65128a)
In mono_btls_x509_get_serial_number(), first call i2c_ASN1_INTEGER()
which takes care of padding and computing the 2s complement for negative
numbers, then reverse the buffer.
needs HANDLES
The method (which also goes by
System.Configuration.DefaultConfig:get_bundled_machine_config) was
converted to use coop handles earlier, but we missed this declaration.
RS takes the stance that redirected streams should not be closed by the Process class because they might
be referenced from the outside.
That would be ok if it was true. If you use Process::BeginOutputReadLine, the output stream becomes inaccessible
with no way to dispose them by any mean but GC.Collect. You get an exception when accessing Process::StandardOutput
once async reading starts.
This change addresses the case where the streams are not accessible. This is tracked by the *StreamReadMode variables
that are set on stream getters or Begin*ReadLine.
* [BTLS]: Allow fallback to MX.X509Certificate in X509CertificateImplBtls.
* [BTLS]: Fix X509Certificate2.MonoCertificate.
* X509Certificate2Impl.FallbackImpl: make abstract.
* X509Helper2.GetMonoCertificate(): new internal helper method.
* X509Certificate2.MonoCertificate: use X509Helper2.GetMonoCertificate().
This is only used by X509ChainImplMono; we should investigate whether we
could implement the missing APIs in BTLS.
* [BTLS]: Fully implement X509CertificateImplBtls.Import() and use correct semantics.
.NET's X509Certificate2.Import() uses several implicit fallbacks which we previously
did not handle in BTLS.
* you may provide a (unused) password when importing an unencrypted PKCS#8.
* when using null as password, it will attempt to use string.Empty if importing
without password fails.
* [BTLS]: Implement X509CertificateImplBtls.PrivateKey.
* [BTLS]: Implement X509CertificateImplBtls.Export().
* [BTLS]: Implement X509CertificateImplBtls.Verify().
* [BTLS]: Fix X500DistinguishedName quoting.
* MonoBtlsUtils: don't quote special chars in UTF-16 strings.
* X500DistinguishedName: don't decode raw data when passed in a name.
* [ci] Test System.dll with both the legacy and btls TLS provider
* [ci] Only run the additional btls System.dll test run on OSX for now
Otherwise we might run it on platforms where btls isn't supported yet and get errors.
The MonoBtlsProvider.IsSupported() icalls ves_icall_Mono_Btls_Provider_IsSupported()
needs to be available whether BTLS is available or not because it is used at
runtime to decide whether or not to enable BTLS.
This means we can't build Mono.Data.Tds.dll either, since it uses
Mono.Security.dll extensively. In any case Mono.Data.Tds.dll is useless
without a working networking stack, and since it's a Mono-only dll (i.e. not
part of netstandard or facades), just remove the entire assembly from the
watchOS profile.
System.Security.dll uses Mono.Security.dll as well, but the required classes
are available inside mscorlib.dll (as internal classes), so just open up
mscorlib internals to System.Security.dll.
The -arch option is an Apple extension, it caused failures on i386 Linux: "unrecognized command line option '-arch'"
(cherry picked from commit aad01ff4db)
* Don't invoke undefined behavior if an fd is >= FD_SETSIZE.
* Allow multiple command sockets.
* Close the command socket(s) on shutdown.
* Properly run cleanup logic on shutdown.
We need it in all cases now that we use it to periodically flush all buffers.
The only platforms it was disabled on was NaCl, which we haven't bothered to
support in a while anyway.
This also has the nice effect of reducing the amount of times we attempt to
take the reader lock. Now we take it and release it once for a single event.
Only in very few situations is it okay to access this field without first
taking the reader lock or exclusive lock.
Fixes some crashes observed in the profiler stress tests.
This avoids exposing the SGen worker threads through the profiler APIs. Doing
so had two main problems:
* It's easy for a user of the profiler API to accidentally mess with the state
of the worker thread in all sorts of ways, which could break the GC.
* It's been a long-standing guarantee that any profiler API callback will be
invoked on a thread that is, at the very least, attached to the runtime via
the mono-threads infrastructure. This is not the case for SGen workers.
This fixes a deadlock in the log profiler when using major=marksweep-conc.
* Emit an NUnit test report file.
* Set a timeout of 6 hours per test.
* Do a thread dump for timed-out tests.
* Don't print stdout/stderr for passing tests.
* mono_gc_enable_events: This did nothing for SGen and we called it
unconditionally for Boehm. So no point in having it around.
* mono_gc_enable_alloc_events: The performance difference between checking
a static alloc_events variable versus checking mono_profiler_events is
marginal at best and doesn't justify the indirection, especially now that
we support managed allocators while profiling.
Found by checkbashisms(1) as "unsafe echo with backslash". Some shells don't expand embedded \n in a string by default,
which broke the build on systems like ArchLinux or Alpine where /bin/sh points to those shells because the generated .lo
file contained just a single line instead of two.
Thanks to @JustArchi for researching this on the Mono Gitter channel, it made fixing the issue a lot easier!
(cherry picked from commit 1958bb8b34)
The HttpWebResponse stub that is used e.g. on the monotouch_watch profile was missing an overload that this commit was using.
(cherry picked from commit f4475ff013)
* external/boringssl: unmanaged code from Google (added earlier).
* mono/btls: unmanaged BTLS lives here (added earlier).
* System/Mono.Btls: managed BTLS lives here.
* Mono.Btls.Interface: The purpose of this assembly is to allow test suites
such as xamarin/web-tests to test parts of the BTLS APIs without making
System.dll's internals visible. It is not a stable and maintained API.
(cherry picked from commit 0f8f0c543c)
BTLS uses a different format and naming scheme for the local certificate store.
Before using BTLS for the first time on desktop, you need to convert the local
certificate store by either using the new `btls-cert-sync` tool (which will convert
your existing certificate store to the new new format) or `cert-sync` with the
`--btls` argument.
The new format and naming scheme is identically to what you'll find in
'/system/etc/security/cacerts/' on an Android device. It stores the certificates
in PEM format (with a human-readable text section added) using filenames which
are based on the certificate's subject name hash.
Previously, Mono stored the certificates in binary DER format, using a filename
based on either the Subject-Key-Identifier (if available) or the thumbprint.
We cannot access this from BTLS because computing these filenames requires
reading the entire X509 certificate (including the X509v3 extensions), but
their native lookup code only gives us the SubjectName.
This transitioning won't affect Android users as we're using the system
certificate store (including the user certificate store, via a Java callback).
* Mono.Security.X509.X509Store and X509Stores:
Support both the old and the new format. Using the new format requires BTLS.
* Mono.Security.X509.X509StoreManager:
Add `NewCurrentUserPath` and `NewLocalMachinePath` properties for the
new path names (~/.config/.mono/new-certs).
* Mono.Security.X509.X509StoreManager:
Add `NewCurrentUser` and `NewLocalMachine` properties. Requires BTLS.
* tools/security/cert-sync:
Add new `--btls` argument to use the new store.
(cherry picked from commit fb93be4490)
* [Mono.Security]: Remove Mono.Security.Interface.IMonoTlsContext.
This interface is not actually used anywhere anymore, so we can also
just get rid of it.
* [System]: Removed unused private TLS classes.
* [Mono.Security]: Cleanup the MonoTlsProvider's certificate validation code.
Each TLS Provider now has its own certificate validator and there is no
automatic fallback anymore. We only have three different TLS Providers,
"legacy", "appletls" and "btls" - and legacy is the only one which used
the managed validator. Both AppleTls and BTLS provider their own validator,
which invokes some native API.
All of the following were internal APIs.
* MonoTlsProvider.HasCustomSystemCertificateValidator has been removed.
* MonoTlsProvider.InvokeSystemCertificateValidator() has been renamed
into ValidateCertificate() and is now abstract.
* I got confused between [TestFixtureSetup] and [Setup], so fix tests whose
[Setup] methods were changed/removed to properly clean up after each test
(using a [TearDown] method) so that the next time the on-demand creation of
objects works as expected (i.e. objects created in the [Setup] method should
be created on-demand once for each method instead of once for each class).
* Also fix SocketAcceptAsyncTest, the change to properly catch exceptions on
background threads was incorrect.
LegacyTlsProvider now lives in the 'Mono.Net.Security' namespace
(previously 'Mono.Net.Security.Private') and derives directly from
the public 'Mono.Security.Interface.MonoTlsProvider'.
This allows it to be used by products, for instance replacing
xamarin-macios/src/Security/Tls/OldTlsProvider.cs.
The idea is to have all 'MonoTlsProvider' implementations inside
System.dll and not access any of the private APIs from outside of it.
This API is provided to be used by libraries providing crash reporting, so
that those libraries can install signal handlers and at the same time not
interfere with Mono's signals.
The intended pattern is this:
Mono.Runtime.RemoveSignalHandlers ();
InstallThirdPartySignalHandlers ();
Mono.Runtime.InstallSignalHandlers ();
This ensures that Mono is always notified first of any signals, and will then
chain to the third-party signal handlers if the signal did not originate in
managed code.
Mono's signals must first be removed, so that if third-party signal handler
does signal chaining, we don't end up in Mono's signal handler again.
Mailing-list thread: http://lists.dot.net/pipermail/macios-devel/2016-September/000016.html [Signal-chaining & crash reporters]
Example usage: https://github.com/rolfbjarne/HockeySDK-Xamarin/commit/862721199f1adec210bfc7b1943f975ffb8048a9
When there is an abstract class in the middle of an inheritance
hierarchy that doesn't override a method, get_base_methods needs to search
the parent class instead aborting early.
Example:
```
abstract class Root {
public abstract void Foo ();
}
abstract class Abs : Root { }
class Derived : Abs {
public override void Foo () { }
}
```
If we ask for the base method of Derived.Foo we used to get Derived.Foo
back because Abs did not provide an override. Instead we should skip
abs and look in Root.
When running with async suspend (ie, not coop), a user thread might be
suspended while its manipulating the coop handle stack. In that case,
we want to present a consitent view to mono_handle_stack_scan () by
adding memory fences to code that updates the stack chunks. In
particular we want to ensure that the chunk size is increased after
the new slot has a valid (null) value. Also when creating new chunks
make sure their size is initialized before the chunk stack is updated.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=43921 (at least the
one stack trace observed in [comment 1](https://bugzilla.xamarin.com/show_bug.cgi?id=43921#c1)).
The "global" TLS Provider (returned by MonoTlsProviderFactory.GetProvider())
may only be modified at application startup (before any of the TLS / Certificate
code has been used).
* MonoTlsProviderFactory.HasProvider has been removed.
This property used to initialize the TLS Subsystem without throwing any
exceptions. However, properties with side-effects is a bad design principle.
* Add MonoTlsProviderFactory.IsInitialized.
Does not have any side-effects.
* Add MonoTlsProviderFactory.Initialize() and Initialize(string).
May only be called at application startup and explicitly initializes the
TLS Subsystem.
* Rename MonoTlsProviderFactory.GetDefaultProvider() into GetProvider()
and removed MonoTlsProviderFactory.GetCurrentProvider().
We do not distinguish between a "default" and a "current" provider anymore;
there is only one "global" provider.
* Add MonoTlsProviderFactory.IsProviderSupported(string).
Checks whether a specific TLS Provider is supported, without having side-effects.
* Make the old APIs [Obsolete] until products have been updated.
* Add some documentation to MonoTlsProviderFactory.
* [System] Fix line endings for a few files.
Best reviewed by not showing whitespace changes (should be empty diff in that case).
* [System] Fix potential crash in CookieParserTests.
Socket.EndAccept can throw an exception, which will crash the process if it
happens on a threadpool. So handle any exceptions when calling
Socket.EndAccept, and propagate those exceptions out of any threadpool code.
* [System.Net.Http] Throw a PlatformNotSupported exception when using HttpClientHandler on watchOS.
* [System.Net.Http] Tweak tests after watchOS API change.
Unfortunately the [ExpectedException] attribute can't be put on an entire
class, so remove the [Category ("RequiresBSDSockets")] attribute on the class
and instead decorate each test with an [ExpectedException] attribute.
* [System] Extract the HttpListenerResponseHelper class to a separate file.
Extract the HttpListenerResponseHelper class to a separate file so that it's
easier to replace the HttpListenerResponse implementation by using a different
file.
* [System] Throw a PlatformNotSupported exception when using the networking stack on watchOS.
* [System] Tweak tests after watchOS API changes.
Unfortunately the [ExpectedException] attribute can't be put on an entire
class, so remove the [Category ("RequiresBSDSockets")] attribute on the class
and instead decorate each test with an [ExpectedException] attribute.
* [System] Fix potential crash in SocketAcceptAsyncTest.
Fix potential crash in SocketAcceptAsyncTest by catching any exceptions in
code executed on the threadpool so that we never end up with unhandled
exceptions (which may terminate the process).
* [System] Avoid logic in SetUp attributes in tests.
Any type of exception in [SetUp] attributes causes all tests to fail with that
exception.
This even happens for tests that do not need the logic in the [SetUp]
attribute.
So rewrite tests a bit to not use [SetUp] when we can reasonably avoid it, and
instead instantiate class-level variables on first use. This ensures that any
exceptions that occur when initializing those variables are attributed to the
corresponding test, and not all of them.
* [System] Make exception handler (for test) catch more exceptions to avoid unhandled exceptions causing process termination.
* [System] Remove Obsolete attribute on API that throws PNSE.
Introduction of the structured trace logger accidentally broke Android
logcat logging. The reason for this is that the default logging handler
defined in eglib (in goutput.c) for Android was never called, instead it
was superseeded by the log file logger defined in mono-logger.c
The net effect was that Mono tried to write the messages to some file on
Android leaving logcat devoid of any trace of life :)
This commit fixes the issue by introducing a set of structured logging
functions for Android.
As a side note - file logging on Android is not that useful. Writing to
a file on external storage medium requires app permissions and writing
to a location on internal storage medium, while allowed, might be
subject to space limitations and should not be used for possibly
voluminous data. It's therefore not very practical on Android.
* CertificateValidationHelper.GetDefaultValidator(): rename this internal method into
GetInternalValidator() (keeping the old as [Obsolete] until products have been updated).
* CertificateValidationHelper.GetValidator(): remove the 'provider' argument.
This is part of a set of cleanups and simplifications for the upcoming BTLS integration.
(cherry picked from commit 61d76d9642)
It is possible for SSLHandshake() to return SslStatus.Success while we're still
having a pending write (AsyncOperationStatus.WantWrite).
This is because our managed write callback must never return 'WouldBlock', so
SSLHandshake() things that all data have been sent while we still have them in
our write queue.
When this happens, we currently flush the write queue then call SSLHandshake()
again via AsyncOperationStatus.WantWrite -> AsyncOperationStatus.Continue.
This returns SslStatus.Protocol on iOS 10.
(cherry picked from commit 2cc1b887c1)
* Remove Mono.Security.Providers.NewSystemSource and Mono.Security.Providers.NewTls.
* Remove InternalVisible for "Xamarin.BoringTls".
* Remove Mono.Security.Providers.OldTls and Mono.Security.Providers.DotNet.
Satellite assemblies are special in that they all share the same file
name, but they live in separate directories named after the locale the
satellite assemblies pack. mkbundle currently doesn't support "clashing"
assembly filenames and will not be able to store the satellite
assemblies in the bundle for that reason.
This patch adds support for storing satellite assemblies with their
culture path prefix (e.g. en-US/MyAssembly.resources.dll) in the bundle
and also special code in the assembly loader to support such filenames
in the code which loads assemblies from the bundle.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=27061
The generic setter receives the element by reference which means we will never emit the write barrier since a reference to a reference is not a reference (mini_type_is_reference). Check the actual value instead. Fixes#42903.
Previously the tests were written so that some failures might occur on the
threadpool in methods that did not catch exceptions. In that case the process
would terminate due to an unhandled exception. So rewrite the tests to catch
exceptions in any code that doesn't run on the main thread, and forward those
exceptions to the main thread.
Also wait for async operations to complete on the main thread, and remove
asserts that are not supposed to be hit.
This is a backport of PR #3558.
[arm64] fixes around icache flushing
This fixes https://bugzilla.xamarin.com/show_bug.cgi?id=39859
The problem in `emit_thunk` and the workarounds for the Cortex A53 errata are unrelated to the bug above, but they don't hurt either.
The basic problem that we see is that the Exynos 8890 SoC (shipped for example in the world edition of the Samsung Galaxy S7) is a big.LITTLE architecture with two different CPUs, four of each. `__clear_cache` of GCC [1], the built-in that we use to flush the instruction cache, correctly reads the cache line sizes for data- and instruction cache from the CPU and then caches that result. However, the two CPUs on the Exynos 8890 have two different cache line sizes for the instruction cache (128 bytes vs. 64 bytes): If we happen to initialize the cache line sizes with the larger one, we only flush every other cache line if the code runs on the second CPU model.
Neither V8 [2] and LLVM [3] do caching of the cache line sizes, but just read it every time from the register. We have a slightly different solution, where we cache _and_ read the value every time and try to determine a global minimum across CPUs, in case we get scheduled to a different CPU by the kernel during the flushing loop.
[1] https://android.googlesource.com/toolchain/gcc/+/master/gcc-4.9/libgcc/config/aarch64/sync-cache.c#33
[2] https://github.com/v8/v8/commit/fec99c689b8587b863df4a5c4793c601772ef663
[3] https://github.com/llvm-mirror/compiler-rt/blob/ff75f2a0260b1940436a483413091c5770427c04/lib/builtins/clear_cache.c#L146
This reverts commit 2f88b8eb5a.
There is a subtle problem with turning the monitor thread into a simple
MonoThreadInfo: It creates worker threads via mono_thread_create_internal (),
which allocates the thread object from the GC. No thread can allocate from
the GC unless it's actually attached, which the worker thread is not when
created with just mono_threads_create ().
This handle should have a total of maximum 3 copies:
- in MonoThreadInfo->handle
- in the return value of mono_threads_create_thread
- in MonoInternalThread->handle (if it's a managed thread)
Before, there would always be 2 references (in MonoThreadInfo->handle and another another for MonoInternalThread->handle), even if it would be referenced at one place only (MonoThreadInfo->handle), which would lead to a handle leak.
We also need to duplicate the returned handle, to avoid having a use-after-free of the handle (in case the thread exits before we access that handle)
* [threads] Allocate StartInfo in create_thread
* [threads] Put start_notify semaphore in StartInfo
* [threads] Factor common attach code in mono_thread_attach_internal
* [threads] Use same registered mechanism as mono-threads
The tests that fail like this actually work just fine on Mono on Windows.
Added the NotDotNet category to the CurrentThread_Domains() test since it
fails on .NET.
* Avoid race condition between domain unload and threadpool.
Previous logic only ensured that domain->threadpool_jobs was 0.
There was a race condition as follows:
* worker jobs are queued into ThreadPoolDomain outstanding_request's
* domain unload starts
* domain->threadpool_jobs is 0 so unload proceeds
* queued jobs are processed from the ThreadPoolDomain outstanding_request's
* the domain can be set/entered by worker threads since
mono_domain_set only fails for domains with state of MONO_APPDOMAIN_UNLOADED,
while our domain is in the MONO_APPDOMAIN_UNLOADING state
* jobs are now being unexpectedly run in domain that is being shutdown
* Fix a hang when unloading the domain.
This change applies the diff from ludovic-henry to correct the hang.
https://gist.github.com/ludovic-henry/6c8a4bc8951091c0ef36df2e52d5e32e
For some reason preamble length was asumed to be 3 when cp 65001
instead of always looking att the preamble length of the encoding.
This resulted in responses cropped by 3 chars on windows.
* When doing async connect on windows, connect returns `WSAEINPROGRESS`
which needs to be returned to managed code for correct handling. However
on Windows, last error gets reset before `WSAGetLastError` is called.
This fix moves all `WSAGetLastError` calls up closer to the socket
operation to make sure correct error code is returned to the managed code.
* Making sure `TcpNoDelay` is set to false for IPv4 by default and
that `DualMode` is default for IPv6.
* Making sure option value is set to 0 before calling getsockopt
because on Windows, a `BOOL` option value (eg TcpNoDelay) is returned
as a one byte value and the other bytes are left untouched which resulted
in TcpNoDelay always seemed to be true.
* Fixed so `TF_REUSE_SOCKET` is set only if `reuse` parameter is set
when disconnecting a socket.
* [threads] Remove use of CREATE_SUSPENDED for domain unload thread
* [threads] Remove use of CREATE_SUSPENDED for managed thread
* [threads] Remove CREATE_SUSPENDED
If the sweeping mechanism has been triggered by a major collection, then the iteration will wait for the general sweeping to finish and also for the sweeping of all the blocks. If this isn't the case (we are doing a major collection) we will iterate over the current state of the heap, some objects being bound to die during the collection, and the callback must account for this.
It doesn't make much sense to explicitly attempt to iterate without trying to sweep blocks that can be swept.
We never enter this case since we never iterate over the major objects without requiring to sweep, while a sweep is running. Even if we would be in this case, we directly access the mark bits which can be cleared at the same time by another thread sweeping the block, so we could miss live objects.
It doesn't make sense to check the remset during a major since majors dont use the cardtable and we clear it at the start. So we will obviously have missing remsets. For the debugging concurrent case, we should use the mod union consistency checks instead.
We would need to deal with scanning forwarded objects and dead objects which is not worth it. Only verify the heap at the start of a collection, as indicated by the name of the flag.
When verifying the heap we need to have the set of pinned objects otherwise we can observe missing remsets if suspended between the store and the write barrier. If we don't have the set of pinned objects, allow missing remsets.
After https://github.com/mono/mono/pull/3452 was merged, the LinkSdkRegressionTest.SecurityDeclaration
test in XI started failing:
```
[FAIL] LinkSdkRegressionTest.SecurityDeclaration : FileIOPermissionAccess
Expected: null
But was: <System.Security.Permissions.FileIOPermissionAccess>
at LinkSdk.LinkSdkRegressionTest.SecurityDeclaration () [0x00033] in /Users/builder/data/lanes/1411/2a96404f/source/xamarin-macios/tests/linker-ios/link sdk/LinkSdkRegressionTest.cs:990
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /Users/builder/data/lanes/1411/2a96404f/source/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:305
```
Due to the added call in the PR the linker preserved FileIOPermissionAccess enum,
which is apparently something we want to avoid.
Since according to https://github.com/mono/mono/pull/3452#discussion_r76384945 the only point of calling the
FileIOPermission is validating that the path doesn't contain invalid characters (mostly an issue on Windows),
I decided to just copy those checks as it's way cleaner.
Reference source of WindowsIdentity is using `MAXIMUM_ALLOWED`
as desired level by default when getting current token.
Changing to same behaviour to enable duplicate and impersonation
to be made on the token retrieved by GetCurrentToken.
* This improves the marshaling for packed classes and class involved in an inheritance hierarchy.
* Similar changes were made in the Unity fork of mono across a number of commits.
* The tests here have been taken from the repo and converted to work as NUnit tests in Mono.
Refcatored Win32RegistryApi.SetValue() to accept uint/byte/etc values for
DWord registry values.
Added handling of ERROR_CHILD_MUST_BE_VOLATILE errors to GenerateException().
Centralized MarkedForDeletion error handling to GenerateException().
Fixed tests which create volatile subkeys in RegistryKeyTest which used the
same key names but didn't clean up properly.
On Windows the TearDown() cannot delete the temp folder since it contains
loaded assemblies. The next time the test is run it will create a temp folder
inside the previous folder. After a number of runs the temp folder path will
approach the ~260 max path supported on Windows and some of the methods will
start failing.
This patch changes the way the temp folder is created and reduces the depth in
the same way as AssemblyTest was patched to fix a similar issue in
b90617e7b7.
Removed the differing code paths in FixtureSetUp() when running on non-Windows
platforms vs when running Mono on Windows or on .NET. The path previously
taken only on non-Windows platforms has been verified to work as expected on
both Mono on Windows and on .NET.
It failed a couple of times according to the babysitter logs with the following error:
```
MESSAGE:
Expected and actual are both <System.Int32[10]>
Values differ at index [1]
Expected: -1
But was: 0
```
Bump timeouts a little bit to see if this helps.
environment variables
Relying on the HOMEDRIVE/HOMEPATH environment variables is fragile as those
variables don't always have to be set. One such scenario is when running on a
Jenkins slave. When not set g_get_home_dir() in eglib would return NULL which
led to a crash in ves_icall_System_Environment_InternalGetHome().
This patch changes g_get_home_dir() to first try to call
SHGetKnownFolderPath() to get the current user's profile directory (i.e. home
dir). If that fails it falls back to the USERPROFILE environment variable. If
that isn't available the function will fallback to the original behavior and
try with HOMEDRIVE/HOMEPATH.
Creating new sockets between each connect instead of
trying to reuse the client socket. The original tests
was not working in the MS .NET runtime and socket reuse
is not recommended to use unless in very special cases.
On Windows the test used URLs like `file://C:\tmp\foo` which the
Mono.Security.Uri class was incapable of parsing. System.Uri from
referencesource parses such URLs just fine. This patch fixes the Uri class to
be able to parse such URLs. It also changes the Zone class to not throw an
exception if the Zone.CreateFromUrl() method is passed an invalid URL. Instead
the zone is set to NoZone. This seems to be how Zone in .NET behaves.
There are a few places in the runtime that should not be aborted as it would render it unusable.
cctor execution was the initial trigger for this changeset, but then edge triggering of TAE turned out
to expose this issue on other places.
This patch introduces those regions and apply them to the following places:
ves_icall_System_Runtime_Remoting_Messaging_AsyncResult_Invoke: Not delivering the completed CB
screws up any completion logic the user setup up.
mono_unhandled_exception: Protect delivery of the unhandled exception. Can't fail during it or we'll
effectively double fault and abort.
A thread abort must not start if the thread is running a .cctor.
OTOH, if the cctor self aborts, it should start.
Threads now track how many cctors they are running in the cctor_exec_depth variable.
The abort machinery (in async_abort_critical) checks that flag and bail from
starting the abort.
The cctor code (in mono_runtime_class_init_full) ensures that cctor_exec_depth is bumped
around execution of the cctor itself. Additionally, error handling was cleaned up as it was pretty
inconsistent.
This callback allows the runtime to query mini on whether there's a block guard in play.
It allows for the runtime to correctly decide when to start an abort.
Call g_log_set_default_handler in mono_trace_set_log_handler and mono_trace_set_log_handler_internal to ensure that g_ logging is properly routed.
Additionally, change the internal logging callback to take the log message as a single string to simplify things.
The patch of mono/metadata/marshal.c makes sure that
Marshal.StringToHGlobalAnsi() allocates a memory area for the returned C
string which is at least as large as the length of the string object. This is
the behavior on non-Windows platforms. Without this patch the
MarshalTest.StringToHGlobalAnsiWithNullValues() test wrote outside of a memory
area and corrupted the heap which later on caused a heap corruption crash when
running the corlib test suite on Windows.
The patch of mono/metadata/mono-security.c fixes the freeing of the memory
returned by GetNamedSecurityInfoW(). When GetNamedSecurityInfoW() is used to
request a DACL one must also pass an argument for the ppSecurityDescriptor
parameter and this is the one that should be freed later on, not the pointer
to the DACL as the original code used to free. Before this patch
DSACryptoServiceProviderTest.UseMachineKeyStore() crashed because of a heap
corruption.
In defer mode, the debugger thread is started as soon as possible before
any clients are attached, but the debugger is_attached flag was being set
immediately. This would cause the runtime (and the System.Diagnostics.Debugger
class) to wrongly report a client as being attached.
Originally fixed in the Unity fork here: https://github.com/Unity-Technologies/mono/commit/797a210e0f2416d779c448bb9e03bb16933e261f
In defer mode, the debugger thread is started as soon as possible before
any clients are attached, but the debugger is_attached flag was being set
immediately. This would cause the runtime (and the System.Diagnostics.Debugger
class) to wrongly report a client as being attached.
Originally fixed in the Unity fork here: https://github.com/Unity-Technologies/mono/commit/797a210e0f2416d779c448bb9e03bb16933e261f
All ia64 compilers have intrinsics that we use. Also, this ia64 code was very
out of date (missing many functions we use now) so it wouldn't compile anyway.
When encoding a UTF-16 C# string into bytes using "latin-1"/"iso-8859-1" encoding we were
not properly taking surrogate pairs into account. MSDN[1] provides a good introduction about the topic.
For example, this string would fail to encode and throw an IndexOutOfRangeException: "\ud83d\ude0a "
However, this string would not: "\ud83d\ude0a" (note missing space at the end). More on that later.
In https://bugzilla.xamarin.com/show_bug.cgi?id=42887 it was noted that encoding the string
didn't cause the exception on Mono 2.10.8.1 and 3.2.8. Indeed, it doesn't happen until Mono 4.0
where we switched most of the string code to referencesources.
While looking at the output of `Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")` on the
"working" Mono verions I noticed that they essentially returned wrong results, e.g.:
```
csharp> Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")
{ 63, 32, 0 }
```
This is wrong, because a) there's only one 63 (i.e. the '?' question mark char which is used as
a replacement char whenever the target encoding doesn't support a specific char,
and b) there is a trailing '\0' which should not be there.
Checking on .NET confirms the expected result: { 63, 63, 32 } i.e. '?? '
Back to the IndexOutOfRangeException in Mono 4.0+: digging into Latin1Encoding.InternalGetBytes() it
turns out that when we detect a surrogate pair we were incrementing charIndex, but we didn't decrement count.
This means that after the fallback characters (the two '?') were added we would still try to encode one more
char in the loop even though we're already at the end of the string, resulting in the exception.
For the "\ud83d\ude0a" example we would not hit the `count > 1` code path which made it work in this case.
Adding the decrement fixes the bug and makes the code consistent with MonoEncoding.cs[2].
The reason why it "worked" in earlier Mono versions was that we had a very simple EncoderReplacementFallbackBuffer
implementation[3] which only output a single '?' for the surrogate pair:
```
csharp> var buf = Encoding.GetEncoding("latin1").EncoderFallback.CreateFallbackBuffer()
csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0
csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0'
```
This means that for the "\ud83d\ude0a " example from above we would only add one '?' character
for the surrogate pair to the result buffer, then the space and then we'd be done, leaving the
uninitialized array value 0 in the buffer on index 2.
I added tests for Latin1Encoding, mostly copied from ASCIIEncoding with additions for these
specific cases.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42887
[1] https://msdn.microsoft.com/en-us/library/ms404377(v=vs.110).aspx
[2] https://github.com/mono/mono/blob/5b69e839e12b3d3512f310aa5a0e873346d7e50e/mcs/class/I18N/Common/MonoEncoding.cs#L62-L66
[3] https://github.com/mono/mono/blob/mono-3.12.0-branch/mcs/class/corlib/System.Text/EncoderReplacementFallbackBuffer.cs
Previously, we would try to satisfy the heapshot request every single time
process_requests () was invoked, but since heap_walk () would never unset
heapshot_requested, we'd end up doing GCs over and over, causing massive
overhead.
All events now come with a time field right after the event byte, and it is
encoded in the same way for all events (based on LogBuffer.time_base).
This allows readers to queue up events until a sync point event, then sort all
of those events by time, and process them in the order they were actually
written in.
Maintaining it for every format version is getting unwieldy and most readers
likely won't care about ancient format versions. Further, we haven't really
stabilized the format yet, so documenting every iteration of it doesn't make
a whole lot of sense. Finally, if somebody does care about older versions,
they can simply look in decode.c to see how mprof-report deals with them.
* Methods in backtraces are now properly encoded based on the method_base
rather than the ptr_base.
* The unused IL and native offset fields in TYPE_SAMPLE_HIT's backtrace frame
format have been removed.
* The unused flags fields in the backtrace format and in metadata events have
been removed.
* Changed a number of fields to be encoded as a single byte instead of LEB128.
These events basically establish points in the event stream where all previous
events can be processed and a complete view of the profiling state can be
formed. For instance, if an allocation event for object A of type T was seen
before a sync point, then it's guaranteed that the class load event for T also
came before that sync point (although it may have arrived after the allocation
event for A).
Other than timely arrival, there was never really a need to force buffers to be
flushed at these arbitrary points. Now that we flush all buffers periodically,
that reason is no longer relevant.
Heapshots have nothing to do with call depth, so they should be processed in
any case.
Also flush the local buffer in all enter/leave callbacks if needed.
This ensures that even for low-activity threads, it won't take ages before the
events from it arrive. We do this flush every 1sec (as with sampling of perf
counters).
The problem is this: Given an allocation of object O on address A1 in thread T1
and a subsequent GC move of O from A1 to A2 in T2, if T2's event buffer is
flushed before T1's buffer, the reader of the log file will a seemingly
nonexistent object being moved, and then see the actual allocation of that
object later. This makes it extremely difficult to keep track of the managed
heap.
The way we solve this is by flushing all thread-local buffers when the GC stops
the world. This way, we know that all prior allocations in the program have
been committed to the log file before any moves that are about to follow as
part of the collection. Similarly, we flush the GC thread's buffer before
restarting the world to ensure that all moves are committed before allocations
that may happen later (which could use some of the addresses freed up for use
due to the moves).
This is reasonably easy now that we have a global LLS of threads which are
attached to the profiler. However, we have to use some kind of lock to ensure
that buffers aren't being written to while we're trying to flush them, or that
a suspended thread was in the middle of modifying its buffer. For this, we use
a reader/writer spin lock of sorts, somewhat similar to the lock used by the
SGen binary protocol. When threads are modifying their own buffer (whether it
be writing to it or flushing it), they take the reader lock. This means that
many different threads can be operating on their own buffers without any lock
contention (which would kinda defeat the purpose of the thread-local buffers).
Now, when a thread -- such as the thread that initiated a GC -- wants to access
the buffers of other threads, it takes the writer lock. In doing so, all other
threads are prevented from modifying their own buffers. This gives the thread
holding the writer lock an opportunity to flush the buffers of other threads.
* Actual thread-local state is now stored in a MonoProfilerThread struct which
is stored in a single TLS slot per thread. This avoids storing things like
call_depth in LogBuffer. The current LogBuffer of a thread is also stored in
the MonoProfilerThread.
* All MonoProfilerThread structs are stored in a global linked list set. This
LLS is used to manually clean up the thread-local state of threads that, for
whatever reason, don't have the thread_end () callback invoked.
* Lifetime management of thread-local buffers is now much more straightforward.
* The thread-local list of referenced methods is now only allocated if needed.
* Fixed a MethodInfo memory leak in handle_writer_queue_entry ().
* Fixed some class load events that could be lost in the writer thread when
fetching the full name of a method.
* TLS slots are now actually freed on shutdown.
Make GC bridge perform well in the "double fan" scenario
The GC bridge currently performs VERY poorly when the object graph has a "double fan" shape. If M Java objects point to 1 C# object which points to N Java objects, the graph exported to Monodroid will currently drop the C# object node and replace it with (M\*N) edges. It is easy to write code where (M\*N) grows ludicrously big.
In testing on device, this patch solves that problem while leaving other cases unaffected. In testing on device, I found:
* All performance tests except double fan: No discernible performance difference after patch
* Double fan test with M=N=1000: Before the patch this took between three and seven seconds. After the patch this took between 0.2 and 0.3 seconds.
* Double fan test with M=N=4000: Before the patch this took anywhere from one to two minutes. After the patch, this took about 1.2 seconds.
* Double fan test with M=N=6000: Before the patch, this took so long I was not able to get it to ever complete. After the patch, this took about 1.3 seconds.
* Double fan test with M=N=20000: I did not attempt this test pre-patch. After the patch, it took about 5.6 seconds.
The commit messages describe the changes in some detail but the short version is:
* Add a mechanism for exporting non-bridged SCCs to the bridge client
* Turn that mechanism on whenever the product fanin*fanout grows too large
* Make the merge cache more aggressive
To function, this patch requires a corresponding change to monodroid. See https://github.com/xamarin/xamarin-android/pull/154
When running msbuild on mono against an architecture-specific
project that also defines satellite assemblies, the /platform
switch will be passed to al.exe. Mono's implementation of al.exe
does not support this switch, which results in a failed build.
This commit adds the /platform switch, which also required
changing the implementation from System.Reflection to
IKVM.Reflection because Mono's implementation of AssemblyBuilder
does not support the necessary options.
While testing this change, I also noticed a few attributes that
the MS version of al.exe will copy from "template" assemblies,
so I added support for those, as well.
SetUp() already sets up a temp folder for the test methods and TearDown()
handles cleaning up and taking into account that on Windows, loaded assemblies
cannot be deleted. However, some of the tests created temp files outside of
this folder and had their own cleanup code which didn't expect temp files to
be undeletable on Windows.
This patch removes any custom temp file creation/cleanup from individual test
methods. It also slightly changes the way SetUp() created the temp folder for
each test, now the paths will be shallower.
When AppDomain.Unload() is called a new thread is created and attached. When
this thread is done the thread handle is immediately closed using
CloseHandle() but the same handle is also passed to CloseHandle() later on by
the InternalThread destructor. On the second CloseHandle() call Windows will
abort the process with an "Invalid handle" error.
This patch removes the CloseHandle() calls in mono_domain_try_unload() which
is called by AppDomain.Unload(). The patch also wires up some more functions
in mono/utils/mono-threads-windows.c to set the MonoThreadInfo.handle field to
NULL when a thread has exited.
* Get rid of the arch-specific headers.
* Do verbose printing in one place.
* Simplify the cross compilation case to avoid repetition.
* Define all variables with a macro in a single header.
* Print detected hwcap info via g_print () so it's visible on mobile.
* Expose plain bool variables in the s390x implementation.
It doesn't work on nunitlite e.g. on XA:
```
src/corlib/Test/System.Runtime.InteropServices/MarshalTest.cs(268,11): error CS0117: `NUnit.Framework.Assert' does not contain a definition for `Less'
```
Previously every time we add a new runtime setting for the bridge
processor we have added a new callback for it. Because this is getting
unweildy, and also because some of these settings might have been
broken by recent init order changes, I created a new set_config
callback and merged the existing set_dump_prefix and enable_processing
with it.
hash_perturb was being mixed in in the wrong place. Fix that, also
make the hash function take some input from high bits on 64-bit
systems, also make the types around the hash more reasonable.
The names were fixed only for the subset of methods that are included in netstandard,
and in cases where netstandard differs from Desktop .NET we match with .NET.
* [runtime] Remove most of the contents of MonoDynamicGenericClass, the elements of the 'fields' array are not used anywhere, making all the other fields unneccesary too.
* [runtime] Remove the MonoDynamicGenericClass structure, merge it into MonoGenericClass. Add some comments.
On Windows we should check the path for invalid characters. Also expand
relative paths set using file:// URLs like .NET does. This patch also removes
the code which prepends "//" on Windows when a file:// URL is given as .NET
doesn't seem to do that in more recent versions.
When having a float variable in memory, we should always access its contents through the fload_r4/fstore_r4 funcs, which do the conversion between float and double and different memsize vs var size. It is common case, to have a temp float, have its address loaded with ldaddr and later do a load_r4_membase on that address to access the contents. If alias analyis lowers this load_r4_membase to an op_fmove directly on the register pair, this would in turn be decomposed into 2 simple int op_moves on soft float, which is wrong if these op_moves are converted to memory accesses.
Even though we register emulations for these opcodes, the native function doesn't not map to the opcode logic, being used instead as part of the opcode decomposition along with other instructions. Make sure we don't try to emulate these opcodes through the general emulation pass.
Add missing source file, sgen-os-coop.c to project.
Add source files and headers into correct filters (Header/Source Files).
Eliminated Visual Studio linker warning LNK4221 for libmonoruntime project generated
by empty source files under certain configurations, boehm-gc.c, lock-tracer.c,
null-gc.c and sgen-os-coop.c
If we happen to have the debugger or the log profiler enabled, we would run into a use before initialization case for the w32handle. The debugger or profiler would call `mono_gc_base_init` which would attach the current thread, and that would crash in `mono_w32handle_new` at w32handle.c:437 with `scan_mutex` being an invalid value (EINVAL).
FileNotFoundException on Mono on Windows and on .NET when target doesn't exist
FileInfoTest.MoveTo_DestFileName_DirectoryDoesNotExist() expects
FileNotFoundException to be thrown but on Windows a DirectoryNotFoundException
is thrown instead.
This patch changes MoveFile() in mono/io-layer/io.c to set the last error to
ERROR_PATH_NOT_FOUND rather than ERROR_FILE_NOT_FOUND when the destination
path doesn't exist. This results in a DirectoryNotFoundException being thrown
in managed code rather than FileNotFoundException. This is the behaviour
observed in .NET.
Also updated FileInfoTest.MoveTo_DestFileName_DirectoryDoesNotExist() to check
for DirectoryNotFoundException rather than FileNotFoundException.
The methods are already available in our implementations of those types so we can just forward the calls
to the "real" types instead of throwing NotImplementedException.
I audited all code in mcs/class/Facades and the only places where we still throw NotImplementedException
are ones where we really need to add a custom implementation.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=43695
It's unlikely the ServiceController APIs will be supported on mobile since they don't really make sense there,
they were initially designed for controlling Windows Services.
If the declaring type of a nested type marked by an xml descriptor
is not otherwise marked, the members of the nested type will not
be preserved, because the linker will not process the nested type
(since the parent type isn't marked, MarkStep:InitializeAssembly
will skip it, and thus skip akk its nested types as well).
https://bugzilla.xamarin.com/show_bug.cgi?id=43658
If sub == 0, i.e. there are no repeated separators, this method just returns
the input string. On Windows, where DirectorySeparatorChar (\) is different
from AltDirectorySeparatorChar (/), this means that the call
Path.GetDirectoryName ("foo/bar/dingus")
won't return "foo\bar" as .NET would but rather returns "foo/bar".
This patch makes sure the '/' -> '\' conversion is run when
AltDirectorySeparatorChar characters have been seen in the input string, even
if sub == 0.
PathTest.GetDirectoryName() triggered this bug.
When a FileStream is opened with the FileOptions.Asynchronous flag set the
code in file-io.c will pass FILE_FLAG_OVERLAPPED to the CreateFile() function.
On Windows this means the file will be in asynchronous mode and when
ReadFile()/WriteFile() are called an OVERLAPPED struct has to be passed. Mono
doesn't do this which means that when reading/writing asynchronous FileStreams
Windows returns an ERROR_INVALID_PARAMETER and an exception is raised in
managed code.
This patch simply removes the code which passes the FILE_FLAG_OVERLAPPED to
CreateFile(). This will work since asynchronous I/O is handled in managed code
anyway, using threads. On the OS level the I/O is still synchronous. The patch
won't affect other platforms since the implementation of
ReadFile()/WriteFile()/etc in mono/io-layer/, which is used on non-Windows, is
always synchronous and ignores the FILE_FLAG_OVERLAPPED.
This bug was triggered by the
MonoTests.System.IO.BinaryWriterTest.AsynchronousModeWrites test.
This test assumes that `new DirectoryInfo (@"c:").Name == "C:\"` when run on
Windows but the actual value depends on the current directory. If current dir
is C:\temp the Name property will return "temp". Similar problem in the
FullName() and Parent() tests.
This patch sets the current directory to C:\ explicitly before running the
failing tests on Windows. The current dir is reset in TearDown().
In an effort to cut down on the number of different scripts we have that try to output NUnit xml.
It also fixes a bug in emitnunit.pl where it would not output anything when one of the tests crashes
the process, because then the final "Overall results: tests: 22425, 100% pass, opt combinations: 25"
line isn't printed and emitnunit.pl just exits. This happens on Alpine.
DoubleTest.Parse() expects "3 5" to be unparseable in the current culture and
expects an exception to be thrown. In sv-SE however "3 5" parses to 35 and
this test fails.
This patch sets the current culture to en-US when the DoubleTest.Parse() test
is run.
When running DirectoryInfo.EnumerateFileSystemInfos() on an empty directory
MonoIO.FindFirst() will return null and the method returns without closing the
handle which is opened by FindFirst(). This patch makes sure the finally block
which closes the handle is run even when FindFirst() returns null.
Also added a call to FindClose() and some more cleanup in the native
implementation of MonoIO.FindFirst() when an allocation fails.
This bug caused an exception in DirectyoryInfoTest.TearDown() and several
other TearDown() methods in the MonoTests.System.IO tests on Windows due to
RemoveDirectory() failing on an empty directory which still had an open handle
associated with it.
The targets depended on a feature that doesn't exist in
xbuild (https://bugzilla.xamarin.com/show_bug.cgi?id=43645)
and as a consequence would fail to add the generated file to
the build when the GenerateTargetFrameworkMonikerAttribute
target was up to date.
With the .NET 4.6.2 merge in https://github.com/mono/mono/commit/b6a30059db8319348ebb454cc303633dfabd6a29#diff-799d44d25450b5b84906cbe7095708a0L86
the RSAPKCS1SignatureDescription class switched from using SHA1CryptoServiceProvider as the DigestAlgorithm to SHA1Cng.
This uncovered an issue when running SignatureDescriptionTest in FullAOT, as we were missing an entry for SHA1Cng
in CryptoConfig.fullaot.cs and so we didn't find the algorithm.
Additionally, 4.6.2 added new variants of the SignatureDescription class using SHA256, SHA384 and 512 which
had to be added alongside the corresponding digest/signature URLs defined by https://msdn.microsoft.com/en-us/library/windows/desktop/dd979768(v=vs.85).aspx
Note that on mobile/FullAOT we're falling back to the *Managed variants of the SHA algorithms as the *Cng/*CryptoServiceProvider variants
aren't exposed there (except for SHA1CryptoServiceProvider since it was in the framework from the beginning) so we need to special case these in the tests.
They were all using the same .cs source files so just including the generic monotouch .sources
makes it easier to reason about and reduces the risk of forgetting to update one of the lists.
SetUp() calls uname() in libc to check whether the test is running on OS X. On
Windows this fails since libc isn't available.
This patch removes the call to uname() completely since the result wasn't used
by the test anyway.
* Fixing linker warning when building libmono DLL.
After splitting mono runtime into a static library used when building libmono
DLL, we got a linker warning since there were no object files left in the
libmono project. There was also a sematic issue with the current organization
of DllMain since it is located in driver.c that will be compiled into the static
library and then consumed by the linker building the DLL. In cases where the
static library was consumed directly it would still include DllMain entry point.
By splitting the DllMain method implementation into a separate specific windows file
we can both resolve the linker warning and remove the DllMain method implementation from the static
library and only include it when building the libmono DLL.
* Visual Studio projects support to optimally link mono applications using static libmono library.
Added a new property, MONO_USE_STATIC_LIBMONO that can be used to link some
mono applications, for example mono(-sgen).exe, towards static version of libmono.
This gives us the option to build a binary without dependencies on other runtime binaries
minimizing what’s needs to be deployed in order to run mono. This in combination with static linked c-runtime
(another option available through a VS property) will produce a mono runtime binary without
addition dependencies except .NET assemblies deployed by mono runtime users.
The default is still using the dynamic version of libmono as before.
* Visual Studio project to test/debug testdriver tests from within Visual Studio.
Added an additional Visual Studio project preconfigured to run testdriver related tests
directly from within Visual Studio.
Commit also includes an additional configuration script that can be used test projects in order
to setup a mono config file. The testdriver project uses this to create a unique configuration
files pointing to the build libtest.dll for running configuration. The configuration file will then
be used when the tests are executed to make sure the correct libtest.dll for current build configuration
gets used when running test different tests from within the Visual Studio debugger.
The project comes with a number of user macros as well that can be used in order to tailor how the tests are run.
Most of the values are preconfigured and shouldn't need to be changed in order to use
current build. In order to change the test that is executed, use the following user macro
defined in mono-testdriver-test property sheet added to the project:
MONO_TESTDRIVER_RUN_TARGET
and point it to the assembly hosting the tests to execute from within Visual Studio.
* Visual Studio project to test/debug nunit tests from within Visual Studio.
Added an additional Visual Studio project preconfigured to run nunit related tests
directly from within Visual Studio.
The project comes with a number of user macros as well that can be used in order to tailor how the tests are run.
Most of the values are preconfigured and shouldn't need to be changed in order to use
current build. In order to change the test that is executed, use the following user macro
defined in mono-nunit-test property sheet added to the project:
MONO_NUNIT_RUN_TARGET
and point it to the assembly hosting the tests to execute from within Visual Studio.
There is also an additional user macro defined:
MONO_NUNIT_FIXTURE
This can be used to limit the number of nunit test run within a test suite.
The merge_cache previously, if it encountered an SCC ("color") with
many outgoing links, would not attempt to merge that SCC with existing
SCCs and instead would just create a new color. Because unbridged SCCs
are now reported to the bridge client, the cost of creating extra
colors has increased. To mitigate this the merge cache now attempts to
merge objects of all sizes, and has been changed to be able to do this
performantly.
The biggest change is that in the case of a hash collision, SCCs will
sometimes be merged without fully checking if they are exactly equal.
This is considered okay because false merges will be infrequent and
nothing worse will happen than short-term failure to reclaim memory.
The bridge-pathologies double fan test has been amended to verify
large-object merging works.
Augments each tarjan-bridge SCC ("color") with a count of how many
SCCs are pointing to it. The mechanism for exporting non-bridged SCCs
uses this count to only export those SCCs where the "double fan"
scenario appears to be occurring. In testing this eliminates the
performance overhead imposed by the previous commit.
Also renamed/moved some variable declarations because namespace bugs
were creeping in.
Introduces a mechanism for non-bridged SCCs which lay between bridged
SCCs in the SCC forest to be reported to the bridge client. In this
version, such non-bridged SCCs are always reported. The result is that
most GCs get a little slower but GCs in which the "double fan" shape
appears (many bridged objects link to one C# object which links to
many bridged objects) become massively faster.
Because before clients were allowed to assume exported SCCs were
always bridged, SGEN_BRIDGE_VERSION has been incremented.
Windows
The error (ERROR_DIRECTORY) wasn't recognized in MonoIO so the error message
became "Win32 IO returned 267. Path: C:\foo\bar" which includes the path. The
test tests that the path isn't include in the message in the thrown
IOException.
Previously we were wrongly trying to achieve this by doing a load-acq followed by a store-rel. Assuming we have memory access on address A (before) and C (after) while the atomic operation happens on address B, the order of the instructions would look like this :
[A], [B]load, load-acq, store-rel, [B]store, [C]
In this case a reordering of memory accesses like the following is possible, which clearly breaks the desired constraints :
[B]load, load-acq, [C], [A], store-rel, [B]store
In order to provide the sequential memory constraint we could emit the barriers like this instead :
[A], membar, [B]load, [B]store, membar, [C]
We can additionally save an instruction and emit the barriers like this instead (this is also what gcc does) :
[A], [B]load, store-rel, [B]store, membar, [C]
In this case we only need to worry about the relation between memory accesses on A and the loading of B. We need to consider what happens if B is loaded before the access on A is fulfilled. This only matters if we load an older value for B (rather than the one visible by other threads) before A. In that case the erroneous store will fail due to using exclusive stores. At the retry the value read from B will be the right one.
Loads with acquire semantics guarantee that following memory accesses are not reordered with the load. Add full membar before it to ensure full sequencing.
Stores with release semantics guarantee that previous memory accesses are not reordered with the store. Add a full membar after it to ensure full sequencing.
NET_2_1 is also defined whenever we define MOBILE, so they have the same effect. Silverlight was 2.1 but it's been dead for a while now, and we do not even support it.
On Win64 Mono doesn't (yet) implement fast JIT TLS (mono_get_jit_tls_offset()
returns -1). This means that the code in
mono_arch_create_handler_block_trampoline() takes the code path in the "else"
which is the same code path taken when AOT compiling. The code generated in
this code path manipulates the stack pointer to ensure it's aligned properly
according to the ABI (RSP-8 is 16-byte aligned on function entry) for the AOT
case where the mono_handler_block_guard_trampoline() C function is called
directly from the handler_block_trampoline.
This patch unifies the JIT and AOT generated code to be as similar as possible
and gets rid of the fast path. There's no need for this code to be fast. The
handler_block_trampoline will after this patch call
mono_handler_block_guard_trampoline() directly, both in JIT and in AOT mode,
and then jump back to the saved return address itself rather than let the
MONO_TRAMPOLINE_HANDLER_BLOCK_GUARD generic trampoline do this.
The tests mono/tests/finally_block_ending_in_dead_bb.exe and
mono/tests/block_guard_restore_aligment_on_exit.exe trigger this bug.
Split up reflection.c into three separate files:
1. reflection-query.c: This contains System.Type and the interrogative System.Reflection APIs.
2. reflection.c: This contains the System.Reflection.Emit APIs
3. reflection-custom-attrs.c: All about dealing with custom attributes.
Simplify code by always accessing tlab information and inside_critial_region through the SgenThreadInfo pointer. This also fixes potential failure to detect that we are in a critical region within the managed allocator.
MONO_THREAD_VAR_OFFSET definitions don't work when loading mono as a dynamic library. This means that we need to be conservative and don't aot code that contains these tls chunks.
In the future, we should remove HAVE_KW_THREAD altogether and use only pthread since it simplifies the code alot.
Added a couple of new/updated test/debug Visual Studio utility projects.
These projects can be used to quicker get up to speed when there is a need
to run/test/debug mono runtime tests inside Visual Studio debugger.
Added utility projects can be used to run mini regression tests using JIT as
well as compile/run full AOT:ed versions of mini regression tests or other assemblies.
The different .prop files are also organized so they can be reused when adding additional
run/test/debug projects going forward.
The code in mono_arch_create_handler_block_trampoline() in tramp-amd64.c emits
a relative CALL which calls the mono_amd64_handler_block_trampoline_helper()
functions via:
amd64_call_code (code, mono_amd64_handler_block_trampoline_helper)
This macro however only works properly for offsets less than 32-bits. For
64-bit offsets it truncates the offset to a 32-bit number. On Win64 the offset
in this case is always a number larger than 2^32. The emitted CALL will jump
to an illegal address causing a crash.
This patch changes the emitted code sequence to load the address of
mono_amd64_handler_block_trampoline_helper() to RAX and then calls the address
in RAX.
The test mono/tests/finally_block_ending_in_dead_bb.exe triggered this bug.
After this patch the test still crashes but in a new way related to another
bug.
* [corlib] Regression test for non-visible custom attributes.
See https://bugzilla.xamarin.com/show_bug.cgi?id=43291
* [reflection] Don't crash when building dynamic custom attributes on dynamic types (Fixes#43291)
1. Don't dereference a NULL custom attribute ctor, throw a type load
exception.
2. When building MonoCustomAttrInfo from an array of
MonoReflectinoCustomAttr*, compute the number of
non-visible (non-public) attributes correctly, and make sure to
iterate over all the attributes when populating the result array.
Fixes [#43291](https://bugzilla.xamarin.com/show_bug.cgi?id=43291)
* [reflection] Marginally better TLE for custom attrs
At least include a message about what went wrong when trying to
construct a custom attribute from a type that isn't finished yet.
I accidentally added this in 3c3ef17376 because I missed that on
full .NET this class inherits from NameValueCollection which provides `this[string name]` already
(the diff tool showed it as missing because it didn't take this into account).
The test fails due an ArgumentException being thrown when the catch handler in
the test's Run() method tries to write to the console. The ArgumentException
is thrown in EncodingTable.GetCodePageFromName(string) since the encoding name
passed to it is null. Normally this exception would be caught and the default
encoding would fall back to UTF8. However, since the thread is aborted the
ThreadAbortException will be rethrown and terminates the test with an uncaught
exception. I have verified that the same behavior is seen on OS X if I mock up
the default encoding code to throw the same ArgumentException.
This patch adds a call to Thread.ResetAbort() just before the call to
Console.WriteLine() in the catch handler. This removes the abort status of the
thread and avoids the rethrow of the ThreadAbortException.
We need a valid MonoThreadInfo in the TLS key while we run
unregister_thread because we may need to block the thread when it tries
to take the sgen GC mutex.
Fixes [#43099](https://bugzilla.xamarin.com/show_bug.cgi?id=43099)
The mono_threads_platform_get_priority()/mono_threads_platform_set_priority()
functions in mono-threads-windows.c assume the "handle" member has been set in
the MonoThreadInfo struct passed to it but it's always NULL. This patch
implements mono_threads_suspend_register() in the same file which sets the
"handle" member at thread registration time, like it's done in
mono-threads-posix.c.
This patch also makes sure the priority of a new thread is set as expected
when the Thread.Priority property has been changed prior to the thread being
started.
Also added a test method to mono/tests/priority.cs which tests that the
priority of the main thread is Normal by default and that it can be changed as
expected.
The changes in this patch fixes the failures in mono/tests/priority.exe on
Windows i386 and x64.
typeof (int).GetType ().GetType ().Name is now RuntimeType since https://github.com/mono/mono/pull/3045.
We did not notice this since most of our bots don't run with en-US locale set which this test requires.
We don't use svn anymore since a very long time and some diff implementations (e.g. in Alpine) don't support --exclude.
Removing the argument is an easy fix.
Previous implementation laid out the BSTR structure
in directly memory in C# code. This layout was not correct
for windows.
Furthermore memory was allocated using AllocCoTaskMem but later freed
using SysFreeString which on windows are different memory areas.
Third issue is that SecureString asumes big-endian when copying the chars
to its internal byte buffer. So on little-endian environments,
a swap of bytes is needed before the buffer is sent to native side
for BSTR conversion. This should probably be handled internally in
SecureString instead.
In the past there were wars between constified and unconstified
second parameter.. NetBSD went with constifying it. POSIX went without
const and most other OSes standarized. NetBSD is still having
incompatible function prototype.
LIBRARY
Standard C Library (libc, -lc)
SYNOPSIS
#include <iconv.h>
size_t
iconv(iconv_t cd, const char ** restrict src, size_t * restrict srcleft,
char ** restrict dst, size_t * restrict dstleft);
iteritems() was removed in py3, we can just use items() instead since we're iterating
over the list anyway so we don't need the lazy evaluation behavior of iteritems().
TODO: there's a gratuitous call to mono_lookup_internal_call_full().
Should really cache the uses_handle info along with the addr since the
lookup by name is rather involved.
Since UNICODE is defined when Mono is compiled under Windows the
GetLocaleInfo() macro will expand to GetLocaleInfoW() which returns the
current locale identifier in g_win32_getlocale() in 16-bit Unicode.
g_win32_getlocale() however expects an 8-bit encoding. By using
GetLocaleInfoA() explicitly we'll get the identifier back as an ordinary 8-bit
C string.
This patch fixes the mono/tests/threadpool-exceptions5.exe test.
MVID and AOTID metadata is no longer displayed at the end of the stack
trace.
Now, every stack frame contains a MVID and AOTID inlined.
An IL offset stackframe looks like:
at StackTraceDumper.Main () [0x00000] in <a3ec873a838c42c29f4eba7cab7d1dfb>:0
A native offset stack frame looks like:
at StackTraceDumper.<Main>m__0 () <0x100e15d50 + 0x00029> in <a3ec873a838c42c29f4eba7cab7d1dfb#b37a9d11247b811ffffdaff8090d8a59>:0
The crash would come from a use-after-free of stack memory: in coop_cond_timedwait_alertable, we would allocate the struct we pass to the mono_thread_info_install_interrupt function on the stack. But there is no guarantee as to when break_coop_alertable_wait is run. The fix is to allocate the data passed to mono_thread_info_install_interrupt on the heap, and free it in the callback, or if it hasn't been interrupted.
the stack.
Windows uses a guard page before the committed stack memory pages to detect
when the stack needs to be grown. If we suspend a thread just after a function
prolog has decremented the stack pointer to point into the guard page but
before the thread has been able to read or write to that page, starting the
stack scan at the thread's ESP/RSP will raise a STATUS_GUARD_PAGE_VIOLATION and
the process will crash.
This patch uses VirtualQuery() to determine whether the thread's stack_start
value points into the guard page. If it does the stack scan will instead be
started at the following, non-guard page.
Windows x64 only has 1 MB of stack per thread which is less than other x86
64-bit OSes. Using 10000 for "depth" in this test causes a stack overflow on
Windows x64. This patch changes "depth" to 5000 only on Windows x64. 5000 will
fit within the default stack size.
This is the recent behavior in .NET/MSBuild as well [1]:
> Starting in Visual Studio 2013, the MSBuild Toolset version is the same
> as the Visual Studio version number. MSBuild defaults to this Toolset
> within Visual Studio and on the command line, regardless of the
> toolset version specified in project file.
With this change xbuild will always use its own version instead
of the one specified in the project, unlessa ToolsVersion is forced
via the /tv switch on the command line.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42938
[1] https://msdn.microsoft.com/en-us/library/bb383796.aspx#Anchor_1
For test harnesses which don't support the full babysitter protocol,
the test harness can now report back paths to NUnit-format XML files
containing test results, or the paths to those XML files can be given
by whoever invoked the babysitter script. Failures listed in the XMLs
are included in the babysitter log.
test-runner.cs has been modified to report its XMLs, so we should now
see mono/tests failures in the babysitter log.
The Process.Exited event would be triggered too early, leading to crash in code using this event to access the process exit code. This is because in Process.ExitCode, we first check that the process has really exited.
ABI states that "All memory beyond the current address of RSP is considered
volatile".
On Windows the stack grows dynamically as the thread needs more stack space.
Windows places a guard page before the page at the top of stack and when this
guard page is accessed by the thread Windows will add a new page to the stack
and resume the thread.
Before this fix SGen on Win64 subtracted 128 from the stack pointer of a
suspended thread and started the stack scan from this address. In the case
when the stack pointer would be less than 128 from the start of the memory
page SGen would attempt to read from the previous page. If this happens to be
the guard page a STATUS_GUARD_PAGE_VIOLATION exception is raised and the
process crashes. The sgen-weakref-stress.exe triggered this bug on almost
every invocation.
We split mono-threads-{posix,windows,mach}.c functions in 2 categories:
- platform: depends on the host the runtime is compiled on (possible values: posix or windows)
- suspend: depends on the mono-threads suspend backend in use (possible values: posix, windows or mach)
On musl __GLIBC__ is not defined, so the conditional logic will
not produce correct result. Add a specific case to handle when
__GLIBC__ is not defined.
It showed the following error before:
```
In file included from wapi.h:23:0,
from wapi.c:2:
../../mono/io-layer/processes.h:218:31: error: unknown type name 'pid_t'
gchar* wapi_process_get_path (pid_t pid);
^
Makefile:603: recipe for target 'wapi.lo' failed
make[3]: *** [wapi.lo] Error 1
```
The NetBSD version of mremap takes additional argument "newp" to hint
new position of a mapped range.
void *
mremap(void *oldp, size_t oldsize, void *newp, size_t newsize,
int flags);
-- NetBSD's mremap(2)
While Linux by default sets old_address and fails if the resize
operation cannot be finished for the original address.
void *mremap(void *old_address, size_t old_size,
size_t new_size, int flags, ... /* void *new_address */);
-- Linux's mremap(2)
NetBSD offers inversed logic with the MAP_FIXED flag to Linux's MREMAP_MAYMOVE:
MAP_FIXED newp is tried and mremap() fails if that address can't
be used as new base address for the range. Otherwise,
oldp and newp are used as hints for the position,
factoring in the given alignment.
-- NetBSD's mremap(2)
MREMAP_MAYMOVE
By default, if there is not sufficient space to expand a mapping
at its current location, then mremap() fails. If this flag is
specified, then the kernel is permitted to relocate the
mapping to a new virtual address, if necessary. If the mapping
is relocated, then absolute pointers into the old mapping
location become invalid (offsets relative to the starting address
of the mapping should be employed).
-- Linux's mremap(2)
The thing that bothers is calling mremap(2) syntax from Linux "posix", but we
cannot do anything with it, for now just simulate the Linux behavior on NetBSD.
for feature in `echo "$enable_minimal" | sed -e "s/,/ /g"`; do
eval "mono_feature_disable_$feature='yes'"
@@ -1091,6 +1097,11 @@ if test "x$mono_feature_disable_security" = "xyes"; then
AC_MSG_NOTICE([Disabled CAS/CoreCLR security manager (used e.g. for Moonlight)])
fi
if test "x$mono_feature_disable_lldb" = "xyes"; then
AC_DEFINE(DISABLE_LLDB, 1, [Disable support code for the LLDB plugin.])
AC_MSG_NOTICE([Disabled LLDB plugin support code.])
fi
if test "x$mono_feature_disable_sgen_remset" = "xyes"; then
AC_DEFINE(DISABLE_SGEN_REMSET, 1, [Disable wbarrier=remset support in SGEN.])
AC_MSG_NOTICE([Disabled wbarrier=remset support in SGEN.])
@@ -1156,20 +1167,6 @@ AC_COMPILE_IFELSE([
AC_MSG_RESULT(no)
])
AC_MSG_CHECKING(for deprecated __attribute__)
AC_TRY_COMPILE([
int doit (void) __attribute__ ((deprecated));
int doit (void) { return 0; }
], [
return 0;
], [
have_deprecated=yes
AC_MSG_RESULT(yes)
], [
have_deprecated=no
AC_MSG_RESULT(no)
])
dnl
dnl Boehm GC configuration
dnl
@@ -2666,12 +2663,6 @@ fi
AC_ARG_ENABLE(bcl-opt, [ --disable-bcl-opt BCL is compiled with no optimizations (allows accurate BCL debugging)], test_bcl_opt=$enableval, test_bcl_opt=yes)
AC_ARG_ENABLE(perf-events, [ --enable-perf-events Enable using `perf` for profiling on Linux], test_perf_events=$enableval, test_perf_events=no)
if test "x$test_perf_events" = "xyes"; then
AC_DEFINE(ENABLE_PERF_EVENTS, 1, [Enable using `perf` for profiling on Linux])
AC_SUBST(ENABLE_PERF_EVENTS)
fi
AC_MSG_CHECKING([if big-arrays are to be enabled])
AC_ARG_ENABLE(big-arrays, [ --enable-big-arrays Enable the allocation and indexing of arrays greater than Int32.MaxValue], enable_big_arrays=$enableval, enable_big_arrays=no)
if test "x$enable_big_arrays" = "xyes" ; then
@@ -2856,13 +2847,12 @@ if test "x$enable_llvm" = "xyes"; then
AC_DEFINE(TARGET_MACH,1,[The JIT/AOT targets Apple platforms])
@@ -3751,6 +3757,45 @@ if test ${TARGET} = unknown; then
AC_MSG_WARN("mono has not been ported to $host: some things may not work.")
fi
if test "x$platform_android" = "xyes"; then
case "x${TARGET}" in
xARM)
case "x$arm_ver" in
xARMv5)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-armv5
;;
xARMv6)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-armv6
;;
xARMv7)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-armv7
;;
*)
BTLS_SUPPORTED=no
;;
esac
;;
xARM64)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-v8a
;;
xX86)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-x86
;;
xAMD64)
BTLS_SUPPORTED=yes
BTLS_PLATFORM=android-x64
;;
*)
BTLS_SUPPORTED=no
;;
esac
fi
if test ${ACCESS_UNALIGNED} = no; then
CPPFLAGS="$CPPFLAGS -DNO_UNALIGNED_ACCESS"
fi
@@ -3873,6 +3918,95 @@ else
fi
AC_SUBST(LIBMONO_LA)
dnl **************
dnl *** Btls ***
dnl **************
AC_ARG_ENABLE(btls, [ --disable-blts Disable the BoringTls provider], enable_btls=$enableval, enable_btls=$BTLS_SUPPORTED)
AC_ARG_WITH(btls_android_ndk, [ --with-btls-android-ndk Android NDK for BoringTls])
AC_ARG_ENABLE(dynamic-btls, [ --enable-dynamic-btls Place the BTLS provider into a separate shared library/archive.], enable_dynamic_btls=$enableval, enable_dynamic_btls=no)
AM_CONDITIONAL(BTLS, test x$enable_btls = xyes)
AM_CONDITIONAL(DYNAMIC_BTLS, test x$enable_dynamic_btls = xyes)
Loaded 100 of 2408 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.