Compare commits

...
Author SHA1 Message Date
Bernhard Urban-Forster 71a7ae4bf0 . 2020-02-12 18:24:12 +01:00
Bernhard Urban-Forster 8824ac84fd initial plumbing for interp support 2020-02-12 18:16:41 +01:00
Bernhard Urban-Forster c49104e842 [sdks] add anroid-interpreter-{arm64-v8a,armeabi-v7a,x86,x86_64} targets
The Xamarin.Android prefers that we ship two different `libmonosgen-2.0.so` for each target, e.g. for arm64 we will have:

* `./sdks/out/android-arm64-v8a-release/lib/libmonosgen-2.0.so`
* `./sdks/out/android-interpreter-arm64-v8a-release/lib/libmonosgen-2.0.so`

As the names indicate one has the interpreter enabled.

This is contrary to what we do for Xamarin.iOS. There, `--enable-minimal=interpreter` has the side-effect that the build will produce a `libmono-ee-interp.dylib` that can be linked next to `libmonosgen-2.0.dylib`.
2020-02-12 18:15:02 +01:00
Alexander Köplinger 071b7259d5 Move offsets-tool into mono/tools (#18783)
This allows us to consume it in the dotnet/runtime build.
2020-02-11 12:18:45 +01:00
Bernhard Urban-Forster 742c414bde [interp] reclaim stack fragments when not used. (#18777)
* [interp] reclaim stack fragments when not used.

When a requested frame size doesn't fit in an already allocated stack fragment, we instead allocate another, larger fragment.
When doing this we missed to free the existing stack fragment, and in combination with a second problem (`current->next` not being cleared when unused) it can accumulate to a lot of unused stack fragments and in worst-case to OOM, as it happened on Linux corlib xunit tests.

Thanks to Jay for tracking it down.

* be cautious for GC

* add barrier
2020-02-11 10:59:49 +01:00
Bernhard Urban-Forster 4cd05a7be4 [cross] support arm-linux-gnueabihf cross compiling with netcore (#18714)
* [cross] support arm-linux-gnueabihf cross compiling with netcore

Some notes on usage:

```console
$ sudo apt install g{++,cc}-gnu-linux-gnueabihf python3-pip

$ ./autogen.sh --disable-boehm --target=arm-linux-gnueabihf \
    --with-cross-offsets=arm-linux-gnueabihf-cross-offsets.h \
    --with-core=only \
    CFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer" \
    CXXFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer"

$ make -C tools/offsets-tool-py/

$ python3 tools/offsets-tool-py/offsets-tool.py \
    --targetdir=/home/lewurm/work/mono \
    --abi=arm-linux-gnueabihf \
    --monodir=/home/lewurm/work/mono \
    --outfile=arm-linux-gnueabihf-cross-offsets.h \
    --libclang=/usr/lib/x86_64-linux-gnu/libclang-7.so.1 \
    --sysroot=/usr/arm-linux-gnueabihf \
    --netcore
```

* better include path guessing
2020-02-11 09:49:21 +01:00
Jay Krell b0c434280d Add volatile for cross thread communication. (#18760) 2020-02-10 21:54:41 -05:00
Alexis Christoforides 72fe53a2ee [MacSDK] Get more verbose output from fc-cache operation (#18782) 2020-02-10 21:53:00 -05:00
dotnet-maestro[bot] 900e77c868 [master] Update dependencies from dotnet/arcade (#18674)
* Update dependencies from https://github.com/dotnet/arcade build 20200201.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20101.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20101.2

* Update dependencies from https://github.com/dotnet/arcade build 20200209.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20109.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20109.1
2020-02-10 17:03:10 +01:00
Jay Krell 8082e87c4e [cxx] Fix C++ build -- int vs. enum. (#18727) 2020-02-09 11:26:52 +01:00
monojenkinsandRyan Lucia 1635c21e59 [loader] Set domain->entry_assembly earlier in startup (#18720)
When locating AppContext.BaseDirectory on netcore, if APP_CONTEXT_BASE_DIRECTORY is not set we call into the runtime for a fallback location, which checks domain->entry_assembly. On legacy this is fine because we set that property in `prepare_thread_to_exec_main`, but on netcore the new managed assembly loading algorithm will call into managed and check the property before that function is run, which will result in AppContext.BaseDirectory returning an empty string. The solution is to set entry_assembly a bit earlier in initialization. This is the appropriate location because it's what embedding API users will call to run an assembly.

In the process, I got rid of some code that would bypass the usual lookup algorithm unnecessarily.

Fixes dotnet/runtime#31800

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-07 17:18:15 +01:00
Alexis Christoforides 96e64ad4e6 [MacSDK] Keep logged post-install script output from previous installs (#18441)
This is helpful for issues such as https://devdiv.visualstudio.com/DevDiv/_workitems/edit/596552 OC, where the user is asked to both provide these logs for debugging, and also reinstall the SDK to resolve the issue
2020-02-07 11:15:48 -05:00
Alex Soto 0c200c5125 [iOS] Replace removed dsymutil -t switch with -num-threads (#18741)
Apple removed `-t` switch from dsymutil in favor of `-num-threads`
2020-02-07 11:13:31 +01:00
Alexis Christoforides d9e090e867 [merp] Add an exception type for managed exceptions (#18685)
Addresses https://github.com/mono/mono/issues/18681
2020-02-06 23:33:27 -05:00
Alexis Christoforides 081498393a [merp] Add tests for crashing via POSIX signal (#18532) 2020-02-06 23:30:11 -05:00
Alexander Köplinger a40e1ae681 Skip WebAssembly build in sdks archive build on non-master branches
WebAssembly doesn't use our usual release branches yet.
2020-02-06 22:36:38 +01:00
monojenkinsandSteve Pfister 88c44e26f6 [mono] Enable debugging for netcore configurations by default. (#18686)
What this means is:

1. --debug is no longer needed since the default is TRUE
2. --debug=ignore is a new switch to go back to the old mono default

Fixes https://github.com/dotnet/runtime/issues/31662

Co-authored-by: Steve Pfister <[email protected]>
2020-02-06 22:19:04 +01:00
Manuel de la Pena b2ad5f34e1 [NUnitLite] Bump nunitlite submodule. (#18731)
Update the nunitlite submodule to pick up some fixes in the NUnit v3
outputwriter that are needed by xamarin-macios.
2020-02-06 15:58:38 +01:00
mdh1418 7936510ec3 [corlib] Silence undocumented exceptions in ReadXdgUserDir (#18684) 2020-02-06 08:49:28 -05:00
Johan Lorensson 6fe343db8c Reduce number of mono header files copied into SDK include folder. (#18702) 2020-02-06 09:11:55 +01:00
Larry Ewing 4cb6ca5599 [wasm][debugger] Use System.Text.Json in the driver 2020-02-05 20:37:46 -06:00
Larry Ewing 8ce81f5113 [wasm][debugger] Let chrome pick an open port by default 2020-02-05 20:37:46 -06:00
Larry Ewing dd151fea35 [wasm][debugger] Silence some logging and extract the null case 2020-02-05 20:37:46 -06:00
Larry Ewing 32826eb418 [wasm][debugger] Make command formatting slightly more sane and reduce c&p code 2020-02-05 20:37:46 -06:00
campersau a2d1aec5d2 [wasm] Close WebSocket on Dispose (#18628) 2020-02-05 18:58:05 -06:00
Alexis Christoforides 31c12db374 [runtime] Improve handling crashing signals (#18243)
The default handlers for SIGTRAP, SIGSYS on macOS and Linux crash the process, so we now have Mono report this.

This patch also changes how the name of the signal is acquired for passing on to the crash reporting functions, allowing code reuse on crashing signals.
2020-02-05 19:04:57 -05:00
Jay Krell 0e49716ff8 [interp] Remove recursion from call_vararg. (#18697)
[interp] Make call_vararg non-recursive.

[interp] Remove recursion from call_vararg.

This contributes to https://github.com/mono/mono/issues/18646 but is not enough to fix it. The interpreter still recurses a lot.
2020-02-05 17:45:53 -05:00
Johan Lorensson b8369e630c Stack corruption running full AOT interpreter using ValueTuple. (#18710)
Calling a method compiled using full AOT from interpreter can lead to
stack corruption if signature include a ValueTuple without fields on
compilers where MONO_ZERO_LEN_ARRAY is not 0 (msvc, clang). This is due
to doing memset on memory allocated using MONO_SIZEOF_GENERIC_INST but
then memset using sizeof (MonoGenericInst). Fix makes sure we allocate
and memset the same size of memory since otherwise it will lead to an
overwrite of allocated memory when type_argc is 0.
2020-02-05 17:35:13 -05:00
Zoltan Varga 64eacfa900 [wasm] Pass the binding assemblies to the linker normally using -d framework_dir instead of explicitly. The latter makes the linker not link them. (#18704) 2020-02-05 11:04:49 -05:00
Zoltan Varga 2c8371c221 [wasm] Rename aot-dummy to aot-instances. (#18692) 2020-02-05 11:04:31 -05:00
monojenkins ab2321c1dc [bcl] Update BCL Linked Size 2020-02-05 08:18:03 +01:00
Zoltan Varga e9830fac8d [jit] Avoid calling mono_aot_direct_icalls_enabled_for_method () in non-aot mode, it might be a stub if aot is disabled. (#18707) 2020-02-05 00:00:27 -05:00
Alexander Köplinger 8027746aea Remove netcore/corerun/Makefile from mono-output.m4
It was removed from DIST_SUBDIRS with https://github.com/mono/mono/commit/2017e493a970a79aa32df2f05e5f1fd877cb3fdc
2020-02-04 23:36:51 +01:00
Bernhard Urban-Forster 71ad5a0771 [mini] iltests.exe depends on TestDriver.dll (#18638) 2020-02-04 23:28:37 +01:00
Matt Sylvia 2f74ab393f Update deprecated query parameter to header (#18705)
Use of `access_token` is now deprecated, switching to HTTP header

```
Please use the Authorization HTTP header instead, as using the `access_token` query parameter is deprecated.
```
2020-02-04 22:57:46 +01:00
Jay Krell b5fb8dbe6a [interp]Factor out common code. (#18672) 2020-02-04 19:53:48 +01:00
Alexander Köplinger 44fe895ab0 Treat 2020-02 as a valid release branch name too
This should give us breathing room until 2100-02 :)
2020-02-04 19:28:24 +01:00
monojenkinsandEgor Bogatov 2017e493a9 Add RunSample target to mono.proj (#18635)
1) Delete `src/mono/netcore/corerun/` folder, we can build/use coreclr's corerun (it has `--clr-path`)
2) Add `RunSample` to build & run HelloWorld with mono runtime.

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-04 19:15:56 +01:00
Zoltan Varga a592b4b871 [wasm] Add linker substitutions for fields. (#18700) 2020-02-04 12:41:31 -05:00
Alexander Köplinger 303122e94a Bump bockbuild 2020-02-04 17:41:44 +01:00
Alexander Köplinger 088692ff3a Bump xsp to get correct version number 2020-02-04 17:41:44 +01:00
Jo Shields 8d00b7cd8b Bump version to 6.13 (#18699) 2020-02-04 17:41:01 +01:00
Jay Krell cdb7944e7b Add --profiling-funcs to release wasm build. (#18663)
[wasm] Put function names in call stacks.

Add --profiling-funcs to release wasm build.
This is a weaker for of -g2 or -g3 or -g4 or --profiling.
It sounds like the weakest form that preserves names.
Documentation is around here: https://emscripten.org/docs/tools_reference/emcc.html#emcc-g1

Now when reproducing for example https://github.com/mono/mono/issues/18646, you get:
```
 e.stack

<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[generate]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke_checked]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke_array]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke_raw]@[wasm code]
<?>.wasm-function[do_icall]@[wasm code]
<?>.wasm-function[do_icall_wrapper]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_wasm_invoke_method]@[wasm code]
wasm-stub@[wasm code]
Db@[native code]
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:189579
call_method@http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:153837
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:156177
callEntryPoint@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:37229
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:33889
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:32396
i@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:31247
promiseReactionJob@[native code]
```
2020-02-04 11:36:22 -05:00
Jay Krell edb18739cb [interp] Remove goto related to code_type being unknown | interp | compiled. (#18678)
[interp] Remove goto for code_type unknown | interp | compiled.

[interp] Remove goto for code_type unknown | interp | compiled.

The goto kinda implies a loop, but it is really just handling one case
before the other, but the source misorders the checks, perhaps for perf,
which maybe, can be done equivalently with G_UNLIKELY, and also remove
some possible extra memory reads.
2020-02-04 09:15:58 -05:00
Jay Krell f85a820677 When the interpreter does recurse, use a different function (#18664)
[interp][wasm]Use per-call recursive functions in interpreter to aid debugging.

 When the interpreter does recurse, use a different function
for each call site, so you can tell from a WebAssembly stack
which case it is. i.e. with function names but no line numbers.

With this, we can see that call_varargs contributes to the problem
here: https://github.com/mono/mono/issues/18646

Also, do not inline the interpreter, it probably/hopefully never was.
2020-02-04 05:56:00 -05:00
monojenkinsandZoltan Varga 1a81bef2bd [runtime] Fix the --enable-minimal=aot build. (#18688)
Fixes https://github.com/mono/mono/issues/18675.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-04 09:51:42 +01:00
Zoltan Vargaandmonojenkins 08f4ef6d0c Bump linker. (#18691)
* Bump linker.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-02-04 09:26:09 +01:00
Jay Krell 8c0f6db26f [interp]Fix the type of clause_args. (#18671)
[interp]Fix the type of clause_args.
2020-02-04 00:25:57 -05:00
Jérôme Laban f3d2cc2176 [Wasm] Update to emscripten 1.39.7 (#18687) 2020-02-03 22:23:47 -05:00
monojenkinsandZoltan Varga 7deefe779f [aot] Avoid inflating gparams with byreflike types during generic sharing. (#18680)
Fixes https://github.com/mono/mono/issues/18676.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-03 21:35:20 -05:00
Zoltan Varga 62a57c687c [wasm] Set the 'top-runtime-invoke-unhandled' option so unhandled exceptions calls AppDomain.UnhandledException. (#18642) 2020-02-03 18:42:49 -05:00
d44a5d3a0b [mini] Add method-triggered printing for runtime stats (#18621)
* [mini] Add optional method name to --stats (#18387)

* Implement parsing for optional method name to --stats command line argument.

* Implement parsing for optional method name to --stats command line argument.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Respond to PR feedback.

* Refactored away helper functions.

* Refactored counter setup into separate function. Moved stats optional method into its own condition.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Renamed mono_setup_counters->mono_enable_counters. Restore whitespace to 0a51d89c42
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* [mini] Print header for stats. (#18442)

* Add an enum for dumping stats at points in time. Print header before dumping stats in mini_cleanup.

* Make print_stats_header private.

* Remove unused variable.

* Refactored print_stats_header to better accomodate optional methods for stats

* Change printf calls to g_printf in response to feedback.

* Undo changes in response to @CoffeeFlux 's feedback. Add g_printf calls to mini_cleanup

* Avoid always printing stats at shutdown.

* Condition on `mono_stats` instead of `mono_jit_stats`.

* [mini] Global JIT Stats Printing (#18483)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Rename mono_print_jit_stats to mono_runtime_print_stats and make it more generic.

* Rename runtime_cleanup_stats back to jit_stats_cleanup.

* Fix a line that was indented with spaces.

* [mini] parse fully qualified method name using mono_method_desc_new (#18499)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Use MonoMethodDesc struct and associated helpers to parse from command
line args. Remove unused stats_method_name and setter function.

Squashed commit of the following:

commit de7410696dab10723c76f545076522daa17e85d1
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 23:01:23 2020 -0500

    Utilize new stats printing function.

commit 1e5038adfec06c670dbaced771a8ff9344502d17
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 22:37:53 2020 -0500

    Remove unused stats_method_name and setter function. Use mono_method_desc_full_match instead of mono_method_desc_match.

commit 04f86c66c139e064bc35738f955f960232eb6cbb
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 18:11:37 2020 -0500

    Modify mono_method_desc_match to actually match namespace and class. Add global MonoMethodDesc, and parse it from command line args. Leave TODO print statement in mono_jit_compile_method_inner.

* Move mono_method_desc to mini-runtime header. Fix whitespace.

* Cherry pick @KyleMarkland 's changes, and resolve merge conflicts.

* Fix leak. Give better error message when parsing empty method name.

* Move freeing of mono_stats_method_desc.

Co-authored-by: KyleMarkland <[email protected]>

* Add a conditional to transform.c to enable stat dumping from the interpreter. (#18540)

* [mini] Use g_printerr instead of g_printf before exiting (#18574)

* Rename g_printf -> g_print

* Use g_printerr on errors instead of g_printf.

* Pass NULL to mono_counters_dump. Revert g_printf to g_print.

* Update manpage

* Feedback

Co-authored-by: rgarc <[email protected]>
Co-authored-by: KyleMarkland <[email protected]>
2020-02-03 16:05:06 -05:00
Vlad Brezae 01b2e15c06 [tests] Reenable enum equals test on interpreter (#18673)
Fixed by https://github.com/mono/mono/commit/9ea350ed3c963c77f0f4f77ec590cc3c84773424
2020-02-03 07:54:10 -08:00
Ryan Lucia 0f47ea5d80 [build] Check for Python 3 in configure.ac (#18625)
I fear this causing packaging problems, but as-is we require Python 3 for most of our Python files but don't actually check for that in configure. We should be consistent about this. Some projects (notably wasm) also use the Python version detected here, so bumping this will also bump the version used in those cases (a good thing, since there are some known bugs when using the scripts with Python 2).
2020-02-03 08:45:49 -05:00
monojenkinsandVlad Brezae 9ea350ed3c [interp] Fix enum constrained calls on netcore (#18659)
We were failing to dereference a managed pointer to the enum, due to incorrect checks, when doing a constrained call.

Fixes https://github.com/mono/mono/issues/18530

Co-authored-by: Vlad Brezae <[email protected]>
2020-02-03 03:08:15 -08:00
Phil Jaenke 20a641e4a0 Fix ppc64 build by fixing -mminimal-toc more correctly. (#18665)
The original fix back at #18554 did not work quite right, which was missed for lack of CI.
Reworked this check to instead test for `-mminimal-toc` on all powerpc hosts, and to only do the ELF version check on FreeBSD.

Fixes #18554 #18578 current CI build failures on AIX
2020-02-03 11:16:33 +01:00
Alex Thibodeau 050340ef1a Adjust mono_class_init_internal to no longer bypass cleanup when a failure is detected. (#18649)
This fixes: https://issuetracker.unity3d.com/issues/unity-physics-collisions-do-not-work-and-errors-are-thrown-when-entering-play-mode
2020-02-02 11:29:57 -05:00
imhameed 4bed68d073 [llvm] Enable jump threading with LLVM JIT. (#18655)
See https://github.com/mono/mono/issues/16243. This will allow elision
of redundant bounds checks. This pass is already used during AOT (as
part of opt -O2).
2020-02-02 01:18:31 -05:00
Fan Yang 3a93ebd6fe Abort cmov optimization, when one of the out blocks is try block (#18626) 2020-01-31 16:40:47 -05:00
mdh1418 9ea6b6a58a [wam] Rename other .stamp-build-debug-sample to debug_sample/dotnet.js (#18653) 2020-01-31 16:34:12 -05:00
Zoltan Varga 2a92593837 [wasm] Bump emscripten to 1.39.6. (#18641) 2020-01-31 13:18:11 -05:00
Zoltan Varga a475b84a3e [aot] Add support for making direct cross-assembly calls in llvm mode. (#18505)
* [aot] Handle '|' during symbol mangling.

* [aot] Clean up the construction of the linker command line.

* [aot] Pass -Wl,-install_name to the linker on osx, otherwise the shared object will be named based on the output file name i.e. <assembly>.dylib.tmp.

* [llvm] Pass the MonoAotFileInfo pointer to the init functions, not used yet.

* [jit] Add support for direct calls between methods in different AOT images in LLVM mode.

* Add a 'direct-extern-calls' AOT option.
* Initialize the PLT/GOT during method initialization,
  this normally happens in load_method () but that is
  not called when a method is directly called.
* Add self initialization support for JITted methods, they can be
  called directly since they are fallbacks if LLVM fails.

* Fix the build.

* Avoid calling BeginInvoke etc. directly, its implemented by the runtime.

* Unify some cases.

* Init PLT at image load time since directly called methods without initializers might make calls through it.

* Fix a startup problem.

* Make the mono_is_corlib_image () check for during startup.

* Check for is_inited in the init icall wrappers, since they might be called from JITted code which doesn't check it.

* Address review comments.
2020-01-31 11:41:49 -05:00
Zoltan Varga 4ef40a91ed [jit] Avoid generating wbarriers for vtype stores to the stack. (#18556)
The existing heuristic doesn't work with llvm.
2020-01-31 09:54:32 -05:00
Zoltan Varga a6707bfeb9 [wasm] Remove .stamp-build-debug-sample, use the output file instead. (#18644) 2020-01-31 01:21:21 -05:00
Zoltan Varga 745c8ff20b [wasm] Avoid overwriting emcc flags when SIMD is enabled. (#18643) 2020-01-31 01:13:17 -05:00
Johan Lorensson 1a2c433b7c Move generic_class_init out of line to assist branch prediction. (#18497)
When calling a generic class we need to make sure it has been inited.
This is done by calling generic_class_init. Since this call happens
frequently, but will only run once, it should be put out of line, assisting
branch predictor, preventing misspredictions (in all places but the first).

Currently this is not the case, since we will generate code like this:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: jne         000000000000048F
000000000000048A: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
000000000000048F: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]

Where we will always (except for first call) take the forward branch.

Instead we should mark the call to the method as out of line, at least
giving the code generator the information so it can move it out of line
and preventing a jmp, unless when it needs to init, the slow path. Doing
that will generate the following code:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: je          00000000000004C1
000000000000048A: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]
...
00000000000004C1: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
00000000000004C6: jmp         000000000000048A
2020-01-31 00:39:37 -05:00
monojenkinsandZoltan Varga 1317cf06da [aot] Fix some races if GOT entries are already set. (#18636)
decode_patch_info () would not decode patches if their corresponding GOT entry
was already set, causing half initialized MonoJumpInfo entries to be returned
to callers. Set the type of the MonoJumpInfo entry to MONO_PATCH_INFO_NONE
in these cases, and have the callers handle it.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 17:12:54 -05:00
monojenkinsandZoltan Varga 04c01efd49 [jit] Add a 'top-runtime-invoke-unhandled' option to treat exceptions which reach the top invoke frame as unhandled when embedding. (#18610)
This is not really a MONO_DEBUG option, but easier to implement it this way.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 15:59:30 -05:00
mdh1418 109a61227c [mono] Prevent direct MonoClass field access (#18634)
* [wasm] Add --enable-checked-build=private_types configure option to disable direct access of MonoClass fields.

* [mono] Indirectly access MonoClass fields
2020-01-30 15:57:55 -05:00
monojenkinsandZoltan Varga 4e90bb7e69 [wasm] Fix mono_vfree (). (#18632)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 13:28:51 -05:00
Steve Pfister e0fcb6093b [wasm][debugger] Cancel Existing Single Step Request When Creating New (#18608)
When stepping out of a c# breakpoint into the runtime (or JS... Not sure if that's accurate),
multiple single step requests can be created when the breakpoint is hit again.  In order to handle this scenario for wasm,
we just cancel the existing one.

NOTE: The debugger args addition seemed a better way to handle this than outright canceling in debugger-engine. If there
are implications to doing this, I can take it out.
2020-01-30 15:19:31 +01:00
Ryan Lucia e3624d356b [wasm] Finish renaming mono.js to dotnet.js (#18623)
This work mostly happened a while back, but this commit finishes the job.
2020-01-29 23:47:46 -05:00
Ryan Lucia cf5204021b [wasm] README updates (#18624)
Use -j with all example make invocations, mention Ninja dependency, mention raising ulimit, recommend py3 for running the server (when using py2, the port isn't properly released).
2020-01-29 23:47:28 -05:00
Larry Ewing b4cfcd5c5e Merge pull request #18602 from lewing/wasm-debug-inspect
[wasm][debugger] Initial support for proxying flat sessions
2020-01-29 17:05:43 -06:00
Alexander Köplinger 5e3e34220c [corlib] Split corlib xunit tests even more for iOS (#18619)
Follow-up to https://github.com/mono/mono/pull/18228 since two parts wasn't enough to fix the size issue on 32bits devices.
We now use three parts.

Fixes https://github.com/mono/mono/issues/18614
2020-01-29 21:28:06 +01:00
Aleksey Kliger (λgeek) 81f184c2ad [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP (#18567)
* [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP

Only prevent `gdb` or `lldb` from being invoked.

MERP is controlled by separate mechanisms --- configure flags for crash
reporting and an explicit opt-in icall.

The existing code collects crashed process data in-process, but it writes the
data out (using mono_merp_invoke) in the forked child process.  This is a bit
surprising, but in the interest of not disturbing the code too much, this PR
doesn't change that.  We should revisit `dump_native_stacktrace` again in the
future and rationalize these decisions.

Addresses https://github.com/mono/mono/issues/18565
2020-01-29 10:03:25 -05:00
Vlad Brezae 60b28b3b56 [interp] Fix virtual calls in mixed mode (#18526)
* [interp] Enable virtual calls to compiled code

If we have an aot-ed version of a method we should always go through it, because the IL for that method could be stripped.

* [interp] Disable jit call for wrappers

Some generic wrappers don't seem to work in mixed mode, probably due to the fact that the transition already reuses a gshared wrapper.
2020-01-29 00:01:58 -08:00
Larry Ewing c75a1364c5 [wasm][debugger] use PdbReaderProvider helper to try to load more pdb forms 2020-01-28 22:32:53 -06:00
Larry Ewing c6e9369de2 Merge remote-tracking branch 'upstream/master' into checkpoint 2020-01-28 22:17:07 -06:00
Larry Ewing edfe20ffc7 [wasm][debugger] start reworking logging logic 2020-01-28 18:53:36 -06:00
Larry Ewing 0fe46d8d35 [wasm] simplify EndOfMessage case 2020-01-28 18:53:36 -06:00
Larry Ewing a6ebd5cb38 Rename Ws* to something more reasonable 2020-01-28 18:53:27 -06:00
Egor Bogatov 883162e861 Use memmove in Buffer.MemoryCopy (#18547)
* Use memmove in Buffer.MemoryCopy

* add test case
2020-01-28 20:00:30 +01:00
Martin Baulig 7d2924adcb [wasm][debugger-proxy]: If we cannot load a PDB file, send a log entry. (#18601) 2020-01-28 12:31:50 -05:00
Nikita Voronchev e2b632e4a7 [WinForms] Incorrect Tree Nodes drawing at the OwnerDrawText mode (#18605)
Assume we have the following form `MyForm`:
```
using System;
using System.Drawing;
using System.Windows.Forms;

namespace test_TreeView
{
    public class MyForm : Form
    {
        TreeView treeView = new TreeView()

        public MyForm()
        {
            treeView.Nodes.Add("root node");
            treeView.Nodes[0].Nodes.Add("first child node");

            treeView.BackColor = Color.Gray;
            treeView.LineColor = Color.Yellow;

            treeView.DrawMode = TreeViewDrawMode.OwnerDrawText;
            treeView.DrawNode += treeView_DrawNode;

            Controls.Add(treeView);
        }

        private void treeView_DrawNode(object sender, DrawTreeNodeEventArgs e)
        {
            var brush = new SolidBrush(Color.FromArgb(100, 255, 0, 0));
            e.Graphics.FillRectangle(brush, e.Bounds);
        }
    }
}
```
The displayed result is
![image](https://user-images.githubusercontent.com/12382656/73263417-92872980-41e1-11ea-8f7c-7cecbb376b60.png)
One can see couple drawing issues:
* White rectangles under each `TreeNode`.
* The area of custom drawing (red rectangles) is shifted with respect to this white rectangles.

Current PR proposes the fix of this issues.
Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:38:27 +01:00
Karl 1bfd317362 [Winforms] Form fixes (#18427)
* Fix Return handling. Ensure correct ClientSize.

When pressing Return / Enter, ensure any target button is both visible
and enabled.

In CreateHandle(), ensure the ClientSize is correct if set. If not,
update it to make it right. Suspend layout while changing it so that
child controls with anchors other than Top Left are not moved.

* Select active control. PerformLayout after updating for client size.

Select the active control with SendControlFocus to avoid upsetting what
is currently selected.

* Improve Form window decoration adjustment handling

Reverts the incorrect attempted fix contained in previous commits.

Basically, when a property is changed that affects window decorations,
ensure the client size stays the same.

* Use UpdateFormStyles on FormBorderStyle too.
2020-01-28 16:32:51 +01:00
Karl 2fdafcf3a4 Don't select ListBox item if SelectionMode == None (#18492)
When the user types a letter or digit, KeySearch looks for and selects
an item starting with that character. However, it should not if
SelectionMode is None.
2020-01-28 16:31:18 +01:00
Nikita Voronchevandmonojenkins c191dba87d [WinForms] Make ContextMenuStrip behavior closer to the reference one (#18598)
* [WinForms] Make `ContextMenuStrip` behavior closer to the reference one

* Bump API snapshot submodule

Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:30:46 +01:00
Alexander Köplinger 726a9134c8 Merge pull request #18491 from PreferLinux/RTF+Textbox-work
I've implemented a heap of missing RTF and RichTextBox stuff, and also made some improvements or fixes to various aspects of it. I'm sorry this is so big...

To give an overview of the changes, here's a list:
- Removed the RTF Charcode / Charset handling that wasn't actually used, had a lot of missing functionality and was basically reimplementing character encoding handling, and replaced it with ensuring the correct character encoding is being used (changing when needed).
- Improved RTF handling of objects with object results (process the given object result). Sometimes pictures can be like this.
- Improved line wrapping and alignment
- Added or greatly improved support for:
  - paragraph spacing
  - line spacing
  - paragraph indents (left, right, hanging)
  - tabstops (left, centre, right, decimal)
  - text background colour (like highlighting)
  - superscript / subscript
  - character offset (vertical)
  - text visibility
- Significantly improved RichTextBox RTF generation
- Cache DPI, rather than doing `CreateGraphics()` every time it is needed
- A variety of other fixes and improvements, with a few I can think of relating to:
  - caret position
  - replacing text
  - invalidation and drawing
  - when tags are added or deleted
2020-01-28 16:26:36 +01:00
monojenkins b87c271b6f [csproj] Update project files 2020-01-28 15:26:03 +00:00
monojenkins e4d69eb248 Bump API snapshot submodule 2020-01-28 15:24:53 +00:00
Steve Pfister 4b4b297450 [wasm][debugger] Handle When BreakpointRequest Commands Come Out of Order
BreakpointRequest commands can come out of order and before the runtime is ready (RuntimeReady). A null check for the DebugStore was added to guard against this since these requests seem to be for JS breakpoints. These seem to be able to be safely ignored.

Fixes https://github.com/mono/mono/issues/18584
2020-01-28 09:57:40 -05:00
Cody Russell a19aaa31da Bump bockbuild for Pango patch (#18566)
Fixes bug #1048838, correct SF font not loading in VSMac
2020-01-28 13:11:04 +01:00
Larry Ewing 4b620fa84d [wasm][debugger] Make the file loading slightly more async 2020-01-27 19:14:38 -06:00
Larry Ewing e1ec438c44 [wasm][debugger] Track the sessionId with all events and commands 2020-01-27 19:14:30 -06:00
Ryan Lucia a5fe7852de [utils] Default to g_print in mono_counters_dump if the stream is NULL (#18582) 2020-01-27 13:24:53 -05:00
Ryan Lucia f41680e395 [eglib] Use the stdout handler with g_printf (#18581)
This ensures it works properly on Android.
2020-01-27 11:57:11 -05:00
Nikita Voronchev ef05316107 [WinForms] Display caret while click on ComboBox (#18523)
If one click on a `ComboBox` the caret does not appear. This is so because `document.PositionCaret` call hides the caret.
2020-01-27 17:00:33 +01:00
Steffen Kieß 1fd0748362 Fix UnixEncoding.GetBytes() for empty strings (#18520)
Current versions of the mono return a null pointer as address of the first
element for empty arrays, i.e.

    fixed (byte* ptr = new byte[0]) Console.WriteLine (ptr == null);

will print "true". This causes a problem in the UnixEncoding class where it
triggers a check for null pointers. Fix this by allowing null pointers when
the number of elements is zero.
2020-01-27 17:00:16 +01:00
dotnet-maestro[bot] 66ff74a1f3 Update dependencies from https://github.com/dotnet/arcade build 20200126.3 (#18597)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20076.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20076.3
2020-01-27 16:53:25 +01:00
felixdoerre 6358345667 Fix open port for X11 applications (#18583)
Whenever mono opens an X11 window, the application opens a tcp socket for controlling the (non-tcp)-connection to X11. This is bad practice, as a tcp socket is not needed.

This pull request cleans the logic, by replacing the tcp socket with a fifo. That way this loopback connection is not visible from outside the .NET application, and all logic surrounding it should remain unchanged.
2020-01-27 14:32:15 +01:00
Karl ba3b6a169b [Winforms] X11 Fix bounds change from within bounds-related events (#18513)
The real change here is, in `XplatUIX11.SetWindowPos()`, simply moving the `SendMessage` call to the end instead of earlier. The problem with it earlier is that any change in bounds from any of the bounds-related events (OnLocationChanged / OnMove, OnSizeChanged / OnResize, OnLayout, OnClientSizeChanged) will be overridden. When `SendMessage` is called, `UpdateBounds` is run, and that raises those events as relevant. After those events finish, `SetWindowPos` continues with setting the size of the window to the value originally specified – which is now potentially wrong. So send the message later on.

There is also no point that I can see in writing the values to `hwnd` twice, because as far as I can see the only one that can be changed is `hwnd.ClientRect` (set by `PerformNCCalc`).
2020-01-27 13:41:00 +01:00
Adrian 23b8f10cb8 Update locale tables - add Tatar CLDR files tt.xml and tt_RU.xml (Fixes #18524) (#18525)
Fixes #18524

The `tt.xml` and `tt_RU.xml` files were added in CLDR 34: http://cldr.unicode.org/index/downloads/cldr-34

Upgrading to v34 (or the latest v36) introduced a larger number of new cultures.  I went with the smallest change possible to have Tatar added.
2020-01-27 11:46:23 +01:00
Phil Jaenke 58b3ac9ad0 Fix for -mminimal-toc detection, enable BTLS on FreeBSD PowerPC (#18578)
Fixes #18554 by making autoconf actually check if `-mminimal-toc` works instead of assuming on powerpc. Tested with FreeBSD 12.x and 13.x on powerpc64 ELFv2.
While we are in here, also enable BTLS which has passed testing.
2020-01-27 11:20:54 +01:00
Vincent Povirk 21e2ebcb06 Allow passing IntPtr values with UnmanagedType.AsAny. (#18544)
This is used by the SplashScreen class of WindowsBase in .NET Core to pass a NULL pointer.
2020-01-27 11:18:55 +01:00
Pranav K c8516b2848 Remove unused variables (#18579)
These show up as build errors in Blazor when they're copied as sources. Seemed trivial enough to fix.
2020-01-27 11:17:44 +01:00
abrevet-dev ed88c509dd [WinForms] Fix #18506 ActiveTracker, do not propagate message to control when it is not enabled and visible. (#18541)
Fixes #18506
2020-01-27 11:07:58 +01:00
Ankit Jain a6a89a5ef4 [master] Bump msbuild to track xplat-master (#18575) 2020-01-27 11:00:26 +01:00
Jérôme Laban d23da5628e [Wasm] Fix threading support file in packager (#18593) 2020-01-26 22:42:52 -05:00
monojenkinsandZoltan Varga c9a97e5258 [runtime] Disable lldb backtrace display on osx, it hangs on attaching in lldb. (#18590)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-26 14:31:07 -05:00
Thays GraziaandAleksey Kliger f8c34dc56b [debugger] Access invalid memory address using PointerValue Command. (#18537)
* Validate the address that came from IDE using PointerValue. The IDE can send an invalid address and it was crashing mono.

Fixes #18191
Fixes #15612

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
2020-01-26 00:00:41 -03:00
Maher Jendoubi 4704c85c9d Fix typos (#18585)
* Contributing: fix typos

* fixed encoding and other typos

* fixed encoding and spotted other typos

* Update TransformationRules.cs
2020-01-25 19:55:22 +01:00
Zoltan Varga 9fc4cae875 [jit] Compute has_references correctly for gshared types whose constraint is a generic valuetype. (#18552)
Also emit write barriers correctly for these types.
2020-01-25 09:20:31 -05:00
Zoltan Varga 9e377af674 [wasm] Fix netcore build. (#18564)
* [wasm] Fix netcore build.

Add a DOTNET_REPO_DIR variable to Make.config which should point to the dotnet/runtime repo. Compile
the netcore runtime and corelib from this repo.

* Disable wasm+netcore on CI.
2020-01-25 06:29:58 -05:00
monojenkinsandZoltan Varga 653db97098 [runtime] Avoid loading ThreadAbortException class on netcore. (#18563)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 20:13:10 -05:00
Phil Jaenke 85b34350c6 Fix for failing compile-runtime-tests (#18569)
compile-runtime-tests is looking for python3, but $(env python3) will return error (no `/usr/local/bin/python3`); if this is the case, create a symlink from 3.6 or 3.7 to /usr/local/bin/python3 so it works. We already have this listed in `SHEBANG_FILES` for the port.
2020-01-24 18:16:44 -05:00
monojenkinsandZoltan Varga 0215d2e331 Initial CMAKE build system support. (#18561)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 17:00:49 -05:00
Radek Doulik 14570f14ac [aprofutil] Update the tool name 2020-01-24 10:04:02 +01:00
mdh1418 ae9335ac21 [mcs] Fix support for digit separators (#18539)
* [mcs] skip "_" when building number

* [mcs] Refactor digit separator feature report

* [mcs] Clean up digit separator code

* [mcs] Throw errors for invalid digit separator usage

* [mcs] Fix hex and bin usage of digit separator tests

* [mcs] Add digit separator usages that throw errors

* [mcs] Update il
2020-01-24 09:18:32 +01:00
Jay Krell 8d69965919 Remove "handles" from System.Threading.Semaphore. (#18512)
Actually, there were no handles.
This reduces MonoError.
2020-01-23 13:58:39 -05:00
monojenkinsandYoussef Victor 05da2da407 Fix typos (#18546)
Found more occurrences of the typos in dotnet/runtime#2061

The work done here is made by simple search & replace.

I'll point to replaces that needs attention in a review.

Co-authored-by: Youssef Victor <[email protected]>
2020-01-23 19:40:58 +01:00
Ankit Jain cc6e927d10 [bockbuild] preinstall: kill any running VBCSCompiler processes (#18488)
- roslyn's shared compiler server hashes it's pipename on the path to
the assembly.
- On macOS, this looks like `/Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/VBCSCompiler.exe`
- When installing a new mono, with that same `x.y.z` version, this
process will effectively be out of sync with the new updated
mono/msbuild/roslyn
- This manifests as:

```
  Using shared compilation with compiler from directory: /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn
  /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : Roslyn compiler server reports different hash version than build task. [/Users/builder/azdo/_work/3/s/xamarin-android/build-tools/xaprepare/xaprepare/xaprepare.csproj]
```

- See https://github.com/xamarin/xamarin-android/pull/4108

- So, we kill any running roslyn shared compiler servers
2020-01-23 19:34:26 +01:00
Thays Grazia 6b475d5d00 [debugger] Native thread not executing managed code considered as terminated (#18501)
* If a thread was suspended and doesn't have any managed stack, it was considered as terminated, but it wasn't really terminated because it can execute managed code again, and stop in a breakpoint so if the execution arrives in debugger_agent_breakpoint_from_context we reset the flag terminated to FALSE.

* Adding terminated = FALSE in single_step_from_context.
2020-01-23 12:41:10 -03:00
Fan Yang 4d9832a399 Add disable_omit_fp support for LLVM (#18489)
Fixes #18417
2020-01-23 16:31:14 +01:00
Alexander Köplinger 0f1ac3e5bb Disable netcore mono builds that now live in dotnet/runtime (#18549) 2020-01-23 14:45:10 +01:00
Zoltan VargaandVlad Brezae 4586c4f091 [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype. (#18496)
* [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype.

Fixes https://github.com/mono/mono/issues/18455.

* [jit] Simplify code

Fix some line damage

Co-authored-by: Vlad Brezae <[email protected]>
2020-01-23 07:57:36 -05:00
Bernhard Urban-Forster 082662cc8a [interp] context can be uninitialized for get_resume_state callback (#18533)
`jit_tls->interp_context` gets initialized lazily, that is, upon the first interpreter execution on a specific thread (e.g. via interp_runtime_invoke). However, with mixed mode the execution can purely happen in AOT code upon the first interaction with the managed debugger.

Stack trace:

```
  thread #1, name = 'tid_407', queue = 'com.apple.main-thread'
    frame #0: 0x0000000190aedc94 libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x0000000190a0f094 libsystem_pthread.dylib`_pthread_cond_wait$VARIANT$armv81 + 672
    frame #2: 0x000000010431318c reloadcontext.iOS`mono_os_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-os-mutex.h:219:8
    frame #3: 0x0000000104312a68 reloadcontext.iOS`mono_coop_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-coop-mutex.h:91:2
    frame #4: 0x0000000104312858 reloadcontext.iOS`suspend_current at debugger-agent.c:3021:4
    frame #5: 0x000000010431be18 reloadcontext.iOS`process_event(event=EVENT_KIND_BREAKPOINT, arg=0x0000000145d09ae8, il_offset=0, ctx=0x0000000149015c20, events=0x0000000000000000, suspend_policy=2) at debugger-agent.c:4058:3
    frame #6: 0x0000000104310cf4 reloadcontext.iOS`process_breakpoint_events(_evts=0x000000028351a680, method=0x0000000145d09ae8, ctx=0x0000000149015c20, il_offset=0) at debugger-agent.c:4722:3
    frame #7: 0x000000010432f1c8 reloadcontext.iOS`mono_de_process_breakpoint(void_tls=0x0000000149014e00, from_signal=0) at debugger-engine.c:1141:2
    frame #8: 0x000000010430f238 reloadcontext.iOS`debugger_agent_breakpoint_from_context(ctx=0x000000016f656790) at debugger-agent.c:4938:2
    frame #9: 0x00000001011b73a4 reloadcontext.iOS`sdb_breakpoint_trampoline + 148
    frame #10: 0x00000001008511b4 reloadcontext.iOS`reloadcontext_iOS_Application_Main_string__(args=0x000000010703a030) at Main.cs:14
    frame #11: 0x00000001010f9730 reloadcontext.iOS`wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 272
    frame #12: 0x00000001042fd8b8 reloadcontext.iOS`mono_jit_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at mini-runtime.c:3162:3
    frame #13: 0x0000000104411950 reloadcontext.iOS`do_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at object.c:3052:11
    frame #14: 0x000000010440c4dc reloadcontext.iOS`mono_runtime_invoke_checked(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, error=0x000000016f656ff8) at object.c:3220:9
    frame #15: 0x0000000104415ae0 reloadcontext.iOS`do_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5184:3
    frame #16: 0x00000001044144ac reloadcontext.iOS`mono_runtime_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5281:9
    frame #17: 0x0000000104414500 reloadcontext.iOS`mono_runtime_run_main_checked(method=0x0000000145d09ae8, argc=1, argv=0x000000016f6570d0, error=0x000000016f656ff8) at object.c:4734:9
    frame #18: 0x00000001042d3b54 reloadcontext.iOS`mono_jit_exec_internal(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1320:13
    frame #19: 0x00000001042d39a4 reloadcontext.iOS`mono_jit_exec(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1265:7
    frame #20: 0x0000000104597994 reloadcontext.iOS`::xamarin_main(argc=5, argv=0x000000016f657a80, launch_mode=XamarinLaunchModeApp) at monotouch-main.m:483:8
    frame #21: 0x00000001008510dc reloadcontext.iOS`main(argc=5, argv=0x000000016f657a80) at main.m:104:11
    frame #22: 0x0000000190af8360 libdyld.dylib`start + 4
[...]
* thread #5, name = 'Debugger agent', stop reason = signal SIGABRT
  * frame #0: 0x0000000190aedec4 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x0000000190a0d724 libsystem_pthread.dylib`pthread_kill$VARIANT$armv81 + 216
    frame #2: 0x000000019095d844 libsystem_c.dylib`abort + 100
    frame #3: 0x00000001045871b4 reloadcontext.iOS`log_callback(log_domain=0x0000000000000000, log_level="error", message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", fatal=4, user_data=0x0000000000000000) at runtime.m:1213:3
    frame #4: 0x0000000104544fc8 reloadcontext.iOS`eglib_log_adapter(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", user_data=0x0000000000000000) at mono-logger.c:405:2
    frame #5: 0x000000010456093c reloadcontext.iOS`monoeg_g_logstr(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, msg="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n") at goutput.c:134:2
    frame #6: 0x0000000104560598 reloadcontext.iOS`monoeg_g_logv_nofree(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, format="* Assertion at %s:%d, condition `%s' not met\n", args="e\x12z\x04\x01") at goutput.c:149:2
    frame #7: 0x000000010456061c reloadcontext.iOS`monoeg_assertion_message(format="* Assertion at %s:%d, condition `%s' not met\n") at goutput.c:184:22
    frame #8: 0x0000000104560674 reloadcontext.iOS`mono_assertion_message(file="../../../../../mono/mini/interp/interp.c", line=7176, condition="context") at goutput.c:203:2
    frame #9: 0x000000010459b570 reloadcontext.iOS`interp_get_resume_state(jit_tls=0x000000014900d000, has_resume_state=0x000000016fc7a9f4, interp_frame=0x000000016fc7a9e8, handler_ip=0x000000016fc7a9e0) at interp.c:7176:2
    frame #10: 0x0000000104319420 reloadcontext.iOS`compute_frame_info(thread=0x0000000104fe4130, tls=0x0000000149014e00, force_update=1) at debugger-agent.c:3422:3
    frame #11: 0x0000000104320d40 reloadcontext.iOS`thread_commands(command=1, p="", end="", buf=0x000000016fc7acf8) at debugger-agent.c:9048:3
    frame #12: 0x000000010431cca0 reloadcontext.iOS`debugger_thread(arg=0x0000000000000000) at debugger-agent.c:10132:10
    frame #13: 0x000000010447eb04 reloadcontext.iOS`start_wrapper_internal(start_info=0x0000000000000000, stack_ptr=0x000000016fc7b000) at threads.c:1232:3
    frame #14: 0x000000010447e788 reloadcontext.iOS`start_wrapper(data=0x000000028203ef40) at threads.c:1305:8
    frame #15: 0x0000000190a11d8c libsystem_pthread.dylib`_pthread_start + 15
[...]
```

Thanks to @drasticactions for helping me to reproduce.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1050615
2020-01-23 13:52:15 +01:00
dotnet-maestro[bot] 4321ba1e3b Update dependencies from https://github.com/dotnet/arcade build 20200118.1 (#18514)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20068.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20068.1
2020-01-23 10:34:41 +01:00
Alexander Köplinger a6f2903d58 configure.ac: remove AC_SEARCH_LIBS for libintl (#18531)
This causes an additional dependency of mono on libintl which we didn't have before.
Reported by a user on gitter, it's problematic on macOS since it makes the mono binary depend on the libintl we ship in the Mono .pkg.

It was introduced in Mono 6.6 by https://github.com/mono/mono/commit/18e0ebfe89be0a175d2f904b9bb1ec6816daa318
2020-01-23 00:29:56 +01:00
Alexander Köplinger b278d88a5f configure.ac: Don't look for X11 in netcore mode 2020-01-22 21:23:54 +01:00
Karl 3d168887a6 Store the current encoding code page, not encoding
The encoding itself could be null, while encoding_code_page
should always be valid.
2020-01-21 10:09:20 +13:00
Karl ff0605fe18 Use correct default encoding. 2020-01-21 10:04:57 +13:00
Karl d826bebd54 Calculate DPI before Recalculate. 2020-01-18 17:11:13 +13:00
Karl f39d652642 Fix NullReferenceException when parsing RTF.
charset_stack was never initialised.
2020-01-18 16:51:32 +13:00
Cody Russell c4aa327737 Bump bockbuild to get GTK NSPasteboardType change. (#18080) 2020-01-17 19:32:26 +01:00
Nikolay Sivov 0a96741f93 Compilation warnings fixes (#18495)
Compilation warnings fixes
2020-01-17 14:16:03 +01:00
Vlad Brezae 92bddddd7b [tests] Re-enable profiler test that used to fail (#18471)
It is not clear if it will fail again or not, but it is not reproducible.

Closes https://github.com/mono/mono/issues/8709
2020-01-17 13:10:59 +01:00
Bernhard Urban-Forster 73eeda130f [prof] remove wrong cast for format string (#18477)
[prof] remove wrong cast for format string

Fixes this warning:
```
  CC       mprof-report.o
mprof-report.c:408:62: warning: format specifies type 'char *' but the argument has type 'guint64' (aka 'unsigned long long') [-Wformat]
                        snprintf (format, sizeof (format), "%s : %%" PRIu64 "\n", (guint64)key_format);
                                                            ~~                    ^~~~~~~~~~~~~~~~~~~
                                                            %llu
/Applications/Xcode112.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:57:62: note: expanded from macro
      'snprintf'
  __builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), __VA_ARGS__)
                                                             ^~~~~~~~~~~
```

mistake in refactoring commit https://github.com/mono/mono/commit/07402062541fa9555bef99dbaac84cfb587926cd#diff-c825627ea9ddb3fa753fa3617eaec7c8R408
2020-01-17 11:05:50 +01:00
Aleksey Kliger (λgeek) 1f0e147cdd [netcore] Mark InitializeCurrentThread NoInlining (#18493)
Follow-up for https://github.com/mono/mono/pull/18480

We might want to replace calls to this with an intrinsic
2020-01-17 10:43:05 +01:00
monojenkins ed6a63bb25 [bcl] Update BCL Linked Size 2020-01-17 10:01:30 +01:00
Bernhard Urban-Forster b4c506c304 [runtime] fix warnings on linux (gcc 8.3.0) (#18479)
* [aot] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
aot-compiler.c: In function ‘emit_trampolines’:
aot-compiler.c:7675:28: warning: ‘_e’ directive writing 2 bytes into a region of size between 1 and 256 [-Wformat-overflow=]
    sprintf (end_symbol, "%s_e", symbol);
                            ^~
aot-compiler.c:7675:4: note: ‘sprintf’ output between 3 and 258 bytes into a destination of size 256
    sprintf (end_symbol, "%s_e", symbol);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At top level:
aot-compiler.c:753:1: warning: ‘link_shared_library’ defined but not used [-Wunused-function]
 link_shared_library (MonoAotCompile *acfg)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:558:1: warning: ‘mangle_symbol_alloc’ defined but not used [-Wunused-function]
 mangle_symbol_alloc (const char * symbol)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:535:1: warning: ‘mangle_symbol’ defined but not used [-Wunused-function]
 mangle_symbol (const char * symbol, char * mangled_symbol, gsize length)
 ^~~~~~~~~~~~~~~~~~~
```

* [amd64] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
tramp-amd64.c: In function ‘mono_arch_create_generic_trampoline’:
tramp-amd64.c:242:65: warning: variable ‘ex_offset’ set but not used [-Wunused-but-set-variable]
  int i, lmf_offset, offset, res_offset, arg_offset, rax_offset, ex_offset, tramp_offset, ctx_offset, saved_regs_offset;
                                                                 ^~~~~~~~~
```

* [lldb] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
lldb.c:243:1: warning: ‘buffer_add_buffer’ defined but not used [-Wunused-function]
 buffer_add_buffer (Buffer *buf, Buffer *data)
 ^~~~~~~~~~~~~~~~~
lldb.c:215:1: warning: ‘buffer_add_id’ defined but not used [-Wunused-function]
 buffer_add_id (Buffer *buf, int id)
 ^~~~~~~~~~~~~
lldb.c:208:1: warning: ‘buffer_add_long’ defined but not used [-Wunused-function]
 buffer_add_long (Buffer *buf, guint64 l)
 ^~~~~~~~~~~~~~~
lldb.c:188:1: warning: ‘buffer_add_short’ defined but not used [-Wunused-function]
 buffer_add_short (Buffer *buf, guint32 val)
 ^~~~~~~~~~~~~~~~

```

* [interp] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
interp/interp.c:3425:1: warning: ‘g_error_xsx’ defined but not used [-Wunused-function]
 g_error_xsx (const char *format, int x1, const char *s, int x2)
 ^~~~~~~~~~~
  CC       interp/libmono_ee_interp_la-mintops.lo
  CC       interp/libmono_ee_interp_la-transform.lo
  CCLD     libmono-ee-interp.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       libmono_dbg_la-debugger-agent.lo
  CC       libmono_dbg_la-debugger-state-machine.lo
  CC       libmono_dbg_la-debugger-engine.lo
  CCLD     libmono-dbg.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CCLD     mono-sgen
  AR       libmain.a
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       interp/libmaintest_a-whitebox.o
In file included from interp/whitebox.c:3:
interp/transform.h:163:12: warning: ‘stack_type’ defined but not used [-Wunused-variable]
 static int stack_type [] = {
            ^~~~~~~~~~
interp/transform.h:155:20: warning: ‘stack_type_string’ defined but not used [-Wunused-variable]
 static const char *stack_type_string [] = { "I4", "I8", "R4", "R8", "O ", "VT", "MP", "F " };
                    ^~~~~~~~~~~~~~~~~

```

* [thread state] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
  CC       libmonoutils_la-mono-threads-state-machine.lo
mono-threads-state-machine.c:33:1: warning: ‘get_thread_no_safepoints’ defined but not used [-Wunused-function]
 get_thread_no_safepoints (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
mono-threads-state-machine.c:26:1: warning: ‘get_thread_suspend_count’ defined but not used [-Wunused-function]
 get_thread_suspend_count (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
```

* [domain] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
appdomain.c:2415:1: warning: ‘get_app_context_base_directory’ defined but not used [-Wunused-function]
 get_app_context_base_directory (MonoError *error)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC       libmonoruntimesgen_la-assembly-load-context.lo
  CC       libmonoruntimesgen_la-domain.lo
domain.c:135:1: warning: ‘mono_domain_alcs_lock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_lock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~
domain.c:138:1: warning: ‘mono_domain_alcs_unlock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_unlock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~~~
domain.c:2129:1: warning: ‘mono_alc_free’ defined but not used [-Wunused-function]
 mono_alc_free (MonoAssemblyLoadContext *alc)
 ^~~~~~~~~~~~~
domain.c:2108:1: warning: ‘mono_domain_create_default_alc’ defined but not used [-Wunused-function]
 mono_domain_create_default_alc (MonoDomain *domain)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
domain.c:2084:1: warning: ‘create_alc’ defined but not used [-Wunused-function]
 create_alc (MonoDomain *domain, gboolean is_default)
 ^~~~~~~~~~

```

* [native library] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libmonoruntimesgen_la-native-library.lo
native-library.c:295:1: warning: ‘mono_loader_register_module_locking’ defined but not used [-Wunused-function]
 mono_loader_register_module_locking (const char *name, MonoDl *module)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* [sample] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libtestlib_la-test-mono-embed.lo
In file included from test-mono-embed.c:7:
../../samples/embed/teste.c:20:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 gimme () {
 ^~~~~

```

* fix for checked builds
2020-01-17 09:36:32 +01:00
Aleksey Kliger (λgeek) c1498b8d4c [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread (#18480)
* [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread

This function (and its call to mono_handle_new) was showing up in the
flamegraph for some async-heavy benchmarks
2020-01-16 21:57:54 -05:00
Aleksey Kliger (λgeek) 2ba25542e3 [interp] Add missing files to tarball (#18482) 2020-01-16 19:27:23 -05:00
Thays Grazia e2d8f5ee91 Revert "On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)" (#18490)
This reverts commit 61894d4a07.
2020-01-16 17:23:58 -05:00
Alexander Köplinger e4552f54fb Bump xsp to fix issues with removed Mono.Security APIs (#18478)
See https://github.com/mono/xsp/pull/88
2020-01-16 21:49:21 +01:00
Thays Grazia 61894d4a07 On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)
Fixes #18106
2020-01-16 17:34:22 -03:00
Alexander Köplinger bafdd7865f Bump corefx to disable two tests. (#18464)
HashTableTests.Ctor_Int_Int_GenerateNewPrime and BitConverterTests.ToString_ByteArrayTooLong_Throws allocate a lot of memory and this causes issues on Windows i386 and other platforms.
2020-01-15 23:52:44 +01:00
Bernhard Urban-Forster bdde94b8e2 [arm] account for signed bit when calculating offset to PLT entry (#18454)
[arm] account for signed bit when calculating offset to PLT entry

See `bl` documentation (make sure you look at the thumb2 one):
<img width="1003" alt="Screenshot 2020-01-14 at 22 43 33" src="https://user-images.githubusercontent.com/75403/72385601-2bb53b00-3720-11ea-9858-d73bc2d02a7e.png">


Fixes https://github.com/mono/mono/issues/18247
2020-01-15 18:35:53 +01:00
Aleksey Kliger (λgeek) e7e81a47c9 [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects (#18457)
* [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Match .NET Framework behavior.

Addresses https://github.com/mono/mono/issues/18388

* [tests] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Regression test for https://github.com/mono/mono/issues/18388
2020-01-15 10:47:27 +01:00
Maher Jendoubi f4567e83dd Contributing: fixing some typos (#18453) 2020-01-14 22:30:06 +01:00
Jay Krell 6eea4a18f0 Reduce some stack use and cleanup. (#17809) 2020-01-14 22:17:00 +01:00
Vlad Brezae fef06df537 [utils] Make sure we include WRAP_ATOMIC_INTRINSIC if needed
Currently, this doesn't impact any targets, but do it for consistency.

Fixes https://github.com/mono/mono/issues/10651
2020-01-14 22:03:48 +01:00
Vlad Brezae 7fb011aa5a [utils] Include missing define of MONO_GNUC_VERSION
We were adding the WRAP_ATOMIC_INTRINSIC all the time on arm64, due to missing MONO_GNUC_VERSION.
2020-01-14 22:03:48 +01:00
Jay Krell 0740206254 Fix many printf warnings (#16484)
* Do not `printf` `long long` or `unsigned long long`.

*Do* printf `gint64` or `guint64`.

Do not printf %ll.
Do not printf %z.

 gint64 ==  int64_t, which could be long, or long long, or maybe other
guint64 == uint64_t, ditto

For printing gint64/guint64, use PRId64, PRIx64, etc.
For printing gsize/gssize, use `G_GSIZE_FORMAT` (seems like excess `G`).

%z definitely would warn for every single use, with Mingw.

And %ll might also, since on 64bit platforms, gint64 tends to be
long, not long long, and would merit %l not %ll.
(No matter that %ll is merited on 32bit, and the behavior
would then be correct on both.).

If there becomes a system w/o PRId64 etc., and it is a 64bit system,
then we could use %l under a portable name.
Or %I64. This appears unlikely (unless reach is extended to older Windows runtime).

Note that PRIxPTR for gsize/intptr_t may be a good idea too.
2020-01-14 22:03:22 +01:00
Ryan Lucia 3b19228a66 [netcore] Implement Thread.GetCurrentProcessorId (#18450)
CoreCLR has a managed cache surrounding this, but for the moment the naive implementation will suffice. We can later move their managed cache to shared on top of this PR with minimal trouble.

We can't guarantee the id will be [0..CpuCount) for a variety of reasons, and a followup PR to the dotnet docs will clarify this. I've chosen not to intentionally add an offset, but no one should be relying on that behavior, and if someone thinks adding an offset would drive the point home the perf cost is irrelevant.
2020-01-14 22:01:13 +01:00
Bernhard Urban-Forster 05f2651a12 [interp] initial infrastructure for whitebox testing of optimization passes (#18268) 2020-01-14 21:59:48 +01:00
Vlad Brezae d9db79b4b2 [sgen] Fixes for max-heap-size option (#18446)
* [sgen] Don't subtract twice the nursery size from max_heap_size

We were subtracting the nursery size when initializing max_heap_size as well as adding it to the allocated_heap when allocating the nursery (alloc_nursery).

* [sgen] Trigger collection if failing to allocate los section

If the memory governor says we are exceeding the max_heap_size by allocating a los section (1MB) for a new los object, attempt to clear some space by triggering a major collection, before completely bailing out.
2020-01-14 11:42:14 -08:00
Jay Krell 5237696f65 [cxx] Make the externC of icalls configuration-dependent instead of always. (#17573)
There are pluses and minuses here:

 + Minimize externC.

 + Maximimze typesafe linkage.

 - Minimize configuration-sensitivity.

 - Maximize commonality across all configurations, so that
   success in one configuration maximally but not 100% implies
   success in others (which is why to avoid `#if` in general,
   to contribute toward compilation success).

 + Use higher level macros to explain reason.
   Even if ICALL_EXTERN_C was always externC (presently not),
   using it instead of plain G_EXTERN_C offers a builtin comment.
2020-01-14 19:28:54 +01:00
Egor Bogatov 08bb5660f6 Optimize typeof(T).IsValueType (#18307)
* Optimize Type.IsValueType

* Ignore gsharedvt types

* use mini_is_gsharedvt_variable_klass

* test

* fix build
2020-01-14 09:37:16 -06:00
Alexander Köplinger 55393c2c9b Add normalized .bat files after the .gitattributes EOL change 2020-01-14 15:52:06 +01:00
Jay Krell c32414966c Remove comment about unexplained C++/LLVM problem. (#18447)
The explanations/fixes were:
 Compile consistently as C++:
   https://github.com/mono/mono/commit/893486638cae8efc46323d6c2209fc99febcd05d

 Make inconsistent compilation safe since it was fragile:
   https://github.com/mono/mono/commit/b2817e6404d573e837e1d8c7e047135974a3b311
2020-01-14 14:20:15 +01:00
Jonathan Chambers 480bef7217 Add mono_gc_thread_detach callback. (#18408)
Add mono_gc_thread_detach callback for Boehm

Use under Boehm to detach without threads lock as Boehm will take it's own lock internally.

Note in `on_gc_notification` we take threads lock after Boehm already has it's own lock. For consistency and to prevent deadlocks always take lock ordering of Boehm then threads lock.
2020-01-14 12:56:09 +01:00
Jay Krell dc866c2aad [cxx] Fixes for all lanes to build as C++ but w/o configure.ac change. (#17452)
* [cxx] dlmalloc remove extern "C" and remove NO_MALLINFO.

* [cxx] Move CallInst bitfields to the end to minimize padding for alignment
and fix an unexplained LLVM problem.
2020-01-14 12:32:08 +01:00
Egor Bogatov f5e1bd3cdf Merge pull request #18293 from EgorBo/vt-gettype-boxing
Optimize boxing for var.GetType()
2020-01-14 14:04:21 +03:00
Zoltan Varga 71a1a84c19 [wasm] Pass --used-attrs-only true to the linker. (#18439) 2020-01-14 09:42:10 +01:00
Alexander Köplinger 53ade46065 [netcore] Fix build warning 2020-01-13 21:27:23 +01:00
Aleksey Kliger (λgeek) 6abab691ec [netcore] Require use of MonoClass getters in Debug builds (#18267)
1. Update `build.sh` to use `--enable-checked-build=private_types` which forces the use of getter methods to access `MonoClass` fields.
2. Fix up some places that were accessing MonoClass directly.
3. Get rid of a spurious comma operator usage from an earlier refactoring:
   ```c
      z = a, b, c;
   ```
   is the same as
   ```c
      z = a; b; c;
   ```
   which is the same as just `z = a;` if `b` and `c` have no side-effects.
2020-01-13 21:15:09 +01:00
Alexander Köplinger a5cc4cd9ec Write version.h for netcore and allow preview VS for msvc builds 2020-01-13 20:47:35 +01:00
Karl 8bd438f318 [Winforms] Calculate GroupBox preferred size the same way as Panel (#18429)
GroupBox's GetPreferredSize() is not great. I've simply copied the code from Panel, and used that. It uses LayoutEngine.GetPreferredSize(), which does things a lot better than the code this used.
2020-01-13 17:15:09 +01:00
Karl 4ee4d368c6 [Winforms] Always draw at least one line of a Label. (#18422)
Previously, if the Label wasn't quite high enough nothing would be drawn as the line didn't quite fit. It should draw the line anyway, clipped if necessary.
2020-01-13 17:12:36 +01:00
Karl 42956f554b [Winforms] Improve X11 keyboard handling. (#18428)
- Ignore key events with vkey == 0. They won't match a real key code so don't try. Besides, trying can cause segfaults on ChromeOS Linux mode.
- Check special-case keys (numpad) before looping to get them. No point leaving them for afterwards...
2020-01-13 17:12:10 +01:00
Karl 7a8242ed36 [Winforms] Don't have non-active controls focused (#18426)
- Make the state of Control.CanSelect depend on whether the control is visible and enabled.
- When setting Control.Visible to false, if this control is focused, move the focus to the next control.
2020-01-13 17:11:38 +01:00
Karl d7e84ac6d1 [Winforms] Implement correct Control.Region / ClipRegion support on X11 (#18421)
Fixes https://xamarin.github.io/bugzilla-archives/20/20233/bug.html

Previously, on Linux, setting Control.Region just set the clipping for painting, but didn't influence the visibility of other controls behind the one it was set on. This implements this correctly via X11 Shape extension.
2020-01-13 17:10:57 +01:00
Maxim Lipnin 87cfaf15ae [netcore] Make System.Diagnostics.TraceSourceTests.DefaultTraceListenerClassTests.EntryAssemblyName_Null_NotIncludedInTrace Pass (#18403)
* [netcore] Extract Assembly.GetEntryAssembly method to shared part
2020-01-13 17:06:55 +01:00
Johan Lorensson 407898621e Make sure .bat and .cmd files have crlf in wd. (#18437)
Looks like https://github.com/mono/mono/commit/c9dc20f59ebe7fbed2a439d69c2d03a2d4500577 broke Windows build for some scenarios using .bat files. Fix makes sure we convert .bat and .cmd files to use crlf in wd.
2020-01-13 15:01:41 +01:00
dotnet-maestro[bot] 49c5101200 Update dependencies from https://github.com/dotnet/arcade build 20200112.1 (#18438)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20062.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20062.1
2020-01-13 15:00:54 +01:00
Egor Bogatov 423192ed85 [netcore][llvm] Implement Sse1-4.2 subsets used by corlib (#18103)
* Implement SSE41 subset used by corlib

* Implement Sse2.MoveMask

* only works with llvm

* Implement Sse3 and Ssse3 subsets used by corlib

* Implement a few SSE1 methods

* Address feedback, also, implement Sse.Add/Subtract

* Fix build

* Implement Sse.Multiply, Sse.Store

* Implement Sse.CompareNotEquals

* Implement Sse.MoveScalar

* Finish SSE1 corlib subset

* Implement Sse.LoadVector128, Sse.Shuffle

* Sse.Shuffle cleanup

* Implement Sse2 APIs

* More of SSE2: LoadAlignedVector128, Compare*

* Implement Sse2.Unpack* and Sse2.StoreScalar

* Implement Sse2.PackUnsignedSaturate

* Implement Sse2.ShiftRightLogical

* Implement Sse2.Shuffle

* Implement Vector128<T>.Zero

* Fix CreateScalarUnsafe

* Implement Vector128.As*, Fix Vector128.CreateScalarUnsafe

* Fix failures

* Fix failures

* Fix Sse.MoveMask

* remove redundant null checks

* Fix AOT failures

* fix compilation warrning

* rename create_vector_mask_*

* Fix failures found via tests

* Index in Sse41.Insert has to be a constant

* add local tests for mono

* Update tests (cleanup)

* Code cleanup

* test

* fix typo

* Clean up

* Clean up

* Implement And, AndNot, Or, Xor, Divide for Sse1

* Cleanup

* Fix build

* limit emit_vector128 with LLVM

* enable IsSupported for corlib

* Fix build on wasm (if-defs issue)

* Don't intrinsify Vector256

* Address feedback
2020-01-13 13:51:51 +01:00
Karl e92fe19fa6 [Winforms] X11Keyboard LookupString fixes (#18423)
Two small fixes to X11 keyboard handling here:
- For Xutf8LookupString, use the actual buffer size rather than the starting buffer size. Using the previous fixed size made the buffer overflow handing code immediately after the change pointless.
- For XLookupString, handle similar to Xutf8LookupString, getting a byte array. The old was sometimes causing problems where input on the numeric keypad didn't enter text. I believe using a StringBuilder similar to how it was _should_ work, but for some reason I could not get it to work reliably. So I changed it to this, which seems to work well for me. The only question I have is whether it should be Encoding.ASCII like I used, Encoding.Default, or another specific encoding.
2020-01-13 12:19:36 +01:00
Zoltan Varga 4b718ca1f2 [jit] Fix an assertion in the Thread.VolatileRead () intrinsic. (#18430)
Fixes https://github.com/mono/mono/issues/18418.
2020-01-13 12:09:09 +01:00
Vlad Brezae a13f5ccbb6 [interp] Fix struct copying during pinvoke transitions (#18402)
If we are in pinvoke wrapper, we need to use the marshalled native size, not the managed size.
2020-01-13 12:05:25 +01:00
Zoltan Varga 09a7317e5a [wasm] Disable a test which OOMs. 2020-01-13 10:00:49 +01:00
Karl ff494e6065 Only calculate scrollbars once handle created 2020-01-13 17:54:00 +13:00
Karl b8f747e452 Significantly improve DPI caching.
Particularly, don't call CreateGraphics() in TextBoxBase's constructor.
2020-01-13 17:54:00 +13:00
Karl e9abb44242 Correct code style problems. 2020-01-13 17:54:00 +13:00
Karl c41ee8050e Improve font name and charset handling.
- Use alternative names as a fallback if present.

- Use the untagged name (\fname, name that doesn't
identify charset ) if present.

- Use font codepage or charset correctly to get the correct encoding.

Rip out the old nonfunctional RTF charset handling.
This was not used, and furthermore was fundamentally flawed.
I have replaced it by simply using the correct text Encoding, mentioned
above.
2020-01-13 17:54:00 +13:00
Karl 9b87a23ee6 Replace Document Height / Width Changed events with SizeChanged 2020-01-13 17:54:00 +13:00
Karl 150d5e209d Round font ascent / descent up in LineTag. 2020-01-13 17:54:00 +13:00
Karl 24c2e505a3 Fix updating to cover lines modified while suspended. 2020-01-13 17:54:00 +13:00
Karl e586084ae8 Rearrage slightly, fix kerning. 2020-01-13 17:54:00 +13:00
Karl 3d9e4fa7f4 Fix CharCount, CharIndexToLineTag, recalculate wrapped lines.
CharCount / length was inconsistent about counting line endings. Fixed
to include them all, as that was much easier than the other way.

When shortening a wrapped line, it is necessary to start the recalculate
at the previous line, in case the wrap point has changed.

In CharIndexToLineTag, if there is a line boundary at the specified
index, return the line starting at the index.

Fixed a few other things too.
2020-01-13 17:54:00 +13:00
Karl 11a89a53ab Recalculate after changing canvas. Fix scrolling. 2020-01-13 17:54:00 +13:00
Karl 0b6dd983dd Later RTF para overrides earlier. Emit \nosupersub.
RTF paragraph controls override earlier instructions, rather than
keeping the maximum.
Emit \nosupersub between superscript and subscript text. Missing it out
technically gives a superscript (or subscript) of the subscript (or
superscript), which is not what we want.
rather than keeping the maximum.
2020-01-13 17:54:00 +13:00
Karl df3b5a26b4 Improve indent and kerning handling. 2020-01-13 17:53:59 +13:00
Karl ca7352ad24 Add tabstop support. Use floats for spacings.
Floats are now used to preserve paragraph spacing and indents. The RTF
format uses twips (1/20") for these, which is usually less than 1px.
2020-01-13 17:53:33 +13:00
Karl b35c560df3 Lots of TextControl stuff, plus others.
* RTF.cs: Add support for JPEG and EMF pictures.
  Allow for attribute etc. groups inside the picture group.

* Line.cs: In InsertString, ensure that the tag passed in is currently
  in the line – not necessarily valid when the passed tag is the
  cursor, for instance.

* LineTag.cs: Only break tags for formatting when needed.

* TextControl.cs: Fix invalidating text that isn't left-aligned.
Fix caret stuff:
  Positioning after the line ending no longer happens
  Up and down when not left-aligned
  Tag used when moving left to start of line
  Page up / down
  Remove empty tags when navigating away from them
Change tag background draw location – I think it is a correction!
When inserting at a position without being given the tag, choose an
  empty tag at that position if one exists.
Change ReplaceSelection order back (corrected some formatting and
  similar stuff), and format the new text to match the tag.
Change IncrementLines to transverse the tree (using a new function)
  rather than repeatedly call GetLine.

* RichTextBox.cs: Don't streamline the line in case it removes tags we
  want to keep – we'll skip them ourselves instead.
Emit pictures.

* TextBoxBase.cs: Remove redundant check – a previous check means
  value != acutal_border_style, therefore at least one of them !=
  BorderStyle.Fixed3D, therefore the if condition was always true.
2020-01-13 16:42:44 +13:00
Karl 681b514e88 Use a float for the RTF font size to avoid rounding error when dividing by two to get the point size. 2020-01-13 16:42:44 +13:00
Karl dbf98ffc93 Add line spacing support. Move dpi to TextControl.
Line spacing seems to be working perfectly.
2020-01-13 16:42:44 +13:00
Karl f25ca77bfe Fix ReplaceSelection() again. Line spacing support. Fix drawing non-multiline selection.
It was inserting at the start, and then removing what had been inserted +/- whatever length as needed.
2020-01-13 16:42:44 +13:00
Karl e01e43025d Fix TextControl.ReplaceSelection to insert before removing so that setting SelectedText doesn't lose formatting. 2020-01-13 16:42:44 +13:00
Karl 1821cc8adf RichTextBox: Cache dpi
The dpi isn't likely to change during the lifetime of the object, so is relatively safe to cache.
Creating a Graphics object each time the dpi is wanted when reading RTF is also rather silly performance-wise.
Even worse, when generating RTF it can cause a recalc, which can mean that some lines <= end_line don't exist.
2020-01-13 16:42:44 +13:00
Karl be446f04f3 Three fixes in RichTextBox, one GenerateRTF, two ContentResized.
In GenerateRTF, check tag != null before trying to generate anything based on it. Not sure
why it hasn't been a problem before, but either way I was getting NullReferenceExceptions
because of it today.

Run ContentSizeChanged() when either width or height changes, irrespective of Multiline status.
This fixes various problems with stuff jumping around vertically.

Correct ContentSizeChanged to consider the left and right margins instead of the left margin twice.
Was probably a typo at an earlier stage, otherwise would have been a * 2.
2020-01-13 16:42:43 +13:00
Karl 42cecb6ed2 Fix line calculation for vertical positioning of tags.
* Line.cs: Fix handling of differing tag Ascents and CharOffsets in recalculateLine.

* TextControl.cs: Remove unneeded assignments.
2020-01-13 16:42:43 +13:00
Karl fee6509476 * Line.cs: Finished off indent support, also added tabstops.
* LineTag.cs: Added support for visibility. Corrected handling of tabs
  and added tabstop support.

* RichTextBox.cs: Added support for text visibility, indents,
  tabstops.

* TextControl.cs: Make invisible text work correctly. Fixed a bug with
  tag backcolor. Added support for indents.
2020-01-13 16:42:35 +13:00
Karl f98c1e0087 Fixed so that it sets measuredText to an empty rectange if not
measuring text.
2020-01-13 16:09:02 +13:00
Karl e4a5b4a720 * RichTextBox.cs: Align Justified as Left instead of as Center.
* TextControl.cs: Provide an output on PictureTag.Draw(... out
  Rectangle measuredText ...) if requested.
2020-01-13 16:08:29 +13:00
Karl cc91b61cea * RTF.cs: More work on ReadObjGroup
* RichTextBox.cs: Fixed up the insertion of pictures.
2020-01-13 16:06:41 +13:00
Karl aab3dca9fc All sorts of work, mostly on RichTextBox. 2020-01-13 16:03:11 +13:00
Zoltan Varga 73468eda91 Bump linker. (#18424) 2020-01-12 10:57:28 +01:00
Ankit Jain fc7b254953 [master] Bump msbuild to track xplat-master (#18432) 2020-01-12 10:56:21 +01:00
monojenkins ee3da3821b [bcl] Update BCL Linked Size 2020-01-11 08:58:48 +01:00
Martin Baulig a702e4ff02 [Sockets]: Always reset internal SAEA completion when reattempting connection in MultipleConnectAsync. (#18389)
This should fix #18030.
2020-01-10 19:01:36 +01:00
Zoltan Varga 3563bd90dc Revert "[wasm] Pass --used-attrs-only true to the linker. (#18401)" (#18415)
This reverts commit 6f4fe05209.
2020-01-10 16:49:43 +01:00
Ankit Jain 9666bcfae7 [master] Bump msbuild to track xplat-master (#18412) 2020-01-10 12:45:01 +01:00
Vlad Brezae d33a100961 [interp] Add gc unsafe transition during pinvokes (#18399)
Native methods can be run in gc safe mode, meaning we can be in gc safe mode while setting/getting the arguments for a native transition. Call context computation requires us to be in unsafe mode.

Fixes https://github.com/mono/mono/issues/16969
2020-01-10 09:32:04 +02:00
Zoltan Varga 5a65a97c58 [corlib] Mark the SRE types with required=false so they can be linked out. (#18379)
Currently SRE types are still kept by the reference in RuntimeType::MakeTypeBuilderInstantiation ().
2020-01-10 04:22:45 +01:00
Larry Ewing 439091eddf [wasm] use updated node to run jsvu (#18386)
* [wasm] use updated node to run jsvu

* [wasm] make the browser tests use the node we've installed

* [wasm] Fix browser tests to reference dotnet.js
2020-01-09 22:42:03 +01:00
Zoltan Varga 6f4fe05209 [wasm] Pass --used-attrs-only true to the linker. (#18401) 2020-01-09 19:12:16 +01:00
Alexander Köplinger c9dc20f59e Fix line endings in .bat files
They need to be CRLF otherwise we'll get conflicts when moving into dotnet/runtime which has a .gitattributes setting forcing .bat to CRLF.
2020-01-09 18:06:53 +01:00
Ankit Jain 46f834fda7 [master] Bump msbuild to track xplat-master (#18391) 2020-01-09 17:45:05 +01:00
Maxim Lipnin f5aca4235e [netcore][xunit tests] Re-enable System.Diagnostics.Tests.ProcessTests.ProcessStart_UseShellExecute_ExecuteOrder (#18397)
#14903
2020-01-09 17:01:57 +01:00
monojenkins e55a1a4f62 [bcl] Update BCL Linked Size 2020-01-09 14:21:06 +01:00
Vlad Brezae 345550fb99 [interp] Throw overflow exception when converting NaN (#18384)
Fixes https://github.com/mono/mono/issues/18061
2020-01-09 12:08:15 +02:00
Bernhard Urban-Forster 207cf805e5 [memory barrier] use pagesize determined by helper (#18383)
[memory barrier] use pagesize determined by helper

`limits.h` doesn't always define `PAGESIZE` properly (e.g. iOS), so it would fall back to `0x1000` (4096) but it's really `0x4000` (16384) on iOS. We already have a helper in the runtime to determine the pagesize, so let's use that instead.


Fixes https://github.com/mono/mono/issues/18273
2020-01-09 10:46:03 +01:00
Marek Safarandmonojenkins 9cc751d618 [corlib] Switch to CharUnicodeInfo from corefx (#18305)
* [corlib] Switch to CharUnicodeInfo from corefx

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-01-09 09:36:55 +01:00
Zoltan Varga 147c7972a8 [wasm] Reenable linker substitutions. (#18381) 2020-01-09 05:18:58 +01:00
Zoltan Varga 46fe5ad3f6 Bump linker. (#18380) 2020-01-09 05:18:17 +01:00
Martin Baulig ba7c00e4a9 Add workaround for #17710. (#18275) 2020-01-08 17:50:55 -05:00
Ryan Lucia 9fd95db929 [reflection] Fix check for Type.FullName to match CoreCLR (#18385)
Updated test to follow in dotnet/runtime
2020-01-08 17:02:25 -05:00
Ryan Lucia 1178ab3a1a [metadata] Propagate error in mono_runtime_try_invoke_array (#18382)
This was prompted by https://github.com/mono/mono/issues/18339, which seems to be failing to locate a native library and then hitting an assertion. We should propagate the error instead, so that if others run into something similar they can see what's actually going wrong instead of just getting a runtime crash.
2020-01-08 15:57:16 -05:00
Egor Bogatov ee5850a2b5 Update FieldInfoTest.cs 2020-01-08 23:21:41 +03:00
dotnet-maestro[bot] 5dbc923cb3 [master] Update dependencies from dotnet/arcade (#18206)
* Update dependencies from https://github.com/dotnet/arcade build 20191215.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19615.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19615.1

* Update dependencies from https://github.com/dotnet/arcade build 20191222.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19622.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19622.1

* Update dependencies from https://github.com/dotnet/arcade build 20191229.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19629.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19629.1

* Update dependencies from https://github.com/dotnet/arcade build 20200105.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20055.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20055.1
2020-01-08 20:47:44 +01:00
Jay Krell e8d38961ee [Coop] Remove handles from Mono.Interop.ComInteropProxy. (#16993) 2020-01-08 13:34:11 -05:00
Alexander Köplinger ba505510f2 FreeBSD: fix provisioning script to force reinstall perl5 at the end
It didn't work otherwise.
2020-01-08 16:46:58 +01:00
Zoltan Varga 34dee0ea4e [runtime] Initialize field info before accessing it in ves_icall_System_RuntimeFieldHandle_SetValueDirect (). (#18378)
Fixes https://github.com/mono/mono/issues/18364.
2020-01-08 16:39:39 +01:00
Ankit Jain d31f6ac4da [master] Bump msbuild to track xplat-master (#18376) 2020-01-08 15:01:03 +01:00
Zoltan Varga e999846088 [jit] Implement fast TLS access in gshared methods. (#18377)
Fixes https://github.com/mono/mono/issues/18370.
2020-01-08 13:32:52 +01:00
Zoltan Varga d0a3fb1b1b [wasm] Rename binaries mono -> dotnet. (#18373) 2020-01-08 03:04:55 +01:00
Ankit Jain aa87d3db6d [master] Bump msbuild to track xplat-master (#18372) 2020-01-07 20:56:41 +01:00
Bernhard Urban-Forster 6481e8f7a3 [interp] run regression with all combinations of available optimizations (#17644)
[interp] run regression with all combinations of available optimizations

Like we do for the JIT.
2020-01-07 19:35:54 +01:00
Zoltan Varga 3d4cd57a52 [wasm] Pass --explicit-reflection to the linker only in aot mode. (#18367) 2020-01-07 18:50:22 +01:00
Phil Jaenke f3f11d1081 FreeBSD: Fix for perl5.26 / perl5-5.30 conflict (#18368)
The base image appears to have perl5.26 which conflicts with perl5-5.30, resulting in the /usr/local/bin/perl binary not being placed by perl5-5.30, which results in failed builds. The `perl5.26` appears to be part of the base image and has no packages actually dependent on it.
2020-01-07 18:14:55 +01:00
Calvin Buckley 524a8298d0 Force Python 3.x from env in shebang lines (#18285)
* Fix byte/string incompatiblity in Python 3
2020-01-07 17:28:06 +01:00
Zoltan Varga ec7f4c68f9 [netcore] Remap String ctors to the String:Ctor () methods. (#18357) 2020-01-07 17:12:34 +01:00
Ankit Jain 71070d34c8 [master] Bump msbuild to track xplat-master (#18297)
This enables parallel build support with msbuild!
2020-01-07 16:21:51 +01:00
Calvin Buckley b3ce5bfa88 More disambiguation of Python in makefiles (#18284) 2020-01-07 15:44:03 +01:00
Vlad Brezae 376c46bcf3 [sgen] Fix xref computation with tarjan bridge (#18239)
* [sgen] Include also derived classes as bridges

When using `MONO_GC_DEBUG=bridge=` debug option

* [sgen] Fix xref computation with tarjan bridge

Between C# and java (on android) there are objects that live on both worlds. This means that there exists a C# object with a corresponding java object. The relationship between them is strong, meaning if C# object is alive then java object must stay alive, and vice-versa. We keep java bridge objects always alive through a GCHandle (on the java gc). When doing a C# collection we select all bridge objects that appear to be dead on the C# side. These objects are candidates for collection, assuming the java side has nothing against it. Before triggering a collection on the java side (following the mono gc) we switch all the strong gchandles to the java objects to be weak (for these objects that are candidate to be collected) and we recreate the reference graph from the C# side to the java side (if C# Bridge1 can reference C# Bridge2 then, on the java side, we add a reference from Java Bridge1 to Java Bridge2; this is done by adding to an array of references inside Java Bridge1). In order to minimize the amount of work that needs to be done on the java side, we compute the minimal amount of references that need to be added, by computing the strongly connected components of the object graph.

An optimized way to construct the SCCs and the xrefs is by using the tarjan algorithm (https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm). Our algorithm is non recursive (scan_stack emulates the recursive order of traversal in a dfs algorithm, while loop_stack is the stack used by the algorithm). A color represents an SCC. We have some optimizations in place where we might merge colors if they don't contain bridges, since the client only cares about SCCs containing bridge objects and the links between them. color_merge_array is used to keep track of all neighbors of a node until we are creating the scc for that node. It is populated when scanning all the refs inside an object (compute_low). All the colors in the color_merge_array will be cross references with that scc.

Before this commit we were only clearing the color_merge_array when creating an SCC. This is problematic because we could end up with xrefs inside of an SCC that belong to another SCC. Consider the simple graph of nodes 0,1,2,3 where 0 <=> 2, 0->1, 2->3. Assume we start scanning with node 0. When creating the SCC for node 3 the followed path is 0 -> 2 -> 3, while the loop stack will contain (0,1,2,3). After creating SCC for node 3, we will finish scanning node 2 which would detect the xref to Bridge3, which would have been added to the color_merge_array. Because node 2 is not the root of the SCC it belongs to (its lowlink points towards node 0 which has a lower index), we are not creating an SCC with it, and the link to node 3 remains in color_merge_array. Because the next node from the scan_stack is node 1, which is also the root of the SCC that it belongs to, we will create an SCC for it and wrongly add the node 3 reference from color_merge_array to it. In order to fix this issue, we will always clear the color_merge_array once we finished scanning the xrefs for a node. If the node in question is not the root of an scc, then we will remember them as xrefs pointing out from this object. When we finally reach node 0 (which will be the root of the SCC containing nodes 0 and 2), we will then know that all xrefs for this color are the union of the xrefs of all objects belonging to this color (which represents the objects that we are popping from the loop_stack until we encounter the root node).

Even though this change adds required bookeeping for xrefs, I didn't notice any change in performance on the bridge tests that we have in mono/tests.

* [sgen] Some logging improvements in tarjan bridge

* [sgen] Disable optimization when comparing bridge outputs

When this optimization is enabled, the tarjan bridge will create more SCCs in order to reduce amount of xrefs in the graph. This would render the `bridge-compare-to` debug flag unusable with tarjan bridge.
2020-01-07 16:31:49 +02:00
Jay Krell 2129ac6c28 Set thread pool thread name just based on string pointer check. (#16637)
* Set thread pool thread name just based on string pointer check.

When the generation-based solution went in, there were not
yet constant thread names, so a pointer based approach could fail
due to reuse. We can do slightly simpler now.

* Remove the generation field which is no longer needed.
Remove the duplicated knowledge of how to set a constant and put
it behind a funny sounding but perhaps reasonable flag.
Revise corlib version.
2020-01-07 09:01:22 -05:00
EgorBo 322ba565b6 disable a test 2020-01-07 17:00:23 +03:00
Steve Pfisterandmonojenkins 0ccf503f0a Add SignedCms to the TypeForwards in the System.Security.Cryptography.Pkcs facade (#18325)
* Add SignedCms to the typeforwards in the System.Security.Cryptography.Pkcs facade

* Added additional missing types from System.Security.Cryptography.Pkcs.

* It would be nice if the files I added had namespaces ;-)

* Bump API snapshot submodule

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>

Fixes https://github.com/mono/mono/issues/18323
2020-01-07 14:03:15 +01:00
EgorBo e6c0cdae0c Address Zoltan's feedback (enable for AOT) 2020-01-07 15:18:51 +03:00
EgorBo 1c5a295082 Merge branch 'master' of github.com:mono/mono into vt-gettype-boxing 2020-01-07 14:50:16 +03:00
Marek SafarandBernhard Urban-Forster adcd62248d [System.Private.CoreLib] Cleanup intrinsic tracking (#17884)
* [System.Private.CoreLib] Cleanup intrinsic tracking

1. Removed icalls where intrinsic version for the method exists

2. Add [Intrinsic] attribute to methods which have runtime intrinsic
   to make them easier to track and to tell linker not to analyze them

3. Use recursive syntax for any intrinsic which can be called via
   reflection to correctly apply the intrinsic when invoked.

```c#
public static bool IsSupported { get => IsSupported; }
```

When one calls X86.IsSupported, the call gets replaced by an
intrinsics. When one calls it with an Invoke, the body of IsSupported
gets replaced by intrinsics. It's just a hack to get Invoke support for free.

* [interp] implement System.Type::op_Equality

* Undo changes which are not yet supported by interpreter

* [interp] Implement System.Runtime.CompilerServices.RuntimeHelpers::OffsetToStringData

Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-01-07 11:17:06 +01:00
Zoltan Varga 487dacce79 [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods. (#18334)
* [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods.

Fixes https://github.com/mono/mono/issues/18276.

* Fix compilation with mcs.
2020-01-07 02:57:10 +01:00
Jay Krell 6a8ca2252b Do not depend on gboolean to have multiple bits. (#17825) 2020-01-06 16:11:21 -05:00
Zoltan Varga 384db9e890 [netcore] Optimize RuntimeType::IsSZArray. (#18347) 2020-01-06 16:36:59 +01:00
Zoltan Varga 7236dadd32 [netcore] Mark SRE types with required=false so they are only retained when they are actually used. 2020-01-06 15:44:50 +01:00
Jo Shields 4c9e28b80a Don't run make install in boehm submodule (#17907)
Don't run `make install` in boehm submodule
2020-01-06 09:20:46 -05:00
Zoltan Varga 090e25ec2d [wasm] Rename the interp-sample targets to interp-hello. (#18342) 2020-01-06 07:40:40 +01:00
Zoltan Varga 47424e34ae Bump emscripten to 1.39.5. 2020-01-05 21:45:10 -06:00
monojenkins 59a1eade7c [csproj] Update project files 2020-01-05 13:43:39 +01:00
monojenkins 36e89d52d1 Bump API snapshot submodule 2020-01-05 13:43:39 +01:00
Zoltan Varga 3acab49ce7 Bump linker+cecil. 2020-01-05 13:43:39 +01:00
Aleksey Kliger (λgeek) 07fee8a91d [metadata] Size 0 Blob heap is ok when resolving assembly refs (#18313)
* [metadata] Size 0 Blob heap is ok when resolving assembly refs

Sometimes ILasm can produce images with a Blob heap of size 0.  In cases where
we're loading assembly references from such an image, the hash (which is
optional) can be at index = 0 and the Blob heap size is also 0.

Also: add the hash value to the output of `monodis --assemblyref`

* [metadata] Add a separate assertion for the index == size == 0 case

And a comment explaining how it is likely to be triggered.  If the assembly is
reasonable the caller of mono_metadata_blob_heap should be updated to use
mono_metadata_blob_heap_null_ok instead

* [bcl] Allow Mono's ILASM to produce a size 0 Blob heap

ECMA 335 II.24.2.4 says that the user string and blob heaps should have an
entry at index 0 consisting of the single byte 0.  However .NET Framework (and
.NET Core) ilasm will entirely omit the Blob heap (that is, create a blob heap
entry of size 0) if it is not needed.  This PR changes Mono's ILASM to emit the
initial byte on demand only if one of the MetaDataStream.Add() methods is
called. Otherwise we will also emit a stream of size 0.

This is needed to compile some test cases.

* [tests] Add regression test for loading assemblies with size 0 Blob heap

Depends on ILASM that can emit a size 0 Blob heap
2020-01-03 10:42:44 -05:00
Zoltan Varga 22afdb80e1 [netcore] Add some linker exclusions. 2020-01-03 13:11:39 +01:00
monojenkins 95101b19fc [bcl] Update BCL Linked Size 2020-01-03 09:22:27 +01:00
Larry Ewing 14a3add47f [wasm][debugger] put the testharness base url in one place 2020-01-02 19:28:18 -06:00
Larry Ewing 11bf192d1a [wasm][deubgger] clean up task usage and remove delay 2020-01-02 19:28:18 -06:00
Marek Safar c9036e7555 Cleanup a few dependencies to reduce wasm bcl size 2020-01-02 14:24:44 -06:00
Ryan Lucia 33ca3d33e1 [loader] Add assembly name matching on netcore to fix DefaultContext test (#18272)
* [loader] Rename assembly name check strictness flag

* [loader] Add predicate for assembly name matching on netcore

* [loader] Enable LoadInDefaultContext test

* Make mono_assembly_names_equal external, change flags in domain search

* Move ifdef inside mono_loader_get_strict_assembly_name_check

* Fix pedump

* No reason to make this external

* Feedback
2020-01-02 15:21:45 -05:00
Egor Bogatov 458d80002d Add !align 8 to all managed allocators (calls) (#18283) 2020-01-02 20:45:40 +01:00
Jérôme Laban 52674e238a [Wasm] Use Path.Combine in Packager (#18332)
Improves Windows compatibility
2020-01-02 20:38:41 +01:00
Ryan Lucia ff6294d231 [reflection] Convert internal GetCustomAttributes calls to Attribute[] (#18176)
* [reflection] Avoid creating object[] in GetCustomAttributesBase

* [reflection] Migrate Attribute type checking to CustomAttribute.cs
2020-01-02 11:58:15 -05:00
Ryan Lucia 675f1eb321 [eglib] Fix whitespace in memory.c (#18320) 2020-01-02 11:56:50 -05:00
Thays Grazia e20956305e The behaviour expected of filter exceptions is: (#18309)
-> throw in a async task with a try catch in the caller -> don't stop
-> throw in a new thread with a try catch in the caller -> stop
-> throw in the main thread without any try catch -> stop

Including a new test to test the fix of #17601.

Fixing #16588, create a while 1 to stops the program execution when an exception is thrown and no try catch is found and MONO_DEBUG=suspend-on-unhandled.

Fixes #17601
Fixes #16588
2020-01-02 11:26:14 -03:00
Jonathan Nagy 5470386775 Set inited flag _after_ TermInfoDriver initialisation completes (#18318)
Fixes #18317
2020-01-01 19:02:48 -05:00
Steve Pfister 6f24832646 Pull in DirectoryInfo Invalidate After Create Fix (#18310)
* Bump to mono/corefx@bff1d77
2019-12-30 21:05:14 -05:00
Martin Baulig 400fb9c2cb Socket.BeginMConnect() should not attempt connections on unsupported address families. (#18302)
Fixes #16513.
2019-12-30 13:08:03 -05:00
Aleksey Kliger (λgeek) 61eb2be2e1 [eglib] Don't define _FORTIFY_SOURCE on Cygwin builds (#18308)
See https://github.com/msys2/MINGW-packages/issues/5803

Without -lssp (or -fstack-protector), -D_FORTIFY_SOURCE=2 leads to linker
errors.
2019-12-30 09:54:57 -05:00
imhameed 9719635e58 Use byte-aligned LLVM loads and stores for unaligned mini IR loads and stores. (#18296)
Should fix https://github.com/mono/mono/issues/18221.
2019-12-26 14:34:02 -05:00
EgorBo 28cae98ef0 Optimize box(T)+GetType 2019-12-25 02:32:46 +03:00
Aleksey Kliger (λgeek) ffdffa54f2 [domain] Don't add NULL runtime to runtimes list (#18260)
Also when iterating over the runtimes, actually look at the current element,
not the first element.

Also mention the unsupported runtime version in the warning message when it is
passed to mono_init_internal.

Fixes https://github.com/mono/mono/issues/17916
2019-12-23 10:44:32 -05:00
Fan Yang a1a4bcc49d Fix llvm jit assertion (#18280)
* Fix assertion when Alignment is greater than 32
2019-12-23 08:38:36 -05:00
Fan Yang 7ffcaaf59a Add cast to ((align) - 1) for Alignment macros (#18255)
* Add cast to ((align) - 1) for Alignment macros

* Add test for Alignment macro change
2019-12-23 08:34:37 -05:00
Alexander Köplinger 720767a9ff Fix invalid assetManifestPlatform in pipeline-netcore-runtime.yml
Spaces are note valid there.
2019-12-23 12:24:53 +01:00
Larry Ewing 1587c49f5f [wasm][debugger] Clean up startup and improve comments
Properly await more of the async logic and add retries when we get
an empty response from the "json/list" endpoint.
2019-12-20 21:41:05 -06:00
Larry Ewing 14b736e942 [wasm] resolve the full path of pdbs so that ../ doesn't break inclusion 2019-12-20 21:41:05 -06:00
Larry Ewing 04b9e5cb8f [wasm][debugger] rename the debugger target to run-debugger-tests 2019-12-20 21:41:05 -06:00
Larry Ewing 1669812689 [wasm][debugger] get the debug harness working again 2019-12-20 21:41:05 -06:00
Maxim Lipnin b55d8f9700 [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes (#18223)
* [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes

* [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate

* [wasm][xunit tests] Exclude XLinqTests.TreeManipulationTests.IEnumerableOfXNodeRemoveWithEvents test method
2019-12-20 19:18:51 -06:00
Ryan Lucia 0f8aac3c62 [loader] Check AppContext.BaseDirectory on netcore in preload hook 2019-12-20 19:17:01 -06:00
Ryan Lucia 57b6c8775b [loader] Remove unnecessary newlines in tracing 2019-12-20 19:16:11 -06:00
Ryan Lucia 5c87445310 [netcore] Re-enable some BinaryFormatter tests (#18277) 2019-12-20 19:52:53 -05:00
Alexander Köplinger cfba7c550f [netcore] Update dotnet sdk version in global.json (#18264)
Should unbreak the publishing build of netcore-mono.
2019-12-20 17:26:09 +01:00
Larry Ewing 47bca852c1 [wasm] fix the wrapped function signatures to match current documentation 2019-12-20 09:38:18 -06:00
Larry Ewing 5db64e27e9 [wasm] fix the mono_clear_bps condition 2019-12-20 09:38:18 -06:00
Aleksey Kliger (λgeek) d6f3de3c4c [tests] Add test files to tarball (#18261)
Some test sources weren't included.

Fixes https://github.com/mono/mono/issues/18202
2019-12-20 11:11:26 +01:00
Alexander Köplinger 7fd24a40bb Bump corefx to get test fix 2019-12-20 00:37:55 +01:00
Larry Ewing 83a898d25b [wasm] Make sure we're comparing the same date (#18251) 2019-12-19 15:35:05 +01:00
Alexander Köplinger c8e19bec68 [System] Add additional hardening against race in MonoTlsStream (#18250)
See https://github.com/mono/mono/issues/15805#issuecomment-567131297
2019-12-19 02:25:06 +01:00
Bernhard Urban-Forster e7b98d3587 [interp] add guard that a local hasn't been modified for stlocfld fusion (#18129)
[interp] add guard that a local hasn't been modified for stlocfld fusion

Fixes https://github.com/mono/mono/issues/18120
2019-12-19 02:05:57 +01:00
Nikita Voronchev 0beb9611df [WinForms][UIA] Add new internal event to track form's owner change (#18117)
This PR is needed for https://github.com/mono/uia2atk/pull/8
2019-12-18 22:24:06 +01:00
Alexander Köplinger a74416d950 [corlib] Move a few more test cases to part2, turns out part1 was still too large
Follow up to 3891fe97.
2019-12-18 22:18:33 +01:00
Alexander Köplinger 3891fe97d2 [bcl] Split corlib and System.Core xunit test assemblies for monotouch profiles (#18228)
They cannot deal with large test assemblies so we need to split them.

We're doing this by introducing a new `#skip <PROFILE>` directive in the .sources format which defines where to split the assembly.
The logic is very simple: files up until the first `#skip` are common files which get included in each part, subsequent files are only in the respective part assembly.

Fixes https://github.com/mono/mono/issues/18213
2019-12-18 21:11:17 +01:00
Larry Ewing 2ff93e4c76 [wasm] Fix test case spelling (#18242) 2019-12-18 13:51:21 -06:00
Maxim Lipnin 56f397bef3 [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate (#18234) 2019-12-18 06:49:45 -06:00
Larry Ewing 60f081c077 [wasm] handle completed Tasks and exceptions when resolving Task prom… (#18215)
* [wasm] handle completed Tasks and exceptions when resolving Task promises

* Preserve the old void invoke behaviour
2019-12-17 22:47:23 -06:00
Larry Ewing f46d5fda1d [wasm][debugger] fix exception when we can't locate debug assembly (#18230) 2019-12-17 21:37:22 -06:00
Nathan Ricci ed7432d988 Change ICONST to I8CONST when propagating constants through sign extension. (#18222)
Change ICONST to I8CONST when propagating constants through sign extension.
2019-12-17 21:35:56 -05:00
Evgenii Stepanov 06673e723e Relax assert in mono_arch_build_imt_trampoline. (#18195)
This assert fails when the trampoline code size worst case is realized.

Observed on a Android device with HWASan [1] enabled, where heap
addresses have a non-zero tag in the most significant byte, and thus
require 4 instructions to materialize in emit_imm64.

[1] https://source.android.com/devices/tech/debug/hwasan
2019-12-17 01:18:53 +01:00
Maxim Lipnin df3837eb7d [wasm][xunit tests] Re-enable System.Buffers.Text.Tests.ParserTests (#18220) 2019-12-17 01:18:34 +01:00
Zoltan Varga aa9227f7c8 [llvm] Generate correct LLVM IR for OP_XEQUAL. (#18217) 2019-12-17 00:52:02 +01:00
Zoltan Varga 970d695718 [wasm] Disable linker substitutions for now, it generates invalid IL when running make build-aot-hello-netcore. (#18218) 2019-12-17 00:10:04 +01:00
Zoltan Varga 6bf2c17f34 [wasm] Pass --deterministic to the linker in msbuild. (#18214) 2019-12-17 00:09:46 +01:00
Zoltan Varga 9f5e38229e [netcore] Add AssemblyLoadContext:OnAssemblyLoad() to linker descriptor. (#18219) 2019-12-17 00:09:24 +01:00
Jay Krell de359d4930 Remove handles/MonoError from some marshal icalls. (#18036)
Remove handles/MonoError from some marshal icall functions.

This does some small other things.
  In particular, if a function is not static for some platforms, make it not static for all.
  static historically was valuable, because of default linker semantics around visibility.
  I believe it is less important now, since visibility is default constrained.
  Also being consistent across platforms seems good.

Maybe split off a separate PR for that.
2019-12-16 23:06:11 +01:00
Ryan Lucia e19914fe69 [metadata] Check for GTDs in setup_generic_array_ifaces (#18101)
Judging by https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/metadata.c#L3515 the cached_class is sometimes intentionally a GTD, which can get passed all the way down to setup_generic_array_ifaces and cause a crash.
2019-12-16 14:17:44 -05:00
Zoltan Varga ac1a12f997 [wasm] Regen wasm_m2n_invoke.g.h. (#18212) 2019-12-16 18:54:54 +01:00
Marek Safar 47199fede8 Revert "[wasm][ci] Deactivate linker on browser tests (#18152)"
This reverts commit 079acc2fb8.
2019-12-16 17:12:55 +01:00
Marek Safar f8266639ea Bump linker 2019-12-16 17:12:55 +01:00
Maxim Lipnin ba69304049 [wasm][xunit tests] Re-enable System.Tests.MathFTests (#18208) 2019-12-16 09:53:02 -06:00
Jérôme Laban 5562065e73 [Wasm] Enable dynamic runtime tests (#18098)
* [Wasm] Enable dynamic runtime tests

- Add dynamic runtime browser tests in CI
- Change `release-dynamic` to `dynamic-release` to align with other runtimes
- Add `-dynamic-runtime` flag for packager.exe
- Add debug configuration for dynamic runtime
- Fix invalid MAIN_MODULE=2 flag, which forces the emscripten linker to pass even on exported functions.
2019-12-16 07:28:26 -06:00
Calvin Buckley 84221f04b5 Fix locating Python interpreter when it isn't "python" (#18197)
Fix locating Python interpreter when it isn't "python"

The "python" interpreter is specified by PEP-394. In it, an OS
can decide not to ship a "python" executable, and instead only
ship python2/3, providing some ambiguity for scripts that are
version-neutral, like the one Mono uses.

This uses an automake macro to locate a suitable Python, and
changes the makefile to use the located Python, whatever it may be.
2019-12-16 11:06:00 +01:00
Zoltan Varga 61a103f371 [wasm] Set the main thread, some EH code depends on it. (#18203) 2019-12-16 08:41:43 +01:00
Marek Safar afcafb6c5e [monolinker] Fix path to wasm references 2019-12-15 17:05:37 +01:00
Maxim Lipnin c454cb92eb [wasm][xunit] Re-enable System.MemoryTests and System.SpanTests tests (#18179) 2019-12-14 14:59:48 +01:00
Jay Krell e76fb5dbf9 Add full compiler barriers to any fence that lacked a compiler barrier. (#18177)
A compiler barrier generates no code.
It "only" inhibits certain compiler optimizations, which I do not consider
particularly worrisome, as most code is "fully" optimized, and
compiler barriers are relatively rare.

It could be a concern in code paths that greatly benefit
from compiler optimizations and for which barriers greatly
inhibits the compiler.

I would further suggest, though it is not here, that any compiler
half barrier (read, write) be a compiler full barrier instead,
as they sound pretty cheap, so simplify.

Similar cannot be said about CPU barriers.
They do not always generate code, but when they do, it can cost.

This is follow-up from https://github.com/mono/mono/pull/17849#issuecomment-556273801
https://github.com/mono/mono/pull/17849#issuecomment-557661604.
2019-12-13 22:49:15 +01:00
Marek Safar 6dc5295ed0 [wasm] Improve virtualenv installation script 2019-12-13 22:37:08 +01:00
monojenkins b0e7a1e4f9 [bcl] Update BCL Linked Size 2019-12-13 22:24:52 +01:00
Marek Safar 15a8c2a7fa [wasm] Run linker tests with wasm substitution settings 2019-12-13 22:15:59 +01:00
Aleksey Kliger (λgeek) d0f27ce4fe [cominterop] Attach to runtime from IUnknown and IDispatch methods (#18175)
* [cominterop][test] Calling QueryInterface from an unattached thread asserts

If we pass a managed COM object to native code and the native code calls one of
the IUnknown methods, we assert because the thread is not attached but we want
to enter GC Unsafe mode.

* [cominterop] Attach to runtime from IUnkown and IDispatch methods

If the COM objects for a managed object is passed to a thread that hasn't
interacted with Mono before, we need to attach before doing GC thread state
transitions.

Addresses https://github.com/mono/mono/issues/18137

* [test] Call managed method through COM wrapper from unattached thread

This works, but I don't think we had a test previously.  Call a managed method
via a COM interface from a thread that is not attached to the runtime, to
ensure that it is attached.
2019-12-13 15:32:06 -05:00
Ankit Jain ea50ca48ba Bump msbuild to track xplat-master (#18183)
- bump roslyn to 3.5.0-beta1-19606-04, to match msbuild
- bump nuget.exe to 5.5.0.6319 (5.5.0-preview1), to match msbuild
2019-12-13 20:06:16 +01:00
Jay Krell 1310043953 Fix on-demand initialization race conditions [threads.c]. (#18166) 2019-12-13 19:57:33 +01:00
Jay Krell 2eca6a1e37 Fix on-demand initialization race conditions [marshal.c]. (#18169)
* Fix on-demand initialization race conditions [marshal.c].

* PR feedback: fix incorrect merge
2019-12-13 19:57:17 +01:00
Ryan Lucia 7bc25cef6d [netcore] Implement AppDomain.AssemblyLoad (#18143) 2019-12-13 11:36:08 -05:00
Jay Krell d2539babf4 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:30:08 +01:00
Jay Krell b637c1db67 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:29:30 +01:00
Alexander Köplinger 7e65a73d4d [sdks] Use Xcode 11.3 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-12-13 17:27:24 +01:00
Kenneth Pouncey 96fa4e8c6f [wasm][debug-proxy] Fix crash and send back failed message. (#18146)
* [wasm][debug-proxy] Fix crash and send back failed message.

* Remove debugging error message

* Remove debug Console.WriteLine
2019-12-13 10:22:55 -06:00
Jay Krell 57c6b81b96 Fix on-demand initialize race conditions [cominterop.c] (#18158) 2019-12-13 12:28:34 +01:00
Jay Krell 64ad646066 Fix on-demand initialize race conditions [custom-attrs.c] (#18159) 2019-12-13 12:27:39 +01:00
Steve Pfister 97bf0a1bcc Relax faultcode parsing for Soap11 message errors. (#18154)
* Relax faultcode parsing for Soap11 message errors.

The mono implementation was reading the faultcode content as an XmlQualifiedName, which applied the rule where you could not
start a name with a number. This caused interoperability problems with the .net framework because their parsing just looks
for <namespace>:<name>, where name can be any string.

Example:

<faultcode>s:1</faultcode>

On mono, a soap response that contains a fault code name of 1 will throw an error.  If you run the same code on the .net framework, it will not.

Fixes https://github.com/mono/mono/issues/12995

* [csproj] Update project files
2019-12-13 12:26:53 +01:00
Jay Krell 0dc19600d1 Fix on-demand initialization race conditions [object.c]. (#18163)
Fix on-demand initialization race conditions [object.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 09:25:56 +01:00
Zoltan Varga e391861c01 [wasm] Add linker substitutions. (#18149)
* [wasm] Add linker substitutions in netcore mode.

* Enable non-netcore mode as well.

* Disable conditionals which are not true in threaded mode for now.
2019-12-13 06:46:33 +01:00
Jay Krell 9cefdf59ad Fix on-demand initialization race conditions [sgen]. (#18170)
Fix on-demand initialization race conditions [sgen].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 04:55:58 +01:00
Jay Krell a93e4712ec Fix on-demand initialization race conditions [remoting.c]. (#18168)
Fix on-demand initialization race conditions [remoting.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:46:07 +01:00
Jay Krell 943a2760ec Fix on-demand initialization race conditions [native-library.c]. (#18162)
Fix on-demand initialization race conditions [native-library.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:26:07 +01:00
Jay Krell 1fe3d7eda6 Fix on-demand initialization race conditions [reflection.c]. (#18167)
Fix on-demand initialization race conditions [reflection.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:45:36 +01:00
Jay Krell 51bc81919f Fix on-demand initialization race conditions [mini] (#18161)
Fix on-demand initialization race conditions [mini].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:36:19 +01:00
Jay Krell 620d7cf84d Fix on-demand initialization race conditions [sre.c]. (#18164)
Fix on-demand initialization race conditions [sre.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:55:33 +01:00
Jay Krell f96ba075a1 Fix on-demand initialization race conditions [threadpool.c]. (#18165)
Fix on-demand initialization race conditions [threadpool.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:46:06 +01:00
Ryan Lucia 88dde8b763 [netcore] Implement AppDomain.TypeResolve (#18141) 2019-12-12 15:00:28 -05:00
Jay Krell a1e16d3273 Remove handles/MonoError from 12 easy socket icalls. (#18035)
Also put _icall back on ends so the search coincides between managed and native.
2019-12-12 13:32:14 -05:00
Maxim Lipnin 6d9c1359ec [wasm] Disable particular failing System.Net.Security.Tests tests rather than the whole namespace (#18151) 2019-12-12 13:07:44 -05:00
Ryan Lucia 7d88d4d502 [netcore] Enable AppDomainTests.AssemblyResolveInvalidAssemblyName (#18157)
I don't remember why this wasn't enabled originally, but it seems to be passing now. I probably accidentally fixed it back when I made changes to our assembly name parsing.
2019-12-12 12:27:51 -05:00
Zoltan Varga fb67916c0f [wasm] Change mono_wasm_runtime_invoke () to return the exception itself in the out argument instead of a boolean. (#18135) 2019-12-12 17:23:42 +01:00
Kenneth Pouncey 079acc2fb8 [wasm][ci] Deactivate linker on browser tests (#18152)
There are problems on the CI when using the linker building the Browser Tests.

https://github.com/mono/mono/commit/3142396c3c6a62f389420113129783bb156de399

PR https://github.com/mono/mono/pull/18128
2019-12-12 12:53:05 +01:00
Ryan Lucia 3ddcbda8b6 [netcore] Enable AppDomainTests.MonitoringIsEnabled (#18142) 2019-12-11 21:19:45 -05:00
Ryan Lucia acc8c0c696 [netcore] Remove MonoDomain.DoAssemblyResolve (#18139)
This is no longer called in netcore builds as of a while ago, so get rid of it and the corresponding linker exclusion.
2019-12-12 00:16:50 +01:00
Jay Krell 256eda8c7f Make Socket.Socket_icall static, as most icalls are, removing the first, unused parameter. (#18130)
Also partial undo recent and append _icall to end.
This is verbose, but maybe worthwhile to aid search.

This is extracted from https://github.com/mono/mono/pull/17978.
2019-12-11 14:40:27 -05:00
Ryan Lucia 0e05bda235 [utils] Correctly convert MONO_DL_LOCAL to RTLD_LOCAL (#18121)
On Linux, neither flag being specified will result in a default of RTLD_LOCAL. Unfortunately, on MacOS it's the opposite - the default is RTLD_GLOBAL. This means currently there is no way to set RTLD_LOCAL on Macs, so despite the potential for this to break someone it directly contradicts our documentation and needs to be changed.
2019-12-11 10:42:10 -05:00
Marek Safar 3142396c3c Bump linker (#18128)
* Bump linker

* [csproj] Update project files
2019-12-11 16:38:32 +01:00
Alexander Köplinger 8ae9708e87 [corlib] Rename internal LogLevel enum to avoid conflict with Android LogLevel (#18124)
After https://github.com/mono/mono/pull/18040 we now have InternalsVisibleTo for the Mono.Android assembly.
This exposed the internal LogLevel enum in BCLDebug which causes an issue when consumed in XA:

```
Java.Lang/Object.cs(119,18): error CS0104: 'LogLevel' is an ambiguous reference between 'Android.Runtime.LogLevel' and 'System.LogLevel'
```

To fix this we rename the internal enum in corlib since it's not really used.
2019-12-11 13:39:38 +01:00
Jay Krell 97dd6cf04c Sockets: Foo_internal to Foo_icall and ves_icall_Foo_internal to ves_icall_Foo. (#18017)
The first is to avoid requiring the signature matching in icall resolution.
The second is just to shorten names.
2019-12-11 10:40:00 +01:00
Maxim Lipnin 93c18d6fd1 [wasm] Re-enabling a test that was fixed by the corefx bump (#18108)
* [wasm] Clean up rsp-file

* Bump corefx
2019-12-11 10:39:35 +01:00
Kenneth Pouncey d6d74e819f [wasm][debugger] Prevent crash on disconnect of client (#18107) 2019-12-11 06:37:19 +01:00
Bernhard Urban-Forster a65b7af6d5 [interp] relop result is always int (#18114)
This can lead to problems on big endian.

```
result.i = (int) 1;
--> 0x0000_0001_XXXX_XXXX;

result.l = (long) 1;
--> 0x0000_0000_0000_0001;
```

On little endian it's always `0x0000_0000_0000_0001`, so it doesn't matter.
2019-12-10 22:06:48 +01:00
Ryan Lucia a778c46362 [utils] Make mono_dl_open default to RTLD_LOCAL on netcore (#17912)
* [utils] Make mono_dl_open default to RTLD_LOCAL

The exception is for fallback handlers added via the embedding API, in which case we convert the flags over to preserve legacy behavior

* Convert flags properly for embedding API use

* Changes

* Turn off -fvisibility=hidden temporarily

TESTING

* Undo some previous crimes, maybe fix failure

* Leave legacy mono unchanged, defaulting to RTLD_GLOBAL
2019-12-10 14:58:58 -05:00
Bernhard Urban-Forster ef422d9cb7 [interp] fix build when DEBUG_INTERP is enabled (#18112)
broke with c4136ebd94
2019-12-10 18:55:09 +01:00
Maxim Lipnin cd2e0e8f7a [wasm] Clean up rsp-file (#18113) 2019-12-10 18:08:52 +01:00
Bernhard Urban-Forster 80b1e103cb [debugger] skip suspend for unattached threads (#18105)
When the debugger tries to suspend all the VM threads, it can happen with cooperative-suspend that it tries to suspend a thread that has previously been attached, but then did a "light" detach (that only unsets the domain). With the domain set to `NULL`, looking up a JitInfo for a given `ip` will result into a crash, but it isn't even necessarily needed for the purpose of suspending a thread.

More details: Consider the following:
```
(lldb) c
error: Process is running.  Use 'process interrupt' to pause execution.
Process 12832 stopped
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
    frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
   2713         MonoDomain *domain = (MonoDomain *) mono_thread_info_get_suspend_state (info)->unwind_data [MONO_UNWIND_DATA_DOMAIN];
   2714         if (!domain) {
   2715                 /* not attached */
-> 2716                 ji = NULL;
   2717         } else {
   2718                 ji = mono_jit_info_table_find_internal ( domain, MONO_CONTEXT_GET_IP (&mono_thread_info_get_suspend_state (info)->ctx), TRUE, TRUE);
   2719         }
Target 0: (WatchWCSessionAppWatchOSExtension) stopped.
(lldb) bt
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  * frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
    frame #1: 0x004e177b WatchWCSessionAppWatchOSExtension`mono_thread_info_safe_suspend_and_run(id=0xb0767000, interrupt_kernel=0, callback=(WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical at debugger-agent.c:2708), user_data=0xb04dc718) at mono-threads.c:1358:19
    frame #2: 0x00222799 WatchWCSessionAppWatchOSExtension`notify_thread(key=0x03994508, value=0x81378c00, user_data=0x00000000) at debugger-agent.c:2747:2
    frame #3: 0x00355bd8 WatchWCSessionAppWatchOSExtension`mono_g_hash_table_foreach(hash=0x8007b4e0, func=(WatchWCSessionAppWatchOSExtension`notify_thread at debugger-agent.c:2733), user_data=0x00000000) at mono-hash.c:310:4
    frame #4: 0x0021e955 WatchWCSessionAppWatchOSExtension`suspend_vm at debugger-agent.c:2844:3
    frame #5: 0x00225ff0 WatchWCSessionAppWatchOSExtension`process_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0, il_offset=0, ctx=0x00000000, events=0x805b28e0, suspend_policy=2) at debugger-agent.c:4012:3
    frame #6: 0x00227c7c WatchWCSessionAppWatchOSExtension`process_profiler_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0) at debugger-agent.c:4072:2
    frame #7: 0x0021b174 WatchWCSessionAppWatchOSExtension`thread_startup(prof=0x00000000, tid=2957889536) at debugger-agent.c:4149:2
    frame #8: 0x0037912d WatchWCSessionAppWatchOSExtension`mono_profiler_raise_thread_started(tid=2957889536) at profiler-events.h:103:1
    frame #9: 0x003d53da WatchWCSessionAppWatchOSExtension`fire_attach_profiler_events(tid=0xb04dd000) at threads.c:1120:2
    frame #10: 0x003d4d83 WatchWCSessionAppWatchOSExtension`mono_thread_attach(domain=0x801750a0) at threads.c:1547:2
    frame #11: 0x003df1a1 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop_internal(domain=0x801750a0, cookie=0xb04dcc0c, stackdata=0xb04dcba8) at threads.c:6008:3
    frame #12: 0x003df287 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop(domain=0x00000000, dummy=0xb04dcc0c) at threads.c:6045:9
    frame #13: 0x005034b8 WatchWCSessionAppWatchOSExtension`::xamarin_switch_gchandle(self=0x80762c20, to_weak=false) at runtime.m:1805:2
    frame #14: 0x005065c1 WatchWCSessionAppWatchOSExtension`::xamarin_retain_trampoline(self=0x80762c20, sel="retain") at trampolines.m:693:2
    frame #15: 0x657ea520 libobjc.A.dylib`objc_retain + 64
    frame #16: 0x4b4d9caa WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke + 279
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb26e Foundation`____addOperations_block_invoke_4 + 20
    frame #24: 0x65de007b libdispatch.dylib`_dispatch_call_block_and_release + 15
    frame #25: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #26: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #27: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #28: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #29: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #30: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #31: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```

Going further, `info` is about this thread:
```
(lldb) p/x *(int *)(((char *) info->node.key) + 0xa0)
(int) $2 = 0x01243f93
(lldb) thread list
Process 12832 stopped
  thread #1: tid = 0x1243ee1, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_303', queue = 'com.apple.main-thread'
  thread #2: tid = 0x1243ee6, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10
  thread #3: tid = 0x1243ee7, 0x65f81aea libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'SGen worker'
  thread #4: tid = 0x1243ee9, 0x65f7e3d2 libsystem_kernel.dylib`semaphore_wait_trap + 10, name = 'Finalizer'
  thread #5: tid = 0x1243eea, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10, name = 'Debugger agent'
  thread #6: tid = 0x1243f1d, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.uikit.eventfetch-thread'
  thread #8: tid = 0x1243f93, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
* thread #9: tid = 0x12443a9, 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  thread #10: tid = 0x1244581, 0x65f7fd32 libsystem_kernel.dylib`__workq_kernreturn + 10
(lldb) thread select 8
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
    frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
libsystem_kernel.dylib`mach_msg_trap:
->  0x65f7e396 <+10>: retl
    0x65f7e397 <+11>: nop

libsystem_kernel.dylib`mach_msg_overwrite_trap:
    0x65f7e398 <+0>:  movl   $0xffffffe0, %eax         ; imm = 0xFFFFFFE0
    0x65f7e39d <+5>:  calll  0x65f85f44                ; _sysenter_trap
(lldb) bt
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
  * frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
    frame #1: 0x65f7e8ff libsystem_kernel.dylib`mach_msg + 47
    frame #2: 0x66079679 libxpc.dylib`_xpc_send_serializer + 104
    frame #3: 0x660794da libxpc.dylib`_xpc_pipe_simpleroutine + 80
    frame #4: 0x66079852 libxpc.dylib`xpc_pipe_simpleroutine + 43
    frame #5: 0x66043a8f libsystem_trace.dylib`___os_activity_stream_reflect_block_invoke_2 + 30
    frame #6: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #7: 0x65de3d71 libdispatch.dylib`_dispatch_block_invoke_direct + 257
    frame #8: 0x65de3c62 libdispatch.dylib`dispatch_block_perform + 112
    frame #9: 0x6604349a libsystem_trace.dylib`_os_activity_stream_reflect + 725
    frame #10: 0x6604ef19 libsystem_trace.dylib`_os_log_impl_stream + 468
    frame #11: 0x6604e44d libsystem_trace.dylib`_os_log_impl_flatten_and_send + 6410
    frame #12: 0x6604cb3b libsystem_trace.dylib`_os_log + 137
    frame #13: 0x6604f4aa libsystem_trace.dylib`_os_log_impl + 31
    frame #14: 0x4b4eb4e9 WatchConnectivity`WCSerializePayloadDictionary + 393
    frame #15: 0x4b4d7c4d WatchConnectivity`-[WCSession onqueue_sendResponseDictionary:identifier:] + 195
    frame #16: 0x4b4da435 WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke.411 + 35
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb067 Foundation`____addOperations_block_invoke_2 + 20
    frame #24: 0x65dedf49 libdispatch.dylib`_dispatch_block_async_invoke2 + 77
    frame #25: 0x65de4461 libdispatch.dylib`_dispatch_block_async_invoke_and_release + 17
    frame #26: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #27: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #28: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #29: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #30: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #31: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #32: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```
which is a thread in a "light" detach state (aka. coop detach), where we only unset the domain:
https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/threads.c#L6084-L6111

Fixes https://github.com/mono/mono/issues/17926
2019-12-10 12:16:05 +01:00
Aleksey Kliger (λgeek) 7704bd1b92 [mini] Fix delegate trampoline virtual call via delgate Invoke (#18073)
* [mini] Fix delegate trampoline virtual call via delgate Invoke

If we need to jit the Invoke method of a delegate, we get tramp_info with a
NULL method.

Background: normally when we create a delegate around a virtual method,
handle_delegate_ctor will just create a virtual invoke trampoline with
mono_arch_get_delegate_virtual_invoke_impl which doesn't get here.  But if
we're asked to compile the delegate's Invoke method, then compile_special ()
will create a tramp_info with a null method, and return a delegate trampoline.

That's the case here - we had
  var del = SomeDelegate(obj.VirtualMethod);
  var invoke_method = del.GetType().GetMethod ("Invoke");
  invoke_method.Invoke (del, args);
or
  var del = SomeDelegate(obj.VirtualMethod);
  var another_del = OtherDelegate (del.Invoke);
  another_del (args);

in both cases, we end up in mono_delegate_trampoline with tramp_info->method ==
NULL.

in the second case the IL is like this:
   newobj instance void Derived::'.ctor'
   ldvirtftn instance void class Base::VirtualMethod()
   newobj instance void class SomeDelegate::'.ctor'(object, native int)

So delegate->target is a derived instance but delegate->method is some base
class method.

Addresses https://github.com/mono/mono/issues/17718

* [tests] Add tests for compiling delegate's Invoke method

Where the delegate calls a virtual method
2019-12-10 12:12:50 +01:00
Bernhard Urban-Forster fd44435a84 [debugger] remove unused thread suspend count (#18102) 2019-12-10 08:25:31 +01:00
Nikolay Sivov d1fc2ffa84 [support] Remove some dead code (#16674)
Both 'ptv' and 'tv' are unused since select() was replaced with poll() in 57b521e.
2019-12-10 00:21:00 +01:00
Steve Pfister 4cefdcb7ce Bump to mono/corefx@f3815a9 (#18090)
Picks up https://github.com/mono/corefx/pull/376
2019-12-09 21:29:40 +01:00
Jay Krell 5cbb1f4be8 [cxx] Fix mono-boehm to compile as C++. (#18086)
https://jenkins.mono-project.com/job/test-mono-pull-request-i386/32263/consoleText

```
      ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
boehm-gc.c: In function 'void mono_gc_base_init()':
boehm-gc.c:202:42: error: invalid conversion from
'void (*)(size_t) {aka void (*)(unsigned int)}' to
GC_on_heap_resize_proc {aka void (*)(long unsigned int)}' [-fpermissive]
  GC_set_on_heap_resize (on_gc_heap_resize);
                                          ^
In file included from ../../mono/utils/gc_wrapper.h:44:0,
                 from ../../mono/metadata/gc-internals.h:16,
                 from boehm-gc.c:16:
../../external/bdwgc/include/gc.h:132:21: note:
 initializing argument 1 of 'void GC_set_on_heap_resize(GC_on_heap_resize_proc)'
 GC_API void GC_CALL GC_set_on_heap_resize(GC_on_heap_resize_proc);
```
2019-12-09 18:59:10 +01:00
abrevet-dev 3d83c7429e [WinForms] No event should be triggered when calling ComboBox.Items.RemoteAt() (#18069)
* [WinForms] Fix wrong behavior when calling ComboBox.Items.RemoteAt() with index equals to SelectedIndex

* [WinForms] Fix #10643 No event should be triggered when calling ComboBox.Items.RemoteAt()

Fixes #10643
2019-12-09 15:51:28 +01:00
dotnet-maestro[bot] 612f2a8e88 Update dependencies from https://github.com/dotnet/arcade build 20191208.1 (#18089)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19608.1
2019-12-09 15:26:16 +01:00
Maxim Lipnin 17bba9e7dd [wasm] Clean up rsp-file (#18087) 2019-12-09 07:57:00 -06:00
Egor Bogatov 2befdcfa52 [netcore] Enable more tests for interpreter and llvm on CI (#17957)
* Enable more tests

* fix build?

* forward XUNIT_MONO_ENV_OPTIONS

* increase timeout

* ignore some tests for interpreter

* ignore more tests

* move interp specific issues to a separate rsp

* fix CI

* fix CI

* undo build.sh chmod

* revert chmod for xunit-summary-py

* improve xunit summary

* Ignore the whole System.Net.Tests namespace

* Ignore System.Text.Json

* disable the whole BinaryFormatterTests

* Disable System.Net.Http.Functional.Tests
2019-12-09 14:55:44 +01:00
Kenneth Pouncey 1462208991 [wasm][bcl][websockets] Add WebSocket support. (#18062)
Add the ability to initialize a `WebAssembly.Net.WebSockets::ClientWebSocket` from `System.Net.WebSockets.ClientWebSocket`.

`System.Net.WebSockets.ClientWebSocket` will delegate all calls to the WebAssembly ClientWebSocket.  

- Add support for marshaling `Uri` class correctly.  This was a cause of failures.
- Add bindings tests for `Uri` marhaling.
2019-12-09 13:32:17 +01:00
Jay Krell ed6b8024b0 fix warning: threadpool-worker-default.c: In function 'worker_try_create': 'now' may be used uninitialized (#18056)
https://jenkins.mono-project.com/job/build-source-tarball-mono-pullrequest/854/console
2019-12-09 08:17:54 +01:00
Jay Krell a6c4eafee8 Add some missing barriers when dealing with global caches (#18032)
* Add some missing barriers when dealing with global caches
accessed from multiple threads without locks.

This is based on earlier PR https://github.com/mono/mono/pull/17849.

Read-side barriers are removed.
  More analysis is needed to determine if they are needed (which pointer
  read vs. written).
Write-side barriers are compiler barrier + store-release instead of full barrier.

Note all candidates are converted to new form yet.

Cases that do motivate read-wide barriers are skipped.
e.g.
`mono_arch_start_dyn_call`
`get_agent_domain_info`
`init_jit_info_dbg_attrs`

* PR: Remove 'Try one more time..'
2019-12-08 15:30:48 +01:00
Sam Patel 74567a0a34 Update glib-remap.h to include g_module_address (#18076) 2019-12-07 07:11:59 +01:00
abrevet-dev 4be2a9a207 [WinForms] Handle menu for wheel button click (close the contextmenu) (#18049)
Fixes #18019
2019-12-06 18:41:47 +01:00
Ankit Jain d1db937e78 Bump msbuild to track xplat-master (#18050)
.. to get changes from https://github.com/mono/msbuild/pull/162 .

And bump roslyn to 3.4.0-beta4-19569-03 to match msbuild.
2019-12-06 16:55:44 +01:00
Aleksey Kliger (λgeek) 3dabeddfc6 [threads] Add back mono_threads_attach_tools_thread as a public API (#18048)
* [utils] Add back mono_threads_attach_tools_thread

In https://github.com/mono/mono/commit/a5da7b21f4b6dbc5eaa09c2addee91b84dc1dbd5
we got rid of "tools" threads internally to the runtime.

However since the API was previously marked with MONO_API it was an internal
API that some embedders depended on.

This PR adds back an (external-only) limited form of tools thread.

The runtime is aware of the Tools thread in that FOREACH_THREAD_* macros will
iterate over them, and the thread has a coop thread state machine. (That is,
mono_thread_info_current() and GC Safe and GC Unsafe transitions all work.)

However the thread is:
1. Not stopped by the GC
2. Is not interrupted by profiler sampling.
3. Does not have a "current domain"
4. (As a consequence of the above) cannot call managed methods or touch managed
objects.

Such threads are useful for low-level interaction with the runtime such as
querying metadata, the JIT state and other coordination.

mono_threads_attach_tools_thread should be called no more than once.  It should
not be called on a thread that is already attached with mono_thread_atach, and
vice versa.

Addresses https://github.com/mono/mono/issues/18011

* [threads] Make mono_threads_attach_tools_thread into a public API
2019-12-06 10:50:34 -05:00
Bernhard Urban-Forster 121928d29a [ppc32] delete leftover code from pre-monoctx area (#18072)
Also update assert which doesn't make sense since we have moved over to MonoContext

Some context: e7011c780f

Fixes: https://github.com/mono/mono/issues/18064
2019-12-06 11:35:11 +01:00
Jay Krell 12bc4717fd [cxx] Remove use of register keyword. (#18057)
[cxx] Remove use of register keyword.

https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html

```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
4 warnings generated.
```
2019-12-06 03:55:56 +01:00
Jay Krell a131e9bbb0 [cxx][netcore] Fix goto around init. (#18058)
https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html
```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
```
2019-12-05 15:41:09 -06:00
Egor Bogatov 63f155ac46 Let user overwrite opt -Ox level (#18012) 2019-12-05 13:44:05 -06:00
Jay Krell b99ea70e43 Remove handles/MonoError from two socket icalls. (#17977)
Remove handles/MonoError from two socket icalls.

Ongoing removal of handles() icalls.
This is small, to pause and remove `mono_class_get_field_from_name_full` from w32socket.c.
2019-12-05 19:36:11 +01:00
Egor Bogatov 84839ab297 Merge pull request #18053 from EgorBo/fix-fneg-llvm
LLVM: use fneg instead of fsub for `-x`
2019-12-05 21:13:11 +03:00
Jay Krell ae9192f96f [netcore] Remove approx. 10 icalls. (#18018)
* [netcore] Remove ves_icall_System_Runtime_InteropServices_Marshal_PtrToStringUni and _len icalls.
They are implemented in C#.
The C# is equivalent to the C, and could be used for classic runtime, except for the treatment of large/negative lengths.

* Remove a few more icalls.
Instead of ifdef and leaving them uncompiled when compiling netcore,
declare them and put asserts in them.
2019-12-05 13:10:29 -05:00
Alexander Köplinger 269d509e42 Remove DNS lookups of the local hostname in tests (#18059)
This allows us to workaround issues we currently have in the macOS bot network where DNS lookup of the hostname fails.
2019-12-05 18:08:26 +01:00
Zoltan Varga e67673fac8 [llvm] Avoid decomposing OP_STOREV_MEMBASE early for types which contains no references. (#18041) 2019-12-05 16:02:43 +01:00
Martin Baulig cc720b83aa [System.Net.Http]: Add workaround for mono/mono#17897. (#17992)
Use `(UriKind)300` (which is the same as setting `MONO_URI_DOTNETRELATIVEORABSOLUTE=true`)
when parsing Uri headers.
2019-12-05 14:36:07 +01:00
Jay Krell 5a5e092a53 Fix ifndef include guard in icall-decl.h. (#18055)
There was no clash so ok either way, but this is more like intended.
2019-12-05 10:19:24 +01:00
Egor Bogatov a3fbb64493 Convert to 0/1 in OP_XEQUAL (#18038) 2019-12-05 02:38:34 +01:00
abrevet-dev 186f9cd8c4 [WinForms] Fix wrong behavior of RadioButton auto-check (#18044)
Fixes #18020
2019-12-05 00:41:49 +01:00
Alexander Köplinger 4590bf6cac Bump bockbuild to get https://github.com/mono/bockbuild/pull/136 (#18043) 2019-12-05 00:33:08 +01:00
Alexander Köplinger f965e5d3e8 Fix formatting of README.netcore.md 2019-12-05 00:11:14 +01:00
EgorBo 55a3c10e72 remove some items from rsp 2019-12-05 02:09:11 +03:00
EgorBo 92047c453f use fneg instead of fsub 2019-12-05 02:03:24 +03:00
Bernhard Urban-Forster 9d58bc1d12 [amd64] align application stack pointer in signal handler (#18021)
* [amd64] align application stack pointer in signal handler

16byte stack pointer alignment is only required in certain places by the ABI, so the assumption that it holds all the time is wrong (though, it passed CI 😅). We have to setup the frame properly, i.e. we have to ensure the stack pointer is properly aligned. That's what we have done previously too.

Regression of https://github.com/mono/mono/pull/17922
Fixes https://github.com/mono/mono/issues/18006

* add ALIGN_DOWN_TO macro

* remove unnecessary ifndefs
2019-12-04 15:23:19 -06:00
Alexander Köplinger 59554712cc Update submodules before running external/bdwgc/autogen.sh (#18042)
Otherwise we'll run into issues because in configure.ac we update submodules _after_ we try to call external/bdwgc/configure (which won't be there if autogen.sh didn't run).
This would manifest itself as `*** No rule to make target 'all'.` in external/bdwgc when you build as there's no Makefile.
2019-12-04 22:16:21 +01:00
Jonathan Peppers 8511ae20f0 [corlib] add InternalsVisibleTo for Mono.Android (#18040)
* [corlib] add InternalsVisibleTo for Mono.Android

Context: https://github.com/xamarin/xamarin-android/pull/3991

In Xamarin.Android, we have a bit of System.Reflection usage on
startup:

https://github.com/xamarin/xamarin-android/blob/bbbe2ed36e6eeb7cce0905dcbfdaeaa30754070f/src/Mono.Android/Android.Runtime/UncaughtExceptionHandler.cs#L47-L67

These methods are in `mscorlib.dll` and `internal`. If we had
`[assembly: InternalsVisibleTo ("Mono.Android")]` we could avoid the
System.Reflection.

This is already done for `Xamarin.iOS`, `Xamarin.Mac`, etc. so
hopefully it's OK to add one more?

We could then, just call these methods directly:

* `System.Diagnostics.Debugger.Mono_UnhandledException`
* `AppDomain.CurrentDomain.DoUnhandledException`

The results are possibly a 100-250ms improvement to startup:

    Before:
    12-03 14:27:51.450  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s727ms
    12-03 14:27:57.310  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s723ms
    12-03 14:28:03.216  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s703ms
    12-03 14:28:09.054  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s679ms
    12-03 14:28:14.953  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s711ms
    After:
    12-04 09:13:18.248  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s608ms
    12-04 09:13:24.087  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s570ms
    12-04 09:13:29.997  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s566ms
    12-04 09:13:35.805  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s498ms
    12-04 09:13:41.670  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s475ms

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Bump API snapshot submodule
2019-12-04 21:01:20 +01:00
Kenneth Pouncey 83cac0306c [wasm][build] Modify to use -s USE_ZLIB=1 instead of zlib support headers (#18031)
* [wasm] Modify to use -s USE_ZLIB=1 instead of zlib support headers

- This will remove the dependency on the mono zlib header files.
- Moves the zlibtest.cs to tests/src directory to help clean up the main directory.

* [wasm][packager] Remove the zlib headers from the zlib build

- use `-s USE_ZLIB=1` instead when building.
2019-12-04 20:11:43 +01:00
Alexander Köplinger 4e8ead7caf Remove mk/ folder (#18039)
The plan to put additional helpers into this directory didn't materialize and we can just inline the one thing we have in common.mk where needed.
2019-12-04 19:55:22 +01:00
Kenneth Pouncey 42c5ccc298 [wasm][reference] Update binary references to include wasm websocket assemblies. (#18029) 2019-12-04 11:52:19 +01:00
Jonathan Peppers b816189b94 [android] don't probe directories for .config files (#18024)
* [android] don't probe directories for .config files

In Xamarin.Android, on startup we see logging like this for every .NET
assembly:

    12-03 14:27:42.104 26842 26842 D Mono    : Prepared to set up assembly 'FormsViewGroup' (/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll)
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly FormsViewGroup[0xebcee9c0] added to domain RootDomain, ref_count=1
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly Loader loaded assembly from location: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll.config'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config'.

In most cases, no `.dll.config` file exists and so it probes this
macOS-looking path:

    /Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config

It looks like this is happening because nothing in Xamarin.Android
calls `mono_set_dirs` or `mono_set_config_dir` and so it falls back to
using `MONO_CFG_DIR`. `MONO_CFG_DIR` is set to a path on the build
machine.

Even if Xamarin.Android *did* call `mono_set_config_dir`, the code
appends `mono/assemblies`:

    cfg = g_build_filename (cfg_dir, "mono", "assemblies", aname, cfg_name, (const char*)NULL);

So it doesn't seem useful to use this fallback at all, there would
never be a file found.

I wrapped the appropriate code in `#ifndef TARGET_ANDROID` blocks.

I was able to see a minor *maybe* ~20ms performance improvement on
startup:

    Before:
    12-03 14:27:48.494 26903 26903 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:601::443900
    12-03 14:27:54.356 26963 26963 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::495400
    12-03 14:28:00.263 27021 27021 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:586::147100
    12-03 14:28:06.135 27074 27074 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:593::639100
    12-03 14:28:11.988 27129 27129 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:571::194900
    After:
    12-03 14:46:50.590 27737 27737 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:503::918700
    12-03 14:46:56.517 27794 27794 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:570::533600
    12-03 14:47:02.368 27850 27850 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:537::435000
    12-03 14:47:08.289 27904 27904 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::830900
    12-03 14:47:14.130 27965 27965 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:555::593400

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Add --enable-minimal=cfgdir_config

* This also defines `DISABLE_CFGDIR_CONFIG`
* Used this setting for android
2019-12-04 11:50:59 +01:00
abrevet-dev a79c4d11a4 [WinForms] Editing the DialogResult of the form in the button click event was ignored. (#18025)
Fixes #12022
2019-12-04 11:49:43 +01:00
Jérôme Laban 91d3d63858 Update to emscripten 1.39.4 (#18016)
Fixes for i64 returning values with dynamic runtime configuration.

See https://github.com/emscripten-core/emscripten/issues/9850
2019-12-04 09:11:22 +01:00
monojenkins 37ba834ee1 [bcl] Update BCL Linked Size 2019-12-04 08:29:56 +01:00
Aleksey Kliger (λgeek) 467d46f8ab [configure.ac] Build on FreeBSD/PowerPC64 (#18015)
Community contribution from https://github.com/mono/mono/issues/18009
from @igorzigorz
2019-12-03 19:56:30 -05:00
Egor Bogatov 85826e5c27 Merge pull request #18013 from EgorBo/intrin-string-memcpy
[netcore] Improve jitdiff tool
2019-12-04 02:19:32 +03:00
Maxim Lipnin e7ef2f3d5f [Wasm] Enable system nunit tests on ci (#17716)
* [wasm][nunit tests] Enable nunit tests for System on CI

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Mail.SmtpClientTest tests

* Fix test suit name

* [wasm][nunit tests] Exclude MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest nunit tests

* [wasm][nunit tests] Exclude MonoTests.System.Net.NetworkInformation.NetworkInterfaceTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.EventDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.AsyncOperationManagerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.ServicePointTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebClientTestAsync nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Timers.TimerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest.PropertiesWhenClosedSet3 test

* [wasm][nunit tests] Increase timeout for System nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest.Find_FindByIssuerName test

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Mail.SmtpClientTest tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerTest nunit tests

* Increase CI timeout for System.Core nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Update NetworkInterface class to throw PNSE

* [wasm][nunit tests] Ignore failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Adjust the use of Ignore attribute

* [wasm][nunit tests] Categorize failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Remove the category for passing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing part of MonoTests.System.Net.ServicePointTest.DnsRefreshTimeout test

* [wasm][nunit tests] Remove PNSE expectation in failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests.

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests due to libc issue

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.WebRequestTest.DefaultWebProxy test

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests due to compare info issue
2019-12-03 22:40:09 +01:00
abrevet-dev 174f2f6a70 [WinForms] SetRowSpan and SetColumnSpan does not trigger the good exception. (#18002)
According to .Net documentation:

Exceptions
ArgumentOutOfRangeException

value is less than 1.
2019-12-03 21:38:38 +01:00
EgorBo 5f58bb0959 Improve TryParseFunctionName in jitdiff 2019-12-03 18:13:48 +03:00
EgorBo ea0824cb22 Remove dead docs 2019-12-03 18:11:42 +03:00
Egor Bogatov 8150a38dae Don't throw NRE when len == 0 in Buffer.Memmove (#18004) 2019-12-03 15:21:41 +01:00
Steve Pfister a3c25e7b4f Added Obsolete to AppDomain.CreateDomain for monodroid. (#18000)
* Added Obsolete to AppDomain.CreateDomain for monodroid.  The intent is to be helpful for when .NET 5 comes along

* Changed AssemblyLoadContext link to a better doc

* Bump API snapshot submodule
2019-12-03 14:23:49 +01:00
Jo Shields a2f9fb77a3 Bump version to 6.11 (#17990) 2019-12-02 15:58:38 -05:00
Di Wu 0370372ce3 Fix raw security descriptor parsing (#17482)
Fix security identifier parsing error with the following SDDL:

```
O:S-1-5-21-1356517589-3987021482-2501375073-1001G:S-1-5-21-1356517589-3987021482-2501375073-513D:(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;S-1-5-21-1356517589-3987021482-2501375073-1001)
```
2019-12-02 20:03:49 +01:00
Jay Krell 72aa81e901 Remove handles/MonoError from Mono.RuntimeClassHandle. (#17805) 2019-12-02 19:18:03 +01:00
Bernhard Urban-Forster a9a0a1e5b9 [profiler] disable inlining when coverage profiler is used and fix allocation sizes (#17986)
* [profiler] let compiler decide about the size of an entry

* [profiler] disable inlining when coverage profiler is used
2019-12-02 19:04:13 +01:00
Vlad Brezae 640ffa7a62 [sgen] Don't trigger collections during allocation of thread objects (#17970)
Allocation of thread objects usese the mono_object_new_mature API. These objects are allocated before the thread is finished attaching to the runtime. This means that a collection can happen on an unattached thread as well as all its callbacks, which is a counter-intuitive behavior.

On android this is problematic because at the end of the SGen collection we need to call the java collection, which needs to have the thread attached. This could probably be fixed instead from the Xamarin.Android side, but this approach seems simpler and saner.

Fixes https://github.com/mono/mono/issues/17878
2019-12-02 12:54:21 -05:00
Jay Krell d7f5a5fdfe Remove handles/MonoError from Mono.Security.Cryptography.KeyPairPersistence icalls. (#17967) 2019-12-02 12:51:16 -05:00
dotnet-maestro[bot] 9bd1b2e065 Update dependencies from https://github.com/dotnet/arcade build 20191201.1 (#17985)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19601.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19601.1
2019-12-02 18:03:11 +01:00
abrevet-dev 1454b010ae [WinForms] the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true (#17981)
* [WinForms] Fix #17980 Prevent from crashing when DataGridView  is null.

Reproducer:

DataGridViewLinkColumn links = new DataGridViewLinkColumn();
links.UseColumnTextForLinkValue = true;

* [WinForms] Fix #17980 The field useColumnTextForLinkValue of DataGridViewLinkCell was not cloned.

* [WinForms] Fix #17980 the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true

Fixes #17980
2019-12-02 17:39:43 +01:00
abrevet-dev 865e955840 [WinForms] Fix #9726 imageLayout of DataGridViewImageCell was not cloned. (#17974)
Fixes #9726
2019-12-02 17:38:08 +01:00
Thays Grazia 3834be3d22 [DIM] Fix behavior when there is an override of a method in a generic interface. (#17963)
* Fix behavior when there is an override of a method in a generic interface.
Fixes #17869
2019-12-02 13:20:00 -03:00
Jay Krell 7356aedc77 Refactor Monitor to eliminate near identical duplicated code. (#17964)
The only difference in the copies, was that one had an infinite timeout
so implied TRUE result unless interrupted.

Where the other copy had a timeout and could return FALSE as a result.

They both used the same underlying, with timeout, so it wasn't really an optimization,
just duplicated code.

Also there is subtelty around having a return value or an out parameter.
That is carefully preserved.

Also, coop is fixed, by creating a frame only in a slow path.
Presumably creating a thread is quick and stateless enough such that this
is not too late.
2019-12-02 11:06:11 -05:00
Calvin Buckley 369806e3e6 Disable ConnectFailAsync test on AIX (#17975)
Hangs runtime; prevents CI from completing run. (Another test can
also hang the runtime, but is hard to reproduce and inconsistent
about that.)

This should be fixed eventually, but...
2019-12-02 10:34:07 +01:00
Zoltan Varga f13e6ba91a [wasm] Add beginnings of SIMD support in AOT mode. (#17971)
* [jit] Implement beginnings of SIMD support for WASM.

* [wasm] Add a DISABLE_WASM_NO_NETCORE Make.local conditional to disable non-netcore builds during development.

* [wasm] Add a --simd argument to the packager.

* Remove some duplicate defines.

* Fix an windows assertion.
2019-12-02 07:21:17 +01:00
campersau b0f0b14023 [wasm][bcl][http] Fix descriptive message for the PlatformNotSupportedException (#17919)
* Fix descriptive message for the `PlatformNotSupportedException`

* Add more descriptive message to the `Properties` `PlatformNotSupportedException`
2019-12-02 05:20:15 +01:00
EgorBo d33ce1c3ec enable only for netcore 2019-12-01 17:38:33 -06:00
EgorBo 376d3acb29 fix failing tests 2019-12-01 17:38:33 -06:00
EgorBo b77f757679 fix build 2019-12-01 17:38:33 -06:00
EgorBo 1b7c7e6075 Address feedback and fix C++ warnings 2019-12-01 17:38:33 -06:00
EgorBo e2241969f5 Optimize box(X) == null to false 2019-12-01 17:38:33 -06:00
Jay Krell fffeb23239 sockets: Remove piecemeal ifndef ENABLE_NETCORE since the file has it broadly. 2019-12-01 17:33:24 -06:00
Zoltan Varga ae4b678b94 [wasm] Avoid rebuilding corelib in build-wasm-bcl-netcore. (#17969) 2019-11-30 03:18:42 +01:00
Bernhard Urban-Forster 4d0feffa42 [interp] track valuetype stack for CEE_RET too (#17968)
[interp] track valuetype stack for CEE_RET too

Usually the stack depth is determined via basic block predecessors, but  that only works for forward branches.

Fixes https://github.com/mono/mono/issues/17924
2019-11-30 00:06:05 +01:00
Zoltan Varga 6c0bfdc3f3 [wasm] Add more netcore test suites. (#17965)
* [wasm] Handle more exceptions in the xunit runner.

* [wasm] Add more corefx test suites.

* Add more icall signatures.

* Add support for running the System.Compression test suites.
2019-11-29 18:17:40 +01:00
Zoltan Varga d1735a8356 Interpreter fixes. (#17966)
Interpreter fixes.
2019-11-29 16:26:05 +01:00
Ankit Jain c92a96ccb5 Bump msbuild to track xplat-master (#17950)
* Bump msbuild to track xplat-master

* Bump nuget.exe to 5.4.0
2019-11-29 12:01:06 +01:00
Egor Bogatov 9a18b6fd6f [netcore] Improve Jitdiff tool (#17960)
use `objdump` arguments to reduce verbosity (no raw bytes, etc) and make the dasm more diff-friendly.
On Linux all functions have a random prefix (address) - replace it with `0xD1FFAB1E` constant). On macOS it's fine by default.

Also, allow to generate diffs for a single assembly via `dump-asm-lib` rule.
2019-11-29 11:54:01 +01:00
Kenneth Pouncey 608c93643e [wasm][http] Fix race condition as a result of using TaskCompletionSource: RangeError: Maximum call stack size on Safari (#17946)
There is a race condition on Safari as a result of using TaskCompletionSource that causes a stack exceeded error being thrown during Blazor startup.

Safari throws the following error:

```
[Error] Unhandled Promise Rejection: RangeError: Maximum call stack size exceeded.
    step (blazor.webassembly.js:2530)
    fulfilled (blazor.webassembly.js:2500)
    promiseReactionJob

```

- Modify `WebAssemblyHttpMessageHandler.cs` to create the TaskCompletionSource to run continuations asynchronously.
   - `TaskCompletionSource<HttpResponseMessage>(TaskCreationOptions.RunContinuationsAsynchronously);`

- There is a second part to this on AspNetCore in `Renderer.cs` to process it's pending startup tasks to run continuations asynchronously to compliment the TaskCompetionSource of the Http response message.

The following link discusses the potential stack/race problem that can occur when using TaskCompletionSource:  https://devblogs.microsoft.com/premier-developer/the-danger-of-taskcompletionsourcet-class/
2019-11-29 07:14:19 +01:00
Zoltan Varga 59a9af6d7c Revert "Use range metadata (#17956)" (#17958)
This reverts commit fde058518d.
2019-11-28 21:52:01 +01:00
Egor Bogatov fde058518d Use range metadata (#17956) 2019-11-28 14:26:35 +01:00
Kenneth Pouncey d6dea097ca [wasm] Marshal null string correctly. (#17955) 2019-11-28 07:31:59 +01:00
Zoltan Varga 79db9b2a87 [interp] Make calli non recursive as well. (#17953) 2019-11-28 06:26:36 +01:00
EgorBo 6360ccbfe3 Add quick diff 2019-11-27 19:15:56 -06:00
EgorBo e333f13710 Add jit-diff tools 2019-11-27 19:15:56 -06:00
Ryan Lucia 9c1cae02ca Change signature, add cast 2019-11-27 19:13:30 -06:00
Ryan Lucia 0c3036abc7 [mini] Move bundled native library loading into the runtime 2019-11-27 19:13:30 -06:00
Jay Krell b2817e6404 [cxx] MonoJitICallId same between C and C++.
This is optional.
The real problem was -enable-cxx not actually compiling some files as C++.
This is still reasonable, to ward off the danger of that occuring again.
2019-11-27 19:12:32 -06:00
Jay Krell c3da0ad518 Update netcore. 2019-11-27 19:11:35 -06:00
Jay Krell 4a15e73429 Remove handles/MonoError from Mono.RuntimeMarshal and Mono.SafeStringMarshal. 2019-11-27 19:11:35 -06:00
Nikita Voronchev 7db2389846 [WinForms][UIA] Add to the PropertyGrid new internal event to track grid items expanded state update (#17731)
This PR is needed for the https://github.com/mono/uia2atk/pull/6 one.
2019-11-27 23:59:04 +01:00
abrevet-dev f135312b44 [WinForms] DrawToBitmap() did not draw children controls (#17676)
Fixes #13777
2019-11-27 23:38:04 +01:00
Sergey Podobry abc75476d7 [Mono.Security] Do not decode data beyond detected length in ASN1 parser (#17751)
If `byte[] data` has additional data at the end `Decode` will try to parse it and will generate incorrect tags or throw an exception. This fix makes `Decode` to use only `nLength` bytes from `byte[] data`.
2019-11-27 23:37:20 +01:00
Aleksey Kliger (λgeek) b6a351c330 [threadpool] Decrement max_working when worker times out (#17927)
[threadpool] Decrement max_working when worker times out

The corresponding code in CoreCLR:
https://github.com/dotnet/runtime/blob/bf88f146412d1e2d41779422337184a802f186c9/src/coreclr/src/vm/win32threadpool.cpp#L2156-L2164

The issue is that counter._.max_working is only ever changed by:
1. the monitor_thread when it detects starvation (increment by 1).
2. the hill climbing algorithm (increment or decrement).

Creating a periodic load of creating many workers at once followed by a minute
or more of quiescence increases the variable worker.counters._.max_working in
monitor_thread(). Once that counter matches worker.limit_worker_max,
monitor_thread() keeps looping without unparking or creating a thread, even
though the actual number of threads/workers is small. If the existing threads
are all waiting on work that needs a new thread, then a deadlock occurs.

With this change, when a parked worker times out, it will lower max_working to
the number of active (working + parked + starting) threads minus itself (but no
less than limit_worker_min). As a result, monitor_thread will only increment
max_working as long as none of the already running workers are timing out.

Attempt to address https://github.com/mono/mono/issues/17833

---

Also allow monitor_thread to start workers if there aren't enough active, even if the max worker limit has been reached.

Work around a pathological condition where the work_item_count is non-zero, and
the max_working limit has been reached, but the number of active threads is
still below the max.  In that case, unpark some workers and start some new
threads.

May address http://work.azdo.io/827206
2019-11-27 22:06:17 +01:00
Jay Krell 3e369ba8c9 Cleanup initialize_object_slots, remove unused GetHashCode (ghc) handling. (#17834)
Fix possible race conditions involving multiple writes to globals, by
removing one of the two globals involved in finalize.
2019-11-27 14:25:14 -05:00
abrevet-dev 6145a397d6 [WinForms] Refactor selected indexes in TabControl.Remove (#17608)
Fixes #12337
2019-11-27 19:55:00 +01:00
abrevet-dev 5944e72d72 [WinForms] Returns real installed input languages on Windows (#17653)
The method InstalledInputLanguages() always returned InvariantCulture. Now, installed input languages are detected properly on Windows.

Reproducer:

`Console.WriteLine(InputLanguage.CurrentInputLanguage.Culture.TwoLetterISOLanguageName.ToUpper());`

Related issue: #13552
2019-11-27 19:46:04 +01:00
abrevet-dev 41ba5846b3 [WinForms] DefaultCellStyle was not cloned deeply in DataGridViewRow.Clone() (#17755)
Fixes #16557
2019-11-27 19:41:15 +01:00
HebaruSan d62eed676e [WinForms] Fix X11 theming (#17888)
## Background

In Mono 5 and earlier, the active GTK or KDE theme would be sourced for system colors, allowing dark themes to be obeyed:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

This allowed WinForms apps to blend in and look more like native apps.

## Problem

In Mono 6, this capability was lost (as noted in passing in #14325). WinForms apps now only look like this, regardless of the system color settings:

![image](https://user-images.githubusercontent.com/1559108/69486114-dccce280-0e0d-11ea-8107-fe1653b2d816.png)

This looks somewhat jarring and worse than previous Mono releases, for users with a dark system theme.

## Cause

During load of the system settings (in the static constructor of `X11DesktopColors`), `Theme.SetSystemColors` passes system color data to `System.Drawing.KnownColors.Update`, which populates them into `System.Drawing.KnownColors.ArgbValues`. In Mono 5 and earlier, this flowed through to the visible UI.

It seems that as of Mono 6, this table no longer affects the visible WinForms UI (although it is still being updated!). Instead, a same-but-different table in `System.Drawing.KnownColorTable` drives the system colors now. I was not able to identify the commit in which this change was made.

## Changes

Now `Theme.SetSystemColors` accesses `System.Drawing.KnownColorTable.s_colorTable` instead, and WinForms apps once again obey GTK and KDE themes:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

Fixes KSP-CKAN/CKAN#2881.
2019-11-27 19:13:11 +01:00
RizvanKuliev a73e7b8af2 [WinForms] Fix Recalculate in ScrollableControl (#17163) 2019-11-27 19:09:19 +01:00
Alexander Köplinger 9954579c2a [sdks] Call mono_gc_init_finalizer_thread() on iOS device runtimes
These runtimes are configured with `--with-lazy-gc-thread-creation=yes` so they don't start a GC finalizer threads by themselves.
XI does this because they want to delay creating that thread for some time after startup.

We can just do it immediately, the important part is that it _does_ exist.
2019-11-27 18:36:02 +01:00
Alexander Köplinger c66c7814d0 [sdks] Add option to enable debug server for iOS test app
Include .pdb files in iOS test app if the DEBUG variable is set and wait for managed debugger to attach.
2019-11-27 18:36:02 +01:00
Alexander Köplinger 18ddc2e45d [sdks] Fix loading of assemblies in assembly_preload_hook()
We need to check whether the assembly name passed to us already contains .exe/.dll and append it otherwise.
Fixes loading of assemblies via the hook, we were falling back to the usual path resolving logic (which doesn't work in netcore mode).
2019-11-27 18:36:02 +01:00
Alexander Köplinger a0d79a4886 [sdks] Add netcore option for iOS SDK 2019-11-27 18:36:02 +01:00
Alexis Christoforides 167913524f [merp] Use timeline (and generate timeline breadcrumbs) for SendExceptionToTelemetry() (#17913)
Fixes https://github.com/mono/mono/issues/17790
2019-11-27 11:56:12 -05:00
Bernhard Urban-Forster 4c595069e7 [arm] if mtriple is provided, do not set -march (#17937)
[arm] if mtriple is provided, do not set -march

E.g. when passing `-march=arm -mtriple=armv7s-ios`, `-march` will win and thus ignore `armv7s`.

Fixes https://github.com/mono/mono/issues/17931

* Caused by 5e318afd2e
* Related with 4cd5068230
* Almost fixed by e431093f73

Hopefully we are done with it.
2019-11-27 17:36:03 +01:00
monojenkins 9dfeb545d5 [bcl] Update BCL Linked Size (#17322) 2019-11-27 13:43:31 +01:00
Bernhard Urban-Forster 690af29ad9 [amd64] do not stack allocate on the application stack for the transtion from alstack handling (#17922)
### Backstory
This test is failing in the interpreter (ONLY on Debian9/amd64. Not Ubuntu/amd64. Not macOS/amd64):
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mcs/class/Mono.Debugger.Soft/Test/dtest.cs#L2484-L2499
Where the debuggee is just some unsafe code that tries to read from unmapped memory, thus causing a segfault.

So the test expects the runtime to send a CRASH event via the managed debugger interface. This happens very late in the crash handling machinery:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L1105

However, the runtime would never reach that, because it silently crashes when calling `backtrace`:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L939

At the point where the SIGSEGV happens, the native stack trace looks like this:
```
Thread 1 "mono-sgen" received signal SIGSEGV, Segmentation fault.
interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
4463                            sp[-1].data.i = *(gint32*)sp[-1].data.p;
(gdb) bt
%0  interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
%1  0x000055555569237a in interp_runtime_invoke (method=<optimized out>, obj=<optimized out>, params=<optimized out>, exc=0x0, error=0x7fffffffd8b0) at interp/interp.c:1900
%2  0x0000555555590745 in mono_jit_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=0x7fffffffd858, exc=0x0, error=0x7fffffffd8b0) at mini-runtime.c:3001
%3  0x0000555555778c4c in do_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=<optimized out>, exc=<optimized out>, error=0x7fffffffd8b0) at object.c:3052
%4  0x000055555577b7d0 in do_exec_main_checked (method=0x555555bfdc38, args=<optimized out>, error=0x7fffffffd8b0) at object.c:5184
%5  0x000055555559a2aa in mono_jit_exec_internal (argv=0x7fffffffdcb8, argc=1, assembly=0x0, domain=0x555555bf9d40) at driver.c:1320
%6  mono_jit_exec (domain=domain@entry=0x555555bf9d40, assembly=assembly@entry=0x555555c66ae0, argc=argc@entry=1, argv=argv@entry=0x7fffffffdcb8) at driver.c:1265
%7  0x000055555559b8e2 in main_thread_handler (user_data=<synthetic pointer>) at driver.c:1402
%8  mono_main (argc=<optimized out>, argv=<optimized out>) at driver.c:2622
%9  0x000055555558b757 in mono_main_with_options (argv=<optimized out>, argc=<optimized out>) at main.c:52
%10 main (argc=<optimized out>, argv=<optimized out>) at main.c:434
```
Stepping into the crash handling machinery, the picture looks a bit different:
```
Thread 1 "mono-sgen" hit Breakpoint 1, altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
871     {
(gdb) bt
%0  altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
%1  0x00005555556957de in interp_exec_method_full (frame=0x555555c53f08, context=0x555555c38350, clause_args=0x0, error=0x555555c53f18) at interp/interp.c:4461
%2  0x0000000000000000 in ?? ()
```
So it looks like we messed up in the transition from altstack back to the program stack.

 ### The fix

The problem is that `backtrace` consults the unwind information emitted at the point where SIGSEGV happens in `interp_exec_method_full ()`. However at that given program code, there is no function call that allocates space to pass arguments on the stack. Still, we massage the stack to do that in the signal handler, which does not add up with the emitted unwind information by the C compiler.

Note that without the fix, crashing in `backtrace` is the worst case. In other cases we might (1) get still a correct stack trace, or (2) we get a wrong stack trace, and `backtrace` will not crash. So this PR might improve the crash experience overall.

This also reverts commit cff400f3fae9775ac734f8a691339e59a272872c.
2019-11-27 12:20:47 +01:00
Zoltan Varga c4136ebd94 [interp] Non-recursive interpreter (#16461)
* Aling alloca_size to 8.

* Prevent some false pinning in the monitor-resurrection.cs test when using the non-recursive interpreter.

* [runtime] Add a mono_compiler_barrier () function.

* Add a MONO_GET_SP opcode to pass to enter/exit_gc_safe_region, ldloca might not return a native stack address with the interpreter.

* Add an interpreter/gc callback to mark the interpreter stack.

* Copy only an integer in ENDFILTER.

* Add a native_stack_addr field to InterpFrame so the EH code has something to compare against native stack addresses.

* Add a FrameStack structure to allow allocation of InterpFrame structures and stack data outside the native stack.

* Allocate InterpFrames/stack data from a frame stack instead of the native stack.

* Make some calls non-recursive.

* Remove an assert.

* Use non-recursive calls in CALLVIRT_FAST as well.

* Fix stack size allocation for vararg calls.

* Add some micro optimizations to the calling code.

* Add more micro optimizations.

* [dtest] disable Crash test on interpreter

* Add some implicit conversions to STSFLD to fix a corefx test failure.

* Disable the new test.
2019-11-26 04:31:32 +01:00
Zoltan Varga 5b4cd5ce50 WASM + Netcore. (#17915)
* [runtime] Throw an exception from Thread::StartInternal () if threads are disabled.

* Add more test suites and a corefx exclusions file.
2019-11-26 02:59:31 +01:00
Larry Ewing e917c95053 Add the reference assembly path for the Http implementation (#17908) 2019-11-25 19:21:38 -06:00
dotnet-maestro[bot] cd190481bd Update dependencies from https://github.com/dotnet/arcade build 20191124.1 (#17899)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19574.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19574.1
2019-11-25 21:22:41 +01:00
Alexander Köplinger e437330808 Remove mkinstalldirs invocation from two places (#17910)
These can be easily replaced so we don't need to mirror over the mkinstalldirs file to the new repo
2019-11-25 21:10:26 +01:00
Alexander Köplinger 8aa1cdfe17 [configure.ac] Don't run compiler server check in netcore mode (#17911)
We don't need it there.
2019-11-25 21:10:08 +01:00
Calvin Buckley 836f453906 Fix BinaryReader on BE (#17893)
Fix BinaryReader on BE

The ints passed into the int array ctor needed to be endian swapped.
Interestingly, the BinaryWriter implementation currently being used
(not refsrc AFAIK) does this on write just fine.

Fixes ReadDecimal test on AIX, and adds another test to ensure that
Write(decimal) can be read back by ReadDecimal in case.
2019-11-25 19:15:55 +01:00
Alexander Köplinger bdf4790fa0 Fix boehm on Windows i386 builds (#17901)
After https://github.com/mono/mono/pull/16832 was merged the run-msbuild.bat
script has an additional required parameter for specifying the GC.

However the script only passed the MONO_TARGET_GC property to msbuild
if the script had no additional parameters which doesn't make sense.

Always pass the parameter, this should fix the boehm build on Windows i386.
2019-11-25 18:58:42 +01:00
Larry Ewing b42845c567 Fix issue in debugger variable inspection (#17900) 2019-11-25 11:55:21 -06:00
Marek Habersack 10795da1c0 [WIP] Make debugger agent startup faster (#17898)
Mono SDB's debugger agent takes a `host:port` parameter to specify the host and
port on which to listen for debugger connection. `host` is passed, via
`mono_get_address_info`, to the `getaddrinfo(3)` standard library call which
does the job of translating the name to the IP address. However, there's no
special treatment for situations when `host` is already an IP address. In this
case `getaddrinfo` takes a lot of time to perform at least two DNS lookups (if
there's just a single DNS server configured) which will time out as the IP
address in `host` will not be resolved by any of the servers.

`getaddrinfo` takes a "hints" parameter where caller can specify some
information about the `host:port` pair to make the lookup faster/easier.
However, Mono doesn't specify the address family in the `hints` parameter, thus
forcing the slow path of DNS lookups on `getaddrinfo`.

This commit adds a new hint named `MONO_HINT_NUMERIC_HOST` (which maps to
`AI_NUMERICHOST` for `getaddrinfo`) which will allow, if set, `getaddrinfo` to
take a faster route in address discovery. Debugger agent will, in most cases,
get an IP address to instead of host name so we call `mono_get_address_info` up
to 3 times (IPv4, IPv6 and unspecified) in order to make the lookup faster.
2019-11-25 17:52:58 +01:00
Calvin Buckley f47b7bbf4f Fix flawed BufferTest on big-endian (#17894)
Fix flawed BufferTest on big-endian

Test ignored byte order when considering integer comparison. This
tests different cases for each endian, and explains why the byte
order is different.
2019-11-25 16:25:55 +01:00
Kenneth Pouncey 5d981c2679 [wasm] [bindings] Method invoke signature argument validation. (#17853)
* Implement validation for signature and argument count consistency.

- throw error `Parameter count mismatch.` when arguments and signature counts are not consistent.

* Fix tests that have Parameter count mismatch.

* Add browser binding tests for `Parameter count mismatch
2019-11-25 08:24:33 -06:00
Kenneth Pouncey e8d70c333f Add tests for Promise library.
- Right now only Bluebird promise library is tested right now.
- If more are needed later then we can add them.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 2f1e2f8132 Add a more reliable way of identifying a Promise to be marshaled.
- This is left as generic as possible for now.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 64e2864fef Add a more descriptive error in case of fetch response not returning back what it should.
- An example of this is when a Promise library is being used and the Promise can not be identified correctly.  Thus the internal error.
2019-11-25 08:23:45 -06:00
Vlad Brezae 1ef7a29836 [interp] Resolve tokens also from wrappers during optimization (#17880)
[interp] Resolve tokens also from wrappers during optimization

Remove some duplicated code.
2019-11-25 14:45:56 +01:00
Calvin Buckley 004fd06b96 [eglib] In AIX/generic POSIX, handle null addresses better (#17892)
Fixes crash in tests on AIX.

Matches b0d966bd7a
2019-11-25 02:52:54 -05:00
Calvin Buckley a3f4def8b6 Allow runtime to be built with C++ on AIX (#17672)
* Allow runtime to be built with C++ on AIX

Many AIX/PPC/BE specific codepaths didn't do casting properly,
since C++ is much stricter than C about pointer typing.

Also specify the C99 format macros early as possible, since
inttypes may get included before eglib gets a chance to set the
macros.

Also explicitly use -pthread, since not using it is the cause of
many libstdc++ crashes on AIX.

* Oops, should have been that define instead

* Suggested changes from Jay on style

* Remove unneeded cast
2019-11-24 17:08:19 -05:00
Alexander Köplinger 280e9d2423 Fix warnings in runtime build (#17887)
This makes at least the netcore-mono build in dotnet/runtime warning-free.
2019-11-24 00:16:20 +01:00
Alexander Köplinger 165e28ae97 [netcore] Disable libmonoruntime-support.la build on netcore (#17885)
It's used for the zlib compression helpers used by System.IO.Compression in the old Mono BCL.
On corefx this is handled differently so we don't need it there.

To avoid automake complaining about missing files we include the .c files directly instead of in the Makefile.
2019-11-23 10:06:40 +01:00
Alexander Köplinger 2ccd6122bd [netcore] Disable a test which fails on the interpreter
The test is wrong, will be fixed upstream.
2019-11-22 23:00:28 +01:00
Alexander Köplinger 81e305421d Fix run-msbuild.bat invocation in pipeline-netcore-runtime.yml 2019-11-22 22:48:53 +01:00
Alexander Köplinger 0d80372173 Make configure.ac and Makefile.am work without certain directories (#17871)
Those folders won't be present in the dotnet/runtime repository.

We need to factor out AC_OUTPUT into a separate file so we can continue
to share configure.ac but not the list of files in AC_OUTPUT since
autotools doesn't support variables there.
2019-11-22 22:48:11 +01:00
Zoltan Varga 93e46c687b [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case. (#17832)
* [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case.

* [amd64] Fix the return value mapping in gsharedvt calls, the return address might be in different registers in the caller and callee.

* [runtime] Always initialize del->method_ptr even if interp_method is set, JITted code makes calls through it in mixed mode.

* Add a test for open delegates and gsharedvt.

* Increase nrgctx trampoline count, its needed by dynamic-method-churn.exe in full-aot-interp mode.
2019-11-22 20:34:14 +01:00
Jonathan Chambers cc137237d8 Replace embedded libgc with Unity fork of recent Boehm (bdwgc) (#16832)
* Initial commit to migrate libgc to bdwgc.

* Fix submodule url

* Don't force Boehm as runtime wrapper. Was just for testing locally.

* Build bdwgc as single object file. Helps with performance.

* Bump bdwgc submodule.

* Attempt to fix windows MSVC based build.

* Remove any references to GC_INSIDE_DLL as we manually managed threads on Windows.

* Allow building boehm on Windows amd64.

* Allow building boehm with msvc.

* Bump bdwgc to fix line endings

* Use LF for sh, am, m4, and ac files in bdwgc

* Bump bdwgc & libatomic_ops with fix for cygwin

* Bump bdwgc to fix mono-boehm being generated as libtool wrapper script.

* Link the static boehm library.

* Revert "Link the static boehm library."

This reverts commit 1f90d81d08308a61c890743cd79d8f6bb791bc99.

* Use --export-all-symbols on cygwin/mingw to fix issues with boehm causing mono symbols not to be exported.

* Fix
2019-11-22 18:03:23 +01:00
Filip Navara 0fed03ed63 Fix InternalGetHashCode for boehm/null GC (#17859)
It was unintentionally shifting the address to left instead of right. It should be shift right to get rid of zeroes from the alignment.
2019-11-22 07:51:43 -05:00
Alexander Köplinger 5fedc42ccf Fix automake warning 2019-11-22 13:18:10 +01:00
Kenneth Pouncey 9e1d59b8cc Remove unused GetHttpMessageHandler from WebAssembly.Net.Http.
- Remove linker.xml from project
2019-11-22 13:12:17 +01:00
Zoltan Varga ecd67a91c4 [wasm] Add aot+netcore support. (#17798)
* [wasm] Compile corelib as 32 bit.

* [wasm] Add a netcore cross compiler.

* [wasm] Add netcore+aot support.

* [aot] Load the dedup module lazily instead of at startup.

The previous version didn't work on netcore because it couldn't resolve references to corlib etc.
since they were not yet registered in the root domain at the time of loading.

* Add more icall trampoline signatures.

* Add more netcore pinvokes.

* Fix the build.

* Add more icall trampoline signatures.

* Add an --embed-file option to the packager.

* Add more netcore pinvokes.

* [interp] Return FALSE for IsDynamicCodeCompiled.

* Add a netcore project and targets for running BenchmarkDotNet.

* Fix pinvoke tables.

* Add missing files.

* Fix make clean.

* Add a comment.

* Revert "Fix pinvoke tables."

This reverts commit 998065e77614b7e621d2873510aabc3930669ca9.

* Make xunit-runner into a dotnet project. Add a netcore version.
2019-11-22 12:42:55 +01:00
Egor Bogatov 6a5c44b15e [netcore] Run tests with Interpreter (#17862)
* Run tests with interp and llvm

* fix build

* Run tests with llvm

* Add a few tests to rsp for LLVM
2019-11-22 09:14:23 +01:00
Kenneth Pouncey 202cb6a648 Add more descriptive message to the PlatformNotSupportedException (#17864) 2019-11-22 08:40:30 +01:00
Jay Krell dadc3d3117 Remove handles/MonoError from Mono.RuntimeGPtrArrayHandle. (#17806) 2019-11-21 14:15:29 -05:00
Jo Shields 9e00330af3 Stop referencing files from Mono BCL in Netcore System.Private.CoreLib (#17866) 2019-11-21 19:48:00 +01:00
Bernhard Urban-Forster 058d157648 [interp] fix array_element_size intrinsic (#17857)
[interp] fix array_element_size intrinsic

`mono_class_array_element_size` gives us the size of the provided MonoClass if it would be an array element. But here we want the element size of a given array's MonoClass. That's what `mono_array_element_size` is returning.

This leads to all kind of weird crashes otherwise, specifically here:
https://github.com/mono/mono/blob/2c20649539ac16e069a65f2a750c793eb341e50f/netcore/System.Private.CoreLib/src/System/Array.Mono.cs#L64

Which is then used later to compute the size to memset in order to clear the content of an array. If it's larger than it should be, then this will cause memory corruption. For example this would crash eventually:

```csharp
using System;
using System.Collections.Generic;

namespace HelloWorld {
    class Program {
        static void Main(string[] args) {
            for (int j = 0; j < 0x1000; j++) {
                var d = new Dictionary<string, string> ();
                for (int i = 0; i < 197; i++)
                    d.Add (i + "", i + "foo");
                d.Clear ();
            }
        }
    }
}
```

Thanks to @EgorBo for reporting.
2019-11-21 11:56:03 +01:00
Alexander Köplinger b6e4ddd73c [metadata] Remove native threadpool from netcore build (#17856)
We use the managed threadpool there.
2019-11-21 10:01:39 +01:00
Egor Bogatov 2c20649539 [llvm] Recognize GEP (#17844)
* initial impl

* improve jit

* fix copy-paste

* disable for non-netcore

* clean up
2019-11-20 21:25:24 +01:00
Zoltan Varga 4961d411e7 [interp] Move the freeing of interp method info into a callback. (#17852)
[interp] Move the freeing of interp method info into a callback.
2019-11-20 19:56:09 +01:00
Egor Bogatov 27e6e812f1 enable gss on android (#17838) 2019-11-20 10:53:14 -05:00
Alexander Köplinger 0396037d31 [debugger-agent] Fix CMD_VM_ALL_THREADS returning wrong value due to unitialized variable (#17847)
`remove_gc_finalizing` wasn't initialized so its value is undefined.
The `count_thread_check_gc_finalizer()` function only sets it in some cases so we could end up with the variable having an undefined (and probably not 0) value.
This resulted in the returned thread count from the debuggee being wrong because we'd decrement `count` even though we shouldn't.

Regression from https://github.com/mono/mono/pull/15618
2019-11-20 08:28:58 +01:00
dotnet-maestro[bot] a6db160596 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17690)
* Update dependencies from https://github.com/dotnet/arcade build 20191102.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19552.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19552.1

* Update dependencies from https://github.com/dotnet/core-setup build 20191104.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19554.1

* Update dependencies from https://github.com/dotnet/corefx build 20191103.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19553.8

* Update dependencies from https://github.com/dotnet/arcade build 20191108.11

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19558.11
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19558.11

* Update dependencies from https://github.com/dotnet/core-setup build 20191109.5

- Microsoft.NETCore.App - 5.0.0-alpha.1.19559.5

* Update dependencies from https://github.com/dotnet/corefx build 20191109.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19559.6

* Update dependencies from https://github.com/dotnet/arcade build 20191117.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19567.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19567.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191114.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19564.1

* Update dependencies from https://github.com/dotnet/corefx build 20191113.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19563.6

* Use NETCoreApp version 5.0.0-alpha.1.19563.3

Later versions have an issue and the new dotnet/runtime assets aren't ready yet.

* Disable failing tests

* Fix merge error
2019-11-19 16:47:57 +01:00
Steve Pfister 0c3ffb9cc9 Fixes https://github.com/mono/mono/issues/15805 at least in terms of what may be making it crash. (#17270)
Adds a null check around the sslStream before trying to dispose.
2019-11-19 14:23:51 +01:00
Zoltan Varga b8fe43da3e [interp] Add some missing netcore intrinsics. (#17782) 2019-11-18 13:55:15 +01:00
Steve Pfister 0269f7b479 Bump corefx to pick up https://github.com/mono/corefx/pull/370 (#17758)
Backport of https://github.com/dotnet/corefx/pull/34560

Added HAVE_LCHFLAGS & HAVE_STAT_FLAGS to configure.
2019-11-18 12:13:16 +01:00
Filip Navara e2ccdac3f4 [llvm] Add support for LLVM JIT on ARM64 (#17827)
* [llvm] Add support for LLVM JIT on ARM64

* Fix tabs
2019-11-18 00:24:11 +01:00
Jérôme Laban 965efade84 Update emscripten to 1.39.3 (#17816)
Includes updates for https://github.com/emscripten-core/emscripten/issues/9562 and side modules.
2019-11-16 13:59:28 +01:00
Dotnet-GitSync-Bot 6785b27567 Mirror changes from mono/runtime (#17803)
It looks like the mirror didn't fully quiesce prior to snapping coreclr/corefx for the runtime consolidation.  As a result the libraries build is breaking.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-16 11:27:32 +01:00
Nikolay Sivov 18dd48cad8 [mini] Fix NULL pointer crash in mono_decompose_vtype_opts(). (#17801)
* [mini] Fix NULL pointer crash in mono_decompose_vtype_opts().

All other call sites for mono_type_size() are already passing non-null pointer there.

* [mini] Rename variable dummy -> align.
2019-11-15 17:56:55 +01:00
Marcos Cobeña Morián 92dd4d3883 [Wasm] Forced filesystem creation (#17789)
* Forced filesystem creation

* [Wasm] Added new hello-io sample
- Goal is testing file preloading through System.IO
- Filesystem is now always forced to support later file preloading
- build-io-sample target showcases such

* Added --enable-fs flag to packager.exe

* Update packager.md
2019-11-15 15:24:50 +01:00
Jan Kotas 3ceecb8db0 Fix build breaks
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Carlos Sanchez Lopez d3fbf4a1d3 Fix NetStandard issue in System.IO.FileSystem.AccessControl in 5.0 (#42546)
Fix NetStandard issue in System.IO.FileSystem.AccessControl

We built the fix incorrectly so that it would only apply for netcoreapp3.0 but customer needs it on netstandard2.0 (desktop + core).  Customer impact is that they’ll see a PlatformNotSupported exception rather than the new API.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 00b0170bdc Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#42531)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Tanner Gooding c93801d458 Adding tests for the new apis to convert between S.Numerics and S.R.Intrinsic vectors (#27481)
* Updating Vector128.AsVector3 to have a functioning implementation

* Adding test template metadata covering the new methods for converting between S.Numerics and S.R.Intrinsics

* Regenerating the existing tests

* Generating the new Vector conversion tests

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Emmanuel AndréandStephen Toub 5645032fc6 Add span-based CopyTo and CopyToAsync methods (dotnet/coreclr#27639)
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <[email protected]>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
John adcaf88db0 Fix Vector128 and Vector256 WithUpper documentation (dotnet/coreclr#27274)
* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector256.cs

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 1f2f9b929b Add tracing for handling of dependencies for Assembly.LoadFrom (#27833)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 3242b5f354 Add tracing for Resolving/AssemblyResolve event handler invocation (#27788)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
ts2do 35d94dc8ff Address bugs in BigInteger (dotnet/coreclr#27280)
* Method Add(ref BigInteger lhs, uint value, ref BigInteger result) would store most of the result blocks into lhs instead of result.
* Method ShiftLeft(ulong input, uint shift, ref BigInteger output) with a shift argument exceeding 32 would generally compute the higher blocks incorrectly.
* Multiply(ref BigInteger lhs, uint value, ref BigInteger result) would not set result._length in some cases.
* IsZero() would incorrectly return false for non-canonical zeros with _length > 0.

Fix:
* Inline Add(ref BigInteger, uint, ref BigInteger) into Add(uint).
* Inline ShiftLeft(ulong, uint, ref BigInteger) into Pow2.
* Inline ExtendBlock and ExtendBlocks into Pow2.
* Properly handle 0 in SetUInt32 and SetUInt64.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 2a108e958f Use Stream's BeginReadInternal from FileStream (dotnet/coreclr#27737)
FileStream has two modes, decided upon at construction time.  When it's created in non-async mode, the Read/WriteAsync methods end up queueing work items to invoke the synchronous Read/Write methods.  To do this, the base methods on Stream delegate to Begin/EndRead/Write (since they were around first) and then the resulting IAsyncResult is wrapped.  However, Stream has an optimization that checks to see whether the derived stream actually overrides Begin/EndXx, and if it doesn't, then it skips using those and goes straight to queueing a work item to Read/Write.  However, FileStream does override those, but when it delegates to the base implementation because it's in non-async mode (rather than because it's a type derived from FileStream), going through Begin/EndXx is just unnecessary overhead.  So, in the right circumstances, we can call to Stream's special helper instead.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Zoltan Varga 25b7e6bd42 [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files. (#17795)
* [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files.

* Fix a typo.

* Revert unrelated changes.
2019-11-15 09:04:35 +01:00
Vsevolod Kukol 805c7f5436 [MacSDK] Bump xamarin-gtk-theme.py to latest revision from private bockbuild (#17785) 2019-11-14 09:27:33 +01:00
Jay Krell 19a44cd32b Remove handles from System.Diagnostics.Debugger. (#17713)
* Remove handles from System.Diagnostics.Debugger.
memset (&x, 0, sizeof (type)) to memset (&x, 0, sizeof (x)).

* Bump API snapshot submodule
2019-11-14 08:18:02 +01:00
Zoltan Varga 1d81534d27 [interp] Add some inline checks from the JIT. (#17784) 2019-11-13 22:34:01 +01:00
Zoltan Varga a0fcc29d5d [runtime] Fix some handle usage in native-library.c (#17781) 2019-11-13 18:12:09 +01:00
Jay Krell 73431403c7 Remove handles from ves_icall_System_Array_CanChangePrimitive. (#17708)
Remove handles from ves_icall_System_Array_CanChangePrimitive.
2019-11-13 17:55:52 +01:00
Thays Grazia dd18ec40dd [debugger] Assert when async debug a generic method (#17727)
* When we try to call a method to get the async_id to do an async debug and we are trying to do this in a generic method like this:

async Task<T> ExecuteAsync_Broken<T>()
 {
            await Task.Delay(2);
            return default;
  }

We need to inflate the generic type before call the method or we will get the error:  Could not execute the method because the containing type 'System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[T_REF]’, is not fully instantiated.

Fixes #17549
Fixes #17569
2019-11-13 16:29:33 +01:00
imhameed fe0f824db5 [llvm] Use explicit null checks with LLVM. (#16954)
* [llvm] Use explicit null checks with LLVM.

When the LLVM backend is used for a method, explicit null checks will
unconditionally be used at the mini IR level. We had previously used
implicit null checks at the mini IR level combined with llvm-volatile
loads and stores to avoid mis-optimization of the generated code.

During AOT compilation, LLVM's `ImplicitNullChecks` pass is enabled.
This conservatively elides these explicit null checks. This pass only
modifies branches tagged with `make.implicit` metadata.

Null check branches are now tagged with `make.implicit` metadata, with
three exceptions:

1. For platforms with backends that do not support zero page fault
handling, null check branches are not tagged with `make.implicit`.

2. This commit also adds a `MONO_DEBUG` option:
`llvm-disable-implicit-null-checks`. When it is enabled, null check
branches are not tagged with `make.implicit`.

3. This commit alters the behavior of the `explicit-null-checks`
`MONO_DEBUG` option. Enabling `explicit-null-checks` also implies
`llvm-disable-implicit-null-checks`, because this option is documented
(e.g. in https://www.mono-project.com/docs/debug+profile/debug/) to
completely disable signal/seh-based null checks.

* Add llvm-disable-implicit-null-checks to the MONO_DEBUG help string.

* Add llvm-disable-implicit-null-checks to mono.1.

* Don't tag null checks in EH regions with `make.implicit`.

This relocates the logic formerly used to guard individual LLVM-level
loads and stores with explicit null checks inside EH regions.

* Use atomic/volatile loads and stores for the terimation flag in thread-suspend-suspended.

LLVM was hoisting the load of and branch against `finished` out of
thread t1's loop, resulting in an infinite spin.

* Use ImplicitNullChecks with LLVM JIT.

* Suppress emission of the .llvm_faultmaps section.

Bump external/llvm to release_60
(7a8dc89adbe7e123220e070a527e096ee91e66d5).

* [ci] use LLVM built from repository
2019-11-13 01:35:44 +01:00
Zoltan Varga eaf895f2d0 [wasm] Change netcore support to use a prebuilt corefx runtime. (#17773)
* [wasm] Change netcore support to use a prebuilt corefx runtime.

* [wasm] Build and test netcore support on CI.

* [wasm] Disable netcore tests on CI for now.

* [wasm] Add an error check.

* [wasm] Add a linker exclusion.

* [wasm] Add netcore runtimes to the nupkg.
2019-11-13 01:30:09 +01:00
Filip Navara bd278dd00d Mirror changes from mono/corefx,coreclr (#17777)
* Rewrite the rest of the array non-generic fast paths in C# (#27703)

* Rewrite the rest of the array non-generic fast paths in C#

* Add missing resource string

* Use CorElementType and handle native ints to match the unmanaged implementation

* Avoid try pattern for Sort and Reverse

* Match the native implementation more closely

Signed-off-by: dotnet-bot <[email protected]>

* Rewrite Enum.CompareTo in C# (dotnet/coreclr#27792)

* Rewrite Enum.CompareTo in C#

* Handle floating point too for match the existing native implementation

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused argument from GetRegionDisplayName (dotnet/coreclr#27806)

* Remove unused argument from GetRegionDisplayName

* Address race conditions in CultureData

While most of the race conditions in this file are benign, in these cases, a thread could end up returning an intermediate value due to multiple potential writes to the same field.

Signed-off-by: dotnet-bot <[email protected]>

* Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#27716)

Will enable the rule in corefx once these changes have propagated.

Signed-off-by: dotnet-bot <[email protected]>

* Remove non-shared Enum.CompareTo/InternalCompareTo implementation, add new Array icalls used by shared code
2019-11-12 18:46:00 +01:00
Bernhard Urban-Forster 4728cec01f [llvm] use multiple cores to build llvm if ninja is not available (#17778) 2019-11-12 18:45:45 +01:00
Jo Shields 8bceb40878 Move zlib to its own directory, and re-enable subdir-objects
This has the following benefits:

* Clearer structure for mirroring
* Slight build time improvement on bundled zlib use (as it only builds once)
* Removes pages of subdir-objects spew from autogen
2019-11-12 09:31:08 +01:00
abrevet-dev 8683fa6db7 [WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask setted (#17660)
[WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask se…





<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-11-12 01:55:53 +01:00
abrevet-dev 2b4801d9f1 [WinForms]: Fix #17546 Fix TabPage position when enabling MultiLine (#17670)
[WinForms] Fix TabPage position when enabling MultiLine

Fixes #17546
2019-11-12 00:15:51 +01:00
abrevet-dev 61811075ca [WinForms]: Fix #16632 special values (-1 and -2) of ListView Column header Width was not handled when using List.View. (#17669) 2019-11-11 23:00:45 +01:00
abrevet-dev 9eb0bcae57 [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs (#17662)
* [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs of ScrollBar.cs

* [WinForms] Fix 12249 scroll orientation was not defined in ScrollEventArgs of DataGridView.cs
2019-11-11 22:59:53 +01:00
Jo Shields c256fc86b6 Delete some LLVM test cases from make dist (#17772)
Many of the tests from the llvm-project monorepo cause problems to `make dist`
2019-11-11 17:42:51 +01:00
Alexis Christoforides 722c6f7dd5 [netcore] Complete Monitor.LockContentionCount implementation (#17719)
* [netcore] Complete Monitor.LockContentionCount implementation

Fixes https://github.com/mono/mono/pull/16538
2019-11-11 13:03:03 +01:00
Jan Kotas 594bbbef25 Relax visibility
Signed-off-by: dotnet-bot <[email protected]>
2019-11-10 09:29:18 +01:00
Marek Safar d90030d273 Fixes build 2019-11-09 20:45:42 +01:00
Ryan Lucia 9664df7eb3 Move AssemblyLoadContext.GetResolvedUnmanagedDll to shared (#27775)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
rob313663 e46f29313d Fixed typos in comments. (dotnet/coreclr#27789)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
Joseph Musser 5bf661d7f9 StackTrace.GetFrames should return an array of non-nullable frames (#27787)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 09:37:25 +01:00
Levi Broderick ae02b6677b Improve Encoding.UTF8.GetString / GetChars performance for small inputs (#27268)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 08:13:11 +01:00
Alexander Köplinger f9b3b5276d Fix llvm path in CODEOWNERS file 2019-11-09 00:44:57 +01:00
Alexander Köplinger ed2d5f69e0 [netcore] Fix RuntimePropertyInfo.GetValue() in FullAOT scenarios (#17757)
It tried to use dynamic codegen which is not available in FullAOT.
We can use the `RuntimeFeature.IsDynamicCodeSupported` to check for this, added a JIT intrinsic which returns false in FullAOT.

Used a linker friendly way to check for TypeBuilder by adding an `internal virtual IsTypeBuilder` method in the `Type` class which returns false, and overwrote it in `TypeBuilder`.
2019-11-09 00:33:00 +01:00
Zoltan Varga 73b50890cc [wasm] Bump emscripten. (#17748)
* [wasm] Bump emscipten to 1.39.2.

* Disable some code in libtest.c which causes emscripten to fail.

* [wasm] Source emscripten env before make install as well.
2019-11-08 14:43:32 -05:00
Nathan Ricci 5b0a822afb Implement GC.GetGCMemoryInfo (#17607)
Implement the GC.GetGCMemoryInfo API.
2019-11-08 13:58:37 -05:00
Ryan Lucia d5b6cf33b3 [loader] Unmanaged library refactoring and NativeLibrary implementation (#17369) 2019-11-08 13:42:56 -05:00
Ryan Lucia 68d142912d [netcore] Run individual CoreCLR test suites (#17753)
This is implemented in a lazy way for convenience when manually testing until the tests are wired up to xunit
2019-11-08 13:14:06 -05:00
Alexis Christoforides 23f2baa71e [merp] Remove extraneous waitpid invocation (#17723)
The extraneous call basically guarantees that the call right after will fail. Fixes merp stating that its process has failed when it hasn't.
2019-11-08 11:40:04 -05:00
Jo Shields 3948a065af Merge pull request #17749 from directhex/dotnet-llvm-monorepo
[LLVM] Change llvm submodule to dotnet-org fork of official LLVM git repo
2019-11-08 10:42:20 -05:00
lateralusX fad91a1c1e Fix Windows LLVM build using new repository structure. 2019-11-08 09:21:30 -05:00
Dotnet-GitSync-Bot 759a8c1847 Mirror changes from mono/corefx,coreclr,corert (#17761)
* Add EventWaitHandle creation extension method that takes an ACL (#42213)

* Add EventWaitHandle creation extension method that takes an ACL

* Call OpenExisting, add basic unit tests, move MAX_PATH in csproj for all Windows platforms, ensure same exceptions are thrown in both netcore and netfx.

* Address suggestions: using for the created handle, simplify try finally, merge methods, remove an unnecessary Debug.Assert

* simplify using, reorganize PInvokes a bit, let VS format resx as it expects it, remove BOM from csproj

* Add more unit tests

* Save basic access rights in a constant, generate name randomly

* Use FullControl (EVENT_ALL_ACCESS); update unit tests to use EventWaitHandleRights

* Apply suggestion of creating an EWH, then replacing its SWH.

* Remove using causing chaos, add more unit tests, remove null string unit test.

* spacing

* Address comments

* Remove unnecessary checks/exceptions, update unit tests

* Fix netfx x86 ut failure - different exception for mode validation.

* Small documentation fix

* Remove documentation for exception thrown with null security

* more documentation fixes

* New file for unit tests

* Dispose and remove windows only attributes in tests

* suggestion to not use var

* Remove duplicate resx after merge

* Readd cs files lost during merge

* Remove resx modification since it was untouched for this PR

* Address test comments

Signed-off-by: dotnet-bot <[email protected]>

* Move NativeLibrary.Free null check to managed (dotnet/coreclr#27744)

Signed-off-by: dotnet-bot <[email protected]>

* Arm64: Add S.P.CoreLib instrinsics UnzipEven and UnzipOdd (#27731)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 07:11:13 -05:00
Tamar Christina 771b5fb18d Arm64: Update S.P.CoreLib for ZipLow and ZipHigh (dotnet/coreclr#27730)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 11:01:17 +01:00
Carlos Sanchez Lopez 229d3cc097 Add Semaphore creation extension methods that take an ACL (#42377)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Semaphore with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Semaphore, then set its ACLs. This presents a few problems:
- Potential security hole as semaphores can be accessed between creation and modification.
- Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Semaphore and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 01:59:07 +01:00
Zoltan Varga c0a9d6bf76 [aot] Improve the aot mangler a bit, handle bool/char as a primitive type and avoid emitting a System prefix. (#17739) 2019-11-07 18:18:12 -05:00
Tanner Gooding 8637912a99 Updating NumberToFloatingPointBitsSlow to handle the remaining parsed fractional digits being zero. (#27688)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 23:39:21 +01:00
Jo Shields 4d579d00ff Bump LLVM submodule 2019-11-07 15:52:56 -05:00
Tamar Christina b924ffa5e0 Arm64: Add S.P.CoreLib for AddV. (dotnet/coreclr#27663)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 21:47:22 +01:00
Ryan Lucia 2c22d0de2e [netcore] Propagate ALCs through reflection functions (#16949)
* [reflection] Pass assembly load context through mono_reflection_get_type_with_rootimage

The entry points are mono_reflection_type_from_name_checked and
mono_reflection_get_type_checked which for now pass in an ambient ALC, but
should be updated to take an ALC argument.

Internally mono_reflection_get_type_with_rootimage is recursive through a half
dozen helper functions which all get an ALC argument too.

The principle place where the ALC is used is in
_mono_reflection_get_type_from_info which passes it to
mono_assembly_loaded_internal and to replace the call to mono_assembly_load by
a direct call to mono_assembly_request_byname with an explicit ALC set in the
request.

* Mark mono_assembly_load as external only

* Pass an alc through non-static reflection functions

* Fix usage in wasm debugger

* Relax assertions due to wasm
2019-11-07 15:02:23 -05:00
Marcos Cobeña Morián 8cc0402967 Enabled --preload-files without AOT (#17732) 2019-11-07 15:00:36 -05:00
Jo Shields fe69f5b699 Fix .gitmodules typo 2019-11-07 14:03:06 -05:00
Jo Shields c1c483cef8 LLVM changes for SDKs 2019-11-07 12:16:29 -05:00
Jo Shields d0d779b8d9 Update AzDo pipeine for new LLVM location 2019-11-07 12:09:58 -05:00
Alexander Köplinger 71b78536ad [sdks] Add xunit to iOS test runner and add results reporting (#17744)
- Allows the on-device runner to work with the xunit framework in preparation for netcore.

- Added a simple UI to show the test run progress

- Report the test result XML back to the host so we can process it on Jenkins/AzDO
2019-11-07 18:09:51 +01:00
Jo Shields b61263e994 [llvm] redirect from mono llvm-mirror fork to dotnet llvm-project fork 2019-11-07 12:03:29 -05:00
Jo Shields 9e6b57ce5a delete old llvm 2019-11-07 11:47:32 -05:00
Aleksey Kliger (λgeek) 2c56ac54c0 [profiler] Fix log profiling of native to managed wrappers (#17740)
* [profiler] Add test case for pinvokes and reverse pinvokes

* [profiler] Fix crash in native to managed wrappers

When we call back to managed from a pinvoke, we're in GC Safe mode and the
profiler instrumentation is inserted in the wrapper before it does the
transition to GC Unsafe.  So we need an extra transition before the profiler
calls its locking functions

Fixes https://github.com/mono/mono/issues/17687

* [coop] Mark internal GC Unsafe functions with MONO_PROFILER_API

Allow the profiler to transition to GC Unsafe mode

* [profiler] ptestrunner.pl fix profiler modules directory

we used to specify mono/mini/.libs but that's not where the profiler plugins live, they're in mono/profiler/.libs.   We probably picked up the profiler plugins from the system Mono install.
2019-11-07 11:42:52 -05:00
Thays Grazia 4bdacea237 [embed] Assert when call mono_runtime_object_init (#17589)
* Fix assert when calling externally call mono_runtime_object_init, mono_object_get_domain, mono_string_to_utf8
* Adding samples/embed as unit-test
2019-11-07 11:22:25 -03:00
Carlos Sanchez Lopez 3c84633c7c Add Mutex creation extension methods that take an ACL (#42281)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Mutex with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Mutex, then set its ACLs. This presents a few problems:

Potential security hole as mutexes can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Mutex and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 12:27:50 +01:00
Elinor Fung 0886ddb3a0 Populate properties of AssemblyLoadStart/Stop events (dotnet/coreclr#27558)
* Populate properties of AssemblyLoadStart/Stop events
* Add tests for basic start/stop binder tracing events
* Add property for ALC of requesting assembly

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 09:54:23 +01:00
Jay Krell 7b4b5e72d9 Remove handles from ves_icall_System_Array_InternalCreate. (#17681)
Remove handles from ves_icall_System_Array_InternalCreate.

Also combine two allocas.
2019-11-07 07:36:07 +01:00
Zoltan Varga 0680073b87 [runtime] Fix locking in mono_get_seq_points (). (#17738)
Fixes https://github.com/mono/mono/issues/17737.
2019-11-06 19:50:59 -05:00
Johan Lorensson bda239f44c Switch away from Start-Process, Wait-Process in build.ps1. (#17730)
Running Start-Process, Wait-Process on msbuild.exe have showed a couple
of hangs in the build script. This is most likely due to msbuild.exe
spawned child processes (like compile servers) running longer than
spawned msbuild.exe process. Since there is no specific requirements
when spawning the process from build.ps1, switch to regular process
exec and LastExitCode.
2019-11-06 16:51:31 +01:00
Johan Lorensson 1a6c092019 Explicit update/init only LLVM BTLS repro on external MSVC build. (#17691)
Reduce the scope of git submodule update --init to only LLVM and/or BTLS
repositories when building external MSVC dependencies.
2019-11-06 16:51:14 +01:00
Marek Safar 965beadc1e [System.Private.CoreLib] Remove unused Array methods 2019-11-06 12:46:30 +01:00
Stephen Toub e506008ebb Add Span.Sort, and make Array.Sort span-based (dotnet/coreclr#27700)
* Add Span<T>.Sort

Shares existing Array-based implementation by changing that implementation to be span based.

* Additional work to enable span-based sorts

- Cleans up changes from the previous commit, e.g. corrects nullable annotations, removes TODOs, using GetRawSzArrayData instead of GetRawArrayData, etc.
- Passes spans around rather than spans+lo+hi.
- Deletes the native TrySZSort, preferring to use the managed implementation in all cases.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-06 12:46:30 +01:00
Bernhard Urban-Forster 29ba75c154 Bump Corefx (#17694)
* Bump Corefx

New commits in mono/corefx:

* mono/corefx@c44efe7297 [mono] limit DegreeOfParallelism to 16 (#369)

Diff: https://github.com/mono/corefx/compare/6e65509a17da898933705899677c22eae437d68a..c44efe7297f8e11cab157c7e96b998dc936d53b7

* another bump
2019-11-06 07:36:17 +01:00
Jay Krell 1e8e357831 Handles reduction -- 4 MERP functions. (#17711)
Handles reduction -- 4 MERP functions.
2019-11-06 03:56:10 +01:00
Jay Krell aa506ca1bb error_init reduction. (#17712) 2019-11-05 16:38:40 -05:00
JesperTreetopandStephen Toub 104abd7fe7 Prevent DivideByZeroException in EventCounter.ToString() (#27581)
* Prevent DivideByZeroException in EventCounter.ToString()

Maybe it should print something other than 0. Maybe "Mean" should be excluded. But at least with this it doesn't divide by zero. Ternary operator in interpolated string is not super readable; didn't know if making the code more descriptive would have performance implications.

* Update src/System.Private.CoreLib/shared/System/Diagnostics/Tracing/EventCounter.cs

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 21:18:22 +01:00
Ryan Lucia f477bce548 [loader] Skip the full pinvoke resolution process for __Internal (#17701) 2019-11-05 14:35:00 -05:00
Tanner Gooding 3d4f44d85c Fixing the return type of the ArmBase.ReverseElementBits methods in the PlatformNotSupported file (#27672)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 20:20:17 +01:00
Bernhard Urban-Forster 93b97d5582 [mini] Fix Coverity CID 1455161 & 1455162 (#17615)
* [mini] Fix Coverity CID 1455161 & 1455162

> Null-checking "ji" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.

* do not rely on assert to guard ji-nullness.  refactor a bit
2019-11-05 19:09:32 +01:00
Johan Lorensson 6f73c2dab8 Fix MSVC intellisense for LLVM sources. (#17706) 2019-11-05 13:16:32 +01:00
Johan Lorensson 44e6226c31 Fix correct accumulated time for parallelized jobs in SGEN. (#17518)
* Fix correct accumulated time for parallelized jobs in SGEN.

Current implementation incorrectly accumulate time for scan jobs for
the follow metrics when running parallel minor GC:

time_minor_scan_major_blocks,
time_minor_scan_los,
time_major_scan_mod_union_blocks,
time_major_scan_mod_union_los

Commit fix so updates are atomic making sure values are correct in cases
where parallel minor GC is used.

* Implement 64-bit atommic add fallback in SGEN.
2019-11-05 01:57:06 -05:00
Nikolay Sivov 95743afa80 [mono] Fix ProcessExit handler argument. (#17680)
Problem was spotted with mono tracing hitting ProcessExit handler,
and crashing on first argument.
2019-11-05 01:56:43 -05:00
Egor Bogatov fdc6495a54 [netcore] Intrinsify Buffer.IsPrimitiveTypeArray (#17438)
* Improve IsPrimitive intrinsic

* Implement it for interpreter

* clean up

* clean up
2019-11-05 01:50:37 -05:00
Zoltan Varga ebb4598096 [wasm] Build the tests with --no-native-strip. (#17698) 2019-11-04 17:46:52 -05:00
Vlad Brezae b3b9960ed1 [interp] Use GetType instrinsic also on net4x (#17654)
We didn't do this because it had some issues when GetType was invoked through reflection. This issue would also be present with jit but, because jit doesn't use direct wrappers, the GetType call can't be intrinsified in that scenario. See https://github.com/mono/mono/pull/5350 for the context of the issue.
2019-11-04 12:58:38 -05:00
Dotnet-GitSync-Bot db49be3cbb Update only S.P.CoreLib for ReverseElementBits (dotnet/coreclr#27582) (#17695)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 12:55:57 -05:00
Egor Bogatov cf8495e2f7 [netcore] Copy Array.Clear impl from CoreCLR (#17411)
* Address feedback

* fix build error

* Fix has_reference usage in object.c

* Fix fallback impl

* Rename to ArrayHasReferences

* Rename to ObjectHasReferences

* Rollback to ArrayHasReferences

* Rename to ObjectHasReference

* remove redundant is_array variable

* ObjectHasReferences can accept any object

* undo changes in Program.cs

* undo changes in Program.cs
2019-11-04 12:48:04 -05:00
Filip Navara 07fa0bd3b4 Intrinsify Activator.CreateInstance<T> for value types with no ctor (#17688)
* Intrinsify Activator.CreateInstance<T> for value types with no ctor

* Fix failed test, address feedback
2019-11-04 12:40:13 -05:00
Zoltan Varga 929a8a7fd8 [jit] Allow Unsafe.As<TFrom, TTo> on gsharedvt types. (#17692) 2019-11-04 18:14:59 +01:00
Bernhard Urban-Forster 237f210ea2 [interp] use mask instead of bool expression (#17683)
Discovered by Coverity:
```
>>>     CID 1455205:    (CONSTANT_EXPRESSION_RESULT)
>>>     The expression "flag && 2" is suspicious because it performs a
        Boolean operation on a constant other than 0 or 1.
```
2019-11-04 15:56:46 +01:00
Alexander Köplinger 49d5c4d14d [sdks] Use Xcode 11.2 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-11-04 15:27:42 +01:00
Tarek Mahmoud Sayed 6630bcc67f Fix Buggy code in TimeZoneInfo (dotnet/coreclr#27591)
Although couldn’t reach the corner case cause executing the buggy code, the fix here is done according to the similar code in TimeZoneInfo which show the intention of the change

https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1638
https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1568

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 09:59:54 +01:00
Jan Kotas f3e30a0ad0 Delete two more FCalls (dotnet/coreclr#27640)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas deb3115b15 Use simple Array.Copy overload where possible (dotnet/coreclr#27641)
The simple Array.Copy overload does not come with a performance penalty anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas 6ecf9e87ba Consolidate *.Fast.cs (dotnet/coreclr#27644)
System.Memory OOB package that this split was originally done for does not exist anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Johan Lorensson c0e237b734 Mono NetCore Windows only build/test. (#17646)
Support for Windows Mono NetCore build/test wihtout any need for
cygwin/wsl/automake just MSBuild and MSVC. Windows build support mirror
technologies used by coreclr/aracade builds msbuild + powershell.

Windows only Mono NetCore build, makes a port of existing build.sh and
Makefile, implementing a build.ps1 and build.targets. So far full build
support + execution of corefx test suite is implemented. Other make file
rules can be added when needed.

MSVC build runtime seems to hits a couple of additional test failures
in corefx test suite that needs to be investigated further.

Building NetCore Mono on Windows can now be done without cygwin or wsl.
Build depends on regular VS install (or build tools), if xunit summary
is requested, python needs to be installed.
2019-11-04 08:55:36 +01:00
Egor Bogatov 137ffe8b7c [netcore] Improve Array.CreateInstance (#17673)
* Rewrite Array.CreateInstance

* Rewrite Array.CreateInstance

* Address feedback

* Convert the icall to NOHANDLES

* fix build errors, remove NOHANDLES

* Fix GC.KeepAlive usage

* Remove icall definition for GC.KeepAlive (it's no an icall)

* Fix failing tests (expect details in the AOORE)
2019-11-03 22:39:19 -05:00
Filip Navara 8131518bb1 [netcore] Improve default constructor lookup, (#17666)
* [netcore] Improve default constructor lookup, cache when no default constructor is found (common for struct)

* Address PR feedback
2019-11-03 18:28:32 +01:00
Levi Broderick 8700c04b02 Fix typo in Tuple.cs (dotnet/coreclr#27637)
Reported by https://github.com/dotnet/corefx/issues/42321

Signed-off-by: dotnet-bot <[email protected]>
2019-11-03 18:17:23 +01:00
Alexander Köplinger 4888c49c26 [netcore] Disable shared compilation for System.Private.CoreLib
There's a dotnet process hanging around which seems to lock the build when called from the SDKs Makefiles.
2019-11-03 15:43:40 +01:00
Alexander Köplinger 40c1cd3576 [netcore] Fix strong name key for System.Private.CoreLib
It's using the silverlight key: https://github.com/dotnet/coreclr/blob/34fe045a27e150bde7ee54e0d5f0df635922519d/src/System.Private.CoreLib/System.Private.CoreLib.csproj#L112
2019-11-03 15:43:40 +01:00
Alexander Köplinger d89b96a298 [netcore] Fix out of tree build for netcore/ folder 2019-11-03 15:43:40 +01:00
Jan Kotas ad385f98b3 Normalize trailing whitespaces (dotnet/coreclr#27626)
Normalize trailing whitespaces in frequently changing files (docs and sources)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-02 16:51:03 +01:00
Bernhard Urban-Forster 62ee01b36b [interp] s/MONO_API_ERROR_INIT/error_init_reuse/g (#17650)
[interp] s/MONO_API_ERROR_INIT/error_init_reuse/g
2019-11-02 16:45:36 +01:00
Zoltan Varga 803c657b24 [jit] Call mono_class_setup_fields () before accessing field->offset. (#17667)
Fixes https://github.com/mono/mono/issues/17665.
2019-11-02 03:57:25 -04:00
Manuel de la Pena 36539ed67f [iOS] Match changes done in xamarin-macios in the SDK runtime. (#17664)
* [iOS] Match changes done in xamarin-maciosn in the SDK runtime.

* Address reviews.
2019-11-02 01:40:40 -04:00
Jay Krell 4b9dc08ac1 Move MonoError from managed wrappers to native wrappers. (#15894)
Move MonoError from managed wrappers to native wrappers.

Move MonoError from managed wrappers (ilgen) to native wrappers (C preprocessor).

Alternative to https://github.com/mono/mono/pull/15869.
Which says: Previously, we would emit a call to memset
since the struct is large (> 100 bytes). Only the error code
field needs to be initialized.
Which that PR and this PR both fix, in different ways.

In the past:
- This did not work due to a possible dependency on precise GC,
  interacting with what gets zeroed or not here.
  That is why I put it on the managed side.
  We'll see if that reproduces.
  (https://github.com/mono/mono/pull/11294/files#diff-efe0070415f3f0c0f24ed0221aa1962aR339
  external/coreclr/tests/src/CoreMangLib/cti/system/weakreference/weakreferenceisaliveb.exe.)

- Some indication that embedding API should expose MonoError
  and not set_pending_exception. This does not seem relevant
  presently, and the code seems better structured and
  more efficient this way. It can revisited if/when new public API
  is the dominant concern, or do it this way for these functions,
  and another way for actual public functions.
  (https://github.com/mono/mono/pull/11294#issuecomment-432443485)
2019-11-01 22:36:06 +01:00
Zoltan Varga e9d68e9485 [netcore] Cleanups. (#17661)
* [netcore] Remove the type_info field from RuntimeType, extend the TypeCache structure instead.

* [netcore] Remove some linker exclusions which are not needed.
2019-11-01 12:52:45 -04:00
Steve Pfister b9f185b74a Bump mono/corefx@6e65509 (#17631)
Pulls in PR https://github.com/mono/corefx/pull/368
2019-11-01 11:04:27 -04:00
Jo Shields 16d1a2e6f1 [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. (#17628) 2019-11-01 10:26:08 -04:00
monojenkins a4790c892d [csproj] Update project files 2019-11-01 14:25:26 +00:00
monojenkins cc28706e58 Bump API snapshot submodule 2019-11-01 14:24:11 +00:00
Vlad Brezae 166fdf60b0 [interp] Handle remoting field access same as jit (#17641)
Doing this we avoid using slow ldrmfld in the common case of loading fields from instance methods, when we execute on the real object. We could do this also for strmfld but we need to track on the stack if the instance object is this, which is not worth the trouble.
2019-11-01 14:55:58 +02:00
Vlad Brezae 170d33c35b [interp] Optimize call path (#17623)
* [interp] Optimize void calls

By avoiding setting of retval

* [interp] Avoid computing param count for every call

* [interp] Remove wrong profiler check

From the times when interp had its own EH mechanism. We raise this event from mono_handle_exception_internal for interp.

* [interp] Remove profiling check code from method exit

We emit MINT_PROF_EXIT opcode when under profiler. Unify profiling enter/exit with tracing enter/exit since they are very similar.
2019-11-01 14:55:27 +02:00
Filip Navara 8aa3654de5 Lower min/max SSE operations on x64 (#17536)
Follow up on #17511.
2019-11-01 13:30:41 +01:00
Zoltan Varga dc8d55d446 [llvm] Fix a case where we treated the dreg of a store_membase instruction as a dreg, its actually the base reg. (#17648) 2019-11-01 03:43:42 -04:00
Steve Pfister 9566027142 Modify run-jenkins.sh to adapt wasm build for the OSX CI lane (#16785)
* Modify run-jenkins.sh to adapt wasm build for the OSX CI lane

* Round 2... Go

* Make ENABLE_WASM_DYNAMIC_RUNTIME conditional on non-OSX

* Add runtime-threads target

* Ignore 'multiple rules generate' on osx

* Fix typo

* Remove blank line

* Exclude CoreXml.Test.XLinq.FunctionalTests.EventsTests.EventsReplaceNodes.XElementReplaceNodes

* Revert "Ignore 'multiple rules generate' on osx"

This reverts commit 5010721bd710d6955aaee47aa7f49ad7845763fc.

* Check if OSX lane can follow general build flow
2019-11-01 01:44:50 -04:00
Stephen Toub 8b86349c2c Disable another static analysis rule (dotnet/corefx#42265)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 22:23:35 +01:00
Bernhard Urban-Forster 0be756d609 [threadpool] cache processor count (#17642)
On top of https://github.com/mono/corefx/pull/369 this improves the execution time of `System.Core-xunit` on Linux/ARM64 by 2x, so from:

```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 131.143s
```
to
```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 74.636s
```

This is only relevant for non-netcore. The CoreCLR folks just recently fixed something similar (thanks to Marek sharing this link): https://github.com/dotnet/coreclr/pull/27543
2019-10-31 22:08:14 +01:00
Vlad Brezae b484979aa9 [interp] Avoid emitting MINT_SAFEPOINT for every single call (#17625) 2019-10-31 16:49:16 +01:00
Jonathan Chambers 3648cb087b Avoid caching of System.dll image and types as they may be unloaded with appdomain. (#17602) 2019-10-31 10:13:01 -04:00
Zoltan Varga dff84a2464 [wasm] Fix xunit test ninja errors. (#17639)
* [wasm] Error out on duplicate assemblies in the packager.

* [wasm] Switch search path order to avoid duplicate references to assemblies.

* Copy all xunit binaries to test_lib_dir.

* Revert "Copy all xunit binaries to test_lib_dir."

This reverts commit b15b03c16b508c04dc6121196fddca0e6d949e2f.
2019-10-31 10:10:51 -04:00
Alexander Köplinger ca72deed17 Fix build 2019-10-31 14:36:53 +01:00
Ben Adams c9b7f767a8 Don't needlessly refresh proc count (dotnet/coreclr#27543)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 14:36:53 +01:00
Vlad Brezae 1d043a31e7 [interp] Fix interp logging (#17636)
* [interp] Don't print NOPs during cprop

They can become numerous and distracting.

* [interp] Fix mono_interp_print_code

Regressed after code_size was changed to byte count instead of short count.
2019-10-31 15:32:50 +02:00
abrevet-dev 6b3aa3a05b [WinForms] Change HotkeyPrefix default value in TabControlPainter.cs (#17579)
The default behavior on .NET Framework is _HotkeyPrefix.None_ when creating a _TabControl_ element (ampersands not escaped). 
If an user wants to override this behavior, he has to declare a _DrawItemEventHandler_ (with _DrawMode = OwnerDrawFixed_).

Fixes #16206
2019-10-31 13:48:12 +01:00
Stephen Toub a27aa7afd0 Update analyzer versions (dotnet/corefx#42249)
* Update analyzer packages to latest NuGet versions

* Fix new warnings

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Stephen Toub 6ae3360342 Clean up Uri.UnescapeDataString (dotnet/corefx#42225)
- Use string.IndexOf rather than an open-coded, unsafe loop.
- Avoid an unnecessary SequenceEquals at the end: we're only at this point if a `%` was found highlighting that something escaped was found.
- Use stack memory for smaller inputs if possible, to avoid unnecessary ArrayPool interaction
- Remove an unnecessary argument to a helper function.
- Fix ValueStringBuilder.Grow to only copy the contained data.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Egor Chesakov a94dda5b5a Add BitwiseSelect (dotnet/coreclr#27430)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 08:51:46 +01:00
Zoltan Varga dc76a95aa9 [runtime] Add a --enable-minimal=threads configure option to disable threading support. Use it on wasm. (#17611) 2019-10-31 01:33:50 -04:00
Zoltan Varga 5811404e55 [eglib] Assert in g_strlcpy () if src/dest is NULL, instead of printing a warning. (#17507)
Related to https://github.com/mono/mono/issues/17283.
2019-10-30 16:07:44 -04:00
Martin Baulig 578e0df42f [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. 2019-10-30 15:37:25 -04:00
Jo Shields 27921fbbd0 Rearrange System.Private.CoreLib for easier monorepo integration (#17517)
* [netcore] Move corlib around to match CoreCLR directory layout better

This is a prerequisite to making a sane migration of this whole
directory from mono.git to coreclr.git
2019-10-30 15:09:06 -04:00
Thays Grazia fffb1201bd [dim][regression] Explicit interface override (#17583)
* Precedence order:
-> Override interface method in class
-> Method in class
-> DIM
2019-10-30 14:47:15 -03:00
Egor Chesakov feca5e042d Add ComputeCrc32, ComputeCrc32C (dotnet/coreclr#27421)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Egor Chesakov 744d77ea7f Remove unsigned overloads of LeadingSignCount (dotnet/coreclr#27533)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Martin Baulig c6f71167a1 Bump CoreFX to pickup corefx PR #367 to fix #17133. (#17590) 2019-10-30 12:32:21 -04:00
Bernhard Urban-Forster 95cad24526 [interp] fix signature mismatch between jit<>interp for string constructor (#17595)
[interp] fix signature mismatch between jit<>interp for string constructor
2019-10-30 16:36:04 +01:00
Marek Safar e4c0b50dec [System.Private.CoreLib] Remove NativeRuntimeEventSource 2019-10-30 16:07:43 +01:00
Jan Kotas c9b88b45fe Move runtime specific methods out of the runtime partition
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 16:07:43 +01:00
Jay Krell 10ac2750e2 Inline TLS access. (#17119)
Inline TLS access.

This attempt differs from the previous in that the TLS accessor functions are presented to far less code, i.e. not to main.c, so their data references do not need to be resolved so much.

See https://github.com/mono/mono/commit/bffd4c6b99213a0b154d7efac9b0581df67775e8 etc.
2019-10-30 16:06:24 +01:00
Zoltan Varga 79f5e6c288 Remove some unused icalls. (#17612) 2019-10-30 10:08:50 -04:00
Filip Navara 3d397805d4 [netcore] Report errors on Windows CI (#17616)
* [netcore] Report errors on Windows CI

* Fix Windows build
2019-10-30 14:39:47 +01:00
Zoltan Varga 2823ac5d95 [wasm] Print a useful error message instead of a signature mismatch error on missing icalls. (#17600) 2019-10-30 00:04:15 -04:00
Dotnet-GitSync-Bot 3695d8704e Mirror changes from mono/coreclr,corefx (#17609)
* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp() (#27473)

* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp()

* Insert GC_POLL before statement with unmanaged call.

* JIT test for insertion of GCPoll

Signed-off-by: dotnet-bot <[email protected]>

* Add file creation method that takes an ACL (dotnet/corefx#42099)

Approved API Proposal: #41614
Related change for directory creation method that takes an ACL: #41834 -merged and ported to 3.1 Prev2

Description
We have extension methods in System.IO.FileSystem.AclExtensions that let the user get and set ACLs for existing files, but we do not have methods that create files with predefined ACLs.
.NET ACL (Access Control List) support is Windows specific. This change will reside inside the System.IO.FileSystem.AccessControl assembly.

Customer impact
Before this change, customers had to create a file or filestream, then set its ACLs. This presents a few problems:

Potential security hole as files can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
Stability issues with background processes (file filters) can prevent modifying ACLs right after creation (typically surfaces as a security exception).
This change addresses those problems by adding a new extension method that allows creating a file and ensuring the provided ACLs are set during creation.
This change is expected to be backported to 3.1.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 00:03:47 -04:00
Zoltan Varga 9adce0b632 [wasm] Fix build problems. (#17596)
* [runtime] Add a stub for MacOsIPInterfaceProperties::ParseRouteInfo icall on non-osx platforms to fix the wasm aot build.

* [wasm] Avoid generating invalid ninja build files if an assembly is specified on the command line and its also an implicit reference.

* Put back and ifdef.

* Fix the windows build.
2019-10-30 00:03:08 -04:00
Filip Navara d49d3efd5d Temporary build fix 2019-10-29 23:34:11 +01:00
Noah Falk 67ff64a381 Support large EventSource filter args (dotnet/coreclr#27500)
1. Fix NullReferenceException. When the filter args exceeded the
hard-coded size limit GetDataFromController would return data = null.
The code previously asserted that data was not null and triggered NRE
when it was. The fix correctly null checks the value instead of
asserting and uses an empty args dictionary in this case, the same as if
filter args had been empty to begin with.

2. ETW has always limited filter args to 1024 bytes but EventPipe has no
such restriction. When using DiagnosticSourceEventSource it can be
useful to specify a larger filter arg blob. I can't do anything about
ETW's restriction but there is no need for the runtime to force
EventPipe to be equally limited. The larger size also reduces the chance
that we need to hit the fallback path above causing filter args to be
ignored.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 8a03f60897 Cleanup (dotnet/coreclr#27502)
* Delete empty FireCustomerDebugProbe FCall

* Delete duplicate C++ compiler command line options

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Elinor Fung 17cef4c4f9 Basic infrastructure for binder tracing (dotnet/coreclr#27383)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub 249522036d Fix StreamReader to pass cancellation token into ReadBufferAsync (#27464)
ReadAsyncInternal is correctly passing its token into the two call sites where it delegates directly to the underlying stream, but in one place it calls through ReadBufferAsync, which is currently not defined to take a token.  Fix that, and pass the token through.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 447cb03399 Delete unnecessary abstraction from stubbed out SecurityElement (#27474)
The stubbed out SecurityElement implementation does not support extensibility that ISecurityElementFactory was designed for.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Dan Moseley 9aeef958c6 Add path to not-absolute-path exception (dotnet/coreclr#27470)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Filip Navara db02ebc88b Fix building portable thread pool on Windows (dotnet/corert#7849)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub f26e73b49b Move PreserveDependencyAttribute to shared (dotnet/corefx#42174)
We need to use it in corelib, too.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Bernhard Urban-Forster d274cfbacd [mini] Initial tiered compilation work (#17551)
[mini] Initial tiered compilation work

Enable it with `./autogen.sh --enable-experiment=tiered`.

Let's consider `Simple.cs`:
```csharp
using System.Runtime.CompilerServices;
using System;

public class Simple {
    public static void Main (string []args) {
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
    }

    static int cnt = 0;

    [MethodImplAttribute (MethodImplOptions.NoInlining)]
    public static void HotMethod () {
        for (int i = 0; i <= 1000; i++)
            cnt += i;
    }
}
```

```console
$ csc Simple.cs
$ MONO_LOG_LEVEL=debug MONO_LOG_MASK=tiered ./mono/mini/mono-sgen --trace=M:Simple:HotMethod --interp=-all Simple.exe
[0x10ec275c0: 0.00000 0] ENTER:i Simple:HotMethod ()()
Mono: tiered: queued Simple:HotMethod ()
[0x10ec275c0: 0.00010 0] LEAVE:i Simple:HotMethod ()(
Mono: tiered: patching 0x7fe855803224 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x34 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
Mono: tiered: patching 0x7fe8558031f2 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x2 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
cnt: 500500
[0x10ec275c0: 0.04093 0] ENTER:c Simple:HotMethod ()()
[0x10ec275c0: 0.04095 0] LEAVE:c Simple:HotMethod ()(
cnt: 1001000
```
Note the suffix after `ENTER:`
* `i` indicates it's executed by the interpreter
* `c` indicates it's a compiled method (JIT)

Another example:
```console
$ make -C mono/mini gshared.exe
$ ./mono/mini/mono-sgen --interp=-all --stats ./mono/mini/gshared.exe
Regression tests: 84 ran, 0 failed in Tests
[...]
Tiered statistics
Methods promoted                    : 68
```

It's a basic proof-of-concept for now. An incomplete list of future work items:

* Right now it only works for direct calls, need to expand it to virtual/interface calls.
* Calls of the JIT leading into the interpreter again can't be patched yet.
* Kind of related, no concept of versioning compiled methods does exist yet. The interpreter maintains its own table of "transformed" methods, however, when doing the transition from JIT->interpreter, the wrapper+interpmethod will end up in the JIT table. We need a way to have multiple JIT compilation results for the same MonoMethod exist.
* Investigate actual performance. We might have to optimize the interp<>JIT transition.
* Patching is racy. Need to make that atomic.
* All the FIXMEs in this PR



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-29 22:36:01 +01:00
Alexander Köplinger 1340095581 [acceptance-tests] Bump mono/roslyn to get build fix
See https://github.com/mono/roslyn/commit/6bfcd4e770609a38b8523e83ce1ec943162f30bb
2019-10-29 20:27:09 +01:00
Zoltan Varga b0d966bd7a [eglib] Handle dli.dli_sname being NULL in g_module_address (). (#17592)
Should fix the warnings/assertions in g_strlcpy ().
2019-10-29 09:21:23 -04:00
Aniruddha Shastri 7f91972ded Support platform-independent newlines in line wrapping in Mono… (#17431)
* Add newLine parameter to StringCoda methods

* Update Options.cs to handle both kinds of newlines

* Add test to OptionSetTest

* Directly handle \r\n instead of using the TextWriter.NewLine
2019-10-29 13:53:41 +01:00
dotnet-maestro[bot] cddf90dda4 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17577)
* Update dependencies from https://github.com/dotnet/arcade build 20191027.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19527.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19527.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191027.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19527.2

* Update dependencies from https://github.com/dotnet/corefx build 20191027.5

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19527.5

* Update CoreFX.issues.rsp

GetGCMemoryInfo was moved to GCExtendedTests in https://github.com/dotnet/corefx/commit/d6dac9e6a7875862b91139334c8ec620f599cd4d.

GetTotalAllocatedBytes and GetAllocatedBytesForCurrentThread seem to pass now.
2019-10-29 12:51:33 +01:00
Zoltan Varga e4cc70f4b5 [runtime] Implement support for SuppressGCTransitionAttribute. (#17534)
Fixes https://github.com/mono/mono/issues/17367.
2019-10-28 10:59:52 -04:00
Johan Lorensson c5edcd1ed4 Additional and adjusted logging for parallel minor GC. (#17522)
* Additional and adjusted logging for parallel minor GC.

When running parallel minor GC we currently get incorrect timing
on several loggings and counters due to work being put on thread pool
and accumulated. This commit fix so that the logging is accurate
when running with and without parallelization. In parallel mode a new
logging is also added to show the accumulated split between major and los
remset scan jobs.

The counters time_minor_scan_remsets and time_minor_scan_roots are only
updated when not running parallel GC since they are not correct when
running parallel GC.

* Fix WASM build error.
2019-10-28 15:34:42 +01:00
Alexander Köplinger 450d474684 [netcore] Fix offsets generation when cross compiling
The offsets-tool.py uses some magic to map `typename` to `<underscore>typename` for looking up runtime types.
This caused issues on netcore because we aliased it to a different name.

To fix this let's keep the `_MonoInternalThread` name instead and just fix the one location where `_MonoThread` is used.
2019-10-28 13:57:17 +01:00
Alexander Köplinger fb7b075e49 [netcore] Don't load types in aot-compiler.c that don't exist in corefx
Fixes https://github.com/mono/mono/issues/16026
2019-10-28 13:57:17 +01:00
dotnet-maestro[bot] 6f04054541 Update dependencies from dotnet/corefx (dotnet/coreclr#27431)
* Update dependencies from https://github.com/dotnet/corefx build 20191024.13

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.13
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.13

* Update dependencies from https://github.com/dotnet/corefx build 20191024.14

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.14
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.14

* Update dependencies from https://github.com/dotnet/corefx build 20191025.6

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19525.6
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19525.6

* Workaround https://github.com/dotnet/coreclr/issues/27465

Signed-off-by: dotnet-bot <[email protected]>
2019-10-27 17:54:12 +01:00
Alexis Christoforides db568c6fe0 [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping (#17537)
* dump_native_stacktrace should always register the sigterm handler.

Cherry picked from the https://github.com/mono/mono/pull/17310

* [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping

The primary reason for this is gaining the ability to interpret SIGTERM properly as a crashing signal when outside of 'dump mode', and as a utility signal during the 'thread summarizer' dumping process.
2019-10-26 18:48:40 -04:00
Aleksey Kliger (λgeek) 0e3caf00df [runtime] Unbalanced GC Unsafe transitions before shutdown (#17566)
When embedders call mono_runtime_quit or mono_jit_cleanup, we need to do unbalanced transitions to GC Unsafe, because after mini_cleanup runs we don't have GC thread states anymore and MONO_EXIT_GC_UNSAFE will assert.

* [runtime] Do an unbalanced GC Unsafe transition in mono_jit_cleanup

After mini_cleanup, we don't have GC thread states anymore because all that
stuff was cleaned up, so MONO_EXIT_GC_UNSAFE would assert

* [runtime] Mark mono_runtime_quit external only.

Runtime should use mono_runtime_quit_internal.

After we call the quit_function (aka mini_cleanup) we don't have any GC thread
states anymore because all that stuff got cleaned up.  So MONO_EXIT_GC_UNSAFE
can't work.
2019-10-26 18:45:35 -04:00
Ben Adams 2da4e54a7f Avoid mod operator when fast alternative available (dotnet/coreclr#27299)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 19:07:15 +02:00
Jay Krell 5f3446bd12 [cxx][x86] int/ptr casts (#17570)
[cxx][x86] int/ptr casts
2019-10-26 18:26:15 +02:00
Dotnet-GitSync-Bot c5e3ca90de Mirror changes from mono/coreclr (#17571)
* Use ValueStringBuilder instead of StringBuidler{Cache} in multiple places

Signed-off-by: dotnet-bot <[email protected]>

* Avoid unnecessary allocations in Exception.ToString

Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 07:48:46 -04:00
Dotnet-GitSync-Bot fec907397d Mirror changes from mono/coreclr,corert,corefx (#17565)
* Update Corelib to include Stopwatch (dotnet/coreclr#27417)

* Update Corelib to include Stopwatch

* Delete HighPerformanceCounter

Signed-off-by: dotnet-bot <[email protected]>

* Share completed Task between Task and AsyncTaskMethodBuilder (#27437)

Signed-off-by: dotnet-bot <[email protected]>
2019-10-25 21:23:46 -04:00
Caleb Cornett 1801bdad6a Add --preload-file packager option 2019-10-25 14:53:15 -05:00
Leonard 4ec9bd03e5 Fixes #17190: SerializationException on ListViewItemCount (#17195)
* Fix issue #17190: SerializationException on ListViewItemCount

ListViewGroup doesn't (always) have a serialized member "ListViewItemCount".

This patch accounts for that and also adds support for the "ItemsCount"
property that upstream [1] seems to use.

Also items should be initialized regardless of the count, otherwise we get a NPE.

[1] https://github.com/dotnet/winforms/blob/master/src/System.Windows.Forms/src/System/Windows/Forms/ListViewGroup.cs#L144
2019-10-25 19:30:08 +02:00
Kenneth Pouncey 1bd76c6069 Spacing 2019-10-25 12:19:54 -05:00
Kenneth Pouncey f1a3f3c58d Add CompileFunction to Runtime that returns a WebAssembly.Core.Function
- The code passed to `CompileFunction` is not a function definition. Instead it must create and return a function instance.
- Solves the problem of accessing `Node.js` functionality as defining a JavaScript function executes in the global space and does not get `Node.js` access.
- Returning a function instance allows the function to initialize and maintain encapsulated state.
2019-10-25 12:19:54 -05:00
Jay Krell 893486638c [cxx] Compile mini-llvm.c as C++ if configure -enable-cxx. (#17558)
[cxx] Compile mini-llvm.c as C++ if configure -enable-cxx.

This is presently catastrophic, because of `MonoJitICallId` varying between C and C++, and therefore the layout of `MonoCallInst`.

Related to:
https://github.com/mono/mono/pull/17557

https://github.com/mono/mono/pull/17445#issuecomment-546175352
2019-10-25 17:46:01 +02:00
Jay Krell 84d64d314b [jit][x86][cxx] int/ptr casts (#17382) 2019-10-25 11:34:50 -04:00
Aleksey Kliger (λgeek) d6ee245c94 [runtime] Make mono_thread_manage external only (#17554)
* [runtime] Make mono_thread_manage external only

runtime should use mono_thread_manage_internal
2019-10-25 11:28:18 -04:00
Vlad Brezae 768fefbad1 [interp] Optimize stelem (#17519)
* [interp] Optimize stelem instructions

* [intepr] Optimize stelem_ref and slowpath casts

mono_object_isinst_checked uses coop handles and is very slow. Attempt to use the straightforward mono_class_is_assignable_from_checked.

Makes stelem_ref 5x faster.
2019-10-25 16:25:08 +02:00
Egor Bogatov dc4a7f5ed7 [llvm] Use -place-safepoints in JIT mode too (#17116)
* always use -place-safepoints pass in JIT mode

* ignore loaded llvm

* remove gc.safepoint_poll after optimizations

* fix typo

* cleanup

* cleanup

* cleanup

* cleanup

* Do not emit safepoints on entry thus simple functions (without calls and loops) won't have them at all

* Undo  -spp-no-entry=true

* fix failing tests

* do not emit sp in Allocators
2019-10-25 16:10:57 +02:00
Kenneth Pouncey 6ac749ed20 [wasm][bcl] Fix System.Diagnostics.Trace.TraceError crash. (#17509)
* Fix `System.Diagnostics.Trace.TraceError` crash.

- WASM does not have access to the command line arguments through the browser so the length is 0 causing index out of range error.
- The zero index of the command line arguments is what decides the `AppName` so under WASM the AppName will be blanks.

* Address review comments

* Remove #ifdef for WASM only.

- Addresses review note.
- As per channel discussion some embedding scenarios might be the same.
2019-10-25 11:17:00 +02:00
Zoltan Varga 9d781eb78c [wasm] Propagate exit code from Main in the test runner. (#17559)
* [wasm] Propagate exit code from Main in the test runner.

* Disable some System tests as they currently fail.
2019-10-25 04:59:30 -04:00
Dotnet-GitSync-Bot 8800bfdf9c Fix build break (#17553)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 18:30:09 -04:00
Jay Krell 4ef7c566af [cxx] externC for some icalls, for wasm and ios. (#17444)
Maybe all JIT icalls?
2019-10-24 17:59:11 -04:00
Jay Krell fc390a5f24 Include signal.h unconditionally. (#17430)
Some of the includes already were unconditional, including both on Win32 and Unix.
It is part of ANSI C 89, albeit in a reduced form compared to typical Unix (no sigaction).

Remove configuration of MSG_NOSIGNAL which has no other uses.

Preserve configuration of signal.h temporarily due to other uses of ifdef HAVE_SIGNAL_H.
 i.e. LLVM?
2019-10-24 17:44:57 -04:00
Jay Krell ec81a868a1 [cxx][x86] int/ptr casts. (#17441)
* [cxx][x86] Another int/ptr cast.

* More casts. Notice these are under
`#if defined (MONO_ARCH_USE_SIGACTION) && !defined (MONO_CROSS_COMPILE)`
which would explain them not showing up.
2019-10-24 17:37:12 -04:00
Dotnet-GitSync-Bot aea7214eb4 Optimize GC.AllocateUninitializedArray and use it in StringBuilder (#27364) (#17550)
* use GC.AllocateUninitializedArray for allocating internal char buffers

* force inlining of AllocateUninitializedArray to have no perf hit on the small buffers hot path

* insrease the threshold from 256 to 2048 bytes

* use Unsafe.As instead of a cast

* remove the size precondition from AllocateNewArray method, the called AllocateSzArray is responsible for handling negative size

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:02:11 -04:00
dotnet-maestro[bot] a7287bdae5 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17458)
* Update dependencies from https://github.com/dotnet/arcade build 20191018.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19518.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19518.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191021.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19521.2

* Update dependencies from https://github.com/dotnet/corefx build 20191020.7

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19520.7

* Update dependencies from https://github.com/dotnet/arcade build 20191023.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19523.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19523.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191023.10

- Microsoft.NETCore.App - 5.0.0-alpha1.19523.10

* Update dependencies from https://github.com/dotnet/corefx build 20191023.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19523.8

* Disable a file system watcher test on mac

See https://github.com/mono/mono/issues/17547

* Reenable tests that were fixed
2019-10-24 21:06:46 +02:00
Bernhard Urban-Forster 25b53a642b [interp] fix code length for JitInfo (#17538)
* [interp] fix code length for JitInfo

* fix c++ build
2019-10-24 20:13:27 +02:00
Tanner Gooding 0595c0e6d7 Move down S.N.Vector APIs and expose new conversion support (#27401)
* Updating S.P.Corelib to start building Vector2/3/4 and supporting types

* Removing HAS_INTRINSICS usages from Matrix4x4

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 14:10:07 -04:00
Marek Safar 81a4a19f0a [netcore] Fix corerun build deps 2019-10-24 14:09:14 -04:00
Martin Baulig 1553889bc5 Cleaning up SslStream, MobileAuthenticatedStream and TaskToApm. (#17393)
### Overview

After the removal of the Legacy TLS Provider, we can make some cleanups and improvements to the `SslStream` class.  Since both the remaining providers use the internal `MobileAuthenticatedStream` and support Task-async code, we can get rid of some of the Begin/End async logic and the the underlying
`Stream` class handle it instead.

This also fixes a nasty issue with some of those task objects leaking unobserved exceptions.  The problem was that `SslStream.Dispose()` needs to clear out all resources, but in doing so also nulls out the `Impl` handle to the `MobileAuthenticatedStream` - so a subsequent `End*()` call will not reach it's corresponding `TaskToApm.End()` method.

There are also some cleanups in the internal APIs that were only ever used internally and not intended for public consumption.

### Mono.Security.dll

* `Mono.Security.Interface`: removed internal `ICertificateValidator2` interface.

* `Mono.Security.Interface.IMonoSslStream`: this internal interface is only used by the web-tests, which only use it as a way to get the `SslStream` instance, so all those `Begin/End` async methods are now gone.

* `Mono.Security.Interface.IMonoSslStream2`: removed another internal interface.

* `Mono.Security.Interface.MonoTlsProvider`: removed some `internal abstract` methods; these have been moved into a new `Mono.Net.Security.MobileTlsProvider` inside `System.dll`.

### System.dll

* `Mono.Net.Security.MobileTlsProvider`:
  New abstract internal base class, which received the aforementioned abstract internal methods.  The naming is such to make it consistent with the already existing `MobileTlsContext` and `MobileAuthenticatedStream`.

Since both `AppleTlsProvider` and `BtlsProvider` now have an abstract internal base class inside `System.dll`, they can use the `ChainValidationHelper` class directly and we can also avoid some `Mono.Security` dances like for instance that `MonoSslPolicyErrors` enum.

This will also help make the code easier to read and maintainer.
  
* `Mono.AppleTls.AppleTlsProvider` and `Mono.Btls.MonoBtlsProvider`: change base class into `MobileTlsProvider` and reflect above mentioned overload changes.

* `Mono.AppleTls.AppleCertificateHelper`: use `ChainValidationHelper` as well.

* `Mono.Net.Security.ChainValidationHelper`: only implement `ICertificateValidator`.
  The `ICertificateValidator` interface is still required because it is in use by the web-tests, but the previous `ICertificateValidator2` was an internal dance.

* `Mono.Net.Security.MobileAuthenticatedStream`: largely cleaned up this class.
  - we now implement the current slimmer version of `IMonoSslStream`.
  - moved all those overloads into `SslStream` that only fill in default values for some of their arguments and then call another overloaded version of their own.
  - removed lots of `Begin/End` methods that are not used anymore.

* `Mono.Net.Security.MonoTlsProviderFactory`: use `MobileTlsStream` everywhere.

* `Mono.Net.Security.MonoTlsStream`: cleanup `SslStream` creation logic; instead of going throw `provider.CreateSslStream()`, we can now use an internal constructor directly, thus avoiding the `IMonoSslStream` interface while also making the code a lot easier to read and understand.

* `System.Net.Mail.SmtpClient`: same here.

* `System.Net.HttpWebRequest`: use `MobileTlsProvider`.

* `System.Net.Security.SslStream`: largely cleaned up this class.
  - we now use `MobileAuthenticatedStream` directly instead of the `IMonoSslStream` interface and therefore could also get rid of `IMonoSslStream2`.
  - cleanup constructor logic to allow the internal constructor being used with a `MobileTlsProvider` instead of going through `provider.CreateSslStream()`.
  - all those overloads that are only filling in default values for some of their arguments before calling another overloaded version of their own now live here instead of in the `MobileAuthenticatedStream` class.  The reason is to simply that `MobileAuthenticatedStream` is already quite a complex and difficult class, and removing some of that complexity out of it will make it easier to understand.
  - handle the `TaskToApm.Begin()` / `TaskToApm.End()` here to ensure that we are not leaking any task objects with unobserved exceptions.
  - removed `BeginRead()` and `BeginWrite()` overloads to let the underlying `Stream` class handle these.


* Bump API snapshot submodule

* [csproj] Update project files
2019-10-24 19:42:54 +02:00
Calvin Buckley 638e2c6a86 Fix eglib on AIX/PASE (#17476)
* fix eglib on AIX/PASE

	* Work around access(2) syscall bug on PASE
	  On PASE, X_OK will always returns true. This is contrary to
	  how AIX behaves (though corresponds to documentation!). This
	  is probably because it's an upcall to the ILE version of
	  access, and that seemingly always returns true.

	* Always export the dummy function in the test suite
	  The AIX linker didn't export this properly, without using the
	  giant sledgehammer of -bexpall. Write an export file and use
	  it. This required some build system changes to expose an
	  AIX-only automake conditional.

	* Use a file that exists on PASE for tests
	  PASE doesn't use Unix-style DNS resolution, so /etc/hosts is
	  missing. Use /etc/magic, since that seems to be present.

	* Don't use /bin on AIX/PASE for tests
	  This is a symlink to /usr/bin, and it seems chdir/getcwd
	  follows links. Use /usr instead, that is unlikely to be a
	  symlink.

	* Make a note what directory we're in if we fail test_cwd

* Change manual executable access check change per Jay

* Change test on POSIXy platforms to always use root directory
2019-10-24 18:04:20 +02:00
Vlad Brezae b7ef736338 [interp] Add super instructions for field storing (#17525)
* [interp] Add also floating versions for stloc.np

* [interp] Add super instructions for stfld

When storing into a field of a local or an argument (ldloc/ldarg -> random instructions that end up pushing one value -> stfld). We add this instruction to the cprop pass because the instructions that are optimized together are not consecutive and we need the stack information to be able to access the instruction that loaded the argument / local and to make sure that the local wasn't dirtied. Alternatively we could make the super instruction pass stack aware, which we should probably do if more super instruction candidates turn out to require stack state information.

* [interp] Enable cprop for methods without locals

We do some optimizations here that don't apply to locals.

* [interp] Fix result stack type of CEE_CONV_OVF_I_UN
2019-10-24 18:05:04 +03:00
Vladimir Sadov 6fd07e4745 Moving StopWatch to src\Common\src\Corelib (dotnet/corefx#42072)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:54:28 +02:00
Egor Bogatov 7952e73780 Bump to mono/corefx@fb41040
Includes upstream corefx changes that makes NTLM auth work for more scenarios.
2019-10-24 10:04:24 -04:00
Bernhard Urban-Forster 861812252a [lldb] another fix for handling interp frames (#17535)
[lldb] another fix for handling interp frames

And improved error message in case it breaks again.
2019-10-24 14:07:22 +02:00
Dotnet-GitSync-Bot 2c8e0f68c1 Add AdvSimd (LeadingZeroCount, LeadingSignCount, PopCount) (#27382) (#17532)
* AdvSimd.LeadingSignCount

* AdvSimd.LeadingZeroCount

* AdvSimd.PopCount

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 07:16:49 -04:00
Zoltan Varga 951769b279 [netcore] Add a few more missing SSE intrinsics. (#17511)
* [netcore] Add a few more missing SSE intrinsics.

* Disable some intrinsics for non-llvm.
2019-10-24 06:57:13 -04:00
Johan Lorensson fe7654f745 Adjust restart world logging level to match start major/nursery logging. (#17520)
With this change we will get the following logging on level 1:

2019-10-22 15:46:59 Start nursery collection 38898 00000205A0000000-00000205A1000000, size: 16777216
2019-10-22 15:46:59 restarted (pause time: 2728 usec, max: 689233 usec)

showing the pause time and when runtime was restared, before this fix
we will only see the start log message on level 1.
2019-10-24 04:57:55 -04:00
Filip Navara 7a15b71873 [netcore] Managed ThreadPool implementation (#17387)
* Wire up the managed side of the portable thread pool

* Remove old native thread pool code

* Fix non-netcore build

* Fix Thread.IsThreadPoolThread

* Fix MSVC builds

* Code style fix

* Make the locking primitives uninterruptible

* Revert icall changes for non-netcore

* Implement Thread.ResetThreadPoolThread

* Compile out more thread pool code for netcore

* Remove ThreadPoolBoundHandle, it is shared now

* Address PR feedback on code style
2019-10-24 04:56:25 -04:00
Dotnet-GitSync-Bot 2dfd682ac9 Enable pooling for async ValueTask/ValueTask<T> methods (#26310) (#17531)
Today `async ValueTask/ValueTask<T>` methods use builders that special-case the synchronously completing case (to just return a `default(ValueTask)` or `new ValueTask<T>(result))` but defer to the equivalent of `async Task/Task<T>` for when the operation completes asynchronously. This, however, doesn't take advantage of the fact that value tasks can wrap arbitrary `IValueTaskSource/IValueTaskSource<T>` implementations.

This commit gives `AsyncValueTaskMethodBuilder` and `AsyncValueTaskMethodBuilder<T>` the ability to use pooled `IValueTaskSource/IValueTaskSource<T>` instances, such that calls to an `async ValueTask/ValueTask<T>` method incur 0 allocations (ammortized) as long as there's a cached object available.

Currently the pooling is behind a feature flag, requiring opt-in via the DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKS environment variable (setting it to "true" or "1").  This is done for a few reasons:
- There's a breaking change here, in that while we say/document that `ValueTask/ValueTask<T>`s are more limited in what they can be used for, nothing in the implementation actually stops a `ValueTask` that was wrapping a `Task` from being used as permissively as `Task`, e.g. if you were to `await` such a `ValueTask` multiple times, it would happen to work, even though we say "never do that". This change means that anyone who was relying on such undocumented behaviors will now be broken. I think this is a reasonable thing to do in a major release, but I also want feedback and a lot of runway on it.
- Policy around pooling. Pooling is always a difficult thing to tune. Right now I've chosen a policy that limits the number of pooled objects per state machine type to an arbitrary multiple of the processor count, and that tries to strike a balance between contention and garbage by using a spin lock and if there's any contention while trying to get or return a pooled object, the cache is ignored. We will need to think hard about what policy to use here. It's also possible it could be tuned per state machine, e.g. by having an attribute that's looked up via reflection when creating the cache for a state machine, but that'll add a lot of first-access overhead to any `async ValueTask/ValueTask<T>` method.  For now, it's tunable via the `DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKSLIMIT` environment variable, which may be set to the maximum number of pooled objects desired per state machine.
- Performance validation.  Preliminary numbers show that this accomplishes its goal, having on-par throughput with the current implementation but with significantly less allocation.  That needs to be validated at scale and across a variety of workloads.
- Diagnostics.  There are several diagnostics-related abilities available for `async Task/Task<T>` methods that are not included here when using `async ValueTask/ValueTask<T>` when pooling.  We need to evaluate these (e.g. tracing) and determine which pieces need to be enabled and which we're fine omitting.

Before shipping .NET 5, we could choose to flip the polarity of the switch (making it opt-out rather than opt-in), remove the fallback altogether and just have it be always on, or revert this change, all based on experimentation and data we receive between now and then.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 03:09:50 -04:00
Kenneth Pouncey 1b5658d590 [wasm][sdk] Bump SDK version to 0.2.2 (#17516)
* [wasm][sdk] Bump SDK version to 2.2

* Update SDK version for browser tests
2019-10-24 07:37:28 +02:00
Dotnet-GitSync-Bot e5d88dbd04 Mirror changes from mono/coreclr,corefx,corert (#17526)
* Apply SuppressGCAttribute to some SPCL functions. (dotnet/coreclr#27369)

Apply SuppressGCAttribute to some SPCL functions.

Only insert GC_POLL in first morph call.

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused methods (dotnet/coreclr#27370)

Signed-off-by: dotnet-bot <[email protected]>

* Move Matrix3x2/4x4, Plan, and Quaternion to the shared CoreLib partition (#42021)

Signed-off-by: dotnet-bot <[email protected]>

* Remove Socket.InnerSafeCloseSocket (dotnet/corefx#41888)

* Remove Socket.InnerSafeCloseSocket

* Move IsInvalid checks in SetHandle function

* PR feedback

* Remove unnecessary 'SocketPal.'

* Remove SafeSocketHandle.SetHandle

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 23:01:00 -04:00
Maxim Lipnin 43a938fefb [wasm][xunit tests] Exclude some System.Data tests (#17527) 2019-10-23 22:55:56 -04:00
Thays Grazia 8fda1738f7 [debugger] Changing how debugger handles exception when hybrid suspend is enabled (#17478)
* Removing all fixes to fix #15203 and #17083, and trying a new approach to fix this #17084, together.

The thing is when we are executing on hybrid suspend the caller is executed inside try catch in the runtime_invoke_ method that is generated. And this try catch should be ignored for the debugger exception, otherwise it's the last try catch in main thread, but should always be considered to generate the stack trace.

Fixes #17084

* This test check the case that we were missing. On preemptive this test always runs perfectly, on Hybrid, never run well until this PR.

* Changing how call unit test.
Changing variable name.
2019-10-23 17:56:58 -03:00
Jay Krell 4d7adf4d2a [cxx] Put extern "C" in MONO_LLVM_INTERNAL. (#17443) 2019-10-23 15:36:35 -04:00
Johan Lorensson df30c7228c Optimize LOS for better parallelization. (#17173)
Scanning LOS list as part of minor GC didn't parallelize well since
all scan jobs needed to walk the complete LOS list. This led to touching
a lot of memory walking the list, increasing minor GC pause times
when the number of items in the LOS list increased. As an example,
stressing LOS data structure using ~600 MB's of random sized byte arrays
caused minor GC pause times between ~20ms without parallelization and ~19ms
with parallelization. NOTE, the more work each job gets (more memory to scan)
we will get a better parallelization, but still a lot of room for improvement.

Changing from a normal linked list to a SgenArrayList makes it more effective
to parallelize the scan jobs and since the list is using sequential
memory, this will also reduce cache misses for each thread iterating LOS
objects. Tagging objects in SgenArrayList with reference information reduce
cache misses dramatically in cases where objects not including references
exists on the list, since scanning jobs can skip these items without any
need to touch object memory causing cache misses.

Same scenario running with fix gives us minor GC pause times, without
parallelization in ~7ms (down from 20~ms) and with parallelization,
< 5ms (down from 19ms). NOTE, the more work each job gets (more memory to scan)
we will get a better result using parallelization, so the delta will increase
between using and not using parallelization, this scenario primarily stress
the extreme case where most LOS objects doesn't have references.

In total, this optimization reduce minor GC pause time 4x, from ~19ms when
using parallelization down to < 5ms.
2019-10-23 14:58:04 -04:00
Zoltan Varga 31b475e25a [wasm] Avoid adding an extra arg to native-to-managed wrappers. (#17508)
[wasm] Avoid adding an extra arg to native-to-managed wrappers.

Fixes https://github.com/mono/mono/issues/17389.




<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-23 17:06:18 +02:00
Jay Krell 8b8b8b5010 [coop] Fix big endian problem in MonoThradStateMachine and cleanup. (#17502) 2019-10-23 10:21:54 -04:00
Filip Navara e29f8e32b7 [netcore] Add Unsafe.SkipInit intrinsic 2019-10-23 09:26:09 -04:00
Tanner Gooding e73aa17e23 Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe (#27377)
* Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe

* Updating the runtime to handle Unsafe.SkipInit

* Updating Decimal.DecCalc to use Unsafe.SkipInit

* Removing 'workaround for CS0165' comments

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 09:26:09 -04:00
Marius Ungureanu bed1b39c6e Bump Bockbuild to pick-up gtk# binding change (#17486)
Fixes https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1005448
Which causes quite a few broken clipboard bugs and possible memory corruption

(cherry picked from commit 989333d6ae)
2019-10-23 14:36:46 +02:00
Kenneth Pouncey bc769e7376 [wasm] move jsvu repo back after fixes for v8 (#17514) 2019-10-23 13:34:36 +02:00
Kenneth Pouncey cdf8b839ae [wasm][binding] Add bindings for calling the entry point of an '.exe' (#17510)
- Add `mono_bind_assembly_entry_point`
- Add `mono_call_assembly_entry_point`
   - Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);

Example:

```
   Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);
```

```
   var main = Module.mono_bind_assembly_entry_point("WasmHttpStreamSync", []);
   main();
```
2019-10-23 12:43:16 +02:00
Vlad Brezae 479d2dff36 [interp] Add a new super instructions pass (#17489)
Together with a few very simple super instructions. We replace common instruction patterns to avoid indirection via interp stack, instruction dispatch overhead and to allow the compiler to generate better code for the instruction.
2019-10-23 11:57:07 +03:00
Vlad Brezae 4419dc919f [interp] Retry cprop pass if killing stloc (#17491)
When replacing a stloc with a pop, we should retry the pass, since it is very likely that we will be able to kill the instruction that pushed that value on the stack.
2019-10-23 10:39:27 +03:00
Jay Krell 92c08fab4d SetLastError slightly later, in case last error not preserved, and it is just as good. (#17483)
SetLastError slightly later, in case last error not preserved, and it is just as good.
2019-10-23 09:36:15 +02:00
Vlad Brezae 6159407846 [interp] Optimize ldelem instructions (#17494)
Remove the switch and have each instruction have its own code. Seems to improve speed on a benchmark that heavily uses ldelem by about 10-20%.
2019-10-23 10:23:49 +03:00
Zoltan Varga a2cd8b2495 [wasm] Enable threads again on CI, it seems to work again. (#17503) 2019-10-23 02:41:46 -04:00
Dotnet-GitSync-Bot b99a3e9514 Move Vector2/3/4 to the shared CoreLib partition (dotnet/corefx#42005) (#17505)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 00:36:20 -04:00
Filip Navara 1e60703a19 [netcore] Disable debugging code in lookup_intrins (#17464) 2019-10-23 00:01:51 -04:00
Aaron Robinson 11dca2a3b8 Add documentation for SuppressGCTransition (dotnet/coreclr#27368)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 21:30:52 -04:00
Zoltan Varga 745e5ba3c3 [wasm] Bump emscripten to 1.39.0. (#17484)
* [wasm] Bump emscripten to 1.39.0.

* [wasm] Fix threads build.
2019-10-22 20:09:07 -04:00
Larry Ewing d08713198f [wasm] fix package target 2019-10-22 18:19:19 -05:00
Larry Ewing 433d7c5004 [wasm] Bump DebugProxy to netstandard 2.1 / netcore 3.0.0 2019-10-22 18:19:19 -05:00
Larry Ewing 15d61d3f47 [wasm] work around jsvu / v8 changes 2019-10-22 15:50:18 -05:00
Tanner Gooding 69407517bc Adding back more arm intrinsics that existing before refactoring (#27153)
* Adding AdvSimd.And

* Adding AdvSimd.AndNot

* Adding AdvSimd.Or

* Adding AdvSimd.OrNot

* Adding AdvSimd.Xor

* Adding AdvSimd.Not

* Removing a trailing whitespace from AdvSimd.PlatformNotSupported.cs

* Adding AdvSimd.Subtract

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Filip Navara fdca1630dc Fix nullability error in portable thread pool (dotnet/coreclr#27366)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Jan Kotas b98911d4f8 Delete Uap quirks from CoreLib (dotnet/coreclr#27356)
This matches cleanup done in CoreFX.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Adam Sitnik 0663433389 remove double bound check from StringBuilder.Append(char) (#27340)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Aleksey Kliger (λgeek) fb5b2c949d [reflection] mono_method_get_base_method on a GTD should use canonical ginst (#17475)
* [test] GetCustomAttributes on generic type definition

* [reflection] mono_method_get_base_method on a GTD should use canonical ginst

The issue is in how we pass along the generic instantiation.  So there are two
parts here: we start with a method on a GTD which has to decompose into the GTD
and it's instantiation with its own generic parameters, and then we go up to
its parent and grandparent inflating and decomposing the resulting
instantiation. The instantiations keep passing the last parameter as the number
of parameters shrinks so that the gparam index from the original type is now
out of bounds for the parent and grandparent.  If we mess up, we'll get a
BadImageFormatException from the runtime

Fixes https://github.com/mono/mono/issues/17278
2019-10-22 14:07:44 -04:00
Maxim Lipnin bc9b64b807 Get resource string 2019-10-22 12:13:38 -05:00
Aleksey Kliger (λgeek) ebb5846e0d [coop] Use bitfields for coop state machine state representation (#17298)
* [checked] Mask the thread state in GC thread state assertions

Because we have extra bits (like suspend count and the no_checkpoints bit) on
the thread state, we have to mask it to get the actual thread state enum value

* [coop] Use bitfields to represent coop thread state

Instead of using a 32-bit int, use bitfields and type-punning through a union
type.  We need to type pun because we need to access the entire state as a 32-bit int so that we can do atomic CAS.

* use atomic load of thread state in unwrap_thread_state.

Saw one stack trace on CI on the full coop lane where check_thread_state saw STATE_RUNNING
but with a positive suspend count.  Another thread was starting a GC a
the time.  Seems like we saw the state as runnning, but the suspend
count as if we were already in async_suspend_requested.

Thread 22 (Thread 0x7f43c03ff700 (LWP 39860)):
2019-10-22 12:14:05 -04:00
Marek Safar f34fd20fbf [System.Private.CoreLib] Remove unused methods 2019-10-22 11:28:22 -04:00
Filip Navara cf1a9e0033 Update Mono for changes in shared CoreLib partition 2019-10-22 09:46:29 -04:00
Jan Kotas 64c3405594 Cleanup memory copy helpers (dotnet/coreclr#27307)
- Merge RuntimeImports that has just a few FCalls into Buffer.
- Delete assembly implementation of memcpy for ARM. The implementation was outdated and it was used only on Windows ARM and only for fraction of memcpy callsites.
- Delete unnecessary PInvoke into SysStringLen. We have a managed equivalent on System.Runtime.InteropServices.Marshal.
- Delete large amount of redundant code between SecureString.Windows.cs and SecureString.Unix.cs, and switch SecureString to use Spans
- Fold null check into IsWin32Atom

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas d4a2f01064 Rewrite Buffer.BlockCopy in C# (dotnet/coreclr#27216)
* Rewrite Buffer.BlockCopy in C#

Fixes #27106

* Workaround to enable type check optimizations for BlockCopy only

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Tanner Gooding 8ea0426488 Improve codegen for IsNan (dotnet/coreclr#27272)
* Updating double.IsNaN and float.IsNaN to just `return x != x`

* Updating x86 codegen to optimize `x != x` for floating-point

* Changing the isnan optimization to only check the operand assigned registers

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas 113d855adb Delete ENABLE_WINRT define (dotnet/coreclr#27240)
.NET Native specific code not needed anymore

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Ben Adams d5b182c6d8 Improve Dictionary TryGetValue size/perfomance (dotnet/coreclr#27195)
* Dictionary avoid second bounds check in Get methods

* Add NullRef methods to Unsafe

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 274db87e60 Fix DecoderNLS.Convert to out the correct value for 'completed' (#27210)
This is a complementary fix to https://github.com/dotnet/coreclr/issues/23020

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Del Myers 0e02fa55af Enable ETW/EventSource logging of task IDs for boxed state machines (#27115)
* Wrap MoveNext Action for TPL event tracing

Wraps the MoveNext action of the AsyncStateMachineBox in a
continuation wrapper when async causality tracing is on so that the
TPL event source can find the task that is associated with a
continuation. Does not wrap otherwise.

* Clarifying comment

* removing trailing whitespace

* code review feedback.

Makes AsyncMethodBuilderCore.TryFindContinuationTask check to
whether the target of a continuation is itself a task, as a fall-back
to checking to see if the continuation is a ContinuationWrapper

* Got rid of unnecessary null checks

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 5fdea0b84b Update Utf8String to match latest prototype (dotnet/coreclr#27046)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Stephen Toub d34ba480f9 Use stackalloc expressions with builder ctors (dotnet/coreclr#27194)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Aaron Robinson 296b8d40cf Conditionally remove the GC transition from a P/Invoke (#26458)
* Provide mechanism to remove the GC transition from a P/Invoke.
  New Attribute: 'System.Runtime.InteropServices.SuppressGCTransitionAttribute'

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas cd2388fd6f Replace primitive Array fcalls with managed implementations (#27123)
Prep-work for #27106

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Filip Navara 858fc11900 Move portable thread pool to shared partition (dotnet/corert#7828)
* Move portable thread pool to shared partition

* Rename ClrThreadPool to PortableThreadPool

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Dan Moseley 565b81e26f Annotate System.Resources.ResourceWriter for nullable ref types (#41880)
* Annotate System.Resources.ResourceWriter for nullable ref types

* ref

* Mark common as nullable enable explicitly

* _resourceList

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Eriawan Kusumawardhono f731db32dd Replace tfm based constants with SDK ones (dotnet/corefx#41601)
* replace DefineConstants with SDK ones

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Zoltan Varga b96260e603 [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now. (#17466)
* [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now.

* Fix checks so stack overflows work again.
2019-10-22 06:31:23 -04:00
Larry Ewing 20752df324 [debugger][wasm] Check the length of the vars array. Fixes #17473 2019-10-21 18:45:57 -05:00
Vlad Brezae c1461e4a2d [interp] Replace loading of static readonly fields with LDC (#17420)
LDC is faster and also allows for constant propagation. Cleanup code a little bit.
2019-10-21 23:42:24 +03:00
Jay Krell fd82c6036e No type_traits on Android.
In file included from /Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall.c:42:
/Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall-internals.h:89:10: fatal error: 'type_traits' file not found
2019-10-21 11:15:16 -05:00
Kenneth Pouncey 7e522ef179 Modify the check for streaming to also check that the browser Response object exists. 2019-10-21 10:27:05 -05:00
Jay Krell 21d7d41bfe [cxx][debug-interp] Cast once from void* to non-void*. 2019-10-21 10:22:43 -05:00
Kenneth Pouncey 2cea0a89d1 Fix host object existence check.
- The checking for the existence of the object is incorrect: Missing parens around undefined.
- Modify error description returned back to managed code.
- Fix capitalization in global object check error.
2019-10-21 10:21:16 -05:00
Thays Grazia 0d357c1347 [tests] Replace __WATCHOS__ with MONOTOUCH_WATCH (#17459)
Fixes the conditional from mono/mono#17423, we don't define __WATCHOS__ in the Mono BCL builds. We use MONOTOUCH_WATCH instead.
2019-10-21 16:20:12 +02:00
Kenneth Pouncey f07e93da2f [wasm] Make sure message pump is working when executing via NODEJS (#17453)
- Tests if we are running in ENVIRONMENT_IS_NODE
   - run message pump if we are.

- This also fixes GC not running.

- Clean up some messages.
2019-10-21 14:29:18 +02:00
Kenneth Pouncey 3ff90ac0c2 Export mono_load_runtime_and_bcl so that is available in nodejs. (#17456) 2019-10-21 13:50:57 +02:00
Jay Krell 9f50bdb2d9 [Coop] Unconvert Microsoft.Win32.NativeMethods. (#16992)
* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_CloseProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcessId.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetExitCodeProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessTimes.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_TerminateProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_WaitForInputIdle.
2019-10-21 10:29:39 +02:00
Zoltan Varga 4b9226619c [interp] Small fixes. (#17442)
* [interp] Really abort in ves_real_abort ().

* [interp] Fix stack usage in TRACE_EXIT.

* [interp] Add a callback to free ThreadContext.

* Fix warnings.

* Remove an empty statement.

* Fix a double free.
2019-10-20 23:28:25 -04:00
Filip Navara 32ab6bcf00 [netcore] Implement Array.GetElementSize as intrinsic (#17437)
* [netcore] Implement Array.GetElementSize as intrinsic

* Remove unnecessary icall

* alloc_preg -> alloc_ireg

* Implement Array.GetElementSize as intrinsic in interpreter

* Address PR feedback
2019-10-20 18:39:39 -04:00
Filip Navara 67c8d0d812 [netcore] Remove unused code for Array.Unsafe[Mov/Load/Store] intrinsics 2019-10-20 12:44:52 +02:00
monojenkins 5fcd00669c [csproj] Update project files 2019-10-20 12:43:59 +02:00
monojenkins b19d9aa658 Bump API snapshot submodule 2019-10-20 12:43:59 +02:00
Zoltan Varga 1d44139d01 Bump cecil. 2019-10-20 12:43:59 +02:00
Filip Navara bc678a2c10 Use correct variable in mini_get_cpu_features (#17436)
* Use correct variable in mini_get_cpu_features

* Address PR feedback
2019-10-20 01:26:44 -04:00
Filip Navara 91f10aeee4 Remove dead code left out in #17341 (#17433) 2019-10-20 01:23:05 -04:00
Jay Krell 91413f78f3 [jit][x86][cxx] int/ptr casts in mini-x86.c (#17380) 2019-10-19 19:20:56 +02:00
Jay Krell 84f4e1dcbf Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same, and it is only used once. (#17429)
Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same and it is only used once.
2019-10-19 13:25:54 +02:00
Thays Grazia 105ead8707 Changing behavior of some tests in WatchOs
Needs this ifdefs because of this https://github.com/mono/mono/pull/17260. (#17423)
2019-10-18 21:23:44 -04:00
Aleksey Kliger (λgeek) d8ae32a9ee [bcl] Add null reference checks to Interlocked.Exchange<T> (#17400)
* [bcl] Add null reference checks to Interlocked.Exchange<T>

and Interlocked.CompareExchange<T>

This is to mimic the old behavior in unmanaged when the intrinsic cannot be
used.

The issue was detected via the coreclr acceptance test:
https://github.com/mono/coreclr/blob/mono/tests/src/baseservices/threading/interlocked/exchange/exchangetneg.il

* and netcore
2019-10-18 19:11:22 -04:00
Jay Krell a2992d634b [cxx] [wasm] int/ptr casts esp. m2n-gen. (#17379) 2019-10-19 00:03:00 +02:00
Martin Baulig fffad7e3a8 Remove the Legacy TLS Provider. (#17391)
* `MonoTlsProviderFactory` now treats "legacy" as an alias for "default".

* `Mono.Net.Security/LegacySslStream`: Removed.  This class also wasn't using the `MobileAuthenticatedStream` base class, thus preventing cleanups simplifications in that area.

* `Mono.Net.Security/LegacyTlsProvider`: Removed.

* `Mono.Security/Mono.Security.Protocol.Tls*`: Removed all the old legacy code.

* `Mono.Security.Interface.MonoTlsProviderFactory`: Bump internal version to 4.
2019-10-19 00:02:48 +02:00
imhameed 8836231459 Merge pull request #17279 from imhameed/bug-gh-16935-netcore-equalsall-codegen
Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers.
2019-10-18 12:47:41 -07:00
Alex Rønne Petersen d44e0e80d7 Some updates to make RISC-V buildable again. (#17408)
Some updates to make RISC-V buildable again.

Also some updates to the toolchain setup documentation.
2019-10-18 20:16:14 +02:00
Kenneth Pouncey a0c7d73d6f [sdks] add .gitignore python __pycache__ (#17414)
This will ignore the __pycache__ files generated after the tools offset build.
2019-10-18 18:08:57 +02:00
Vlad Brezae 37f628750a [profiler] Fix coverage profiler on macos (#17396)
If we have coverage profiling enabled for a method (mono_profiler_coverage_instrumentation_enabled, with coverage_filter callback), then we should be able to allocate a coverage info structure (mono_profiler_coverage_alloc). After https://github.com/mono/mono/commit/52429673dc617fd437518757c04c7448211075e9, coverage_filter was reporting wrapper methods as ok to profile, while mono_profiler_coverage_alloc was refusing to allocate the necessary structure, leading to crashes.
2019-10-18 16:31:47 +02:00
Jay Krell 958b9b2658 [interp][wasm] Remove more varargs to conserve stack. (#17388) 2019-10-18 10:39:05 +03:00
Egor Bogatov 4908d1f19a Enable GSS on Linux and Android (#17223) 2019-10-17 19:01:44 -04:00
Alexander Köplinger e5721bbb23 [corlib] Mark YieldAwaitable struct readonly (#17386)
* [corlib] Mark YieldAwaitable struct readonly

Make it consistent with .NET Standard/Core which makes future API Compat runs report one less difference.

* Bump API snapshot submodule
2019-10-17 16:01:22 +02:00
Vlad Brezae ec653e70bf [interp] Constant folding for integers (#17362)
* [interp] Move instructions together

For easy check if an instruction is unary or binary operation. We consider conversion instructions to be unary operators and compare instructions to be binary operators.

* [interp] Add stloc.np also for longs

* [interp] Add stats for total number of emitted instructions

Also make the killed_instructions counter actually track the total amount of killed instructions. It was not previously tracking instructions killed as part of addition of stloc.np / movloc.

* [interp] Improve logging during cprop

Make dumping instruction support also InterpInst's not just compacted instructions.

* [interp] Constant folding of common unary operators

* [interp] Constant fold binary operations

* [interp] Remove MINT_CONV_U8_I4 opcode

It is equivalent with MINT_CONV_I8_U4.
2019-10-17 14:23:12 +03:00
Jay Krell e8b6fcc945 [cxx] Int vs. enum, static for efficiency, cleanup, fix typo. (#17377)
[cxx] Int vs. enum, static for efficiency, cleanup, fix typo.
2019-10-17 12:25:18 +02:00
Jay Krell e97e1f994d [cxx][x86] int/ptr casts. (#17345) 2019-10-17 12:24:46 +02:00
Vlad Brezae ae0fbd366e [interp] Always intrinsify MemoryBarrier method
As we do it on jit.
2019-10-17 12:53:19 +03:00
Vlad Brezae 0102263645 [interp] Avoid using full barriers for volatile operations
This commit avoid unnecessary memory barriers on x86 and amd64.

Volatile writes have release semantics, volatile reads have acquire semantics. Organize the code a little bit to indicate this. On x86 and amd64 we don't require barrier for acquire and release semantics since they are provided by the arch, so we can skip them. Ideally we should have 3 memory barrier interp instructions for each type, but, currently, they would all seem to call mono_memory_barrier so it is useless at this point.
2019-10-17 12:53:19 +03:00
Johan Lorensson 4f8a82c9fd Upgrade build scripts to support VS2019. (#17269)
Changes will default Windows SDK version as well as Platform Toolset to
the default versions used in the targeted VS version. If the projects are
opened up in VS2015, it should default to Windows SDK 8.1 and v140, but if
the same projects are opened in VS2019, it will default to latest Windows SDK 10
and v142. This way the project files should adapt to used VS version, meaning
that we could still build them using VS2015 (what's currently used on CI) but also
using VS2017 and VS2019. There should not be a need to install any previous versions
of build tools, unless an older version is targeted. It is also possible to set
PlatformToolset when calling msbuild and that should adapt to corresponding default
Windows SDK version for targeted toolset version.

Commit makes many changes and adjustments, aligning all vcxproj files but changes
should not affect build output.
2019-10-17 08:55:50 +02:00
Jay Krell edafa021d3 Use functions instead of macros for is_in/is_out. (#17350)
Use functions instead of macros for is_in/is_out.
2019-10-17 06:46:03 +02:00
Aleksey Kliger (λgeek) b6388e271f [bcl] Remove CompareExchange_T (#17370)
In https://github.com/mono/mono/pull/17341 we change the caller of
CompareExchange_T<T>(ref T, ...) to call CompareExchange (ref object, ...), but
didn't delete the declaration.

It's already deleted in unmanaged and in the netcore version of this file.
2019-10-16 21:11:36 -04:00
Maxim Lipnin 02bc95a51a [wasm][xunit tests] Disable System.IO.Compression.Tests.BrotliEncoderTests (#17351)
Fixes https://github.com/mono/mono/issues/17053
2019-10-16 18:47:16 -04:00
dotnet-maestro[bot] 1dbad7cd88 [master] Update dependencies from dotnet/core-setup dotnet/corefx (#17355)
* Update dependencies from https://github.com/dotnet/core-setup build 20191015.17

- Microsoft.NETCore.App - 5.0.0-alpha1.19515.17

* Update dependencies from https://github.com/dotnet/corefx build 20191015.14

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19515.14

* Reenable a test that was fixed
2019-10-16 23:12:15 +02:00
Jay Krell c0d50719ea [cxx][x86][amd64] Remove unused STORE_MEM_IMM. (#17347)
[cxx][x86][amd64] Remove unused STORE_MEM_IMM.

And there was an int/ptr cast error within.
2019-10-16 22:26:02 +02:00
Thays Grazia 3f9d501a78 Removing execution of network tests from WatchOs. (#17358) 2019-10-16 17:10:27 -03:00
Aleksey Kliger (λgeek) f017835ac1 [bcl][jit] implement Interlocked.Exchange<T> in terms of object (#17341)
* [bcl][jit] implement Interlocked.Exchange<T> in terms of object

Exchange<T> is supposed to be picked up as an intrinsic.  But if it isn't,
since T has a class constraint, call the `Exchange (ref object, ref object, ref
object)` overload instead.

This works around a limitation in the JIT:
https://github.com/mono/mono/blob/82a07273c22b996b08fa88ee2e6632d782ac2242/mono/mini/mini-trampolines.c#L704-L713

if we're in the common call trampoline, and the caller is generic method and
the callee is a generic method, if we're using generic sharing for the caller,
but there's no generic jit info for the callee, we will not patch the call
site.

In this case we had Exchange<T> calling Exchange_T<T>
where the callee is an icall (and evidently didn't have generic jit info).

So every time we called Exchange<T>, we would go through the trampoline when
trying to call Exchange_T<T> without patching the call site.

Addresses part of https://github.com/mono/mono/issues/17334

* Bump Mono

* Also drop CompareExchange_T<T>

* Sprinkle some Unsafe magic.

Mark the non-icall methods that we expect the JIT to treet as intrinsics with [Intrinsic]

* aot-runtime doesn't need Volatile.Read<T>/Write<T> code

Both of those methods are implemented as calls to the object overload of Volatile.Read/Write
2019-10-16 15:35:49 -04:00
Egor Bogatov 58b53299b7 remove GenericVectorTests from rsp (#17366) 2019-10-16 15:24:16 -04:00
Jay Krell de9ccfa63f [cxx][netcore] Goto around init. (#17348)
[cxx][netcore] Goto around init.
2019-10-16 20:56:09 +02:00
Jay Krell 7067524141 [cxx][x86] ifndef DISABLE_JIT around mono_arch_emit_prolog. (#17346) 2019-10-16 20:28:24 +02:00
Egor Bogatov c13f36bdfe fix netcore build with cygwin (#17361) 2019-10-16 14:20:48 -04:00
Egor Bogatov 886cd73784 [netcore] Implement System.IO.HasOverriddenBeginEndRead(Write)… (#17237)
* Implement HasOverriddenBeginEndRead and HasOverriddenBeginEndWrite

* fix typo

* clean up

* more clean up

* more clean up

* convert into intrinsics

* handle end* methods

* Update intrinsics.c

* make cache_System_IO_Stream_slots static

* fix race condition

* Rollback to icall

* Update icall.c

* Address feedback

* Address feedback
2019-10-16 20:16:51 +02:00
Marek Safar be6f43c80e [System] Add reflection dependency tracking
for built-in type converters
2019-10-16 20:05:03 +02:00
Alexander Köplinger cd6e574b87 [ci] Improve netcore build telemetry by running nupkg and tests through build.sh (#17349)
Follow-up to https://github.com/mono/mono/pull/17331.

We were running nuget pack and tests explicitly in the Azure Pipelines
config so we weren't capturing failure telemetry.
2019-10-16 18:52:13 +02:00
Cody Russell 37dc33a257 [GTK] Bump bockbuild for GtkViewport autoscrolling patch. (#17321) 2019-10-16 16:44:26 +02:00
Jay Krell a03027cab0 Replace mono_assembly_name_free use with mono_assembly_name_free_internal. (#16982)
Replace mono_assembly_name_free use with mono_assembly_name_free_internal.

Maybe just remove the GC mode change and remove the internal form.
We don't really equate malloc/free with GC/locks/hangs, though arguably we should. Arguably this change is right.
2019-10-16 14:25:55 +02:00
Vlad Brezae 0aabccef9b [interp] Add constant propagation of integers (#17326)
* [interp] Make StackValue more intuitive

Replace the opcode field with an enum which will represent what type of value is contained (STACK_VALUE_LOCAL, STACK_VALUE_I4 etc)

* [interp] Add the mint type to the local data

Makes the emitting code easier and we no longer compute the mint type for each local access, which is fairly expensive.

* [interp] Handle indirect locals saner during cprop

Instead of checking for the indirect flag when loading from a local, simply never store a StackValue into such a local since we can't use it. We use the same logic when not pushing on the stack values for such locals.

* [interp] Add constant folding for integers

We decode the values from LDC_I4 and LDC_I8 opcodes and track/propagate the values on the stack and in the locals using the existing infrastructure. When loading a local, we try to load using the known constant value instead, so we might get to kill that local.

* [interp] Similar to stloc.np we can optimize pop/push pairs

If the pushing instruction pushes the same value that was popped by the previous instruction, we can kill them both

* [tests] Remove issue that seems to have been fixed for a while
2019-10-16 13:56:14 +03:00
Jay Krell 7d3fc17e81 Fix C++ WebAssembly build. (#17302) 2019-10-16 10:12:41 +02:00
Filip Navara 0c02bb1207 Fix SafeHandle marshalling in ref/in/out parameters (#17330)
* Fix SafeHandle marshalling in ref/in/out parameters

* Reorder code to make it exception safe, update comments

* Revert the code reordering because it breaks the invariant that new handle returned from native code should always create new SafeHandle on the managed side

* Fix test_0_safehandle_ref test. The test was expecting incorrect behavior, verified on .NET Core 3 and .NET Framework 4.8.

* Address PR feedback

* Fix cut & paste error

* Fix build

* Really fix the test_0_safehandle_ref tests

* Add more tests for SafeHandle marshalling

* Add exclusion for broken CoreFX test
2019-10-16 01:43:42 -04:00
Zoltan Varga d4d93b9699 [wasm] Add a --native-lib option to the packager to allow linking additional native libraries. (#17344) 2019-10-16 01:31:03 -04:00
Kenneth Pouncey 2b4790a1f3 [wasm][tests] WASM Safari browser tests (#16816)
* Add support for tests via safari browser

* Run safari browser tests on CI for Mac lanes

* Update to fix syntax error using UNAME to `if [[ ${CI_TAGS} == *'osx-amd64'* ]];`

* Fix syntax error with `if`
2019-10-16 04:59:55 +02:00
Kenneth Pouncey 2ddcf8cf16 Set default StreamingEnabled property of the WasmHttpMessageHandler to be false. (#17328) 2019-10-16 04:39:06 +02:00
Zoltan Varga 27247739c6 [wasm] Bump emscripten. Remove generated python cache files. (#17340)
* [wasm] Bump emscripten to 1.38.48.

* Remove generated python cache files.
2019-10-15 21:57:52 -04:00
Zoltan Varga 08e8a7f2c3 [jit] Fix is_reference checks for intrinsics with byref parameters. (#17338)
Found by https://github.com/mono/mono/issues/17335.
2019-10-15 21:45:04 -04:00
Ryan Lucia 3fa5f6c66e [loader] Add an explicit define DISABLE_DLLMAP to control dllmap usage (#17315)
* [loader] Add an explicit define ENABLE_DLLMAP to control dllmap usage

* Update configure.ac

* Update configure.ac

* Add dllmap to enable-minimal

* Change to DISABLE_DLLMAP

* Grab some missing MonoDllMap uses
2019-10-15 19:40:24 -04:00
Bernhard Urban-Forster 73ae9dbc36 [bcl] add WriteLine(string) override to CStreamWriter needed due to corefx import (#17336)
In https://github.com/mono/mono/commit/fd424c5dd07b2dbbd4aafbf9165a42166d789438 we switched over to the `TextWriter.cs`/`StreamWriter.cs` implementation of corefx. One subtle difference is that an override for `WriteLine(string)` in `StreamWriter` was introduced:
https://github.com/mono/corefx/blob/8e3b279377d1f01d2d089f24548e33855bf6502e/src/Common/src/CoreLib/System/IO/StreamWriter.cs#L526-L531
Previously the definition in `TextWriter` was used, which would call the base case `Write(char[],int,int)`.  The new override however won't call the base implementation at all, but directly uses the `Span`-variant.

This is a problem for our `CStreamWriter` implementation, since `WriteLine(string)` won't end up in the special handling for `Write(char[],int,int)`, which accounts for terminal cursor changes.

This is the real fix for https://github.com/mono/mono/issues/16701

This also reverts commit 68f5613422. ("[TermInfoDriver] update top/left value on Console.{CursorTop,CursorLeft} access (#16745)")
While it kind of masked the issue, it also posed a performance issue on some terminals (e.g. GNOME Terminal).
2019-10-15 22:29:41 +02:00
Jo Shields dc0eea6fd1 Initial telemetry for netcore builds (#17331)
* Load VSTS telemetry helpers into build.sh script

* First cut adding some basic VSTS-compatible telemetry on failures

* Don't make `ci=true` the default, make it a build.sh flag
2019-10-15 22:21:21 +02:00
Alexander Köplinger 82a07273c2 [ci] Always initialize an AzDO variable
Otherwise the script will not replace the variable and we get confusing error messages like this:

```
[command]/bin/bash --noprofile --norc /home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh
/home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh: line 2: llvm: command not found
/home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh: line 2: [: =: unary operator expected
```
2019-10-15 19:44:03 +02:00
Alexander Köplinger 3b56247afd [ci] Only install android-specific brew packages when building for android 2019-10-15 13:59:34 +02:00
Steve Pfister 806f75fe4f Bumps corefx to mono/corefx@8e3b279 (#17307)
Resolves https://github.com/mono/mono/issues/17288 where RequestUri
in HttpRequestMessage did not allow file:// based Uri's the same
way its constructor did.
2019-10-15 11:06:24 +02:00
Ryan Lucia a2ef1274c3 Update codeowners
Get notifications for various files in metadata
2019-10-15 10:28:31 +02:00
dotnet-maestro[bot] 55e87320f0 Update dependencies from https://github.com/dotnet/corefx build 20191013.1
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19513.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 61db5aafdd Update dependencies from https://github.com/dotnet/core-setup build 20191014.1
- Microsoft.NETCore.App - 5.0.0-alpha1.19514.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 140750f543 Update dependencies from https://github.com/dotnet/arcade build 20191011.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19511.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19511.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 5a6fb5abe7 Update dependencies from https://github.com/dotnet/corefx build 20191008.3
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19508.3
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] f068c89ca2 Update dependencies from https://github.com/dotnet/core-setup build 20191008.2
- Microsoft.NETCore.App - 5.0.0-alpha1.19508.2
2019-10-15 09:50:26 +02:00
Filip Navara ac1d05ad40 Update CoreFX.issues.rsp
See https://github.com/mono/mono/pull/17232#issuecomment-541973035.
2019-10-14 19:46:39 -05:00
Filip Navara a194de4cdf [netcore] Avoid suspending threads in Environment.Exit, it can hang process (#17318)
CoreCLR doesn't suspend threads on exit (https://github.com/dotnet/corefx/pull/41450#issuecomment-536755093, https://github.com/dotnet/coreclr/pull/1813).

Fixes #17317 
Fixes #15006
Fixes #15010
Fixes #15181

Unblocks #17232
2019-10-15 00:35:26 +02:00
Alexander Köplinger 1a21ad7dd0 [ci] Restore previous values for no Xcode in CI_TAGS
It is still used by the non-pipeline PR builds (we should unify this at some point).
2019-10-14 20:38:46 +02:00
Filip Navara 76068d8d61 Remove broken sin/cos optimization for x86 2019-10-14 12:21:15 -05:00
Filip Navara 05df3970c2 Remove broken sin/cos optimization for x64 2019-10-14 12:21:15 -05:00
Kenneth Pouncey 44e99c5eec [wasm] Continue loading app even when .pdb files are not found (#17266)
* This adds the ability to continue loading the application even though the .pdb files are not found.

When uploading an app built with mono-wasm the `.pdb` may not get committed so they will not exist when trying to load them.

- Add new boolean variable to MONO library support to control skipping these files on errors.
	- `mono_wasm_ignore_pdb_load_errors` default is true

- Modify the assembly file fetching routine `mono_load_runtime_and_bcl` within our MONO module.
	- Allows skipping the .pdb files if the `mono_wasm_ignore_pdb_load_errors` flag is true.
2019-10-14 12:10:43 -05:00
Fan Yang dcb522d181 [netcore] Update dotnet sdk version (#17312)
The previous one was broken.
2019-10-14 18:55:39 +02:00
Johan Lorensson 734b776884 Fix msvc build warning, empty source main-core.c. (#17305) 2019-10-14 18:55:07 +02:00
Alexander Köplinger 7de84059b9 [ci] Fix ios versions in Mono SDKs and fix typo 2019-10-14 18:54:25 +02:00
Ryan Lucia e7b199377c [loader] Locking and coop related cleanup; fix leak 2019-10-14 12:51:29 -04:00
Ryan Lucia bf3afd6cad [loader] Move PInvoke and DllMap code into separate source file 2019-10-14 12:51:29 -04:00
Ryan Lucia 80db55591a [netcore] Remove unused InternalLoadUnmanagedDllFromPath
As of https://github.com/dotnet/coreclr/pull/27090
2019-10-14 12:51:29 -04:00
Ryan Lucia 96050effa6 [netcore] Use a single NativeLibrary.cs for all platforms 2019-10-14 12:51:29 -04:00
Alexander Köplinger 850def2f3b [ci] Use Xcode11.1 for XI/XM Mono SDK builds (#17313)
As requested by the xamarin-macios team.
2019-10-14 18:38:06 +02:00
Marek Safar cae52aa4a6 [System.Net.Http] Clean up HttpMessageHandler setup (#17252)
* [System.Net.Http] Clean up HttpMessageHandler setup

to avoid reflection and not to be recursive

Superseded #15051 and #15922

* [csproj] Update project files
2019-10-14 17:19:22 +02:00
Dotnet-GitSync-Bot 4e81814709 Mirror changes from mono/coreclr (#17297)
* Enable fast-paths on Mono for Array (dotnet/coreclr#27144)

Signed-off-by: dotnet-bot <[email protected]>

* [CoreLib] Remove String fields moved to shared

* Stub Array Try* methods
2019-10-12 08:50:00 +02:00
Ryan Lucia 9832bbb0ac [loader] Fix gnu/lib-names.h define (#17294) 2019-10-11 20:49:24 -04:00
Jay Krell 33d62537d4 [runtime] Add portable cached array creation functions and replace gcc-specific impl. (#16855)
* Remove unused mono_class_get_field_from_name_cached.

* Replace gcc-specific:
  mono_array_class_get_cached
  mono_array_new_cached
  mono_array_new_cached_handle

with portable forms that are easy enough to use and general enough.

Before was "easier" because cache provided for you -- possibly multiple.
Before was more general because rank was a parameter -- and had to be a constant.
It was always 1.
2019-10-11 19:47:20 -04:00
Egor Bogatov 9cc5ea3862 Enable more hw intrinsics for AOT (#17160)
* Enable more hw intrinsics for AOT

* Address feedback

* does it fix the failure?

* fix loadedllvm

* Address feedback

* fix copy-paste

* forgot MONO_LLVM_INTERNAL
2019-10-11 15:09:08 -04:00
Aleksey Kliger (λgeek) 7f95f57c43 [sdks] Android runner: properly shutdown runtime for Mono.Debugger.Soft test app (#17261)
* [sdks] properly shutdown runtime for Mono.Debugger.Soft test app

mini_jit_cleanup will cause mono_runtime_try_shutdown to be called before
mono_runtime_quit which will set the shutting down flags in the runtime.

* [test] Re-enable disabled tests
2019-10-11 13:54:19 -04:00
Kyle White 481b3005c6 Update MERP event type to MonoAppCrash (#17131)
* Update parameters in MERP WerReportMetadata xml

To enable backend analysis, we need to update our parameters to match our back end event type, which is MonoAppCrash.

It doesn't make sense to me that the previous implementation constructed the XML here in Mono, but depended on the user (VSMac in this case) to add the "correct" event type.  The parameters in the XML, must match exactly what is expected for the given EventType, and so we should ensure they are always in-sync by defining them here.  Thus, I removed the usage of `eventType` in `mono_merp_enable` and hard coded the expected eventType.

* [merp] Don't pass eventType from managed

The eventType must me MONO_MERP_EVENT_TYPE_STR ("MonoAppCrash") - the backend
determines the format of the other fields based on the event type, so it
doesn't make sense to pass other values from managed since the runtime controls
the other fields.

For compatability Mono.Runtime.EnableMicrosoftTelemetry will still take a
string parameter (now called "unused" instead of "eventType_str") but it will
not pass it down to the internal call.

* Bump corlib version

* [merp] Revert paramerters in WerReportMetadata.xml

Still need event type to be `MonoAppCrash` but the parameters will be the same as before.
2019-10-11 11:15:21 -04:00
Alexander Köplinger d457346e06 [tests] Bump corefx to get Azure testhost change (#17275)
* Bump corefx to get Azure testhost change

See https://github.com/mono/corefx/pull/360

* Update ClientWebSocketTest as well
2019-10-11 09:34:28 +02:00
Tanner Gooding 5784d064e7 Refactoring the ARM intrinsics to match API review and share code with x86 (#25508)
* Rewriting the ARM64 HWIntrinsic JIT infrastructure to share with and mirror the x86 infrastructure where possible.

* Lowercasing the hwintrinsicarm64 and hwintrinsiclistarm64 files.

* Applying formatting patch.

* Fixing up some names in the PAL and VM that were missed.

* Adding support for AdvSimd.Add

* Ensure impSpecialIntrinsic returns nullptr for unsupported base types.

* Rewording various method headers and simplifying some code.

* Fixing getEmitter to GetEmitter

* Adding support for the ld1 instruction

* Applying formatting patch

* Clarifying a todo comment and removing a stray newline

* Fixing the Arm64 impSpecialIntrinsic to not assert for helper intrinsics

* Fixing some intrinsic signatures that were incorrect

* Adding the initial template files for generating Arm HWIntrinsic tests

* Generating the initial batch of Arm HWIntrinsic tests

* Replace gtRegNum with GetRegNum()

* Fixing the test template alignment checks for Arm

* Regenerating the Arm HWIntrinsic tests

* Fixing the namespace checked for Arm intrinsics

* Changing Base to ArmBase and fixing some Arm hwintrinsic metadata

* Fixing the arm emitter and lowering to not assert for hwintrinsics

* Fixing the arm intrinsics to not set the number of vector elements for SIMDScalar intrinsics

* Updating the AdvSimd.Abs tests to use negative input values

* Regenerating the arm hwintrinsic tests

* Fixing the arm hwintrinsic codegen to use the element size for simd scalar operations.

* Fixing the arm scalar intrinsic tests to check for 0 on upper bits

* Regenerating the ARM intrinsic tests

* Fixing AdvSimd.AbsScalar and AdvSimd.AddScalar to be recursive

* Marking AdvSimd.Arm64 and ArmBase.Arm64 as implemented

* Ensure that AdvSimd.Arm64.Abs is marked as variable size

Signed-off-by: dotnet-bot <[email protected]>
2019-10-11 08:14:11 +02:00
Imran Hameed 7c7d406fc0 Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers. 2019-10-10 16:05:34 -07:00
Jo Shields 5567c83049 Update Linker. This fixes a Xamarin.Android breakage (#17272) 2019-10-10 16:26:10 -04:00
Vlad Brezae 165c114aaa [interp] Kill more instructions (#17226)
* [interp] Replace STLOC to dead local with MINT_POP

* [interp] Optimize MINT_POP

It was only used for 0 and 1 values. The implementation for values larger than 1 was broken anyway.

* [interp] Remove unnecessary usage of MINT_POP1

When storing a static field from a CEE_STFLD instruction.

* [interp] Prevent optimizing away the stack, after a branch

We know the contents of the stack following a branch, so we could have optimized away some instructions preceding the branch. We shouldn't do this because the code we are branching to expects a certain stack size.

* [interp] Clear instructions which push unused values on stack

* [interp] Properly initialize stack when pushing local
2019-10-10 21:29:19 +02:00
monojenkins 3ef0544e7a [csproj] Update project files 2019-10-10 19:43:11 +02:00
Marek Safar 502115435d [System.Net.Http] Make HttpClientHandler on WatchOS always fallback
to platform handler. This simplifies HttpClientHandler in shared libraries
and the "default" socket handler would always throw PNSE anyway
2019-10-10 19:43:11 +02:00
Ryan Lucia 1b6bbfaf36 [eglib] Add newline for failure check prints (#17262) 2019-10-10 09:23:16 -04:00
Johan Lorensson 0db52289a3 Incorrect constrained virtual call in method using gsharedvt for reference type. (#17227)
* Incorrect constrained virtual call in method using gsharedvt for reference type.

For generic constrained virtual calls using reference type's, the
virtual call will be dispatched using instance vtable. When the same
virtual call is taking place in a gsharedvt method, mono_gsharedvt_constrained_call
gets called that will assist making the call, the problem is that it will
use the constrained class for reference types meaning that it will not call
the most derived implementation of the method. This difference in the behavior
compared to none gsharedvt versions of methods, gives inconsistent results.

One way this will reproduce is by using Dictionary<Type, EmptyStruct>. For
methods that will take both Type (reference type) and EmptyStruct (value type)
like Add, calling that through IDictionary (hiding details from compiler) will
use the gsharedvt version of the method. Internally that method will call
Type.GetHashCode (), and since Type really is a RuntimeType it overrides
GetHashCode. But since mono_gsharedvt_constrained_call will use the constrained
class instead of the instance class, the called method will be
Object.GetHashCode, that gives a different result. When later ContainsKey
is called using the same Type, it only takes a reference value, so not
using a gsharedvt version of the method, then the most derived version of
GetHashCode will be called, given a different hash, not finding item
in dictionary.

Fix is to make sure behavior in mono_gsharedvt_constrained_call match
behavior for regular constrained virtual calls when having a reference type.
This makes sure we always call the most derived version of the method.

Commit also adds a test for this specific scenario using a dictionary.

* Updated to a more low-level test.
2019-10-10 09:22:54 -04:00
Bernhard Urban-Forster 6ebf07f2cd [mini] trace snippet should restore return value (#17251)
* [mini] trace snippet should restore return value

This is a problem on architectures with a floating point stack (x86).

Without this PR and without `--trace` the managed-to-native wrapper for `Math.Ceiling` has a basic block like this at the end:

```
AFTER LOWER-VTYPE-OPTS  4: [IN:  BB6(5) BB3(1), OUT:  BB1(3) ]
 fmove R34 <- R16
 ldaddr R35 <- R10
 load_membase R37 <- [R35 + 0x0]
 store_membase_reg [R36] <- R37
 il_seq_point il: 0x2d
```

`R16` contains the return value. `ldaddr`/`load_membase`/`store_membase_reg` is the sequence for LMF popping.  `mono_spill_global_vars` does this with it:

```
 SPILL BLOCK 4:
 fmove R34 <- R16
         ff  34 16
        0  loadr8_membase R34 <- [%ebp + 0xffffffd4]
        1  nop
 ldaddr R35 <- R10
         ii  35 5
        1  add_imm R35 <- %ebp [-36] clobbers: 1
 load_membase R37 <- [R35 + 0x0]
         ii  37 35
        1  load_membase R37 <- [R35 + 0x0]
 store_membase_reg [R36] <- R37
          ii -1 37
        1  store_membase_reg [%edi] <- R37
 il_seq_point il: 0x2d
             -1
        1  il_seq_point il: 0x2d
```

The local register allocator takes care of the FP stack handling:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        [%fr0]
processing:     0  store_membase_reg [%edi] <- %eax
        [%fr0]
processing:     0  il_seq_point il: 0x2d
        [%fr0]
```

At the end of the basic block we have one value left on the FP stack. That's good as we are in the `cfg->bb_exit` basic block, so this is ought to be the return value.

Let's enable `--trace=M:System.Math:Ceiling`. It will insert some code in order to copy arguments/return value to some datastructure that can then be printed by the trace code:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  iconst %eax <- [11116832]
        [%fr0]
processing:     0  iconst %eax <- [52]
        [%fr0]
processing:     0  localloc_imm %eax <-
        [%fr0]
processing:     0  localloc_imm %ecx <-
        [%fr0]
processing:     0  store_membase_reg [%eax + 0x30] <- %ecx
        [%fr0]
processing:     0  x86_lea_membase %edx <- %ebp
        [%fr0]
processing:     0  store_membase_reg [%ecx] <- %edx
        [%fr0]
processing:     0  store_membase_imm [%eax + 0x28] <- [11116832]
        [%fr0]
processing:     0  storer8_membase_reg [%ebp + 0xffffffcc] <- %fr0
        []
processing:     0  x86_lea_membase %ecx <- %ebp
        []
processing:     0  store_membase_reg [%eax + 0x2c] <- %ecx
        []
processing:     0  store_membase_reg [%esp + 0x4] <- %eax
        []
processing:     0  store_membase_imm [%esp] <- [11116832]
        []
processing:     0  voidcall [mono_trace_leave_method] clobbers: c
        []
processing:     0  x86_lea_membase %eax <- %ebp
        []
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        []
processing:     0  store_membase_reg [%edi] <- %eax
        []
processing:     0  il_seq_point il: 0x2d
        []
```

At the very beginning we push some value on the FP stack, same as before. However the code inserted by the tracing code will pop that value. Now, when returning to the, it will pop again something from the FP stack, but alas it's gonna be garbage.

With this PR it looks like this:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  iconst %eax <- [11116832]
        [%fr0]
processing:     0  iconst %eax <- [52]
        [%fr0]
processing:     0  localloc_imm %eax <-
        [%fr0]
processing:     0  localloc_imm %ecx <-
        [%fr0]
processing:     0  store_membase_reg [%eax + 0x30] <- %ecx
        [%fr0]
processing:     0  x86_lea_membase %edx <- %ebp
        [%fr0]
processing:     0  store_membase_reg [%ecx] <- %edx
        [%fr0]
processing:     0  store_membase_imm [%eax + 0x28] <- [11116832]
        [%fr0]
processing:     0  storer8_membase_reg [%ebp + 0xffffffcc] <- %fr0
        []
processing:     0  x86_lea_membase %ecx <- %ebp
        []
processing:     0  store_membase_reg [%eax + 0x2c] <- %ecx
        []
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffcc]
        [%fr0]
processing:     0  store_membase_reg [%esp + 0x4] <- %eax
        [%fr0]
processing:     0  store_membase_imm [%esp] <- [11116832]
        [%fr0]
processing:     0  voidcall [mono_trace_leave_method] clobbers: c
        [%fr0]
processing:     0  x86_lea_membase %eax <- %ebp
        [%fr0]
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        [%fr0]
processing:     0  store_membase_reg [%edi] <- %eax
        [%fr0]
processing:     0  il_seq_point il: 0x2d
        [%fr0]
```

Leaving us the return value on the FP stack before returning.

Fixes https://github.com/mono/mono/issues/16410
2019-10-10 11:21:23 +02:00
Jay Krell 8c5581a6f2 Fix g_assert_not_reached message regression. (#17263)
Fix g_assert_not_reached message regression.
2019-10-10 10:26:14 +02:00
Vlad Brezae d4fbbed106 [tests] Disable tests that crash on android sdks (#17255)
Running these tests makes the application crash, probably at shutdown. Even if the tests are reported as passing, check if disabling them fixes the suite timeout on android osx sdks.
2019-10-10 10:51:53 +03:00
Jay Krell 8718b75b7a Remove varargs from g_assert and g_assert_not_reachable. (#17254)
Remove varargs from g_assert and g_assert_not_reachable (save 200+ bytes per frame in wasm interp).

g_assertf unchanged.

I have a strong suspicion, this will save approximately 432 - sizeof (InterpFrame) bytes of stack
per function call in the WebAssembly interpreter.

It will not significantly benefit or hurt any other system.
Systems with disabled asserts will receive a small size improvement.

WebAssembly has no provision for varargs.
Emscripten emulates it.
It appears the emulation allocates room in the frame per outgoing varargs call.
No stack packing.
So each g_assert takes space.

Alternative would be to discourage or disable or remove asserts.
2019-10-09 22:56:06 +02:00
David Bueno 7df7ceb6d3 [System.Web] Now IsExpired property for FormsAuthenticationTicket compares two dates with same kind (UTC) to avoid unexpected results (#17162)
Fixes #17161
2019-10-09 20:50:46 +02:00
Steffen Kieß 30886f997f [Mono.Posix] Add support for memfd_create() and file sealing (#17225)
* [Mono.Posix] Add support for memfd_create() and file sealing

Add the linux syscall memfd_create() and add support for file sealing with
fcntl().

* Bump API snapshot submodule
2019-10-09 20:49:14 +02:00
Bernhard Urban-Forster 1cf081529e [mini] print inserted instruction in verbose logging (#17249)
[mini] print inserted instruction in verbose logging

before:
```
 fmove R11 <- R9
         ff  11 9
        1  nop
```

after:
```
 fmove R11 <- R9
         ff  11 9
        0  loadr8_membase R11 <- [%ebp + 0x8]
        1  nop
```
2019-10-09 20:16:04 +02:00
Filip Navara cbc8cc658b [mini] Fix check for FastAllocateString that used old name (#17219)
* Fix check for FastAllocateString that used old name

* Fix linker scripts

* Update the icall name for FastAllocateString to match managed name

* Update size check in managed allocator

* One more 64-bit target fix
2019-10-09 15:47:43 +02:00
Vlad Brezae 8e649c3545 [sgen] Fix invalid value passed to write barrier (#17236)
When doing memory copy dest is the destination address and src is the source address. If we are copying a single reference, we need to emit a write barrier and pass to the write barrier the value, not the pointer where the value is stored.
2019-10-09 14:37:41 +03:00
Alexander Köplinger 1e9ba3a026 Add drawing type converters to mobile profiles (#17238)
* Add drawing type converters to mobile profiles

Needed for https://github.com/xamarin/xamarin-android/issues/3726

* Bump API snapshot submodule

* [csproj] Update project files
2019-10-09 12:17:53 +02:00
Johan Lorensson 7fad4a12e2 Running --enable-msvc-only didn't find jay.vcxproj. (#17228) 2019-10-09 10:06:09 +02:00
Egor Bogatov 12ff246297 Optimize ToString() for byte, ushort, uint and ulong (dotnet/coreclr#27056)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-09 09:25:18 +02:00
Filip Navara 28b3446d9e [netcore] Fix NULL dereference when running LLVM JIT 2019-10-09 08:39:22 +02:00
Ryan Lucia 1cfb39203b Replace InternalLoadUnmanagedDllFromPath with NativeLibrary.Load (#27090)
Both just call into NDirect::LoadLibraryFromPath, and so this simplifies the code by removing a superfluous QCall

Signed-off-by: dotnet-bot <[email protected]>
2019-10-09 08:23:30 +02:00
Jay Krell cacf99b973 Revert mono_runtime_set_main_args in 44ff0597b8. (#17233)
It breaks Xamarin.Android.
2019-10-09 00:54:44 -04:00
Vlad Brezae 579e385b96 [interp] Small cleanups (#17229)
* [intepr] Add define for constant

* [interp] Pass separately pop_vt_sp argument

Cleans up the code a bit. There is no need to do fancy bit operations for such a slow opcode.
2019-10-09 00:08:44 +02:00
Filip Navara 63025d93fd [netcore] Port CoreCLR implementation of Exception.SetCurrentStackTrace (#17243)
* Port CoreCLR implementation of Exception.SetCurrentStackTrace

* Update SerializationRemoteStackTraceString

* Update Exception.cs
2019-10-08 23:45:17 +02:00
Alexander Köplinger 4ca42a933f [man] Update MONO_IOMAP docs as it no longer works with corefx System.IO (#17231)
Fixes https://github.com/mono/mono/issues/15845
2019-10-08 21:39:31 +02:00
Alexander Köplinger 209c4bae3e [netcore] Disable some SafeWaitHandle tests everywhere, not just Linux (#17230)
The tests are kinda bogus since they create a SafeWaitHandle from `new IntPtr(1)` which causes a crash during finalization when releasing that handle.

See https://github.com/mono/mono/issues/17224
2019-10-08 17:39:41 +02:00
dotnet-maestro[bot] 52c4d87633 Update dependencies from https://github.com/dotnet/arcade build 20191006.1 (#17214)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19506.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19506.1
2019-10-08 13:30:46 +02:00
Filip Navara e8ed2494ca Fix check in fix_libc_name to trigger only for libc, not or other names with same prefix (#17222)
See https://github.com/mono/mono/pull/17210#pullrequestreview-298663220
2019-10-08 13:06:38 +02:00
Alexander Köplinger faf100da27 [support] Make sure map-impl.h ends up in the tarball 2019-10-08 12:05:47 +02:00
Steffen Kieß 8bb7b8645a Clean up map.c /map.h (#16746)
* [Mono.Posix] Move Mono_Posix_{From,To}MremapFlags() to support/sys-mman.c

Since b522eab5ff Mono_Posix_FromMremapFlags()
and Mono_Posix_ToMremapFlags() contain manual changes for NetBSD. Move the
functions from support/map.c to support/sys-mman.c so that they won't be
overwritten when create-native-map is rerun.

* [Mono.Posix] Fix prototypes in support/stdio.c

In ed892ccf27 wrapper functions for several
stdio functions were added, but the prototypes in map.h were manually
written, not using create-native-map. This commit changes the prototypes so
that they match the output of create-native-map.

* [Mono.Posix] Undefine HAVE_STRUCT_SOCKADDR_IN6 in support/map.c on MSVC

In 22b6b95814 a manual change to support/map.c
was added to prevent Mono_Posix_FromSockaddrIn6() and
Mono_Posix_ToSockaddrIn6() from being built for windows even if
HAVE_STRUCT_SOCKADDR_IN6 is defined. Instead undefine it in a header file
to avoid manual changes to map.c.

With this commit map.c, map.h and NativeConvert.generated.cs contain no
manual changes anymore.
2019-10-08 12:04:45 +02:00
monojenkins ad2132e611 [bcl] Update BCL Linked Size 2019-10-08 10:23:41 +02:00
Ryan Lucia 3355d2dd81 [netcore] Add libc name transition 2019-10-08 10:22:18 +02:00
Alexis Christoforides 37c197f225 Remove unused variable 2019-10-07 21:04:47 -05:00
Alexis Christoforides 2ca93bd2f9 [merp] Print missing status marker file for stage 1 (setup) 2019-10-07 21:04:47 -05:00
Alexis Christoforides b18d52bf1c Zeroing is not necessary, snprintf() always null-terminates in this case 2019-10-07 21:04:47 -05:00
Vlad Brezae d9c9a53332 [interp] Rerun cprop pass if stloc.np intructions were eliminated
If we eliminate these instructions, it means we removed instructions that use the top of stack. Without them we might be able to optimize away that stack slot usage. If we retraverse the instruction stream, then we must handle movloc and stloc.np separately.
2019-10-07 19:35:57 -05:00
Vlad Brezae f2006f61dc [interp] Handle more cases for stloc.np optimization
We emit stloc.np if we have "stloc local; ldloc local". Bu we can also emit if we have "stloc loc1; ldloc loc2" and loc2 == loc1.
2019-10-07 19:35:57 -05:00
Vlad Brezae fc100ae043 [interp] Skip NOPs when checking previous instruction
When emitting stloc.np, which looks for stloc/ldloc pair.
2019-10-07 19:35:57 -05:00
Calvin Buckley 01dd63c440 dladdr shim for gmodule; try to enable crash reporter on AIX (#15808)
* Introduce new dladdr wrapper in glib, AIX reimpl, and convert usage

Provides a wrapper around dladdr that should be platform-neutral.

Also provided is a reimplementation for AIX, which should enable
that platform to get some previously dladdr-specific functions like
the crash reporter. It is somewhat flawed in that it only gets info
for symbols in .text, returns non-constant allocated strings, and
allocates heap memory for buffers.

Then convert usages (except a macOS specific one) of dladdr to the
new eglib function. It has a different signature, which should
simplify intent and reduce structiness. Also free memory on AIX due
to limitations of its reimplmenetation.

* Enable (and fix build of) crash reporter on AIX

So far crashes don't seem any different, but the code for it is
built.

* typo fix (guess this isn't compiled on aix)

* attempt to fix macOS and win32 builds

* Implement more of g_module_address on Win32

Implements the filename stuff. Untested; don't have Windows.

* Look for backtrace_symbols in an IBM compat library for i

Enables i to natively dump a stack trace. (AIX "works" too if you
copy the compat lib's code and hack it in - it probably needs to be
in EGlib.) Unfortunately, it's not perfect. The backchain from the
signal handler is empty on i, and AIX sometimes mangles the stack
frame in a way that severely confuses the backtrace code, causing
it to dereference invalid memory by interpreting instructions as a
pointer. (I have seen somewhat similar for the sigaltstack case,
where the native memory dump attempts to read something it should
not.) Perhaps mincore could be used, but that feels like a sloppy
workaround, especially in the "AIX making an invalid stack frame"
case.

* If the file isn't an archive, don't format its name like one

* Fix up search for backtrace_symbols to look in libexecinfo

FreeBSD ships it in base and Haiku has a package; likely others too.

* Win32 suffers in g_module_address as well

* That should habe been guarded with AIX due to limitations

Otherwise seems to free things that shouldn't be freed

* Casting for win32

* Fix up Win32 impl of module address

* Free HMODULE to lower reference count after using it

* Change dladdr based impl to always dup for impl consistency

Means freeing afterwards shouldn't be ifdefable.

* probably shouldn't strdup NULL

Likely why macOS has failures.

* Don't dup/return const strs, but copy to caller-provided buffers

...which can be on the stack. This makes it far less risky to use
in a crash reporter scenario, because malloc can be hosed then.

Also fix profiler up better to work with this new reality.
2019-10-08 00:19:28 +02:00
Zoltan Varga c7f8e5bdb7 [offsets-tool] Update the README. (#17212) 2019-10-07 22:37:09 +02:00
Jo Shields d28d72cb96 Bump version to 6.9 (#17207)
* Bump version to 6.9

* Add missing new linker source file

* [csproj] Update project files

* No monolinker on MCS mode, new linker version needs Roslyn
2019-10-07 22:36:13 +02:00
Ryan Lucia 1d18c27dbe Don't declare mono_get_module* on non-win32 2019-10-07 09:44:41 -05:00
Ryan Lucia ca5926fa3f Use eglib win32 helper functions without MAX_PATH limit 2019-10-07 09:44:41 -05:00
Ryan Lucia d12db16723 Escape type namespaces, document that we aren't escaping names for now 2019-10-07 09:43:47 -05:00
Ryan Lucia 8760cd446c Make the mono identifier escape function work like unescape and public 2019-10-07 09:43:47 -05:00
Ryan Lucia 7c2d2a551c Unescape type names in the managed parser 2019-10-07 09:43:47 -05:00
Ryan Lucia f37c3a649f Make TypeNameParser consistently use tabs
Is this worth the line damage? Unsure.
2019-10-07 09:43:47 -05:00
Ryan Lucia fa8fbbbc36 Partially revert "Remove redundant function call of EscapeTypeName to make (#16425)"
This partially reverts commit abe48e3fd4.
2019-10-07 09:43:47 -05:00
Jay Krell f0ff9f3860 Do not zero localalloc if not asked. (#16881)
Note that we will still touch the pages (mono_emit_stack_alloc).
2019-10-07 09:42:13 -05:00
Jay Krell 957e6fa4dd [Coop] Convert Get/SetGenericValueImpl. (#17034)
This is an alternative to https://github.com/mono/mono/pull/16994
and does *not* depend on
   https://github.com/mono/mono/pull/16987
or https://github.com/mono/mono/pull/17009
or similar.
2019-10-07 09:40:22 -05:00
Alexander Köplinger 495b08a3c7 Default --host in autogen.sh on cygwin when missing. (#17164) 2019-10-07 16:38:49 +02:00
Ryan Lucia 6f72363932 [IO] Remove read-only logic in mono_w32_get_disk_free_space (#17177)
This brings us in line with GetDiskFreeSpaceExW on Windows as well as .NET Core.
2019-10-07 09:38:23 -05:00
Junil Um 026d82fac3 Fixed a bug in mono-service.cs (#17095)
There is a bug in mono-service.cs file.

`setup.ApplicationBase = Environment.CurrentDirectory;`;

General `Environment.CurrentDirectory` does returns `Directory.GetCurrentDirectory()`.
And retuened value 'current path' without last character `/` but `AppDomain.CurrentDomain.BaseDirectory`;

It have a problem in my service application.

In my service application code,

```cs
using System;
using System.IO;

namespace BaseDirTest
{
    internal class Program
    {
        public static void Main(string[] args)
        {
            var baseDir = AppDomain.CurrentDomain.BaseDirectory;
            Console.WriteLine($"baseDir={baseDir}");
            Console.WriteLine($"Path.GetDirectoryName(baseDir)={Path.GetDirectoryName(baseDir)}");
        }
    }
}
```

When I launch `mono app.exe`,
```bash
baseDir=/workspace/test/mono-owin-test/ConsoleApplication1/BaseDirTest/bin/Debug/
Path.GetDirectoryName(baseDir)=/workspace/test/mono-owin-test/ConsoleApplication1/BaseDirTest/bin/Debug
```

But when I launch `mono-service --no-daemon app.exe`,
```bash
eDir=/workspace/test/mono-owin-test/ConsoleApplication1/mono-service-umc/bin/Debug
Path.GetDirectoryName(baseDir)=/workspace/test/mono-owin-test/ConsoleApplication1/mono-service-umc/bin
```

Because my service application can not founds plugin directory and scans it.
2019-10-07 16:28:40 +02:00
Bernhard Urban-Forster 6861610662 [lldb] use mono_method_full_name helper (#16934)
[lldb] use mono_method_full_name helper

Instead of:

```
(lldb) mbt 20
* thread #11
  * frame #0: 0x1bf28f28 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x1bf9caac libsystem_pthread.dylib`pthread_kill + 300
    frame #2: 0x1be66080 libsystem_c.dylib`abort + 140
    frame #3: 0x024bdfa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="../../../../../mono/metadata/object.c:1905: Expected GC Unsafe mode but was in STATE_BLOCKING state", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #4: 0x02487f44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #5: 0x02487ee0 monotouchtest`monoeg_g_logv(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>, args=<unavailable>) at goutput.c:156:10 [opt]
    frame #6: 0x02487f74 monotouchtest`monoeg_g_log(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>) at goutput.c:165:2 [opt]
    frame #7: 0x024694c4 monotouchtest`assert_gc_unsafe_mode(file="../../../../../mono/metadata/object.c", lineno=1905) at checked-build.c:396:3 [opt]
    frame #8: 0x023d2f64 monotouchtest`mono_class_vtable_checked(domain=0x16d64800, klass=0x173c1b98, error=0x194b0ee8) at object.c:1905:2 [opt]
    frame #9: 0x024130a8 monotouchtest`get_current_thread_ptr_for_domain(domain=0x16d64800, thread=0x02db45d0) at threads.c:635:2 [opt]
    frame #10: 0x024119a8 monotouchtest`mono_thread_current at threads.c:2026:23 [opt]
    frame #11: 0x02416998 monotouchtest`mono_thread_interruption_checkpoint_request(bypass_abort_protection=0) at threads.c:5101:35 [opt]
    Messaging::void_objc_msgSendSuper @ 388435850 "calli.nat.fast" || frame #12: 0x024d5de8 monotouchtest`interp_exec_method_full(frame=0x194b11c0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3230:4 [opt]
    ObjCExceptionTest::InvokeManagedExceptionThrower @ 388435752 "vcall" || frame #13: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1380, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    ExceptionsTest::ManagedExceptionPassthrough @ 388502134 "vcallvirt.fast" || frame #14: 0x024d61c0 monotouchtest`interp_exec_method_full(frame=0x194b14e0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3325:4 [opt]
    Object::runtime_invoke_direct_void__this__ @ 388462550 "vcall" || frame #15: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1598, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    frame #16: 0x024d46ec monotouchtest`interp_runtime_invoke(method=<unavailable>, obj=0x0302c5f0, params=0x00000000, exc=0x194b166c, error=0x194b18c8) at interp.c:1766:2 [opt]
    frame #17: 0x0232b39c monotouchtest`mono_jit_runtime_invoke(method=0x174fda58, obj=<unavailable>, params=0x00000000, exc=<unavailable>, error=0x194b18c8) at mini-runtime.c:3170:12 [opt]
    frame #18: 0x023d5cc8 monotouchtest`do_runtime_invoke(method=0x174fda58, obj=0x0302c5f0, params=0x00000000, exc=0x00000000, error=0x194b18c8) at object.c:3017:11 [opt]
    frame #19: 0x023d26a0 monotouchtest`mono_runtime_invoke_checked(method=<unavailable>, obj=<unavailable>, params=<unavailable>, error=<unavailable>) at class-getters.h:24:1 [opt] [artificial]
```

It prints now:

```
(lldb) mbt 20
* thread #11
  * frame #0: 0x1bf28f28 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x1bf9caac libsystem_pthread.dylib`pthread_kill + 300
    frame #2: 0x1be66080 libsystem_c.dylib`abort + 140
    frame #3: 0x024bdfa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="../../../../../mono/metadata/object.c:1905: Expected GC Unsafe mode but was in STATE_BLOCKING state", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #4: 0x02487f44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #5: 0x02487ee0 monotouchtest`monoeg_g_logv(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>, args=<unavailable>) at goutput.c:156:10 [opt]
    frame #6: 0x02487f74 monotouchtest`monoeg_g_log(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>) at goutput.c:165:2 [opt]
    frame #7: 0x024694c4 monotouchtest`assert_gc_unsafe_mode(file="../../../../../mono/metadata/object.c", lineno=1905) at checked-build.c:396:3 [opt]
    frame #8: 0x023d2f64 monotouchtest`mono_class_vtable_checked(domain=0x16d64800, klass=0x173c1b98, error=0x194b0ee8) at object.c:1905:2 [opt]
    frame #9: 0x024130a8 monotouchtest`get_current_thread_ptr_for_domain(domain=0x16d64800, thread=0x02db45d0) at threads.c:635:2 [opt]
    frame #10: 0x024119a8 monotouchtest`mono_thread_current at threads.c:2026:23 [opt]
    frame #11: 0x02416998 monotouchtest`mono_thread_interruption_checkpoint_request(bypass_abort_protection=0) at threads.c:5101:35 [opt]
    (wrapper managed-to-native) ApiDefinition.Messaging:void_objc_msgSendSuper (intptr,intptr) @ 388435850 "calli.nat.fast" || frame #12: 0x024d5de8 monotouchtest`interp_exec_method_full(frame=0x194b11c0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3230:4 [opt]
    Bindings.Test.ObjCExceptionTest:InvokeManagedExceptionThrower () @ 388435752 "vcall" || frame #13: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1380, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    MonoTouchFixtures.ObjCRuntime.ExceptionsTest:ManagedExceptionPassthrough () @ 388502134 "vcallvirt.fast" || frame #14: 0x024d61c0 monotouchtest`interp_exec_method_full(frame=0x194b14e0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3325:4 [opt]
    (wrapper runtime-invoke) object:runtime_invoke_direct_void__this__ (object,intptr,intptr,intptr) @ 388462550 "vcall" || frame #15: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1598, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    frame #16: 0x024d46ec monotouchtest`interp_runtime_invoke(method=<unavailable>, obj=0x0302c5f0, params=0x00000000, exc=0x194b166c, error=0x194b18c8) at interp.c:1766:2 [opt]
    frame #17: 0x0232b39c monotouchtest`mono_jit_runtime_invoke(method=0x174fda58, obj=<unavailable>, params=0x00000000, exc=<unavailable>, error=0x194b18c8) at mini-runtime.c:3170:12 [opt]
    frame #18: 0x023d5cc8 monotouchtest`do_runtime_invoke(method=0x174fda58, obj=0x0302c5f0, params=0x00000000, exc=0x00000000, error=0x194b18c8) at object.c:3017:11 [opt]
    frame #19: 0x023d26a0 monotouchtest`mono_runtime_invoke_checked(method=<unavailable>, obj=<unavailable>, params=<unavailable>, error=<unavailable>) at class-getters.h:24:1 [opt] [artificial]
```



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-07 16:16:13 +02:00
Thays Grazia 18fac0a241 [debugger][exception] Debugger breaks on handled exceptions (#17106)
* If there is a perform_wait_callback in the stack there will be another catch generated by the owner thread, so we don't need to throw, we can continue and find the next catch.
Fixes #17083

* Reverting unit test changed on commit 405d521.

* Fixing assert when calling mono_jit_info_get_method if it was a trampoline.
2019-10-07 08:22:06 -04:00
Egor Bogatov 35c6b7691f [netcore] Optimize Buffer.BlockCopy and Buffer.IsPrimitiveArray (#16692)
* Optimize Buffer.BlockCopy and IsPrimitiveArray (make them icalls)

* wrap with ENABLE_NETCORE

* fix formatting

* fix check for "is primitive"

* use m_class_is_primitive

* fix build errors

* fix typo

* fix failing test

* use mono_error_set_argument_out_of_range

* fix argument names for exceptions
2019-10-07 12:29:22 +02:00
Steve Pfister 2f6d69e2e6 Re-ordered the * IsValid check in SiteIdentityPermissions. We were throwing an IndexOutOfRangeException
Fixes https://github.com/mono/mono/issues/17017
2019-10-07 09:43:53 +02:00
Jan Kotas c6dcd3de56 More CompareInfo.IndexOf globalization fixes (dotnet/coreclr#27049)
Apply earlier CompareInfo.StartsWith fixes onto CompareInfo.IndexOf as well

Signed-off-by: dotnet-bot <[email protected]>
2019-10-07 09:41:24 +02:00
Zoltan Varga dea3a20b05 [wasm] Bump emscripten version. (#17179)
* [offfsets-tool] Embed a copy of the clang bindings, the version in PIP doesn't work with newer clang versions.

* [wasm] Bump emscripten version to 1.38.47.
2019-10-06 20:10:43 -04:00
Jay Krell f57af77e69 Optimize and extend System.Array intrinsic identification. (#17047)
* Optimize and extend System.Array intrinsic identification.
i.e. do the strcmp last.
Add GetRawArrayData.

* Use [Intrinsic] attribute instead of comment.
2019-10-06 10:27:42 +02:00
Zoltan Varga d2f543b4b1 [mixed] Add interp entry wrappers for devirtualized calls as well. (#17159)
* [mixed] Add interp entry wrappers for devirtualized calls as well.

* [aot] Generate code for string:memcpy/memset/bzero even in profile mode, they are called by the generated code.

* [aot] Treat string:memcpy as always AOTed in mono_aot_can_enter_interp () as well.

* [aot] Always aot string ctors as well, the JIT doesn't generate interp entry code for them.

* [wasm] Update print-table.py.
2019-10-05 20:51:52 -04:00
Aleksey Kliger (λgeek) cb52186661 [merp] Put thread into async context before running summarizer (#17194)
Followup work for https://github.com/mono/mono/pull/17090

In order to get managed method info for unmanaged stacktraces, we call
mini_jit_info_table_find_ext which calls decode_exception_debug_info which may
allocate if it's not in async context.

Do the switch in mono_threads_summarize_execute so that non-supervising threads
all switch when they run the sigterm_signal_handler.  mono_threads_summarize
already does it for the supervisor thread.

Fixes https://github.com/mono/mono/issues/17180
2019-10-05 13:01:56 -04:00
Alexander Köplinger 0600ea1256 Bump libgdiplus to 6.0.4 2019-10-05 17:36:40 +02:00
Aleksey Kliger (λgeek) 2f7fd4f03e [merp] exit_status is 0 if we ran the uploader successfully (#17185)
Not 1 (TRUE)
2019-10-05 09:59:05 -04:00
Marek Safar fce0e64de9 [System.Private.CoreLib] Fixes build 2019-10-05 14:09:54 +02:00
Stephen Toub 2c8d5f0a39 Add ExceptionDispatchInfo.SetCurrentStackTrace (dotnet/coreclr#27004)
* Add ExceptionDispatchInfo.SetCurrentStackTrace

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub d8c115d07a Fix stackalloc reuse bug in Environment.UserDomainName (#26994)
* Fix stackalloc reuse bug in Environment.UserDomainName

According to the comment, this code is never expected to be used (so it should either be deleted or the comment fixed), but in the meantime, if it were to be executed, things could go badly due to potentially passing the same buffer into LookupAccountNameW for two different arguments.

* Fix existing ValueStringBuilder usage to dispose the builder

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub 670738b55f Use const internally in corelib for Environment.NewLine (#27013)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub 1e3d017479 More corelib cleanup (dotnet/coreclr#26993)
* Make relevant types static (mostly Interop)

* Fix a few stray semicolons

* Remove unnecessary locals

Some can be removed entirely, some can be replaced by discards.

* Remove unnecessary usings

* Move Unix-specific helper method to TimeZoneInfo.Unix.cs

* Remove duplicated code

* Remove duplicative checks

* Remove [ThreadStatic] initialization

These are all the default values and wouldn't be necessary for any static, but inline initialization for [ThreadStatic]s is particularly unuseful (and in non-default cases problematic) because the fields are only set to these values on the initializing thread.

* Fix methods whose return value is never used

* Use string.IsNullOrEmpty

* Remove unnecessary field initialization

All ctors overwrite these values.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Boris Parfenenkov 8205424667 Improve TimeSpan precision (dotnet/coreclr#26992)
Change multiplying by (x * (1.0 / BigValue)) on (x / BigValue).

Fix #41380

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Larry Ewing b45d0a1084 [aprofutil] Add skip regexes to simplify expression of removal matching (#17146)
* [aprofutil] Add skip regexes to simplify expression of removal matching

* [aprofutil] arguments to skip and take by count for automatic profile testing
2019-10-04 22:23:24 +02:00
Maxim Lipnin 06effa7d7e [wasm] Enable CoreXml.Test.XLinq.LoadSaveAsyncTests.RoundtripSyncAsyncMatches_Stream test (#17183) 2019-10-04 12:42:45 -05:00
Ryan Lucia 05217395db [reflection] Only duplicate MonoMarshalSpec strings for custom types (#17175)
* [reflection] only duplicate MonoMarshalSpec strings for custom types

* Add test
2019-10-04 13:41:56 -04:00
Egor Bogatov 26754eecff [llvm] Propogate --enable-llvm-asserts to llvm/build.mk (#17149)
* fix Enable llvm asserts

* fix chmod
2019-10-04 10:27:01 -05:00
Vlad Brezae d1b3ee2e32 [interp] Improve copy propagation (#17154)
* [interp] Refactor tracking of stack/local values

Previously we had just an InterpInst* inside StackContentInfo, which was representing the instruction that pushed a certain local/value on the stack. This makes many things awkward, since an instruction is logically different from a value, that a local or a stack slot have. If we clear an instruction, it doesn't necessarily mean that the value that the instruction produced can't be stored on stack or in a local.

This commit creates a new structure StackValue, which holds the value.

* [interp] Generalize contents of StackValue

StackValue contains an opcode and some data to enable reconstruction of the value. For example instead of doing a LDLOC for a local, we can see if the local has a known value and use it instead (this could be a LDLOC from another local, whose value was propagated, or in the future a LDC). This will make more sense when we also start to track constant values.

Also decouple MOVLOC instructions from the cprop pass. They serve no purpose there. They are useful though when we do deadce, since we currently don't know how to kill instructions that change the stack.

* [interp] Handle dup opcode during copy propagation

* [interp] Avoid losing track of stack values for some opcodes

For some opcodes, that access stack values below the top of the stack, we were assuming they pop everything and then push the stack back, in order to prevent optimizing away instructions that pushed some of these values, since the original instruction expects those values to reside on the stack. We handle these instruction separately and keep track of the values of the stack, so we can further propagate the stack values, even though we currently can't optimize away those instructions.

* [interp] Propagate values for ctor arguments

* [interp] MINT_CASTCLASS no longer clobbers top of stack
2019-10-04 14:20:52 +03:00
Johan Lorensson 2aa7adb269 Always do copy_stack_data on entering GC safe/unsafe mode. (#17150) 2019-10-04 09:16:05 +02:00
Piotr Zierhoffer 4e8c496082 mkbundle: fix 17139 mono aot register module fix definitions (#17152)
* [mkbundle] Fix mono_aot_register_module declaration (#17139)

This change reflects the declaration introduced in jit.h in #15132.

* [mkbundle] Remove spurious function declaration (#17139)

The comment is clearly outdated after the introduction of #15132.
2019-10-04 08:46:58 +02:00
Aleksey Kliger (λgeek) ea2a460cf6 [merp] Don't overrun buffer in copy_summary_string_safe … (#17176)
* [merp] Don't overrun buffer in copy_summary_string_safe

MonoFrameSummary:str_destr is an array of MONO_MAX_SUMMARY_NAME_LEN bytes, not
MONO_MAX_SUMMARY_NAME_LEN + 1 bytes.

Fixes Coverity CID 1454563

* [merp] Use g_strlcpy for copy_summary_string_safe

Fixes Coverity CID 1454563

We would sometimes write to MonoSummaryFrame:str_descr which is
MONO_MAX_SUMMARY_NAME_LEN bytes long at index MONO_MAX_SUMMARY_NAME_LEN which
is one past the end of the array.

* nit: rename confusing parameter names

old names were confusing - we were copying from 'out' to 'in'.  Now we copy to
'dest' from 'src'
2019-10-04 01:17:49 -04:00
Vlad Brezae 8c1ef74bb1 [arm] Fix fetching of method addresses (#17099)
After https://github.com/mono/mono/commit/9ff3b0d65ee4, in an aot image we can emit addresses either as bl offset (which uses 4 bytes) or as ldr pc, =<label>; <addr> (which uses 8 bytes). Before this commit we were dereferencing an instruction pointer, in get_call_table_entry, assuming 8 bytes per entry. This could overflow and crash if we used in fact only 4 bytes per entry. Fix this by including the entry size in the aot image.

Fixes random crashes on arm CI on full-aotmixed suite.
2019-10-03 22:29:55 +03:00
Alexander Köplinger ea9dad7249 Bump ikvm to get https://github.com/mono/ikvm-fork/pull/13 (#17169) 2019-10-03 20:46:37 +02:00
dotnet-maestro[bot] 06c3637681 [master] Update dependencies from dotnet/corefx (#17166)
* Update dependencies from https://github.com/dotnet/corefx build 20191002.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19502.6

* Reenable libgdiplus/System.Drawing.Common tests

* Update dependencies from https://github.com/dotnet/corefx build 20191002.7

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19502.7
2019-10-03 20:45:29 +02:00
Bernhard Urban-Forster 40b6f1ad2b [sdks] .gitignore python virtualenv (#17168) 2019-10-03 20:34:15 +02:00
Alexander Köplinger 43ccc6c26b [netcore] Update .gitignore 2019-10-03 18:45:14 +02:00
Alexander Köplinger 5bf18642ff [netcore] Move ignored test to global ignore file instead of the Linux-specific
It affects all platform
2019-10-03 18:44:07 +02:00
Marek Safar 0ccfd7806c [System.Runtime.Serialization] Make reflection calls linker friendly (#17165)
* [System.Runtime.Serialization] Make reflection calls linker friendly

Fixes #12586

* [csproj] Update project files
2019-10-03 18:40:14 +02:00
Sam Patel 83a5343d12 Merge pull request #17105 from naricc/naricc/UnitializedArray-Comment
Add a comment explaining why GC.AllocateUninitializedArray is not an icall
2019-10-03 10:43:45 -04:00
lateralusX aaa383b3e1 Default --host in autogen.sh on cygwin when missing. 2019-10-03 12:25:30 +02:00
Johan Lorensson 55be8289ff Optional only build MSVC runtime using make file system on Windows. (#16915)
* Build MSVC only runtime using make file system.

Add a new option to only build MSVC version of mono runtime and native
tools, --enable-msvc-only. In the past, --enable-msvc build both
mingw as well as MSVC runtime, this commit adds an option to only build
MSVC version of runtime and native tools using make file system.

Commit updates all native make file making sure mingw/gcc builds are
not done if --enable-msvc-only has been used. It also makes sure we
build and use MSVC based tools as part of build and tests.

Result will be a build without any mingw/gcc artifacts, but where all
existing make files still works with the MSVC build artifacts.

This commits is also preparing to switch the .NETCore build over to use
MSVC build runtime in order to get LLVM support, something that is
currently only supported on Windows MSVC build.

* Add link to mono-sgen.exe and clean windows binaries.

* Fix failures in mono/tests test-aot on Windows.

* Add fullaot-mixed tests on Windows x64.

* Run msbuild in parallel.

* Default to x64 mingw host on cygwin build.
2019-10-03 11:21:36 +02:00
Nathan Ricci d0461a9d8d Spelling corrections. 2019-10-02 13:14:48 -04:00
Jo Shields d8040ba865 Fix missing corerun/ directory in make dist tarballs (#17126)
```
 configure.ac:6695: error: required file 'netcore/corerun/Makefile.in' not found
 Makefile.am:24: error: required directory ./netcore/corerun does not exist
```
2019-10-02 10:46:42 -04:00
Jay Krell 6fb01aa269 Some external_only/internal split, and inlining. (#17012)
mono_type_get_type
 mono_type_get_signature
 mono_type_is_byref
 mono_type_get_class
 mono_type_get_array_type
2019-10-02 12:59:45 +02:00
Jay Krell 8d84b6fd38 [jit amd64] Add const and remove x87. (#16700) 2019-10-02 11:51:59 +02:00
Vlad Brezae 0ce8c78e0c [mini] Use clr memory model by default (#17136)
* [mini] Use clr memory model by default

Otherwise we can have potential crashes in the bcl on arm targets, since a lot of code that assumes this memory model is now shared with mono.

Keep the old clr-memory-model option alive for now, to avoid potential build problems, if building older mono with newer mono.

* [build] Remove clr-memory-model option

It is default now

* [man] Add entry for weak-memory-model
2019-10-02 09:23:45 +02:00
Alexis Christoforides 10e9159eaa [merp] Include any managed methods in the 'unmanaged_frames' portion … (#17090)
* [merp] Include any managed methods in the 'unmanaged_frames' portion of the output also

This can help correlate the two lists when we have mixed managed & unmanaged stacks in our crash output.

* Wehn symbol/function address is not found, still output the IaP

* Don't set native_offset to ip, that's not what it means

* [merp] Bump protocol version

* Minor formatting fix

* Remove paths from assembly image names

* Remove unused var

* Refactor filling in managed info for stack frame

* Revert "Remove paths from assembly image names"

This reverts commit defe86b471f37e12fa66a104747cac932943dca1.
2019-10-02 01:11:35 -04:00
Kyle White 0004522028 [merp] Use macOS version not Darwin version in MERP reports (#17130)
* Use macOS version not Darwin version in MERP

Fixes #17004

* Update mono-merp.c

Ensure this is only for Mac

* Don't allocate in macos_version_string

Use a static buffer.

Also some formatting fixes.
2019-10-01 22:01:35 -04:00
Tanner Gooding c76b84e80c Fixing AdvSimd.Arm64 to have an internal constructor in the "supported" file. (#26953)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 19:22:28 +02:00
Maxim Lipnin 41a89af31d [wasm][xunit tests] Exclude some tests due to PNSE-implemented members. (#17137) 2019-10-01 10:13:46 -05:00
Vlad Brezae 5af0ad900c Update codeowners
Get notifications for profiler and exceptions.
2019-10-01 16:40:48 +02:00
Maxim Lipnin db2e8ea79f [wasm][xunit test] Exclude RemoteExecutorTests in bcl profile (#17120)
`System/RemoteExecutorTests.cs` tests are excluded in the mobile test profiles. It makes sense to do the same for wasm.

Fixes https://github.com/mono/mono/issues/17038.
2019-10-01 15:34:49 +02:00
Adam SitnikandJan Kotas 20ab9bf581 allow ASCII equality ordinal fast path for en-* cultures (#26905)
* allow ASCII equality ordinal fast path for en-* cultures

* dont use StartsWith in the initialization execution path

* Update src/System.Private.CoreLib/shared/System/Globalization/CompareInfo.Unix.cs

Co-Authored-By: Jan Kotas <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 11:36:07 +02:00
Vlad Brezae f9ca01e3a5 [sgen] Fix allocator spelling (#17111) 2019-10-01 12:09:54 +03:00
Marek Safar e5f218786d [netcore] Update coreclr tests package name 2019-10-01 10:57:02 +02:00
Santiago Fernandez Madero b74f7b936e Make ResolveEventArgs.Name not nullable (dotnet/corefx#41415)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 10:13:53 +02:00
Youssef Victor 4d6596359c Typo: retreive => retrieve (dotnet/coreclr#26915)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 10:13:53 +02:00
Egor Bogatov 5537a7c2ba Enable hw intrinsics in AOT mode (#17005)
* Introduce -mattr flag

* fix build on arm targets

* remove mono_memory_barrier

* Address feedback

* Address feedback

* cleanup

* fix crash

* fix "bmi" (it's not "bmi1")

* cleanup

* ignore System.Drawing.Tests.IconTests.CorrectColorDepthExtracted test

* fix build on arm

* fix build

* fix build
2019-09-30 21:28:55 -04:00
Maxim Lipnin e416137dfb [wasm][xunit tests] Other fix for "MissingMethodException: Default constructor not found" issue (#17122)
Fixes https://github.com/mono/mono/issues/17040
2019-09-30 15:21:32 -04:00
Ankit Jain 410cb28177 Bump msbuild to track xplat-master (#16975)
.. to pick up https://github.com/mono/msbuild/pull/142 and
https://github.com/mono/msbuild/pull/139 .
2019-09-30 21:11:09 +02:00
Vlad Brezae eb887a8c74 [sgen] Add stats for allocated gchandles (#17074) 2019-09-30 21:15:42 +03:00
dotnet-maestro[bot] 5818100a80 Update dependencies from https://github.com/dotnet/roslyn build 20190926.6 (#17103)
- Microsoft.Net.Compilers - 3.3.1-beta4-19476-06
2019-09-30 18:40:57 +02:00
Egor Bogatov 3768181879 Update NETStandard2.1 APIs (#17080)
Contributes to #17064

* Add System.Data.DataSetExtensions to mobile profiles
2019-09-30 16:07:02 +02:00
Alexander Köplinger 7f37f76b6d [netcore] Disable System.Drawing.Tests.IconTests.CorrectColorDepthExtracted test
It is broken on newer libgdiplus.
2019-09-30 15:32:18 +02:00
Marek Safar f88da6cc14 [netcore] Add target for running coreclr tests 2019-09-27 21:27:08 +02:00
Nathan Ricci bc63dc68f8 Updated comment in GC.cs about unitialized array. 2019-09-27 14:48:00 -04:00
Tanner Gooding bf5f6174ad Refactoring the ARM Hardware Intrinsics based on the latest design decisions. (#26895)
* Moving the Arm64 intrinsic files into the Arm folder.

* Refactoring the ARM Hardware Intrinsics based on the new design.

* Adding support for AdvSimd.Add

* Adding LoadVector64 and LoadVector128 APIs to Arm.AdvSimd

* Marking the LoadVector64 and LoadVector128 methods as unsafe

* Removing the ARM64 HWIntrinsic tests

* Renaming Arm.Base to Arm.ArmBase, as per the design

* Fixing Base.cs to ArmBase.cs in the shared projitems

* Fixing the doc comments for the AdvSimd.LoadVector methods

* Marking various IsSupported methods as new, now that they inherit from ArmBase

* Fixing a malformed doc comment that was missed

* Marking AdvSimd.Arm64 as [Intrinsic] and new

Signed-off-by: dotnet-bot <[email protected]>
2019-09-27 20:47:17 +02:00
Jo Shields 2072e4ce81 Special case in GetFullPath for UNC roots of \\.\ (#17066)
A valid path might be something like \\\\.\\pipe\\pipename (i.e. \\\\.\\ is a valid root). If we blindly strip all instances of `.\\` then we end up confusing our path cleaning algorithm. So permit a directory component of `.` in UNC path roots.

Closes: https://github.com/mono/mono/issues/16460
2019-09-27 13:35:18 -05:00
Zoltan Varga ac2ee5b8fc [wasm] Bump emscripten to 1.38.46. (#17094) 2019-09-27 13:34:02 -05:00
Maxim Lipnin 643bcbc3fc [wasm][xunit tests] Fix issue "System.MissingMethodException: Default constructor not found for type" (#17082)
Fixes https://github.com/mono/mono/issues/16879
2019-09-27 13:59:15 -04:00
Alexander Köplinger 1b9d8ec9eb Fix mono-extensions build (#17100)
It needed an update because https://github.com/mono/mono/commit/5b108b5726ec196f4b61d30f406919ce70211754 touched code the patch modified.
2019-09-27 16:45:57 +02:00
Vlad Brezae 194506d495 [mini] Expand clr-memory-model effect (#17093)
* [mini] Add memory barriers to more stores inside objects

We should now have membars for all stores of an object ref inside another object (for the stores done by the jit). Used by clr-memory-model debug option.

* [mini] Add memory barrier for storing objref in static fields

For clr-memory-model debug option.
2019-09-27 12:36:04 +03:00
dotnet-maestro[bot] ba093a07b7 Update dependencies from https://github.com/dotnet/arcade build 20190923.5
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19473.5
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19473.5
2019-09-27 10:32:57 +02:00
dotnet-maestro[bot] d5c49121b6 Update dependencies from https://github.com/dotnet/corefx build 20190926.2
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19476.2
2019-09-27 10:32:57 +02:00
dotnet-maestro[bot] e062ebf096 Update dependencies from https://github.com/dotnet/core-setup build 20190926.2
- Microsoft.NETCore.App - 5.0.0-alpha1.19476.2
2019-09-27 10:32:57 +02:00
Aleksey Kliger (λgeek) 1dc63ced73 [runtime] Respect runtime_version in mono_init_internal (#17085)
In 6d2c77fb33 we changed the representation of
the runtimes that we will probe from a NULL-terminated array to a linked list.
Unfortunately we made a typo, and in the case where the caller of
mono_init_internal provided a runtime_version, we still picked
DEFAULT_RUNTIME_VERSION instead of the given runtime_version.

This is an issue for embedders like XA that use the "mobile" runtime - all
their assemblies are compiled with the "mobile" version number (2.0.5.0) but
Mono running with the default runtime configuration will remap to 4.0.0.0.  As
a result, when there are references to 2.0.5.0 assemblies, the load requests
will be remapped to 4.0.0.0 which will never see the already loaded 2.0.5.0
assemblies, and we will fall back on the embedder's preload hook or filesystem
probing every single time.  This is a performance fail.
2019-09-26 17:30:37 -04:00
Alexander Köplinger bffd4c6b99 Revert "Inline TLS access. (#16882)" (#17092)
Reverts mono/mono#16882

This broke the Mono Android SDKs build in master though for some reason it doesn't show up in the PR build:

```
[2019-09-26T03:19:55.366Z] Undefined symbols for architecture x86_64:
[2019-09-26T03:19:55.366Z]   "_mono_tls_key_jit_tls", referenced from:
[2019-09-26T03:19:55.366Z]       _mono_tls_get_jit_tls in libmain_a-main-sgen.o
[2019-09-26T03:19:55.366Z]   "_mono_tls_key_lmf_addr", referenced from:
[2019-09-26T03:19:55.366Z]       _mono_tls_get_lmf_addr in libmain_a-main-sgen.o
[2019-09-26T03:19:55.366Z] ld: symbol(s) not found for architecture x86_64
[2019-09-26T03:19:55.366Z] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[2019-09-26T03:19:55.366Z] gmake[6]: *** [Makefile:1634: mono-sgen] Error 1
[2019-09-26T03:19:55.366Z] gmake[6]: Leaving directory '/Users/builder/jenkins/workspace/archive-mono/master/android/debug/sdks/builds/android-host-Darwin-debug/mono/mini'
```.
2019-09-26 23:10:51 +02:00
Vlad Brezae bada362454 [sgen] Report roots from togglerefs (#17063) 2019-09-26 22:38:59 +03:00
Marek Safar 8da9eebcc6 [netcore] Correctly send exit code to the caller 2019-09-26 20:59:50 +02:00
Jan 3132cbd98b Add loading the default Mono configuration file. (#16961)
Add loading the default Mono configuration file.

Add loading the default Mono configuration file in the examples of Mono Embedded.
Also include header in existing usage of `mono_config_parse`.

This fixes problems under Raspbian, when extending the sample with your own code. 

The `teste.c` is still not compiling, because the `mono_set_allocator_vtable` function is not up to date. But I was not able to find the documentation for this function.
2019-09-26 20:46:16 +02:00
Vlad Brezae 470e53f6b4 [sgen] Fix heavy binary protocol build (#17037)
[sgen] Fix heavy binary protocol build
2019-09-26 19:16:04 +02:00
imhameed 5e30301ff3 Merge pull request #17031 from EgorBo/bmi-mulx
[LLVM] Implement Bmi2.MultiplyNoFlags
2019-09-26 09:29:54 -07:00
Alexis Christoforides f52678864d [crashing] Remove Mono signal handlers when starting to handle a crash
Fixes https://github.com/mono/mono/issues/16803
2019-09-26 16:41:35 +02:00
Alexis Christoforides 4727b2475f [merp] Output native library module name when 'whitelist all' mode is enabled (#16899)
Partially fixes https://github.com/mono/mono/issues/16689
2019-09-26 09:30:01 -04:00
Vlad Brezae 4a17f5ca11 [Mono.Debugger.Soft] Fix VirtualMachine detaching (#17020)
* Revert "[Mono.Debugger.Soft] Properly close connections"

This reverts commit 2da7707373.

This was previously reverted because it was causing issues and it still does. It is a hack fix. We signal the connection for closing and next we send the dispose command over the network. This is obviously racy.

* [Mono.Debugger.Soft] Make sure receiving thread always closes

When we want to close the connection, setting the close flag is not enough, because a thread can be already stuck in a blocking receive, without getting to check the flag. We fix this by additionaly breaking the connection, by using socket.Shutdown.

Fixes https://github.com/mono/mono/issues/7377

* Bump API snapshot submodule
2019-09-26 14:40:42 +02:00
Johan Lorensson 9b094e660c BeginConnect complete to early when not using callback. (#17035)
* BeginConnect complete to early when not using callback.

The following commit, https://github.com/mono/mono/commit/068e8fe0424806753d7ab72a7b9bf0e54b58408b#diff-33b28a9dc0b8032650ed77213481a99f
changed the way we do DNS resolve in BeginConnect using host name. Instead
of doing a blocking DNS resolve, it switched to an async DNS resolve, followed
by an async connect. The returned IAsyncResult was just representing the
completion of the DNS resolve task and not the underlying connect. It
also turns out that part of doing EndConnect, we did some validation
of the used IAsyncResult making sure it was an instance of SocketAsyncResult,
meaning that if EndConnect was called on the returned IAsyncResult from
BeginConnect, EndConnect would throw and exception.

NOTE, this only happens if you use pattern like this:

var result = sock.BeginConnect (hostName, port, null, null);
result.AsyncWaitHandle.WaitOne (2000);
sock.EndConnect (result);

It did not happened if you used callbacks, since then EndConnect would
be called as part of callback and that IAsyncResult was of correct type,
or if you did a BeginConnect with an EndPoint or IPAddress[] since that
won't trigger the DNS resolve.

Fix is to make sure we hand out the SocketAsyncResult instance that will
be used in our internal connect calls from BeginConnect. We also make
sure, we chain the DNS resolve task together with a task doing the async
connect (using the same SocketAsyncResult instance). This will make sure
the IAsyncResult returned from BeginConnect is the same one as used in
internal connect call, so it will only complete when the connect call
completes.

* Adding test for BeginConnect using host name.

* Bump API snapshot submodule
2019-09-26 14:34:28 +02:00
monojenkins e5b7473a21 [sdks] Bump min iOS version to 7.0. (#17073)
Because that's the min iOS version in all current versions of Xamarin.iOS.
2019-09-26 14:20:32 +02:00
Bernhard Urban c81b6f5ca9 [interp] always require GC Unsafe mode in exception checkpoint (#17016)
[interp] always require GC Unsafe mode in exception checkpoint

Follow-up for https://github.com/mono/mono/pull/16955

/cc @BrzVlad
2019-09-26 10:45:56 +02:00
Ben Adams 65dc4c364a CollectionsMarshal.AsSpan allow null refs (dotnet/coreclr#26903)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 10:10:42 +02:00
Levi Broderick 23fac785db Introduce Utf8Span, which is a span of UTF-8 text (dotnet/coreclr#26711)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 10:10:42 +02:00
Vlad Brezae f5bb650f78 [debugger] Properly close debugger thread when connection hangs
When we detach from the debugger, we reset the state by starting a new debugger thread (why don't we just use the existing thread?). As per commit https://github.com/mono/mono/commit/540ef385faa230bc226c7a83bfabd0730520911e we should shut down the debugger thread when the connection drops. However, since that commit was acting as a Dispose command was sent instead, it would end up relentlessly spawning and closing new debugger threads up until the runtime is shut down. This commit makes sure we don't respawn once the connection hangs.
2019-09-26 09:53:36 +02:00
Egor Bogatov 4a72dadb34 LLVM: Add comments (string metadata) for calls (#16956)
* Add string metadata to all calls (comments)

* Update mini-llvm.c

* rename tramp_%d to full name too
2019-09-26 09:51:36 +02:00
Jan Kotas 519380d009 Fix CoreRT build break
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 08:59:20 +02:00
Bernhard Urban 1faf71f446 [offsets-tool] install clang python package into a virtualenv (#17013)
[offsets-tool] install clang python package into a virtualenv

So it doesn't mess with the Python installation of the user.

/cc @mandel-macaque
2019-09-26 08:55:33 +02:00
Thays Grazia 5b108b5726 [threading] embedded mono hangs (#16907)
* [threading] embedded mono hangs

When embedded mono is executing a C++ code, it didn't start to execute a C# code, we don't need to suspend the thread to run GC.
Fixes #16192
Fixes #14725

* Changing what was discussed with Aleksey during our 1on1

* Changing what was suggested by Aleksey

* Separating mono_init_version that is being used on tests from the embedded ones
2019-09-25 17:12:33 -03:00
Jay Krell ed0fa35dc2 [Coop] Unconvert ves_icall_Mono_TlsProviderFactory_IsBtlsSupported. (#16995) 2019-09-25 14:40:38 -04:00
Stephen Toub c256e409c9 More Corelib cleanup (dotnet/coreclr#26872)
* Remove unnecessary asserts

* Make several classes static

* Use is instead of as+null check

* Use T? instead of Nullable<T>

* more static classes

* Mark locals as const

* Merge declaration and initialization of some variables

* Remove unnecessary casts

* Remove unnecessary "unsafe"s

* Simplify several lambda expressions

* Remove redundant parentheses

* Remove redundant '== true'

* Remove redundant empty lines

* Simplify boolean comparison with '== false'

* Replace if-statement with return statement

* Use while for infinite loop

* Add static to all partial static class declarations

* Use ++/-- operator instead of assignment

* Use string.IsNullOrEmpty

* Use coalesce expression

* Simplify lazy initialization

* Use coalese expression

* Join string expressions

* Use regular string literal instead of verbatim string literal

* Optimize StringBuilder.Append calls

* Remove redundant assignment

* Remove unnecessary unsafe context

* Merge processor directives

* Use String.Equals instead of String.Compare

* Use Debug.Fail instead of Debug.Assert(false

* Remove Attribute suffix

* Use predefined type

* Use compound assignment

* Use while statement to create an infinite loop

* Remove redundant base ctor call

* Avoid using catch (Exception)

* Remove empty regions

* Span comparison to null

* Avoid unnecessary boxing of value type

* Expression is always equal to 'true'

* Remove unused method

* update coalesce assignment

* fix unsafe

* fix redundant parens

* Fix whitespace errors introduced

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 20:13:47 +02:00
Egor Bogatov 39ff579432 [netcore] Disable annoying SourceLink warrnings (#17055)
Currently netcore-build doesn't need submodules (except LLVM) so a task somewhere in arcade/microsoft.build.tasks.git always complain if those are not cloned.

Set EnableSourceControlManagerQueries=false to ignore the warnings.
2019-09-25 12:28:53 -04:00
Alexander Köplinger ea6ea67351 Bump bockbuild to get recent fixes (#17054)
- https://github.com/mono/bockbuild/pull/121
- https://github.com/mono/bockbuild/pull/122
2019-09-25 16:52:42 +02:00
EgorBo 2ceb062c9b revise all X86 intrinsics 2019-09-25 14:58:53 +03:00
Alexander Köplinger aeca4a6cc3 Update System.csproj 2019-09-25 12:32:42 +02:00
Jay Krell baa9adc06a Fix comment about mono_thread_interruption_request_flag. (#17036)
Fix https://github.com/mono/mono/pull/16906/files#r327660329.
2019-09-25 12:30:27 +02:00
Ben Adams e48a285cfd Add CollectionsMarshal (dotnet/coreclr#26867)
* Add CollectionsMarshal

* Feedback

* Feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 09:55:20 +02:00
Levi Broderick e082719e3f Pack additional useful data into char's "Unicode categories" RVA static info (#26848)
Also improves the performance of char.IsWhiteSpace / char.IsUpper / char.IsLower

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 07:48:24 +02:00
EgorBo ba471607a3 Implement Bmi2.MultiplyNoFlags 2019-09-25 04:04:38 +03:00
Steve Pfister da02f1330f Enable additional wasm corefx xunit tests (#16860)
Add the other corefx xunit test assemblies to the build and adjust jenkins to run them on CI.
2019-09-24 17:27:50 -04:00
Egor Bogatov 37e46aa9ae LLVM: Intrinsify MathF.Round (#16840)
* improve MathF.Round

* prefer vroundss

* enable round only for jit
2019-09-24 20:53:23 +02:00
Bernhard Urban 153b5def01 [checked-build] update allowed thread states (#16971) 2019-09-24 14:53:10 -04:00
Marius Ungureanu 394d8d1f21 Add RenamedEvent* to FSW sources from CoreFX (#16755)
* Add RenamedEvent* to FSW sources from CoreFX

Fixes #16486 - This was only partially fixed until now

* [csproj] Update project files

* Add test files which test these code paths on corefx

* Remove old RenamedEvent* files
2019-09-24 20:49:02 +02:00
Egor Bogatov 8f85967612 Add loop related optimizations to LLVM JIT (#16436)
* Add loop-related optimizations

* Update llvm-jit.cpp

* Update llvm-jit.cpp

* Update llvm-jit.cpp
2019-09-24 20:45:32 +02:00
Egor Bogatov ed622e9112 [netcore] Implement missing Bmi1/Bmi2 intrinsics (#16919)
* Finalize BMI1

* cleanup

* Update simd-intrinsics-netcore.c

* Update simd-intrinsics-netcore.c

* fix LLVM-AOT
2019-09-24 20:44:03 +02:00
Stephen Toub c4aa67cbd9 Change several internal/private instance methods to be static (#26835)
* Replace ToLower(CultureInfo.InvariantCulture) with ToLowerInvariant()

* Mark several members static

Signed-off-by: dotnet-bot <[email protected]>
2019-09-24 20:06:06 +02:00
Jan Kotas 639f30f99b Remove IsFastSort optimizations from String (dotnet/coreclr#26759)
* Remove IsFastSort optimizations from String
- Fix bugs in the Span-based globalization fast paths. Note that some of these bug fixes are going to affect performance of the globalization fast paths.
- Use Span-based globalization fast paths for strings
- Avoid static array allocation for HighCharTable

Fixes #26758

Signed-off-by: dotnet-bot <[email protected]>
2019-09-24 20:06:06 +02:00
Maxim Lipnin f91dcb9c10 Fix time zone issue when jumping into DST (#16430)
Addresses an issue with jumping into DST for some time zones when the incorrect date-time offset is returned for date-time in UTC (which comes from DateTime.Now). The fix is to just check if the incoming date-time is in UTC.

Also added a set of tests for some time zones to verify jumping into DST in general.

Fixes https://github.com/mono/mono/issues/16395
2019-09-24 13:58:59 -04:00
Alexander Köplinger 0a3c9c451b [ci] Make ios/mac sdks archive URL more predictable
Requested by xamarin-macios team.
2019-09-24 15:26:32 +02:00
Bernhard UrbanandAleksey Kliger 749493d4b3 [threads] clear small_id_key TLS when unregistering a thread (#16973)
* [threads] clear small_id_key TLS when unregistering a thread

Fixes
```
* thread #12, name = 'tid_a507', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1be66144)
  * frame #0: 0x1be66144 libsystem_c.dylib`__abort + 184
    frame #1: 0x1be6608c libsystem_c.dylib`abort + 152
    frame #2: 0x003e1fa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="* Assertion at ../../../../../mono/utils/hazard-pointer.c:158, condition `mono_bitset_test_fast (small_id_table, id)' not met\n", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #3: 0x003abf44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #4: 0x003abfb4 monotouchtest`monoeg_assertion_message(format=<unavailable>) at goutput.c:184:22 [opt]
    frame #5: 0x003904dc monotouchtest`mono_thread_small_id_free(id=<unavailable>) at hazard-pointer.c:0:2 [opt]
    frame #6: 0x003a0a74 monotouchtest`unregister_thread(arg=0x15c88400) at mono-threads.c:588:2 [opt]
    frame #7: 0x00336110 monotouchtest`mono_thread_detach_if_exiting at threads.c:1571:4 [opt]
    frame #8: 0x003e7a14 monotouchtest`::xamarin_release_trampoline(self=0x166452f0, sel="release") at trampolines.m:644:3
    frame #9: 0x001cdc40 monotouchtest`::-[__Xamarin_NSTimerActionDispatcher release](self=0x166452f0, _cmd="release") at registrar.m:83445:3
    frame #10: 0x1ce2ae68 Foundation`_timerRelease + 80
    frame #11: 0x1c31b56c CoreFoundation`CFRunLoopTimerInvalidate + 612
    frame #12: 0x1c31a554 CoreFoundation`__CFRunLoopTimerDeallocate + 32
    frame #13: 0x1c31dde4 CoreFoundation`_CFRelease + 220
    frame #14: 0x1c2be6e8 CoreFoundation`__CFArrayReleaseValues + 500
    frame #15: 0x1c2be4c4 CoreFoundation`CFArrayRemoveAllValues + 104
    frame #16: 0x1c31ff64 CoreFoundation`__CFSetApplyFunction_block_invoke + 24
    frame #17: 0x1c3b2e18 CoreFoundation`CFBasicHashApply + 116
    frame #18: 0x1c31ff10 CoreFoundation`CFSetApplyFunction + 160
    frame #19: 0x1c3152cc CoreFoundation`__CFRunLoopDeallocate + 204
    frame #20: 0x1c31dde4 CoreFoundation`_CFRelease + 220
    frame #21: 0x1c304a80 CoreFoundation`__CFTSDFinalize + 144
    frame #22: 0x1bfa324c libsystem_pthread.dylib`_pthread_tsd_cleanup + 644
    frame #23: 0x1bf9cc08 libsystem_pthread.dylib`_pthread_exit + 80
    frame #24: 0x1bf9af24 libsystem_pthread.dylib`pthread_exit + 36
    frame #25: 0x0039df84 monotouchtest`mono_threads_platform_exit(exit_code=<unavailable>) at mono-threads-posix.c:145:2 [opt]
    frame #26: 0x0033bb84 monotouchtest`start_wrapper(data=0x1609e1c0) at threads.c:1296:2 [opt]
    frame #27: 0x1bf9b914 libsystem_pthread.dylib`_pthread_body + 128
    frame #28: 0x1bf9b874 libsystem_pthread.dylib`_pthread_start + 44
    frame #29: 0x1bfa3b94 libsystem_pthread.dylib`thread_start + 4
```

* Update mono/utils/mono-threads.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>
2019-09-24 13:07:19 +02:00
Maxim Lipnin 9988ebc7f2 [System.Xml] Fix deserialization issue (#16965)
[This ](https://github.com/mono/mono/pull/11194/files#diff-ae54d9565e735005686191de0f19ed50R431-R435)change in `XmlTypeMapElementInfo.GetElement`  introduced the logic for handling the case of derived types deserialization. 
For some cases (like https://github.com/mono/mono/issues/16918) it breaks existing functionality because it returns wrong element for given element name after searching withing derived types collection.

![image](https://user-images.githubusercontent.com/3258267/65416085-bedc0500-ddff-11e9-941c-f24045c64121.png)

Unfortunately, what I can do here is just reverting the related changes.

Fixes https://github.com/mono/mono/issues/16918
2019-09-24 11:35:15 +02:00
Thays Grazia 7e6d863686 [debugger] New way to filter exceptions to support VSWin features (#16825)
* Creating a new version of MOD_KIND_EXCEPTION_ONLY to support VsWin features, discussed with Joaquin Jares.
Because we don't have this functionality of get every other exception that is not specified, VsWin was getting every exception and filtering later. But this causes a slowness during the debugger because we stop all thread on each exception that we get, even if they are caught and we don't want to stop.

* Fixing other unit test because now we have 2 nested classes.
Fixing indentation.

* Creating a flag in the MOD_KIND_EXCEPTION_ONLY protocol event and not creating a new protocol event.

* Changing what was suggested by Zoltan.

* Coding style

* Bump API snapshot submodule
2019-09-23 18:16:59 -03:00
Alexander Köplinger 9b98db9ca9 Correctly check for HAVE_STAT_BIRTHTIME in configure.ac checks for System.Native (#16999)
The check was done incorrectly, the include files need to be separated by newlines otherwise we would get a false negative result.

config.log contained this:

```
conftest.c:279:23: warning: extra tokens at end of #include directive [-Wextra-tokens]
#include <sys/types.h>, #include <sys/stat.h>
                      ^
                      //
```

This means that when doing File.GetCreationTime() we'd fall back to returning the last modified time.

It regressed when we switched to System.IO.File to the CoreFX implementation i.e. System.Native in https://github.com/mono/mono/commit/8f5cef936491e8b20888bbf18d426482c890637c.

Fixes https://github.com/mono/mono/issues/16974
2019-09-23 22:44:59 +02:00
Alexis Christoforides bc43f32c9d [merp] Use function names even in 'private crashes' mode (#16897)
Partially fixes https://github.com/mono/mono/issues/16689
2019-09-23 15:49:53 -04:00
Steve Pfister 15e939d542 Restore Previous ExecutionContext Capture Behavior (#16945)
Restores previous ExecutionContext Capture behavior where we were copying a new SynchronizationContext as opposed to a reference to the existing one.

The old behavior caused SynchronizationContextAwaitTaskContinuations to not post to a SynchronizationContext.

Fixes https://github.com/mono/mono/issues/16759

Reverts a fix for https://github.com/mono/mono/issues/12421
2019-09-23 10:52:23 -04:00
Alexander Köplinger 8db7d198b2 [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:53:57 +02:00
Alexander Köplinger 93c67f2a3c [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:48:43 +02:00
Alexander Köplinger 70e266a3af [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:45:02 +02:00
Alexander Köplinger 6ad929c317 [ci] Build iOS/Mac Mono sdks archive using Xcode 11
This was requested by the xamarin-macios team.
2019-09-23 15:42:36 +02:00
Jo Shields 1a8cc98160 Include Mono.WebBrowser.dll in mkbundle bundles (Closes: #7454) (#16972)
* Include Mono.WebBrowser.dll in mkbundle bundles (Closes: #7454)

Because System.Windows.Forms.dll includes an assemblyref on
Mono.WebBrowser.dll, bundles need to be able to include it
in order for WinForms to work (even if they don't actually
use a browser component)

Compare:

```
$ mkbundle hellowin.exe -o hellowin --cross mono-6.0.0-ubuntu-18.04-x64 --i18n all
From: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64
ERROR: Unable to load assembly `Mono.WebBrowser' referenced by `/home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Windows.Forms.dll'
```

to

```
$ mkbundle hellowin.exe -o hellowin --cross mono-6.0.0-ubuntu-18.04-x64 --i18n all
From: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64
Using runtime: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/bin/mono
     Assembly: /tmp/hellowin.exe
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Windows.Forms.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/mscorlib.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.Security.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Configuration.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Xml.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Security.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Core.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Numerics.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Drawing.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Runtime.Serialization.Formatters.Soap.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Accessibility.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.WebBrowser.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.Posix.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Data.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Transactions.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.EnterpriseServices.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.CJK.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.MidEast.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.Other.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.Rare.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.West.dll
systemconfig: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/etc/mono/config
machineconfig: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/etc/mono/4.5/machine.config
Generated hellowin
```

For the test code:
```
using System;
using System.Drawing;
using System.Windows.Forms;

public class HelloWorld : Form
{
    static public void Main ()
    {
        Application.Run (new HelloWorld ());
    }

    public HelloWorld ()
    {
        Button b = new Button ();
        b.Text = "Click Me!";
        b.Click += new EventHandler (Button_Click);
        Controls.Add (b);
    }

    private void Button_Click (object sender, EventArgs e)
    {
        MessageBox.Show ("Button Clicked!");
    }
}
```

* Also add Novell.Directory.Ldap (consumed by System.DirectoryServices)
2019-09-23 09:24:05 -04:00
Jay Krell 7497e4933c [interp] Improve debuggability with enum. (#16721) 2019-09-23 11:10:23 +02:00
Thays Grazia 1a277c31a0 [debugger] Crash when debugging iOS application that throws (#16958)
* Doing the same fix from Android to IOS.
* On mini-exceptions.c check if there is no ji we can continue and generate the exception without checking wrapper_type.

Fixes #16824
2019-09-22 16:12:10 -03:00
EgorBo 6257369adc netcore: fix debug build 2019-09-22 09:09:35 +02:00
Jay Krell aca55a42aa Inline TLS access. (#16882)
The same functions are referenced by JIT and runtime.
The JIT uses cannot be inlined. The runtime can.
Since I could not get anything else to build and the change is simple and mechanical, now two sets of functions are provided, inline and extern. Extern just calls inline.
This PR appears to change all the uses, but it is only the JIT uses. The runtime ones just work and keep the old names.

We should also consider building desktop with LTO/LTCG, which would do all this automatically, smarter, more generally. A quick experiment shows it grows the runtime size, which is why I say desktop.
2019-09-21 18:35:19 +02:00
Jay Krell b20c690e3e Partially inline mono_method_signature_checked/internal. (#16927)
* Inline common fast paths of mono_method_signature_checked and mono_method_signature_internal.

* PR: Restore profilers to use mainstream functions.
2019-09-21 11:09:41 +02:00
Bernhard Urban 0f181a9dc4 [arm64_32] fix pointer size in CallInfo (#16970)
Note that in the output below `a`, `b`, etc. are NSObjects, so they are really passed as pointer.

Before:
```
* thread #1, name = 'tid_303', queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x04f301d4 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=0x00000000) at nsstring-localization.m:81:9
   78   void *
   79   xamarin_localized_string_format_9 (NSString *format, id a, id b, id c, id d, id e, id f, id g, id h, id i)
   80   {
-> 81           return [NSString localizedStringWithFormat: format, a, b, c, d, e, f, g, h, i];
   82   }
   83
   84   }
(lldb) mbt 4
* thread #1
  * frame #0: 0x04f301d4 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=0x00000000) at nsstring-localization.m:81:9
    frame #1: 0x04d9984c aWatchOSExtension`interp_to_native_trampoline + 156
    frame #2: 0x04f41d5c aWatchOSExtension`ves_pinvoke_method(frame=0x05167af8, sig=0x1518afd0, addr=(aWatchOSExtension`::xamarin_localized_string_format_9(NSString *, id, id, id, id, id, id, id, id, id) at nsstring-localization.m:80), string_ctor=0, context=0x14550960, save_last_error=0) at interp.c:1411:2 [opt]
    NSString::xamarin_localized_string_format_9 @ 68 "calli.nat" || frame #3: 0x04f3bfa4 aWatchOSExtension`interp_exec_method_full(frame=0x05167d38, context=<unavailable>, clause_args=0x00000000, error=0x05168540) at interp.c:3290:5 [opt]
```

After:
```
* thread #1, name = 'tid_303', queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x04c2c1e0 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=8) at nsstring-localization.m:81:9
   78   void *
   79   xamarin_localized_string_format_9 (NSString *format, id a, id b, id c, id d, id e, id f, id g, id h, id i)
   80   {
-> 81           return [NSString localizedStringWithFormat: format, a, b, c, d, e, f, g, h, i];
   82   }
   83
   84   }
```
2019-09-20 18:02:20 -04:00
Jay Krell 12d3dcecc6 [Coop] ves_icall_property_info_get_default_value. (#16614)
This includes resolving a somewhat deep problem involving
a void* that is usually to a int/float, but can be a MonoObject** or at least MonoString**.
An additional MonoStringHandleOut parameter is pass around to address this.
Perhaps it should be MonoObjectHandleOut for more generality.
2019-09-20 17:58:16 -04:00
Bernhard Urban e27159174a [interp] avoid exception checkpoint when in thread is in GC Safe state (#16955)
[interp] avoid exception checkpoint when in thread is in GC Safe state 

Usually the interpreter operates in GC Unsafe state. There is an exception however, and that is when it executes a `managed-to-native` wrapper in cooperative (or hybrid) suspend.

The wrapper does essentially:
1. enter GC Safe state via `mono_threads_enter_gc_safe_region_unbalanced`
2. call native function
3. exit GC state via `mono_threads_exit_gc_safe_region_unbalanced`

Usually upon return of a native function call, the interpreter would check for thread interruption and, if applicable, throw an exception.  That, however, would mess with the GC state as the interpreter operates in GC Safe state at this moment.

Note that ignoring thread interruption at that point shouldn't be a problem, because after 3. the `managed-to-native` wrapper will check for thread interruption anyway.

Contributes to https://github.com/mono/mono/issues/16819
2019-09-20 22:15:55 +02:00
Alexander Köplinger 6433ea9cce [ci] Add SymStore directory for netcore publishing stage
It's required by arcade in some channels.
2019-09-20 20:06:11 +02:00
Jay Krell 8b4c1ef828 Use ld -Bsymbolic when linking AOT. (#14511)
The runtime already does this itself.
2019-09-20 13:12:01 -04:00
Alexander Köplinger 6bc988b087 [netcore] Move to new arcade post build using yaml stages (#16966)
This is required by the newer arcade builds.
We can simplify our scripts since we no longer need the special publishwitharcade.proj.

Added copyright and license to the .nuspecs since the new arcade validation now
verifies this against a set of Microsoft policies.

We also need to make the .nupkg pre-release by appending the -preview suffix as stable versions are not permitted in non-stable feeds.
2019-09-20 17:27:15 +02:00
Aleksey Kliger (λgeek) 3757a7296c [System] Make FileSystemWatcher backend non-static (#16922)
* [System] Make FileSystemWatcher backend non-static

The FileSystemWatcher `watcher` field is the backend IFileWatcher that's
supposed to be used for the actual operations.

Each backend has a `bool GetInstance (out watcher)` method that returns a
singleton object, so each new instance of FileSystemWatcher gets the same
IFileWatcher instance.  (They will get different `watcher_handle` instances
which are used by some backends to uniquely identify this FileSystemWatcher).

However when the first FileSystemWatcher instance is `Dispose`d, it will set
`watcher = null` which will prevent the remaining instances from calling

    watcher?.StopDispatching (watcher_handle);
    watcher?.Dispose (watcher_handle);

which means that the backend won't properly cleanup resources for any other FSW
instances that have other `watcher_handle` values.

Addresses https://github.com/mono/mono/issues/16709
2019-09-20 11:21:30 -04:00
Rolf Bjarne Kvinge 95d17b16a8 [offsets-tool] Install clang into the user-specific python directory.
Because the system location isn't world writeable on Catalina.
2019-09-20 16:19:32 +02:00
monojenkins c8450814a2 [bcl] Update BCL Linked Size 2019-09-20 15:02:00 +02:00
Alexander Köplinger 74b9ca426a [ci] Remove more XCODE32_DIR usages (#16964)
Follow-up to https://github.com/mono/mono/pull/16936
2019-09-20 14:00:22 +02:00
Egor Bogatov 32321375ee Implement WriteCore and ReadCore in DeflateStream (#16951)
No idea why these were left unimplemented (probably were not used at that point).
Fixes #16950
2019-09-20 12:06:45 +02:00
Alexander Köplinger 857284b412 Add net_4.8.xml to EXTRA_DIST and bump binary-reference-assemblies again
The file was missed in https://github.com/mono/mono/pull/16912.
Also pull in another reference assemblies update.
2019-09-20 11:59:51 +02:00
Filip Navara 338170ecce Fix check for SafeHandle types 2019-09-20 11:44:25 +02:00
monojenkins 6d3f5732f0 [sdks] Remove the mac32 build. (#16952)
Xamarin.Mac doesn't need nor use it anymore.

This also removes the last part of the mono archive that required Xcode 9.4.
2019-09-19 21:45:57 -05:00
Jay Krell d49d7b4716 [Coop] InterlockedCompare/Exchange (object/T) (#16873)
* [Coop] InterlockedExchange (object and T), InterlockedCompareExchange (object and T).

 - Usually intrinsified.
 - Always?
 - Avoid coop handles by manually always using ref to object.
 - For efficiency on input, and efficiency wrt MonoError.
 - For correctness on output (due to aliasing in the wrappers).
 - To probably avoid assertion failure in runtime around icall marshaling of generics.
 - Unclear on location.

* Balance parens in prints, so they are balanced in the code.

* Reduce local arrays from 16 to auto-sized 1.

* Update and reformat System.Threading.Interlocked signature recognition.

* Rename the generic again to "_T" (or "Generic"?) on the theory
that the aot-runtime.c signature check is too strict/incorrect,
but that if we loosen it, it is not clearly specific to the generic.

* Revise corlib version again.

* Bump API snapshot submodule
2019-09-19 22:36:18 -04:00
Zoltan Varga c730a2a2e3 [llvm] Handle recursive calls as a direct call. (#16944)
Fixes https://github.com/mono/mono/issues/16943.
2019-09-19 21:43:49 -04:00
Jay Krell d02c121f79 Inline mono_thread_interruption_request_flag. (#16906)
Note that the name mono_thread_interruption_request without trailing flag
was already taken.
2019-09-19 15:49:43 -05:00
Filip Navara 81b5819528 [netcore] Show line numbers when xunit tests fail (#16931) 2019-09-19 14:08:07 -04:00
Alexander Köplinger f6a080bb79 Add .NET 4.8 reference assemblies (#16912)
Brings in https://github.com/mono/reference-assemblies/pull/20
2019-09-19 14:06:41 -04:00
Egor Bogatov 5e5033f5e5 Fix StackOverflow in ApplicationTrustEnumerator
ApplicationTrustEnumerator used to call itself. The following snippet crashed with SO:

using System.Security.Policy;
public class Program {
        public static void Main (string[] args)
        {
            ApplicationTrustCollection col = ApplicationSecurityManager.UserApplicationTrusts;
            col.GetEnumerator ();
        }
}
2019-09-19 14:05:15 -04:00
Jay Krell 06a814e66f Balance parens in prints, so they are balanced in the code. (#16930) 2019-09-19 12:19:59 -04:00
Martin Baulig 1256d6c8cf Use NetworkStream from CoreFX. (#16889)
* Use `NetworkStream` from CoreFX.

This brings the `NetworkStream` class and related tests from CoreFX.

* `SocketAsyncEventArgs`: partially replace the following properties
  with the CoreFX version: `Buffer`, `BufferList`, `Offset`, `SetBuffer ()`
  as well as the internal `CopyBufferFrom ()`.

* `SocketAsyncResult`: change `Buffer` from `byte[]` to `Memory<byte>`.

* Add internal `Socket` overloads:
   - `Receive (Memory<byte> buffer, int, int, SocketFlags, out SocketError)`
   - `ReceiveFrom (Memory<byte>, int, int, SocketFlags, ref EndPoint, out SocketError)`
   - `SendTo (Memory<byte>, int, int, SocketFlags, EndPoint)`

* Implement async task overloads from CoreFX (`Socket.Tasks.cs`) to `Socket`:
   - `int Read(Span<byte>)`
   - `Task<int> ReadAsync(byte[], int, int, CancellationToken)`
   - `ValueTask<int> ReadAsync(Memory<byte>, CancellationToken)`
   - `int ReadByte()`
   - `void Write(ReadOnlySpan<byte>)`
   - `Task WriteAsync(byte[], int, int, CancellationToken)`
   - `ValueTask WriteAsync(ReadOnlyMemory<byte>, CancellationToken)`
   - `void WriteByte(byte)`

* remove internal unused `PooledStream` and `ConnectionPool` APIs.

* bring new tests from CoreFX:
  - `System.Net.Sockets/tests/FunctionalTests/NetworkStreamTest.cs`
  - `System.Net.Sockets/tests/FunctionalTests/NetworkStreamTest.netcoreapp.cs`

* corlib: add `System.IO.FileSystem` and `System.Net.Sockets` resources and update SR.

* Update Socket.cs

* [csproj] Update project files

* Bump API snapshot submodule
2019-09-19 14:46:29 +02:00
Maxim Lipnin 12949cf2e6 [wasm] Switch to PNSE version of FileSystemWatcher and exclude xunit tests (#16892)
Fixes https://github.com/mono/mono/issues/16876
2019-09-19 08:13:02 -04:00
Zoltan Varga a3d05c2f88 [ci] Precompile runtime tests on the mac-sdk lane before running them with llvm. (#16916) 2019-09-19 12:58:31 +02:00
Jay Krell 8d23dc71b6 Inline mono_object_unbox_internal and mono_object_get_data. (#16856) 2019-09-19 12:41:29 +02:00
Filip Navara 85056b086f Implement Marshal.SetLastWin32Error 2019-09-19 09:59:49 +02:00
Johan Lorensson ad997d3725 Fix build warnings in libmonoruntime. (#16802) 2019-09-19 08:45:44 +02:00
Johan Lorensson 077e76c470 Explicit disable hybrid suspend when cooperative suspend is enabled. (#16798)
hybrid suspend won't get disabled when using --enable-cooperative-suspend
triggering configure error:

Hybrid suspend and Cooperative suspend cannot be both enabled.

Fix explicit set hybrid suspend default behavior to no if cooperative
suspend has been configured.
2019-09-19 08:45:00 +02:00
Ryan Lucia 529a11cf5b Make case-insensitive class lookup work correctly with exported types (#16652)
* Make case-insensitive class lookup work correctly with exported types

* Search all matching namespaces

* Skip AOT cache for case-insensitive lookups
2019-09-18 17:18:30 -04:00
Egor Bogatov 9a4a624501 [netcore] Implement more System.Runtime.Intrinsics.X86.Bmi1/Bmi2 (#16891)
* Implement more BMI1/BMI2

* implement more bmi1 intrinsics

* Update simd-intrinsics-netcore.c
2019-09-18 15:08:35 -04:00
Alexander Köplinger 2cc7c410a7 [ci] Allow running only msbuild tests in "OSX Extended" runs
Needed for mono/msbuild CI so we can skip the lengthy runtime LLVM tests.
2019-09-18 17:35:57 +02:00
Neale Ferguson 86f7cef7f4 - Some more prep for SIMD on s390x (#16903)
- Use __builtin_bswapxx for s390x rather than inline assembler
- s390x ABI specifies that %f8-%f15 are protected between calls so avoid using or save/restore as required
- Save the "protected" FP registers in the context structure
2019-09-18 15:54:27 +02:00
Vlad Brezae b2739b5196 [mini] Add missing membars when initializing rgctx entries (#16904)
[mini] Add missing membars when initializing rgctx entries

Whenever we are publishing changes to the rgctx arrays, we are racing with rgctx_lazy_fetch_trampoline, so we need to make sure any pointers we set there must have their contents initialized.
2019-09-18 09:16:17 -04:00
Jay Krell 12c48df242 All existing safepoints appear to be mooted by pinned coop handles. (#16695)
Perhaps they are never needed then.
2019-09-17 23:39:09 +02:00
Александр Хухлаев f91b64d9ad [WinForms] fix #16626 ControlBox property didn`t hide Close button and icon when set to false (#16627) 2019-09-17 14:26:05 -07:00
Alexander Köplinger e1f5738371 [netcore] Fix compiler warnings (#16883)
This fixes the compiler warnings when building netcore-only on macOS.
2019-09-17 21:18:54 +02:00
monojenkins 2b98112ce1 Rename bundle identifier for the various Mono.frameworks we create for Xamarin.iOS. Fixes xamarin/xamarin-macios#7005. (#16902)
This is the equivalent of
https://github.com/xamarin/xamarin-macios/commit/a9abc07cb357f9f012e3da66c70fa09d96bf02f7,
just for the Mono.frameworks in the mono archive.

Fixes https://github.com/xamarin/xamarin-macios/issues/7005.
2019-09-17 21:18:22 +02:00
Alexander Köplinger f99acb8fa6 [corlib] Fix building nunit-lite twice (#16894)
We sometimes saw the following build error in Mono SDKs builds on Jenkins:

```
[2019-09-16T01:03:30.709Z] CSC     [xammac] nunit-lite-console.exe
[2019-09-16T01:03:30.709Z] error CS0009: Metadata file '/Users/builder/jenkins/workspace/archive-mono/2019-08/mac/release/mcs/class/lib/xammac/nunitlite.dll' could not be opened -- PE image doesn't contain managed metadata.
```

Taking a closer look we found out that nunit-lite was trying to be built twice for the same profile which caused a race where the file was corrupted.

The reason for this is that the corlib Makefile defines two test assemblies: the normal PROFILE_corlib_test.dll and BinarySerializationOverVersionsTest.dll and
integrating the BinarySerializationOverVersionsTest.dll via the `test-vts` target incorrectly overwrote the `test` target instead of using `test-local`.
2019-09-17 20:25:52 +02:00
Vlad Brezae 906c71b8bf [interp] Fix stack size regression (#16887)
Partly revert https://github.com/mono/mono/commit/b18840e92cb5b1afef6c0e5df05cc026eb32efbf
2019-09-17 19:27:51 +03:00
Stephen Toub 569878fe14 Remove allocations from Dns.* (dotnet/corefx#41061)
This started as an effort to reduce the size of System.Net.NameResolution.dll when publishing a trimmed app.  It's not that big to begin with, but it's carrying around a copy of all of the IAsyncResult helper types, because the Get*Async methods are currently wrappers for the Begin/End* methods.

This PR inverts that, wrapping the Begin/End* methods instead around the Get*Async methods, using the same TaskToApm helper we use in other places in corefx for the same purpose.  This makes the Get*Async methods faster and lighterweight, but it does increase the number/amount of allocation in the Begin/End* APIs.  Since these are considered legacy, I normally would consider that a good trade, however we still use these Begin/End methods in a few places in System.Net.Sockets, and I didn't want to regress those use cases.

So, this also then trims some additional fat, which helps the Get*Async cases even further, and gets the Begin/End* to be even better than before the change.  This includes not allocating an IPHostEntry when we're just going to unwrap it and return its addresses, computing the exact IPAddress[] size we need rather than using a List<> to grow it and ToArray to create the actual array, avoiding creating the HostName if we don't need it, and avoiding an unnecessary SafeHandle allocation.

As part of this, I also noticed that we had some bugs in how some of our interop structures on Windows were defined.  In particular, fields that in the native types were size_t were defined as int rather than IntPtr in the managed code.  It appears we were saved by padding, but I fixed it regardless.

And as long as I was changing pretty much everything else, where I was touching code I also cleaned up some legacy style stuff.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 16:13:18 +02:00
dotnet-maestro[bot] a195b746c6 Update dependencies from https://github.com/dotnet/roslyn build 20190912.11 (#16885)
- Microsoft.Net.Compilers - 3.3.1-beta4-19462-11
2019-09-17 13:18:45 +02:00
Johan Lorensson e4998f5fc4 Exclude AppleTypes from orbis System.dll. (#16814)
* Exclude AppleTypes from orbis System.dll.

* [csproj] Update project files
2019-09-17 13:03:16 +02:00
Jay Krell fb71d47543 [interp] Use +=, *= etc. Seems to generate identical code though. (#16848)
i.e. a = a op b is *not* double evaluating a.
2019-09-17 13:23:19 +03:00
Ryan Lucia 6252794ebe Temporarily disable embedded ppdb data decompression (#16868)
This addresses #16373. We will work on a proper fix, but this at least unblocks Roslyn CI
2019-09-17 11:23:05 +02:00
Bernhard Urban 578e1a9dda [arm64_32] make "Debug Mode" work on Watch series 4 with --interpreter (#16865)
[arm64_32] make "Debug Mode" work on Watch series 4 with --interpreter

Main objective for reviewers: This PR shouldn't change anything for `arm64`.

Context: "Debug Mode" on the Apple Watch allows execution of regular machine code and thus is pretty similar to FullAOT on iPhone, i.e. it doesn't require bitcode, but _still_ doesn't allow `signal(3)` so cooperative suspend is required.

Apple Watch Series 4 and onward use an ARM64 chip, but with a different ABI, also known as `arm64_32`. This PR changes the AOT compiler and related trampolines so that the AOT compiler can generate all necessary code to run the interpreter on this target.


Changes:

--------------------------------------

[arm64_32] set register size to 8 in cross compiler too 

-----------------------------------------

 [machine] update `target_mgreg_t` definition

`SIZEOF_REGISTER` is usually the same as `TARGET_SIZEOF_VOID_P`, except when `MONO_ARCH_ILP32 is defined.

`target_mgreg_t` should represent the width of the hardware register, that is, `SIZEOF_REGISTER`.

---------------------------------------------

[arm64_32] fix AOT image layout 

---------------------------------------------

 [arm64_32] make trampolines bitwidth aware

* replace pointer loads/stores with bitwidth-dependent macro
* reflect `specific` trampolines so that only two instructions are used for ILP32, since the related pair in the data page only needs 2 * 32bits too
* replace some `sizeof (target_mgreg_t)` with `TARGET_SIZEOF_VOID_P` (former is hardware register size, latter is pointer size)

---------------------------------------------
These regression tests are _not_ passing: https://github.com/mono/mono/issues/16864 Also some of monotouch-tests are crashing, see https://github.com/mono/mono/issues/16819

Here is a screenshot of a debug session in VSMac:
<img width="1321" alt="Screenshot 2019-09-16 at 19 13 22" src="https://user-images.githubusercontent.com/75403/64981326-1dfbc000-d8bc-11e9-9d16-48036f36b638.png">


Contributes to https://github.com/mono/mono/issues/10641
2019-09-17 05:15:19 -04:00
Nikolay Sivov 08521a1c43 [cominterop] Default to [in] parameter direction when not specified explicitly.
Treat [In, Out], [In] and unspecified as input arguments. Important change from
current behavior is to allow parameter without directional attributes.

Use case example is _MethodInfo.Invoke() method called from native client.
2019-09-17 11:06:17 +02:00
Jay Krell af185d2c1e [interp] Cleanup dispatch macros. (#16784)
- Factor out reptition into already existing macros.
 - Remove the default for computed goto which is dead.
   Computed goto has no notion of default. It is just
   an unchecked array access.

Extracted from https://github.com/mono/mono/pull/16721.
2019-09-17 10:55:45 +02:00
monojenkins 16ed5ec3aa [csproj] Update project files 2019-09-17 10:53:46 +02:00
Radek Doulik e7f3db9796 [android sdk] Add aprofutil tool 2019-09-17 10:53:46 +02:00
Zoltan Varga 1031c4650d [llvm] Use IntPtrType () for byref arguments instead of a pointer, the latter causes problems when pointer operations are used along with phi nodes. Fixes https://github.com/mono/mono/issues/16808. (#16874) 2019-09-17 03:53:52 -04:00
Vlad Brezae aaaa2bdd5e [interp] Optimize multidimensional array access (#16822)
* [interp] Pass rank instead to LDELEMA

It feels more intuitive and uses less computations.

* [interp] Avoid unnecessary type check

Loading element address of array requires type checks only if the elements of the array are references.

* [interp] Avoid using MINT_CALLRUN for Get intrinsic

It is very slow. Use ldelema/ldobj pair instead.

* [interp] Optimize MINT_LDELEMA

* [interp] Rename ldelema_fast to ldelema1

ldelema_fast was handling arrays with a single dimension. Rename it to better suggest this.

* [interp] Avoid using MINT_CALLRUN for Set intrinsic

* [interp] Remove some duplicated and confusing code

* [interp] Fix stack type of MINT_NEWARR result
2019-09-17 10:32:20 +03:00
Stephen Toub 87072b55c0 Update typo in comment in CultureInfo.cs
Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 09:19:11 +02:00
Tarek Mahmoud Sayed 77473d11b7 Fix calendar instance inside the culture after cloning (#26721)
When creating a new culture, we’ll have the equality CultureInfo.DateTimeInfo.Calendar == CultureInfo.calendar be true. After cloning such culture, this equality would not be true. The fix here is to ensure if the equality is true before cloning then should be true after cloning.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 09:19:11 +02:00
Zoltan Varga 38b2d4f4fe [wasm] Bump emscripten to 1.38.45. (#16871) 2019-09-16 23:37:30 -04:00
Johan Lorensson e8570d9500 Fix warnings in eglib gspawn.c on platforms lacking fork/execv. (#16801) 2019-09-16 17:18:59 -04:00
Filip Navara 17fe4757a3 [netcore] Make Environment calls thread-safe (#16852)
* [netcore] Make Environment calls thread-safe

* Address PR feedback

* Address PR feedback
2019-09-16 17:16:14 -04:00
Ryan Lucia a23da4b8b3 [debugger] remove extraneous ErrorCode declaration (#16862)
With the single point of exit, this means the main ErrorCode isn't being written to and uninitialized memory is returned.
2019-09-16 17:14:42 -04:00
Zoltan Varga 422d559ea4 [wasm] Initial netcore support. (#16718)
* [runtime] Fix configure support for netcore+out of tree builds. Add wasm support for generating netcore/config.make.

* [runtime] Add bundle support for the netcore assembly loader.

* [netcore] Add missing linker exclusion.

* [wasm] Add netcore builds, enabled by defining ENABLE_WASM_NETCORE in Make.config.

The targets are 'package-wasm-runtime-netcore' and 'package-wasm-bcl-netcore'.
The binaries are copied to out/wasm-runtime-netcore-<CONFIG> and out/wasm-bcl/netcore.

* [wasm] Add netcore support to the build system and packager.

* Fix the build.

* Fix the dynamic build.
2019-09-16 15:42:57 -04:00
Kenneth Pouncey e8b2b43f9b Bump sdk and bindings versions (#16809) 2019-09-16 09:34:41 -05:00
Kenneth Pouncey d435a85658 Bump Browser tests Karma version (#16810) 2019-09-16 09:31:44 -05:00
Nikolay Sivov 800e2cc4b4 [corlib] Improve reflection interface definitions (#16324)
* [corlib] Fix _Type interface definition.

* [corlib] Fix Type class attributes.

* [corlib] Fix _MethodInfo interface definition.

* [corlib] Fix MethodInfo class attributes.

* [corlib] Merge MethodInfo partial definitions.

* [corlib] Remove leftovers from testing.

* [corlib] Do not define new attributes for MOBILE build.

* Bump API snapshot submodule

* [corlib] Add explicit _MethodInfo.GetType() implementation.

* Bump API snapshot submodule

* [csproj] Update project files
2019-09-16 13:00:40 +02:00
Jay Krell 9f3effc177 Strike a size and conditional branch balance between (#16836)
https://github.com/mono/mono/pull/16835
and
 https://github.com/mono/mono/pull/16834

The main 4 call opcodes should have no conditionality here.
The non-virtuals no extra branch.
The other opcodes all have one condition and branch into common code.

Everything could be smaller or faster but not both.
2019-09-16 10:10:47 +03:00
Jay Krell 67fb575d11 [Coop] ParseRouteInfo, fix some leaks, cleanup. (#16815) 2019-09-16 00:10:12 -04:00
Johan Lorensson 7fa706a316 Fix infrequent hangs in test-runner. (#16793)
* Fix infrequent hangs in test-runner.

test-runner has been plagued with very infrequent hangs causing long
timeouts on CI. Turns out that there is a race in AsyncStreamReader
used by Process class for handling stdout/stderr when closing down
process. Since AsyncStreamReader::Dispose didn't make sure no pending
async read requests were still in flight before closing the stream
and underlying handle, it led to a race condition deadlocking the
complete process, hanging test-runner. A similar race has also been
observed but instead of causing a deadlock, it could also manifest as an
System.ObjectDisposedException when doing EndRead while another thread
is disposing the underlying stream. Hitting that race didn't deadlock
the process but failed to complete invoke of test-runner.

Fix is to better protect the async callback and the thread doing the
dispose call. In order to make sure we don't close underlying handle
while still having async reads in flight, fix also checks async result
for completion and if not completed when trying to dispose AsyncStreamReader
it will try to cancel outstanding pending IO and wait for completion
before closing the stream.

Since the issue has only been seen on Windows, the fix will try
to cancel pending IO and wait for completion before closing stream
on Windows. On other platforms, old behavior will be preserved, meaning
that the implementation will still be vunarable to race between pending
reads and close of stream, but since problem has not been observed on
other platforms and since there is no good way of cancel outstanding
blocking reads, old behavior is kept.

At some point in future we should look into replacing the implementation
of AsyncStreamReader and use cancelation tokens, but since we don't have
all corefx support around cancelation in Mono BCL, that currently won't
solve the issue, meaning that we need to bring in more classes. Since the
problem also occur in test-runner using the build BCL profile, even more
needs to be added to that profile in order to get similar support. But due
to the infrequency of the problem and that it has only been observed on
Windows, the isolated fix seems more reasonable.

* Build error fix.

* Review feedback.

* Fix type cast not working on linux builds.

* Adjust icall naming in icall-def.h.
2019-09-15 19:51:07 -05:00
Jay Krell 8da3616e7f [Coop] GetValueInternal. (#16811) 2019-09-15 19:50:25 -05:00
Johan Lorensson 9249d38079 Fix warnings in libmono. (#16813) 2019-09-15 19:49:53 -05:00
Zoltan Varga cdb04c042f [netcore] Change an ArrayList to a List<T>. (#16842)
* [netcore] Change an ArrayList to a List<T>.

* [netcore] Call the generic version of Array.IndexOf () in the ICollection wrappers. Update exclusions file.
2019-09-15 20:45:13 -04:00
Egor Bogatov 0438397b8a LLVM: Dump Module instead of Function in JIT mode (#16837)
Since each function has its own module (in JIT mode) it makes sense to dump the whole module for MONO_VERBOSE_METHOD instead of just LLVM::Function.
It adds metadata, attributes and global variables (so it will be possible to copy-paste it to godbolt)

Example

static int Test(int x)
{
    return x / 10;
}

Diff for Unoptimized IR: https://www.diffchecker.com/NrF413Ts
Diff for Optimized IR: https://www.diffchecker.com/jbEkeFFS

So now we can see global variables and metadata for each function (and can easily paste them to godbolt.org)

NOTE: optimized IR for this case contains "dead" variables (dump_module allowed us to see such), we probably need some Module-wide optimizations to strip dead code such as -globaldce

NOTE2: why do we emit exceptions in the first place for this case at all?
2019-09-15 20:43:10 -04:00
Jay Krell 304f3add20 [interp] Speed up passing this parameter. (#16846) 2019-09-15 22:26:41 +03:00
Dotnet-GitSync-Bot 189af5d0c2 Remove some boxing from tuples with >= 8 elements (dotnet/coreclr#26584) (#16844)
Take advantage of https://github.com/dotnet/coreclr/pull/14698 to avoid boxing the TRest argument and improve devirtualization.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-15 12:48:15 -04:00
Calvin Buckley babe881bf2 Fix small sized argument output with tracing enabled (#16841)
This is a legacy of before the profiler was used for the trace
output; it would apparently get the raw stack pointer and try to
dereference things by recalculating the offset as needed (no for
little, yes for big). However, because the profiler returns the
actual fixed-up pointer, this recalculation isn't needed on BE,
but instead grabs garbage a bit ahead on the stack.

To fix, use what was the little endian case unconditionally.

Example before:

[1: 20.70598 14] ENTER: System.Globalization.CultureInfo:.ctor (int,bool,bool)(this:700000000408fb0[System.Globalization.CultureInfo platform.exe], 271824864, -32, -32, )

Example after:

[1: 1.93377 14] ENTER: System.Globalization.CultureInfo:.ctor (int,bool,bool)(this:700000000408fb0[System.Globalization.CultureInfo platform.exe], 127, 0, 1)
2019-09-15 11:50:32 -04:00
Jay Krell 9608debd7b [Coop] ves_icall_InternalExecute. (#16611) 2019-09-13 08:57:45 -05:00
Radek Doulik 131b3a9a92 [aot] Do not wrap tool_prefix path when calling strip (#16792)
[aot] Do not put quotes around wrapped path

Should fix: https://github.com/xamarin/xamarin-android/issues/3619

As it results in double quoting, which vanishes the effect and breaks
AOT for XA on Windows.

The actual error:

    [aot-compiler stdout] Stripping the binary: ""C:\Program Files (x86)\Microsoft Visual Studio\2019\Stable\MSBuild\Xamarin\Android\ndk\arm-linux-androideabi-"strip" --strip-symbol=\$a --strip-symbol=\$d obj\Release\90\aot\armeabi-v7a\libaot-Xamarin.Android.Arch.Core.Common.dll.so.tmp
    [aot-compiler stderr] '""C:\Program' is not recognized as an internal or external command,
    [aot-compiler stderr] operable program or batch file.
    [aot-compiler stderr] AOT of image C:\Users\peter\source\repos\App9\App9\App9.Android\obj\Release\90\android\assets\Xamarin.Android.Support.CoordinaterLayout.dll failed.
2019-09-13 09:05:27 -04:00
Filip Navara 10e72a7ab6 Improve alias analysis for pointer sized variables (#16812) 2019-09-13 08:02:50 -05:00
Vlad Brezae 6908a1941b [interp] Propagate locals and kill dead ones afterwards (#16791)
* [interp] Add flag for locals that have their address taken

The previous handling of LDLOCA was broken because if the address was taken before the instructions that would be optimized we would miss any potential changes to the local value. Right now we just skip optimizing locals that have their address taken. This is achieved simply by not pushing LDLOC on the evaluation stack, for those locals.

* [interp] Add separate counters for optimized instructions

To provide more insight.

* [interp] Propagate locals

Whenever we generate a movloc we remember what local was copied. When loading a local we check if it's currently storing the value of another local, in which case we load the original local instead.

* [interp] Kill instructions that write to dead locals

We track the liveness of a local by counting how many times the local is loaded using ldloc. When such instructions are optimized, the ldloc counter is updated accordingly.

* [interp] Don't optimize code if under debugger
2019-09-13 10:35:44 +03:00
Jay Krell 735b2cd20c [interp] Replace frame_objref with a volatile store to local. (#16790)
* [interp] Replace frame_objref with volatile stores to non-volatile locals.
Non-volatiles subject to volatile read/write still get stack-packed.
Volatiles do not.

This will possibly conserve stack, unless MINT_NEWOBJ_FAST is critical path.
 It looks like it is -- enabling this for Linux/gcc/amd64 grows
  frame from 0x98 to 0xA8.

In either case, it is more efficient as the reads do not have to be volatile.
The value can be on the stack and in registers. The stack value will pin it.

* PR: Comments.

* PR: Rearrange comment for correctness, same length.
2019-09-12 21:39:02 -05:00
Maxim Lipnin fe19944155 [ci][wasm] Use only V8 step to run corlib tests, which can reduce build time 2019-09-12 19:32:01 -05:00
Vlad Brezae aa18379968 [interp] Add percentage to opcode count stats 2019-09-12 19:29:13 -05:00
Vlad Brezae 6d75b066ee [interp] Disable computed goto when debugging interp
It makes tracing and following the code more painful.
2019-09-12 19:29:13 -05:00
Vlad Brezae 54a7e61115 [interp] Fix COUNT_OPS
And make it print the sorted opcode counts when runtime is shutting down
2019-09-12 19:29:13 -05:00
Next Turn 9f93d5d15c Modernize Array constructor (dotnet/coreclr#26684)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 19:27:58 -05:00
naricc bfe5e1efd3 Merge pull request #16388 from naricc/naricc/GetTotalAllocatedBytes
Naricc/get total allocated bytes
2019-09-12 18:12:26 -04:00
Jay Krell b2a1c6cd92 [Coop] Debugger single point of exit. (#16620)
* [Coop] Debugger single point of exit.
Extracted from https://github.com/mono/mono/pull/16614.

* Use a label per error for code size.
2019-09-12 17:57:02 -04:00
Sung Yoon Whang 3f26b9aa6c Improve the performance of Environment.WorkingSet in Windows (#26522)
* Use win32 api directly for workingset counter

* Fix build warnings

* Removing useless code

* more cleanup

* remove size annotation

* remove useless comment

* Move all the changes to Environment.WorkingSet and remove it from RuntimeEventSourceHelper

* removing useless usings

* Use kernel32.dll instead of psapi.dll

* Code review feedback

* Remove newline change

* More code review nits

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Jan Kotas 0055195d1c Change Vector<T> to use more compact code using S.R.CS.Unsafe (#26637)
S.R.CS.Unsafe is used in number of places in Vector<T> already. This change modifies more places to use it to make the code more compact.

Updated Vector.tt template to match while I was on it

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Marek Safar 43c5aa0dad Remove unused Memcpy methods from shared Buffer implementation (#26586)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
obligaron 1b48b44661 Improve logging for EventSource.WriteEvent parameter mismatch (#25533)
* improve logging for EventSource.WriteEvent

* Fix IndexOutOfRangeException in SerializeEventArgs when incorrect arguments got passed

* simplify arg-check; remove nullable-todo

* switch from Debugger.Log to ReportOutOfBandMessage

* Remove unused flush-Paramater from ReportOutOfBandMessage

* fix parameter type nullable-check for reference-types

* fix parameter type nullable-check for nullable-types

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Vlad Brezae 72405f68af [interp] Emit more stloc.np (#16796)
Use STLOC_NP_O also for MINT_TYPE_P. It has the same behavior.
2019-09-12 12:46:22 -05:00
Filip Navara 017f21c2c6 [mini] Fix couple of R4 constant folding checks (#16782)
* Fix couple of R4 constant folding checks

* Address feedback

* Split RMOVE and FMOVE
2019-09-12 12:37:52 -04:00
Jay Krell 9bb3912590 [interp] Remove unused parameters. (#16773) 2019-09-12 15:42:23 +03:00
Jay Krell c7df01524a [interp] Add assert in jmp that sp == stack at start. (#16788)
It makes it true sometimes, only if not calling alloca.
Throwing invalid IL if false might be preferred here.
2019-09-12 15:06:31 +03:00
Egor Bogatov 977a37844b [netcore] Add LLVM to nuget (#16524)
* nuspec for llvm

* Add llvm to matrix

* fix scripts/ci/pipeline-netcore-runtime.yml

* fix scripts/ci/pipeline-netcore-runtime.yml

* fix scripts/ci/pipeline-netcore-runtime.yml

* run tests with llvm

* add macOS lane

* disable llvm tests for now
2019-09-12 13:09:03 +02:00
Maxim Lipnin c241b25bb5 Avoid creating un-representable DateTime for transition point (#16753)
Fixes https://github.com/mono/mono/issues/16742
2019-09-12 05:11:22 -04:00
Steve Pfister 95cd0e74b1 Bump to mono/corefx@e79cf5b 2019-09-12 08:57:15 +02:00
EgorBo 1d324e617a Fix DynamicMethodToString.ToStringTest 2019-09-12 08:55:12 +02:00
EgorBo 313ce184be use full name for llvm ir and asm dumps 2019-09-12 08:55:12 +02:00
Jay Krell 2235e37470 [interp] Fix buffer overreads and overwrites found by valgrind. (#16726)
* [interp] Fix buffer overreads and overwrites found by valgrind.

* Attempt to deal with inlining correctly.

* PR: Dial back IR changes. Only IL relevant.

* PR: Dial back more -- only needed changes to .NET IL manipulation, not interpreter IR manipulation.

* PR: Remove one more line.
2019-09-11 19:35:05 -05:00
Zoltan Varga 6a1ac97dc7 [netcore] Create a separate version of threadpool.c for netcore with the domain support removed. (#16250)
* [netcore] Create a separate version of threadpool.c for netcore with the domain support removed.

* Fix the windows build.

* Fix the build.
2019-09-11 20:04:44 -04:00
Maxim Lipnin a67212d34c [wasm] update run-jenkins.sh (#16771)
* replaced set of run-*-mini targets with more common run-all-mini one;
* removed redundant run-v8-corlib (there is run-all-corlib)
2019-09-11 15:28:07 -04:00
Nathan Ricci 4acfcef038 White space, naming convetions. 2019-09-11 14:43:06 -04:00
Nathan Ricci b88d67d564 White space issues, typos. 2019-09-11 14:28:16 -04:00
Jay Krell f4c1358ed0 [interp] Reduce register pressure in endfinally.
Eventually this could contribute to stack savings. On its own, nothing.
The compiler apparently likes to create temporaries, like ip and ip - 1.
Which then have to survive functions calls, on the stack, if nonvolatile registers are exhausted.
This combats that and in a larger PR helped.
2019-09-11 11:19:44 -05:00
Steve Pfister 5ad1d62367 Use an init-tools script to install a local dotnet for wasm. (#16688)
Similar to what we use in netcore
2019-09-11 09:14:05 -04:00
Zoltan Varga feb0b4ab9a [wasm] Add a separate test for profiled aot. 2019-09-11 07:44:34 -05:00
Zoltan Varga 52d24a57d2 [aot] Emit interp entry code for methods which are not AOTed when using AOT profiling. Also emit most wrappers. 2019-09-11 07:44:34 -05:00
Radek Doulik 52aa2cd3e1 [aprofutil] Add -p and -f options (#16766)
-p PORT allows easier retrieval of AOT profile from the application
 running with aot profiler, using the socket connection on local PORT

-f instructs the tool to try setup adb port forwarding to/from Android
 device or emulator

Example usage, retrieve the AOT profile from Android:

    > mono aprofutil.exe -s -v -f -p 9998 -o my.aprof
    Calling 'adb forward tcp:9998 tcp:9998'...
    Reading from '127.0.0.1:9998'...
    Read total 94903 bytes...
    Summary:
    	Modules:          6
    	Types:          262
    	Methods:      1,162
    Going to write the profile to 'my.aprof'
2019-09-11 07:42:58 -05:00
Tarek Mahmoud Sayed f97ceb5c3e Fix Running .NET Core Under Windows App Compat Shim (dotnet/coreclr#26605)
* Fix Running .NET Core Under Winndows App Compat Shim

.NET Core uses Windows features called sort handles for perf reason to speed up the collation operations (e.g. string comparisons). When enabling Windows app compat shim (e.g. Windows 7 compatibility mode), the calls uses sort handles will fail which cause all collation operations to fail.
The fix here is to detect if the sort handles work before using it to ensure successful run.

* Address the feedback

* More feedback addressing

Signed-off-by: dotnet-bot <[email protected]>
2019-09-11 09:05:40 +02:00
Zoltan Varga 2e78f0c739 [wasm] Add missing aot-profile/runtime.js. (#16762)
* [wasm] Add missing aot-profile/runtime.js.

* [aot] Avoid writing out a zero AOTID in deterministic mode.

* [wasm] Add --aot-profile= option to packager.
2019-09-10 22:24:40 -04:00
Vlad Brezae f1a57de621 [docs] Improve sgen write barrier documentation (#16761) 2019-09-10 20:54:07 -05:00
Zoltan Varga a0ed95fc98 Enable the wasm targets even if ENABLE_WASM is not defined. (#16747) 2019-09-10 20:53:25 -05:00
Larry Ewing b26fd07ff9 [msbuild][roslyn] Bump msbuild and roslyn to pull in new versions (#16744) 2019-09-10 20:30:11 -05:00
Kenneth Pouncey 0847a6d218 [wasm] Dynamic Release RUNTIME configuration (#16749)
* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`

* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`

* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`
2019-09-10 18:04:08 -05:00
Zoltan Varga 697ab15c12 [netcore] Disable more features. (#16748) 2019-09-10 15:36:29 -04:00
Larry Ewing 4790582812 [wasm] test cleanup (#15727)
* [wasm] stop excluding test_0_large_nullable_invoke on wasm

* [wasm] remove wasm exclusion from test_0_simple_double_casts

* Add category MultiThreaded to more of the concurrent collection tests

* Add more MultiThreaded annotations to HttpListener2Test

* Add category MultiThreaded to HttpWebRequestTest

* [wasm] flag more tests that are taking down the runtime

* [wasm] force sockets to throw PNSE in the ctor for now

* [wasm] enable corlib interpreter tests

* [wasm] Increase timeout for corlib tests

* try to fix wasm-cross-win-release-configure on macos
2019-09-10 13:08:58 -04:00
Maxim Lipnin e78ab0b1c0 [wasm] Fix several typos (#16760)
* [wasm] Fix several typos

* add a comment about an error in test class name

* Remove the comment about test class name because it could be passed on xunit command line
2019-09-10 12:36:13 -04:00
Bernhard Urban 68f5613422 [TermInfoDriver] update top/left value on Console.{CursorTop,CursorLeft} access (#16745)
It can be outdated due to writes to `stdout`.

Fixes https://github.com/mono/mono/issues/16701
2019-09-10 17:17:15 +02:00
Kenneth Pouncey eea1ab5994 Remove extraneous -s from EMCC_DEBUG_FLAGS (#16754) 2019-09-10 13:50:20 +02:00
Marek Safar 9894bfa6fa Update tests exclusion 2019-09-10 13:09:56 +02:00
Stephen Toub b434981755 Remove use of WeakReference from corelib (dotnet/coreclr#26597)
Only remaining use is the one place it shows up in public API, in GC.GetGeneration(WeakReference).

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Tyler Brinkley 05d4f301e9 Make StringComparer.Create throw ArgumentNullException (#26570)
* Make StringComparer.Create throw ArgumentNullException

Throws proper exception.

* Update CoreFX.issues.rsp

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Stephen Toub 5f9d09bc90 Remove remaining ToLower() use from corelib (dotnet/coreclr#26602)
Also happens to save a string allocation, but that's a drop in the bucket in this function.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Stephen Toub 3c503413e0 Remove sub-array allocation from GetManifestResourceNamesList (#26601)
Also removes dependency on System.Range and RuntimeHelpers.GetSubArray in a simple console app such that they can be trimmed away.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Marek Safar 96b7fead33 [netcore] Update System.Private.CoreLib public APIs 2019-09-10 12:11:31 +02:00
Bernhard Urban e40e24b4cd [aot] move method_addresses to data.rel.so section to avoid text relocations (#16729)
[aot] move method_addresses to data.rel.so section to avoid text relocations

Another revert of a revert! 🎉 

After some discussions with @grendello and @radekdoulik, we came up with this solution. It's a better attempt to fix https://github.com/mono/mono/issues/16369 as it (1) works within the restrictions of Android 10 and (2) still works for armv7 AOT. 


`.text` relocations aren't allowed in Android 10 (and this change shouldn't hurt on Linux).

AOT compilation:
```console
$ MONO_PATH=/Users/lewurm/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/ \
MONO_ENV_OPTIONS="" \
./sdks/out/android-cross-arm-release/bin/armv7-linux-android-mono-sgen \
--aot=keep-temps,outfile=here.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-,ld-flags= \
~/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll

Mono Ahead of Time compiler - compiling assembly /Users/lewurm/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll
AOTID 7DDD05DA-37E0-813A-8C27-3E1634391BB3
Compiled: 144382/144382
Executing the native assembler: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-as"   -mfpu=vfp3 -o /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ.o /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ
Executing the native linker: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-ld"  -shared -o here.dll.so.tmp  /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ.o
Stripping the binary: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-strip" --strip-symbol=\$a --strip-symbol=\$d here.dll.so.tmp
Retained input file.
JIT time: 15031 ms, Generation time: 7374 ms, Assembly+Link time: 75130 ms.
```

Before:
```console
$ grep -C 6 'method_addresses:' /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ
        .align 3
jit_code_end:

        .byte 0,0,0,0
.text 1
        .align 3
method_addresses:
        .local method_addresses
        .type method_addresses,#function
ldr pc,=.Lm_0
.ltorg
ldr pc,=.Lm_1
.ltorg
$ arm-linux-androideabi-readelf -a here.dll.so | grep -i textrel
 0x00000016 (TEXTREL)                    0x0
 0x0000001e (FLAGS)                      TEXTREL
```

After switching to `.data.rel.so`:
```console
$ grep -C 6 'method_addresses:' /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_1YhAgn  # assembly file produced by AOT compiler
jit_code_end:

        .byte 0,0,0,0
.section ".data.rel.ro"
.subsection 0
        .align 3
method_addresses:
        .local method_addresses
        .type method_addresses,#object
ldr pc,=.Lm_0
.ltorg
ldr pc,=.Lm_1
.ltorg
$ arm-linux-androideabi-readelf -a here.dll.so | grep -i textrel
$ echo $?
1
```
2019-09-10 04:16:07 -04:00
Marek Safar e819930ea2 Disable failing tests 2019-09-10 10:03:42 +02:00
Stephen Toub cfc7988318 Replace HashHelpers.Combine with HashCode.Combine (dotnet/coreclr#26589)
* Replace HashHelpers.Combine with HashCode.Combine

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 10:03:42 +02:00
Jérôme Laban 11f75e5260 [Wasm] Restore release-dynamic configuration (#16655)
* [Wasm] Restore release-dynamic configuration

* Add conditional dynamic linking support, update documentation
2019-09-09 17:21:51 -04:00
Nathan Ricci f7dcc20bc6 Refactored where the varaibles for allocation count live. 2019-09-09 16:39:51 -04:00
Zoltan Varga f5d43ad341 Enable the wasm targets even if ENABLE_WASM is not defined. Update README. (#16743) 2019-09-09 16:37:05 -04:00
Thays Grazia 055818f344 Vtable [i] can be null so this should be check before use it. (#16733)
Fixes #16712
2019-09-09 17:07:34 -03:00
Maxim Lipnin ba2bb209ba [wasm] update build.sh (#16737) 2019-09-09 12:37:54 -05:00
Jay Krell 83f965eca5 [interp] Let compiler decide clear_resume_state inlining. (#16704)
Manual inlining shows no additional stack use any longer.
2019-09-09 12:31:37 -05:00
Zoltan Varga 798af66137 [llvm] Mark Vector128/Vector256 types as SIMD.
These are not currently supported, but marking them as simd helps llvm eliminate dead variables with these types, since
XZERO is easier to eliminate than VZERO.
2019-09-09 19:09:29 +02:00
dotnet-maestro[bot] e563d3c76c Update dependencies from https://github.com/dotnet/arcade build 20190906.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19456.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19456.2
2019-09-09 19:03:51 +02:00
Nikolay Sivov 8590a25b30 Default to SAFEARRAY(VARIANT) as CCW argument for object[]. (#16673)
* [cominterop] Add a helper to create default marshal spec for ccw arguments.

* [cominterop] Default to SAFEARRAY(VARIANT) as CCW argument for object[].

* [cominterop] Fix build with --enable-cxx.
2019-09-09 19:02:46 +02:00
Jay Krell 1f76ab9270 [interp] Embrace that ip is const guint16* and remove casts to that type, (#16708)
[interp] Embrace that ip is const guint16* and remove casts to that type.

or to gint16*. Or is there a reason for this style?

Maybe could change type but still want to cast for the deref?
2019-09-09 11:06:16 -04:00
Vlad Brezae b9dbd2ff91 [interp] Defer local offset allocation to compacted emit phase (#16714)
Before this commit, the only information associated with a local was just the offset at which it resides and instructions had this offset embedded from the start. This makes it awkward to have data structures indexed on the local, or have additional information per local, which we might care at some point (for example if it is multi basic block local, if the address of the local was taken etc). In addition to this, if we will want to remove some instructions that make some locals dead, it means that we will have to compact the whole locals space and update all the offsets referenced by all instructions. We solve all these problems by resolving (and allocating) the local offset at the latest time, while emitting the instruction in the compacted byte stream. At this point dead locals will remain dead, with no additional work needed to handle them.

For now, IL locals of the current method that we are compiling are immortal. We can address that later if needed.
2019-09-09 17:10:50 +03:00
Filip Navara 320f91129d [netcore] Tweak inlining heurestic for memcpy 2019-09-09 10:56:20 +02:00
David Karlaš 13391beeed Fix Xaml reader to parse escaped values correctly 2019-09-09 10:05:57 +02:00
Michal Strehovský d81ec88406 Mark unsafe intrinsics as reflection blocked
Keeping these reflection enabled pulls in quite a bit of junk (generic dictionaries, standalone method bodies, bookkeeping) that adds up to 1.5% of the size of Hello World...

Signed-off-by: dotnet-bot <[email protected]>
2019-09-09 09:24:01 +02:00
Filip Navara cf2c8575e1 [netcore][x64] Implement lowering of new SIMD OPs into SSE opcodes (#16672)
* [netcore][x64] Implement lowering of new SIMD OPs into SSE2 or SSE4.1 opcodes.

* Implement LZCNT/POPCNT in mini JIT

* Fix C++ build
2019-09-08 14:57:21 -05:00
Stephen Toub e15efa37b7 Fix race conditions in CultureInfo.GetCultureInfoHelper (#26567)
The implementation reads the current value of the static cache field and then later overwrites that field with the originally read value.  If ClearCachedData was called in the interim to clear the cached data, this will end up effectively undoing part of that clearing, as it'll end up restoring the previous cache. The fix is to write back the new reference only when we need to, which we could do when we detect it's null.

There's a secondary race condition this fixes as well.  When a name and an altname are used with CultureInfo.GetCurrentCulture(string, string), the implementation was publishig the newly created instance into the dictionary under the lock, but then after the lock was released, it was setting that TextInfo to be read-only.  That provides a window where another thread could see it as non-read-only and mutate it.  The fix is to ensure it's marked as read-only fully before it's published.

As I was doing this, it struck me that the implementation was overly complicated, trying to merge three different code paths for three different callers, presumably to share code, but most of the code ended up not being shared anyway, which just made the code harder to follow for little gain.  So, I split it back out into the three calling functions and removed the helper.

I also changed the locking mechanism.  It had been using a single shared lock to protect all state, but the purpose of the lock is really just to synchronize access to an individual dictionary, so we can just lock on the relevant dictionary.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-08 11:22:09 +02:00
Jay Krell 8c77bcdfaf [interp] Fix debug build. (#16720) 2019-09-08 09:25:33 +03:00
Zoltan Varga d9aaed760f [sdks] Change the DISABLE_ conditionals to ENABLE_ to avoid building extra stuff. (#16719) 2019-09-07 17:30:48 -04:00
Jay Krell 5ba104cf39 [interp] Fix recent warnings I added. (#16722) 2019-09-07 15:30:47 -04:00
Jay Krell b18840e92c [interp] Assign sp once is enough in leave. (#16694)
Move vt_sp there too.
2019-09-06 20:48:15 -05:00
Dotnet-GitSync-Bot 381c187df9 Mirror changes from mono/coreclr (#16713)
* Remove stale SuppressMessage attributes (dotnet/coreclr#26501)

Signed-off-by: dotnet-bot <[email protected]>

* Use null coalescing assignment operator in more places

There's currently no auto-fix for this, so the fixes were done manually.

Signed-off-by: dotnet-bot <[email protected]>

* Fix IDE0026 (use expression-bodied members for indexers)

Signed-off-by: dotnet-bot <[email protected]>
2019-09-06 15:58:38 -04:00
Alexis Christoforides 18fb728c65 Remove accidentally committed debug line 2019-09-06 14:27:57 -04:00
imhameed b4886817ef [jit][llvm] Track variable nullness separately from array length ranges/index ranges in abcrem. (#16499)
* [jit][llvm] Track variable nullness separately from array length ranges/index ranges in abcrem.

"Nullness" is tracked via a separate field that is stored adjacent
variable/array value ranges; values may be "maybe-null" or "not-null";
the intersection of maybe-null and not-null yields not-null, the union of
maybe-null and not-null yields maybe-null. The existing graph traversal logic
is reused as much as possible, which makes the implementation of this
null value approximation slightly more complicated than it would be
otherwise.

Fixes https://github.com/mono/mono/issues/16310, which was caused by
the use of non-empty array length ranges as a way to encode non-null
values.

This also allows null check elision for zero-sized arrays.

* Fix C++ compatibility. Fix a typo in a comment.

* Add symmetric nullness relations.

* Make ldlen imply that the source register is not-null, similar to CHECK_THIS or any other null check.
2019-09-06 08:55:21 -04:00
Vlad Brezae e49be5b9ec [metadata] Fix leaks when handling a few attributes (#16675)
Callers of mono_reflection_create_custom_attr_data_args_noalloc were leaking some of the returned information. Accessed attributes are FixedBufferAttribute and UnmanagedFunctionPointerAttribute.
2019-09-06 10:31:39 +03:00
Thays Grazia c23a8906f3 [debugger][android] It was not initialising seq_points on MonoCompile on Android, so when was compiling dynamic methods, seq_points wasn't created and we got the assert when try to single step. (#16691) 2019-09-05 22:16:50 -03:00
Vlad Brezae f0dabc6618 [interp] Kill InterpFrame.ex (#16666)
* [interp] Reduce unnecessary uses of frame->ex

* [interp] Remove ex from InterpFrame

We save stack space and move the exception where it belongs, in ThreadContext. It was confusing what frame->ex really meant, it was probably not used / initialized properly. We move it now to ThreadContext where it represents the current exception being thrown during EH (when we have a resume state set).
2019-09-05 23:10:13 +03:00
Filip Navara 673c8835fd [netcore] Bump CoreFX, core-setup dependencies (#16681)
* [netcore] Bump CoreFX, core-setup dependencies

* Update .deps.json hacks

* Update test exclusions
2019-09-05 20:03:47 +02:00
Jay Krell 7fc0f8842e [interp] Localize endfinally_ip to leave, and it cannot be null (barring (#16676)
[interp] Localize endfinally_ip to leave, and it cannot likely be null (unless offset carefully formed).
2019-09-05 13:46:12 -04:00
Jo Shields a67839540e Fix distclean (#16667)
* Disable `subdir-objects` mode in Automake

This forces creation of subdirectories' files (e.g. .deps/*.Plo) in the
parent directory listing the source file. This works around a situation
where two different files specify files in a different parent directory
(i.e. ../../support/adler32.c in mono/metadata/ and adler32.c in
support/) both emit files into the same place (support/.deps), causing
`make distclean` to try and delete support/.deps from two different
places, breaking distclean.

This unfortunately adds a pile of Automake warnings, during autogen.

The "correct" fix is to stop specifying zlib as a pile of flat .c files,
and instead put it in its own subdirectory, conditionally including
its .la files as required.

Partially fixes https://github.com/mono/mono/issues/16605

* Ship netcore/Makefile with an empty distclean rule

We need a netcore/ directory because we write to it during configure,
and we need a netcore/Makefile with a distclean rule, so top-level
recursive distclean doesn't fail. So specify both here.

Partially fixes https://github.com/mono/mono/issues/16605
2019-09-05 13:25:48 -04:00
Fan Yang abe48e3fd4 Remove redundant function call of EscapeTypeName to make (#16425)
* Remove redundant function call

* Remove commented out line of code
2019-09-05 18:12:57 +02:00
Maxim Lipnin f617a2692f [wasm] Add build.sh (#16682) 2019-09-05 11:10:05 -05:00
Jay Krell ee85b695c8 [interp] Remove unused parameter to ves_pinvoke_method. (#16677)
[interp] Remove unused parameter to ves_pinvoke_method.

It might actually cause stack pressure (but on its own, this PR does not save stack).
2019-09-05 11:30:59 -04:00
Zoltan Varga c3541c2738 [netcore] Fix the range checks for Vector.CopyTo(). (#16663)
* [netcore] Fix the range checks for Vector.CopyTo().

* [netcore] Fix Vector.get_Item () for u1/u2, the llvm type is a vector of i1/i2, so we need to zero extend.
2019-09-05 13:55:53 +02:00
Next Turn cb1a228a54 Remove unnecessary !s for [DoesNotReturn] (dotnet/coreclr#26512)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen Toub 0770a3d82b Fix StyleCop tuple warnings (dotnet/coreclr#26514)
A few PRs crossed such that PR validation passed but master is issuing lots of warnings-as-errors.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen Toub 1dc58da785 Remove readonly from GCHandle field on Gen2GcCallback (#26509)
GCHandle is mutable, and this type does call Free on the handle, but with it marked as readonly, the mutations performed by Free will be done on a copy rather than on the original.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen ToubandAdam Sitnik ff7dd88cfa Use expression-bodied members in more places (dotnet/coreclr#26500)
* Fix IDE0027 (use expression body for accessors)

* Fix IDE0023 / IDE0024 (use expression body for operators)

* Apply suggestions from code review

Co-Authored-By: Adam Sitnik <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Zoltan Varga db626fe9d3 [llvm] Use branch weight metadata in some cases. (#16668)
* [llvm] Add functions to create weighted branches.

* [llvm] Use weighted branches in GC_SAFE_POINT, some llvm passes doesn't seem to recognize llvm.expect so the LLVM JIT generates the slow inline.
2019-09-04 18:17:07 -04:00
Sam Patel c0a36b62af Update CodeOwners (#16662)
* Update CodeOwners

Add Imraan as code owner for LLVM related files

* Update CODEOWNERS
2019-09-04 19:32:11 +02:00
Jay KrellandLarry Ewing 1f21aae8cb Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 9fc2ade131 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 4359ad66d0 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing e25d2e5536 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 557645fea3 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b3f996d45d Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 8232c85e4c Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing bd9ae10c7c Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b071aa9192 more star placement
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 46ea28c95b Move stars to sometimes but not always onventional and wierd location. C syntax experiment takes it toll for decades on.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 90e2563523 Move stars to conventional but wierd place.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b8313d586a Update mono/eglib/gmodule-win32.c
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing d0817ccdc6 Update mono/eglib/glib.h
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay Krell a7357e8177 Remove unnecessary cast. 2019-09-04 12:31:13 -05:00
Jay Krell 595b66c0eb Allocate MAX_PATH to start, but it usually wastes memory.
Double later for clarity.
2019-09-04 12:31:13 -05:00
Jay Krell 8723e49b0e Fix C++? 2019-09-04 12:31:13 -05:00
Jay Krell fb6e2c75cd cleanup 2019-09-04 12:31:13 -05:00
Jay Krell da2471d001 Test invalid module. 2019-09-04 12:31:13 -05:00
Jay Krell 74f1312cd3 Skip psapi.dll and go right to the Windows 7 or newer versions in kernel32.dll. 2019-09-04 12:31:13 -05:00
Jay Krell 2db0c8c2d4 Remove cast. 2019-09-04 12:31:13 -05:00
Jay Krell 06f4f93aa5 comments 2019-09-04 12:31:13 -05:00
Jay Krell 3ca5b2687f Remove accident. 2019-09-04 12:31:13 -05:00
Jay Krell f8be8c6de7 MAX_PATH reduction. 2019-09-04 12:31:13 -05:00
Jay Krell a55da2a903 Tweak names and fix preexisting leak under low memory. 2019-09-04 10:31:22 -05:00
Jay Krell 14afe79a0a [Coop] GetStackTraces. 2019-09-04 10:31:22 -05:00
Egor Bogatov 30625b25df Implement more Math LLVM intrinsics (#16578)
* Implement more Math intrinsics

* Test on CI

* Update pipeline-netcore-runtime.yml

* undo "test on CI"
2019-09-04 09:54:52 -04:00
Jay Krell 350dd0e2de [Coop] ves_icall_System_IOSelector_Add 2019-09-04 08:36:42 -05:00
Stephen Toub 036ab58b42 Fix several recently added StyleCop tuple rules (dotnet/corefx#40794)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-04 11:15:23 +02:00
Egor Bogatov 1558316cec Throw NRE if src or dst in Memmove are nulls (#16650) 2019-09-04 11:01:53 +02:00
Zoltan Varga 55ac86fe8b [netcore] Fix the exception thrown by the Vector<T>.CopyTo() intrinsic. (#16618)
Fixes https://github.com/mono/mono/issues/16616.
2019-09-04 10:57:39 +02:00
Dotnet-GitSync-Bot dcb90fa1c0 Mirror changes from mono/coreclr,corefx (#16654)
* Refactor BinaryReader.ReadChars using Span (dotnet/coreclr#26325)

Avoids unsafe code and makes the code simpler

Signed-off-by: dotnet-bot <[email protected]>

* Remove unnecessary parens from `return (true);` and `return (false);` (#26488)

Just a search and replace.

Signed-off-by: dotnet-bot <[email protected]>

* Avoid some unnecessary initialization in DateTimeParse (#26487)

Signed-off-by: dotnet-bot <[email protected]>

* Update analyzer versions (dotnet/corefx#40778)

In particular to pick up StyleCop fixes related to C# 8 and nullability.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-04 00:20:35 -04:00
Jay Krell 82d4d8d81d Fix overly sensitive g_ascii_strncasecmp tests. 2019-09-03 22:46:10 -05:00
Jay Krell b66ec468da Add g_asciiz_equal_caseinsensitive, g_ascii_equal_caseinsensitive, g_asciiz_equal.
The default takes a length.
"z" means zero terminated, the usually slower not preferred.
2019-09-03 22:46:10 -05:00
Jay Krell 8c288f0666 Cleanup and augment string handling based on https://github.com/mono/mono/pull/15522.
1. When comparing strings, check pointer equality.
Faster when they match, slower when they do not.

2. Provide string lowercasing into a provided buffer, which can equal
the input, to avoid extra malloc/free.

3. define G_STRING_CONSTANT_AND_LENGTH(x) (x), G_N_ELEMENTS (x) - 1
e.g. strncmp (foo, G_STRING_CONSTANT_AND_LENGTH ("version"))
2019-09-03 22:46:10 -05:00
Zoltan Varga e1b568b529 [netcore] Enable Vector<T>.get_Item () intrinsics for floats/doubles. 2019-09-03 22:05:15 -05:00
Zoltan Varga 4107a15389 [netcore] Disable socket support to fix warnings. 2019-09-03 21:24:23 -05:00
Alexis Christoforides e50a7360e1 Debugging 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4840f4161f [merp] Fix logic of 'is_managed' attribute 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4a52bd6da0 Fix pointing to stack on string && some add tracing 2019-09-03 20:30:30 -05:00
Alexis Christoforides 8c0dc66d25 Guard writing to out_module 2019-09-03 20:30:30 -05:00
Alexis Christoforides 1b66666cc7 Update MONO_CORLIB_VERSION 2019-09-03 20:30:30 -05:00
Alexis Christoforides 001398e05b Remove problematic assert from SIGILL handler
mono_handle_native_crash() function can actually return if the native crash has triggered or has been trigerred by  more than one signal. So it is possible to reach the point of the assert
2019-09-03 20:30:30 -05:00
Alexis Christoforides 8d4c0f3553 Add stub of new function when crash reporting is disabled 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4b02c3eb9d Set out_module, to avoid getting no useful information in the printout
It seems that without this, most/all entries in the unmanaged frames list are have no useful information.
2019-09-03 20:30:30 -05:00
Alexis Christoforides 40a8d88d8e [merp] Add API method that whitelists all native libraries
This is so that complex consumers such as Visual Studio for Mac can forego attempting to enumerate all loaded libraries, register them one at a time and potentially keep the list updated
2019-09-03 20:30:30 -05:00
Next Turn 0fc4d68d8e Use Span<T> in Encoding (dotnet/coreclr#25860)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-03 19:42:55 -05:00
Zoltan Varga 83953ef83d [wasm] Disable threads tests as well. 2019-09-03 18:06:36 -05:00
Zoltan Varga aad5494258 [wasm] Avoid aligning the stack bounds, they are not page aligned on wasm. 2019-09-03 18:06:36 -05:00
Zoltan Varga ea512c1604 [wasm] Bump emscripten to 1.38.43. 2019-09-03 18:06:36 -05:00
Zoltan Varga 5f79b69e34 [wasm] Disable threads support, it doesn't work with 1.38.42. 2019-09-03 18:06:36 -05:00
Larry Ewing 68b7af0539 [wasm] update emscripten 2019-09-03 18:06:36 -05:00
Jay Krell 1780e1d1f8 [coop] CustomAttributeBuilder_GetBlob, etc. (#16579)
mono_reflection_get_custom_attrs_blob_checked
 ModuleBuilder_WriteToFile
 ModuleBuilder_build_metadata
2019-09-03 18:52:41 -04:00
Jay Krell 651eb7696e [Coop] ves_icall_InternalInvoke. (#16625)
* [Coop] ves_icall_InternalInvoke.

(Part taken from ves_icall_InternalExecute https://github.com/mono/mono/pull/16622.)

* error_init_reuse.

* Remove stars from types.
2019-09-03 17:18:21 -05:00
Thays Grazia 1046dd746f [debugger][android] Fixing single step inside Exception on Android (#16596)
* Fixing single step inside Exception on Android...
The method is null because on ss_create_init_args the IL is -1 when calling method Mono_UnhandledException_internal what I think it's expected, because of that I ignore the single step on android if there is no method. (debugger-engine.c)
After that the sequence point cannot be found because it's calling a method of class DynamicMethodNameCounter what I think it's expected, because of that I remove the assert on android. (debugger-agent.c)
Fixes #14772

* Changing comment to make it clear...
2019-09-03 16:28:37 -04:00
Egor Bogatov 625af20cef [netcore] Use CoreRT's Overlapped (#16106)
Made it Nullability friendly

Interop.MemFree  -> Marshal.FreeHGlobal
Interop.MemAlloc -> Marshal.AllocHGlobal

Fixes https://github.com/mono/mono/issues/15308
Fixes https://github.com/mono/mono/issues/15310

Also, copy ThreadPoolBoundHandle.PlatformNotSupported.cs
2019-09-03 16:22:04 -04:00
Jay Krell 37e49876b9 Fix non-netcore warning about add_assembly_to_alc. (#16622) 2019-09-03 16:09:24 -04:00
Filip Navara 4ae804bb33 [netcore] Add intrinsic implementation for Unsafe.InitBlockUnaligned 2019-09-03 18:40:06 +02:00
Vlad Brezae 930a5d7d44 [interp] Transform ldloc/stloc pairs to a movloc instruction (#16546)
* [interp] Add stack information for each instruction

For each instruction PopN indicates how many values from the top of the stack are used by the instruction. PushN signals how many values are pushed back. These are also used to prevent values from the stack being optimized out, since they might still be used by an instruction. VarPop and VarPush mean that the number is not constant, so we need to inspect the data fields of the instruction (typically for an imethod or csignature) to figure out the stack usage.

* [interp] Convert ldloc/stloc pairs to movloc

The movloc instruction operates directly on locals. This is faster due to avoiding dispatch and stack indirection. More important is that it makes it easy to do copy propagation of locals and it has no stack usage, which means that it could even be killed if the destination local is no longer referenced (as well as STLOC_NP). In the future we could reuse this mechanism of tracking the value of a stack slot to do constant propagation and kill even more instructions (we might need to have _IMM versions of opcodes for this to work).

* [interp] Enable disabling of interp optimization

* [interp] Add time tracking stat for cprop optimizations

* [interp] Move stloc.np transformations to the cprop pass

It makes more sense to do such transformations here and we also don't want transformations in the generate phase prevent us from detecting alternative transformations when running the full pass.

* [interp] Add counter for instructions that were optimized away

* [interp] Make it explicit that we are dealing with IR code

It can be confusing in verbose output.
2019-09-03 19:36:21 +03:00
Nikolay Sivov f8718c7a05 [cominterop] Maintain object domain around ccw wrapper calls. 2019-09-03 18:03:42 +02:00
Ryan Lucia 5377f9655d Remove MAX_PATH usage in w32process (#16553)
* Remove MAX_PATH usage in w32process

* Fixes

* Shrink buffer temporarily

* More fixes

* Reset buffer size to 260

* Fix length check and add notes

* Actually free things

* Add tracing and zero len at start

* Typo

* Fix another leak

* Simplify logic
2019-09-03 11:41:27 -04:00
Zoltan Varga 98a4d5ec1e [llvm] Implement more SIMD intrinsics on netcore. (#16449)
* [netcore] Add intrinsics for Vector128/Vector256.Count.

* [netcore] Implement Vector<T>.get_Item () as an intrinsic.
2019-09-03 08:31:16 -04:00
Egor Bogatov 416a31796e Improve Fast-Math mode for LLVM (#16599)
It turns out it's not enough to set global fast math parameters, it's also required to mark some instructions as "fast" (see https://llvm.org/docs/LangRef.html#fast-math-flags) like, for instance, clang does, see https://godbolt.org/z/U8TFST
E.g. all optimizations inside SimplifyLibCalls pass (it's a subpass of InstCombine) need it.
2019-09-03 08:19:03 -04:00
Egor Bogatov 5692180ea2 [netcore] Makefile clean up (#16630)
Remove ASP.NET sample (we currently have to download apsnet runtime everytime)
instead we can do:

dotnet new webapp

and do make patch-app APP_PATH=/path/to/myasp (works for any kind of apps/templates)
2019-09-03 07:55:22 -04:00
Bernhard Urban f23113afcd Relax assembly checking for 64 bit targets to allow running mixed-mode MSVC code. (#16330)
* [jit] Relax assembly checking for 64 bit targets to allow running mixed-mode MSVC code.

Fixes #37913

* STACK_PTR|I8 should be the same as I8|STACK_PTR tuple

* fix i4 to ptr conv on 64bit
2019-09-03 12:14:02 +02:00
Vlad Brezae f491af13f4 [interp] Remove varargs from InterpFrame and recompute it instead (#16598)
It is a waste of stack space.
2019-09-03 11:21:56 +03:00
Filip Navara b9b13f0fc1 Use CMP_GE/CMP_LE/CMP_GE_UN/CMP_LE_UN instead of manual decomposition 2019-09-03 09:43:17 +02:00
Filip Navara 03899588ba [netcore] Add intrinsics for Vector.GreaterThanOrEqual/LessThanOrEqual 2019-09-03 09:43:17 +02:00
Dotnet-GitSync-Bot a6b91f4295 Mirror changes from mono/coreclr (#16547)
* Move FormatTypeName to Type (dotnet/coreclr#25631)

Signed-off-by: dotnet-bot <[email protected]>

* Fix for mono
2019-09-02 16:02:59 -04:00
Egor Bogatov 73c8017b1f Merge pull request #16624 from filipnavara/hebrew
[corlib] Add hack for Hebrew calendar data
2019-09-02 20:43:08 +03:00
monojenkins eaa525ab41 [csproj] Update project files 2019-09-02 10:33:47 +00:00
Filip Navara 6ca4e84758 Fix month names for the Hebrew calendar 2019-09-02 11:19:24 +02:00
Filip Navara ca389a3cae [corlib] Add hack for Hebrew calendar data 2019-09-02 02:45:45 +02:00
Alexander Köplinger 07b7c7d824 [Coop] Monitor. (#16608)
* [Coop] Monitor.

* PR: Rename function; cleanup slightly.

* PR: And then rename the icall back.

* Rename parameter back.
2019-09-01 15:14:17 +02:00
Jay Krell b3e48bcf95 [Coop] Fully inline NULL_HANDLE (do not even access a global variable). (#16612) 2019-09-01 15:13:20 +02:00
Ryan Lucia c9a6dd351a [netcore] Add handle frame to loop body in GetTopLevelForwardedTypes (#16594)
* [netcore] Add handle frame to loop body in GetTopLevelForwardedTypes

* No early returns

* Move scope of cols

* Fix C++ builds, probably
2019-08-31 21:40:52 -04:00
Egor Bogatov 6327274b54 LLVM: Make Buffer.Memmove intrinsic (#16610)
* Add memmove intrinsic

* Fix typo

* Add -lower-expect
2019-08-31 19:42:44 -04:00
Jay Krell 24180e9106 Rename parameter back. 2019-08-31 10:35:14 -07:00
Jay Krell 5d9ca20646 PR: And then rename the icall back. 2019-08-31 10:34:32 -07:00
Jay Krell ea0db154b4 PR: Rename function; cleanup slightly. 2019-08-31 10:13:47 -07:00
Zoltan Varga fb26a68fd1 [aot] Avoid crashing with --aot=llvm is used in a runtime without llvm support. (#16577) 2019-08-31 09:03:45 -04:00
Jay Krell 91445e2286 [Coop] Monitor. 2019-08-31 05:22:45 -07:00
Jay Krell 9b193b9ec4 [coop] Allow SETUP_ICALL_FRAME (i.e. a loop) to reference calling frame w/o HANDLE_FUNCTION_ENTER or SETUP_ICALL_FUNCTION. (#16498)
This is based on my read of https://github.com/mono/mono/pull/16487.
2019-08-31 14:20:44 +02:00
Nikolay Sivov 2f5c8cba6e [corlib] Simplify file: protocol check in Assembly::Load(). (#16494)
Mirrors change made with 068660732f.
2019-08-31 08:19:41 -04:00
Vlad Brezae bf53a3a53b [interp] Fixes for il_offsets associated with instructions (#16587)
* [interp] Remove some some false is_bb_start markings

Having fewer basic blocks makes potential optimizations more effective.

* [interp] Persist variables in TransformData

Thate are not used by inlined code because we don't inline code that has multiple basic blocks or clauses.

* [interp] Enable emitting of MINT_NOP instruction

This instruction will be optimized away when compacting the instruction stream.

* [interp] Clear instructions insead of removing them

Makes everything easier.

* [interp] Fix some instructions size inconsistencies

* [interp] Add more is_bb_start markings

* [interp] Add nop instead of extending bb

We should always have an IR instruction associated with an IL offset, especially when that instruction is a bb_start since we will need to branch to it and resolve the IR instruction from the il_offset.

* [interp] Simplify resolving in offset

We should always have an in_offset associated with the il_offset we are branching to.

* [interp] Add correct il_offset for all instructions

Except for special prolog instructions that will have -1 as the offset. We don't make much use of il_offset for non bb_start instructions, but they might be useful for debugging.

* [interp] Fix il_offset of some conv opcodes
2019-08-31 13:37:06 +03:00
Jay Krell a768b15330 Follow convention -- "static inline" is only for .h files, "static" is for .c. (#15502)
inlining is up to the compiler to decide.
It is used in .h files for linkage/dedup, not to affect codegen.
2019-08-31 09:40:33 +02:00
Jay Krell cba94f28be [coop] InternalGetHashCode (#16583) 2019-08-31 09:38:12 +02:00
Aleksey Kliger (λgeek) 8bc27f6665 [runtime] Make eglib tests run on CI (#16187)
[runtime] Make eglib tests run on CI

Update mono/eglib SUBDIRS to build tests after eglib is built.

Add a check-local automake rule.

Add CI steps to run the eglib tests.
2019-08-31 02:06:25 -04:00
Jay Krell f735571f27 Cleanup x86_patch and amd64_patch. (#16474)
* Cleanup x86_patch and amd64_patch.

amd64_patch has two incorrect looking probably unused cases.
 g_assert(0) and fix them.
 call and jmp indirect (one case)
 move reg RIP relative

x86_patch is large-ish inline macro.
Presumably for sharing with amd64, not inlining.
Restructure as an inline function, called once.
Try to make the code a little clearer also, with same functionality and efficiency
 (modulo the optimized perf of "if i == 4" vs. "if i")
i.e. speak of instruction_size and offset_size, not pos and boolean size.

Add to x86_patch the ability to patch jmp/call indirect,
as amd64_patch has, but which was incorrect.
This would seem useful for a later change (using "thunks"
to patch nearby data instead of patching aligned instructions,
indirect call/jmp vs. direct).

amd64_patch largely delegates to x86_patch.
One case asserts and delegates.
Move the assert to the x86 side, where an optimizing
compiler should know to remove it for x86 anyway.

Change some assert to g_assert.

Remove some unnecessary casts.

Fix an assert (https://github.com/mono/mono/pull/16471).

* Change g_assert back to assert except where making larger changes
and less expansion.
2019-08-31 00:45:55 +03:00
Bernhard Urban 06e63b345f [mini] publish global patches after JitInfo has been added (#16589)
[mini] publish global patches after JitInfo has been added

Fixes https://github.com/mono/mono/issues/14080

Consider the following example:

```csharp
static void CommonCallTarget () { }

static void TailA () {
    tailcall CommonCallTarget ();
}

static void TailB () {
    tailcall CommonCallTarget ();
}
```

Since `TailA` and `TailB` are tailcalling into `CommonCallTarget`, the resolution at patch-time is a bit tricky, that is, since it's a jump-like instruction the patching machinery won't know where it was called from. That's why we maintain a global hashtable `jump_target_hash` where each jump-site is signed up to be patched. At patch-time we know the target method (in the example `CommonCallTarget`), but since we don't know where we are coming from, we will just apply all patches for that target.

This works since ages, so why did it crash on arm64 sometimes?
When the patching happens, we check if the displacement between jump-site and target fits into it (24bit). If not, which happens not very often, we have to allocate a _thunk_:
https://github.com/mono/mono/blob/36296ce291f8a7b19de3eccb7a32c7e4ed9df8f2/mono/mini/mini-arm64.c#L928-L942

So instead of jumping to the target directly, we'll branch to the thunk. This is a little trampoline that loads the full address of the target and then finally branches to it. This one will live close-by the jump-site, because during compilation we will reserve specifically for that scenario some space after the generated code. For this, however, we need the JitInfo of the jump-site. And that's where the origin of the race is. Let's say:

* Thread A compiles `TailA`, and then jumps into it. Thus one patch point is in the `jump_target_hash`.
* Now Thread B compiles `TailB`, registers the patch point but has _not_ yet registered its JitInfo.
* Then Thread A continues, does the tailcall into `CommonCallTarget`, enters the patching machinery, which sees two patches. Now assume when applying the patch for `TailB` the displacement is too large, thus it tries to allocate a thunk but can't find the relevant JitInfo for it that it needs to emit the thunk. So it crashes as reported in https://github.com/mono/mono/issues/14080

As far as I can tell this only affects ARM64, ARM and PPC.



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-08-30 16:46:10 -04:00
Steve Pfister dbce6ac447 Turned the compiler server off on Windows because Named Pipes aren't fully working (#16552)
* Turned the compiler server off on Windows because Named Pipes aren't fully working

* Added win32 to the CS is turned off message
2019-08-30 16:27:14 -04:00
Bernhard Urban 5eb67ac88a Revert "[android] add hardware specific workaround for Nexus9 in armv7 mode" (#14691)
Revert "[android] add hardware specific workaround for Nexus9 in armv7 mode"

This reverts commit ee90fc601e (also see https://github.com/mono/mono/pull/4878 ).

The device was released November 2014.  I think it's fair to assume it isn't widely used anymore.
2019-08-30 16:26:12 -04:00
Zoltan Varga 4997d83291 [netcore] Fix make patch-local-dotnet if .dotnet has more than one netcore install. (#16590) 2019-08-30 14:08:41 -04:00
Jay Krell 86c45ff645 [coop] mono_runtime_class_init_full (#16580) 2019-08-30 13:30:58 -04:00
Steffen Kieß 80b7813d54 [Mono.Posix] Add FcntlCommand.F_OFD_{GETLK,SETLK,SETLKW} (#16571)
Add F_OFD_GETLK, F_OFD_SETLK and F_OFD_SETLKW to FcntlCommand. These
commmands are similar to F_GETLK, F_SETLK and F_SETLKW but operate on a
per-open-file-description lock instead of a per-process lock and avoid the
problem that the locks are released when close(dup(fd)) is called. See
https://lwn.net/Articles/586904/ for more information.

This commit also reruns create-native-map but keeps manual changes from
ed892ccf27,
b522eab5ff and
22b6b95814.

In [create-native-map.diff.txt](https://github.com/mono/mono/files/3557077/create-native-map.diff.txt) you can see the difference between the commited `map.[ch]` and the one `create-native-map.exe` wants to create:
- Manually inserted changes for NetBSD (`MAYMOVE` is turned into a negated `FIXED`)
- `Mono_Posix_ToSockaddrIn6()` and friends get skipped for `HOST_WIN32` (not sure why, when `sockaddr_in6` exists then `Mono_Posix_ToSockaddrIn6()` should work)
- At the bottom of `map.h` some functions are reordered and the parameters change (`const char*` vs. `char*`)

Running create-native-map.exe also moved some O_NOATIME-related stuff (from 8b9033e4d2) to the correct position and added it to Mono_Posix_ToOpenFlags(), which I kept in the commit.
2019-08-30 19:13:33 +02:00
Nikolay Sivov c8707c6e84 [cominterop] Respect ComDefaultInterfaceAttribute when constructing ccw. (#16584) 2019-08-30 18:29:54 +02:00
Nikita Voronchev 94797b95d6 [WinForms] Fix UIA event on node insertions into a TreeView object (#16559)
The UIA event doesn't emited by means of insertion a new `TreeNode` to a `TreeNodeCollection` of the respective `TreeNode`.
2019-08-30 18:29:18 +02:00
Jay Krell 9db00e7495 [Coop] ves_icall_RuntimeFieldInfo_GetRawConstantValue. (#16582) 2019-08-30 10:53:40 -04:00
Bernhard Urban 07045604cf [configure] remove autodection of llvm-config in $PATH (#16569)
[configure] remove autodection of llvm-config in $PATH

Instead `./autogen.sh --with-llvm=<path>` should be used if a build from `external/llvm` is not preferred.

The old behavior was confusing when a `llvm-config` is on `$PATH`, which can be the case when you have LLVM packages installed on a Linux distribution. It picked that up when `./autogen.sh --enable-llvm` was used.

/cc @brianrob
2019-08-30 09:16:03 -04:00
Jay Krell 36296ce291 [interp] Compress interpreter opcode names in usual way, offsets vs. pointers. (#16350)
* [interp] Compress the interpreter opcode names in the usual way, by changing
from an array of pointers to an array of 16 bit offsets into one large "string".

* PR: Make mono_interp_opname a function.
2019-08-30 11:27:19 +03:00
Jay Krell 3db78e829c [coop] Convert System_Runtime_Remoting_Messaging_AsyncResult, etc. (#16566)
* [coop] Convert System_Runtime_Remoting_Messaging_AsyncResult_Invoke, etc.

* Remove mono_load_remote_field_checked from this PR.

* Pin ac->msg earlier (should not matter).

* Add missing barrier.

* Pin slightly earlier -- still should not matter.
2019-08-29 22:51:15 -05:00
Jay Krell cbb9c645eb [Coop] mono_runtime_class_init_full mostly single point of exit. 2019-08-29 22:00:46 -05:00
Zoltan Varga 796a575131 Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5. (#16567)
* Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5.

* [monodis] Add a missing cattr parent case.
2019-08-29 20:46:30 -04:00
Egor Bogatov c3ccb6e027 Make Math.Pow, Min and Max for doubles intrinsics (#16561)
* Add new Math intrinsics

* call EB.setTargetOptions only for fast-math

* Address feedback

* rename float opcodes

* Rename opcodes

* Update CoreFX.issues.rsp
2019-08-29 19:01:15 -04:00
Zoltan Varga 210b324347 [simd] Implement .ctor(T[]) and CopyTo () as intrinsics. (#16558) 2019-08-29 19:00:59 -04:00
Jay Krell 5390b796dd Cleanup trampoline size checks. (#15624)
Cleanup trampoline size checks.

- named `int` everywhere
- `assert` everywhere
- assert `<=` everywhere instead of `<`
- `const` almost everywhere
Names not consistent, following existing code (`size`, `buf_len`, `kMaxCode`.)

Only touching x86/amd64, granted.

Could be better -- make them two pass and avoid the presizing.
2019-08-29 16:16:14 -04:00
Alexander Köplinger 309cf1bcee Build pkg-config as 64bit only, libtiff as universal and remove gtk-quartz-engine.py (#16548)
See https://github.com/mono/bockbuild/pull/116
2019-08-29 20:24:14 +02:00
Ryan Lucia 3b57849541 [netcore] Implement AssemblyLoadContext.LoadUnmanagedDllFromPath (#16525)
* [netcore] Implement AssemblyLoadContext.LoadUnmanagedDllFromPath

* Fixes
2019-08-29 12:35:15 -04:00
Nikolay Sivov 421bd79f06 [cominterop] Implement native-to-managed safearray marshalling. (#16483)
* [cominterop] Implement native-to-managed safearray marshalling.

* [tests] Fix index variable type.

In attempt ot fix MSVC build failure (int* vs LONG* on Windows).

* [cominterop] Use proper statement block for newly added switch entry.

* [marshal] Conditionally compile in mono_cominterop_emit_marshal_safearray().

Hopefully should fix remaining build failures.

* [cominterop] Don't free user array argument
2019-08-29 08:19:08 -04:00
Jay Krell 52c1d4acfc [Coop] mono_load_remote_field_checked single point of exit. (#16526)
Extracted from earlier version of #16504.
Baby steps.
2019-08-29 11:00:44 +02:00
Egor Bogatov 2e9c5df855 LLVM: Add more Math intrinsics, introduce --cpu flag (#16309)
* Add Math Intrinsics, introduce --cpu flag

* cpu -> mcpu
2019-08-29 04:32:43 -04:00
Jay Krell 2f2d9541e5 [coop] Cleanup mono_reflection_get_custom_attrs_blob_checked. (#16535)
- single point of exit
 - local variable cheaper than strcmp
2019-08-29 04:23:20 -04:00
Jay Krell 07c6b1c9f1 Should fix many warnings: sgen-protocol-def.h adding unsigned long to a string does not append to the string (#16288) 2019-08-29 10:08:42 +03:00
Filip Navara e32fb11dbe [netcore] Add Monitor.LockContentionCount icall (#16538)
* Add Monitor.LockContentionCount icall

* Add NOHANDLES around the new icall
2019-08-28 20:35:07 -04:00
Marius Ungureanu 1d23ca14e1 Use the right FileSystemWatcher sources (#16539)
* Use the right FileSystemWatcher sources

The corefx code for FileSystemEventArgs has some changes to support
constructing one with a null filename.

Fixes #16486

* Remove the mono sources for the newly replaced files

* [csproj] Update project files
2019-08-28 19:45:25 -04:00
Jay Krell 7ca965e09c Read only function pointers (MonoThreadInfoRuntimeCallbacks). (#16545)
Read only memory is preferred, for efficiency and security.

Even if it requires relocation first.

If the runtime is loaded in more than one process, physical pages can be shared (at least on Windows, where the kernel does the relocation).

If there is memory corruption, writes to here will sigsegv/accessviolate.

The older code also copied the pointers. For lifetime purposes that could be useful, but generally is not. i.e. they live with the code they point to.

Writable function pointers are also particularly targets for attackers.
There's been tons of defenses put around them through the years.
Read only ones are not vulnerable -- unless the attacker escalates and calls mprotect first.

When there is a JIT, arguably there is just tons of attack surface.
But there is also AOT.

Structs of function pointers are usually read only in the wider world.
2019-08-29 00:10:40 +02:00
Steve Pfister 7c4c30ec00 Implement Read and WriteAsyncMemory in Mono DeflateStream
Resolves https://github.com/mono/mono/issues/16122

Since we put in the corefx version of GZipStream, it is expected that
our version of DeflateStream actually implements Write+ReadAsyncMemory.

This change does that by just calling the plain WriteAsync & ReadAsync
methods until we decide (or not) to pull in the corefx DeflateStream.
2019-08-28 16:20:09 -04:00
Zoltan Varga c4695b8b44 Merge pull request #16323 from EgorBo/ffast-math
Fast Math mode
2019-08-28 16:16:18 -04:00
Jay Krell 056d5ca545 [interp] Fix remaining MonoError reuse. (#16533)
* [interp] Fix remaining MonoError reuse.

This finishes off:
 https://github.com/mono/mono/commit/38aebeae38704df53ef8f2991ff63e293a3ac4de

which is *ok*, but leaves alone the other approach:
 https://github.com/mono/mono/commit/4d9e6ae20e536394c9ff65c61238045db294c08e
 https://github.com/mono/mono/commit/b654aa1ca858666539a677e96d00f594fd81c630
 https://github.com/mono/mono/commit/fd4956fbf277ec56a13137e01b48efb98a5556f5

Several other options are possible.

* self and peer PR: No need to cleanup successful error.
2019-08-28 21:33:48 +02:00
Jay Krell ff73c84482 Portably pass a full pointer and not just an integer to terminate vararg pointer lists. (#15629)
i.e. because of define NULL 0.
Though I think at least Visual C++ special cases this and passes a full pointer to varargs.
2019-08-28 13:35:10 -04:00
Jay Krell a0ba2cbce5 [interp] Possible code size savings via goto instead of macros. (#16531)
[interp] Possible code size savings via goto instead of macros.

Possibly the compilers do this already.
2019-08-28 12:59:11 -04:00
Egor Bogatov ffc3081c3b Merge remote-tracking branch 'origin/master' into ffast-math 2019-08-28 19:01:34 +03:00
imhameed bd3cc61301 Merge pull request #16242 from lateralusX/lateralusX/windows-fix-local-valuetype-copy-llvm
OP_LLVM_OUTARG_VT doesn't always create temporary vt needed by some ABI's.
2019-08-28 08:23:43 -07:00
Bernhard Urban 71f0f0ff82 [mini] do not re-AOT whole BCL for fullaotcheck targets (#16536)
[mini] do not re-AOT whole BCL for fullaotcheck targets

And enable `llvmfullaotcheck` target on CI.

Fixes https://github.com/mono/mono/issues/15999

On my armv7/linux machine with a `--with-runtime-preset=fullaot_llvm` configured build, before this PR:
```console
$ time make -C mono/mini llvmfullaotcheck -j4
[...]
real    9m42.362s
user    23m2.000s
sys     0m18.480s
```
After:
```console
$ time make -C mono/mini llvmfullaotcheck -j4
[...]
real    3m1.984s
user    4m51.648s
sys     0m8.248s
```
2019-08-28 11:19:39 -04:00
Alexander Köplinger 2152f0a16d [netcore] Add missing 'prepare' dependency to run-tests-corefx 2019-08-28 16:17:31 +02:00
Phil Jaenke 1bee4e354c [corlib] Remove/Update deprecated TZs for #16415 in TimeZoneInfoTests (#16450)
This fixes #16415 

All timezones indicated as deprecated have been updated to the current IANA specification.
2019-08-28 16:08:20 +02:00
Zoltan Varga f562c7ef6a Merge pull request #16517 from vargaz/wasm-xunit4
[wasm] Enable xunit tests for corlib/System.Core.
2019-08-28 09:46:09 -04:00
Jay Krell c80a1eca63 Fix assert in x86_patch regarding second byte of Jcc instructions being 0x80-0x8F vs. 0x70-0x8F (6byte conditional branch with 32bit offset). (#16471) 2019-08-28 15:42:44 +02:00
Aleksey Kliger (λgeek) fc21168065 [merp] Initialize hang_watchdog_path earlier; be lax if it's missing (#16298)
* [merp] Initialize hang_watchdog_path earlier; be lax if it's missing

Initialize hang_watchdog_path earlier - not just when
ves_icall_Mono_Runtime_EnableCrashReportingLog is called.  That function is
only expected to be called by apps that explicitly want progress reports from
MERP when it's collecting a crash report.  But we want the mono-hang-watchdog
to execute for all mono crashes (when crash reporting isn't ifdef'd out).

Also if we try to exec mono-hang-watchdog and fail for some reason, print a
nice message and then exit.  We used to call g_assert_not_reached which would
kick off another MERP of the _forked child process_ which would then get
confused because it really can't run in the forked process at all.
2019-08-28 08:34:30 -04:00
Jay Krell 2ca04a9743 [interp] Outline box. (#16456)
With additional reasonable changes this should conserve stack.
A combined PR that demonstrates actual savings probably should/will be presented.
2019-08-28 13:51:23 +03:00
Jay Krell 56b83f489b [interp] Minor cleanup of stackval_from_data. Compiler can do it. (#16534)
I missed this earlier cleaning stackval_to_data.
2019-08-28 12:11:56 +02:00
Jay Krell c870df0aa7 [interp] Asserts pessimize clang. Removing saves 16 bytes on Linux/amd64/clang. (#16530)
Paired with https://github.com/mono/mono/pull/16456, saves 16 bytes on Linux/amd64/clang.
No change with gcc.
Contributes to https://github.com/mono/mono/issues/16172.
2019-08-28 12:10:30 +02:00
Eric Erhardt 39ddea3983 Don't validate empty TransitionTimes on deserialization. (#26405)
On Unix, TransitionTimes are not used, so they are left empty. When deserializing an empty TransitionTime, it is throwing an exception because those are not normally valid. Fix deserialization to not validate empty TransitionTimes.

Fix dotnet/corefx 40578

Signed-off-by: dotnet-bot <[email protected]>
2019-08-28 11:55:07 +02:00
Jay Krell c534511003 Remove dead Boehm code. (#15224)
* Remove dead Boehm code.

Unity looked similarly dead.

The key points making it dead were:

create_allocator:
  g_assert_not_reached() covering everything but local variables

mono_gc_get_managed_allocator:
  always returning NULL, killing a bunch of code
  after it

mono_gc_is_critical_method:
   looking at data that was never filled in

This should have no impact, not even on Boehm.
Boehm should continue working just the same.

It is nice because it is the few/only uses of CEE_MONO_TLS.
2019-08-28 10:56:08 +02:00
Jay Krell 0375acd7c8 [mini] Cleanup MONO_MMAP_32BIT. (#16469)
- It _presumably_ means nothing on 32bit ARM and 32bit MIPS (was there mono/mips64?).
 - Leaving only PowerPC32 and PowerPC64 (in this vicinity).
   - It presumably means nothing on PowerPC32.
   - Leaving PowerPC64 as the only user in this vicinity -- is it needed?
- Notice this is varyingly absent on x86, arm64, amd64, s390x.
  Subject of another PR maybe.

Perhaps this was to enable the debugger to be 32bit against a 64bit target
on a biarch system, so it could do a cross-process manipulation of these pages?
That makes some sense on Windows at least.
(32bit debugger cannot debug 64bit target as of Vista, but that is using the Win32
"hard" debugger API. Mono could do otherwise).

There remains amd64 mono-codeman.c define ARCH_MAP_FLAGS MONO_MMAP_32BIT left alone.
2019-08-28 10:54:00 +02:00
Zoltan Varga 162849f972 Merge pull request #16520 from vargaz/fix-llvm-2
[llvm] Fix ctlz intrinsics, they have 2 arguments.
2019-08-28 03:24:01 -04:00
Bernhard Urban d6fa5ff9c6 fix cast and signature for mono_rconv_u4 2019-08-27 20:35:42 -05:00
Bernhard Urban 460bfa9beb [amd64] use opcode emulation for fconv/rconv 2019-08-27 20:35:42 -05:00
Bernhard Urban 8e6b967391 [llvmaotcheck] require sse3 being available for regression tests 2019-08-27 20:35:42 -05:00
Bernhard Urban 5b20322a84 [arm] use opcode emulation for fconv/rconv 2019-08-27 20:35:42 -05:00
Bernhard Urban ba824cb921 [arm] account only 4 bytes on stack for single precision arguments
Fixes `test_0_arm64_small_stack_args` in gshared.cs when
* `test_0_arm64_small_stack_args` is compiled with LLVM
* `Foo3.Floats` is compiled with Mini
2019-08-27 20:35:42 -05:00
Bernhard Urban 475defca4d [arm64] use opcode emulation for fconv/rconv
LLVM eagerly optimizes constants such as NaN in a way that does not align with .NET behaviour. In the future we can change it so that we only do opcode emulation when LLVM is used.

Fixes https://github.com/mono/mono/issues/16411
2019-08-27 20:35:42 -05:00
Bernhard Urban d164550a61 [ci] add make -C mono/mini llvmaotcheck to FullAOT LLVM lanes 2019-08-27 20:35:42 -05:00
Jay Krell 645c01066c Cleanup arm64 SeqPointInfo. 2019-08-27 20:34:42 -05:00
Stephen Toub 55002c3e02 Fix pragma warning restore (dotnet/coreclr#26389)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-28 00:16:03 +02:00
Jay Krell f516e79dd9 Inline mono_string_length_internal and mono_string_chars_internal (#16374)
* Inline mono_string_length_internal and mono_string_chars_internal for everyone.

* Add MONO_REQ_GC_UNSAFE_MODE.
2019-08-27 17:39:18 -04:00
Zoltan Varga cb5470b40a Fix timeouts. 2019-08-27 17:37:59 -04:00
Tarek Mahmoud Sayed 9ad1f08077 Fix Abbreviated Genitive Month Names Parsing (dotnet/coreclr#26379)
* Fix Abbreviated Genitive Month Names Parsing

In 3.0 we have fixed the formatting to use the abbreviated genitive month names when having "d" format specifier followed by "MMM". This fix is good as original formatting specs required but we needed to support the parsing when we have such genitive names.

* Addressing the feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 22:56:08 +02:00
Filip Navara f4145ce108 Update version information in System.Private.CoreLib to mimic CoreCLR and make BenchmarkDotNet happy (#16515)
BenchmarkDotNet recently updated the algorithms for .NET version detection (https://github.com/dotnet/BenchmarkDotNet/pull/1230) which made it incompatible with the version presented by Mono.

This updates the value of [ProductName](https://github.com/dotnet/coreclr/blob/1f428dbc5554191dadd27511cd5cd181e4a60efb/Directory.Build.targets#L7) and [Version](https://github.com/dotnet/coreclr/blob/c9007e73286456434ba12744e778e87a89fc1d90/eng/Versions.props#L5) to match what CoreCLR is currently reporting and makes the [BDN detection code](https://github.com/dotnet/BenchmarkDotNet/blob/b6d0e0f1d80ed60aa157149a0dfb704a2da4ca41/src/BenchmarkDotNet/Environments/Runtimes/CoreRuntime.cs#L73-L81) happy.
2019-08-27 22:55:37 +02:00
Ryan Lucia f9f5fff40c [netcore] Change exception thrown for SetTypedReference (#16518)
This brings us in line with CoreCLR behavior.
2019-08-27 22:53:37 +02:00
Zoltan Varga 340289d4ae [llvm] Fix ctlz intrinsics, they have 2 arguments. 2019-08-27 16:34:27 -04:00
Zoltan Varga ab1e9be35e [llvm] Fix the mixing of LLVM modules in the llvm JIT, use ctx->lmodule instead of module->lmodule. 2019-08-27 16:31:18 -04:00
Zoltan Varga 44d33673a5 Fix the netcore build. (#16519) 2019-08-27 22:30:16 +02:00
Zoltan Varga 0c015aa815 [wasm] Enable xunit tests for corlib/System.Core. 2019-08-27 15:46:26 -04:00
Zoltan Varga 3108d43051 Merge pull request #16487 from vargaz/cattr-data-handles
[runtime] Convert mono_reflection_create_custom_attr_data_args () to use handles.
2019-08-27 15:22:30 -04:00
Zoltan Varga 79475574a6 [wasm] Add more xunit test exclusions. 2019-08-27 13:41:11 -05:00
Egor Bogatov e9dff75d2f use gunichar2 2019-08-27 20:21:39 +02:00
Alexander Köplinger f914d900f6 Remove AppContext.cs that was moved to shared 2019-08-27 20:21:39 +02:00
Egor Bogatov f97d699183 Move AppContext.CoreCLR.cs to shared, remove static ctor (#26045)
* Remove AppContext.CoreCLR.cs (move to shared)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 20:21:39 +02:00
Steve Pfister 580defd967 Allow file:// based Uri's to be used in HttpClient when the BaseAddress is set
Fixes https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/967582

Bumps the HttpClient part in mono/corefx at: https://github.com/mono/corefx/commit/ee3acec16b37bec5016da129e1e3840097ca629b

The test that would have caught this failure was disabled previously via:
https://github.com/mono/mono/commit/2f411750acfd2132f9b4970ef8cf11d9a0f62df9
2019-08-27 13:54:52 -04:00
Alexander Köplinger 8d422436f5 [acceptance-tests] Bump roslyn test repo to remove a problematic nuget feed 2019-08-27 19:48:08 +02:00
Vlad Brezae ccc4bf8536 [interp] Remove unnecessary recomputation (#16509)
Seems to have been added by mistake in https://github.com/mono/mono/pull/16317, which was meant just to remove the locals variable from InterpFrame.
2019-08-27 20:27:22 +03:00
Jay Krell 00c1ec421a [interp] Error handling refactor (optimize non-exception path) (helps clang save stack) (#16371)
While working on stack reduction, I believe I found that
the compiler could not tell how THROW_EX and/or NULL_CHECK terminate.

That is, essentially, the success paths preserve volatile registers.
They do not have to be spilled/filled.
This tweak should make it clearer to the compiler, optimized common paths, and not slow down uncommon paths.

This also cleans it up a little and reduces macro expansion, and then possibly should allow for reinlining what is now used only once.

Also add some const so it can be casted away less.

Believed to save 32 bytes of stack with Linux/amd64/clang, nothing for gcc.
And a nice cleanup.
2019-08-27 15:31:10 +02:00
Zoltan Varga fa1288794b Merge pull request #16493 from EgorBo/impl-lzcnt
Implement System.Runtime.Intrinsics.X86.Lzcnt
2019-08-27 06:51:03 -04:00
Johan Lorensson 70aba13173 Fix multiple SGEN_LOG to correctly scale elapsed time in usec. (#16485)
Several SGEN_LOG entries logged usec based on result returend from TV_ELAPSED.
On SGEN's implementation of TV_ELAPSED, the result is not scaled to usec
but returned as 100ns ticks. This was not handled correctly by SGEN_LOG,
but other measures, like profile counters and SGEN binary protocol handles
it correctly.

Fix will adjust the SGEN_LOG calls currently presenting usec to correctly
scale the result. Fix also adds one additional logging measuring total
amount of time spend scanning thread data.
2019-08-27 12:14:12 +02:00
Jay Krell bfe1b6182c [interp] Replace imethod with frame->imethod. (#16459)
This conserves stack on Linux/amd64/gcc.
2019-08-27 11:51:21 +02:00
Jay Krell 6cf8b6e738 CodeManager cleanup: (#16468)
- Chunks do not need flags, only codemanagers do.
 - Sort by size to minimize padding for alignment.
 - Booleans only need one bit.
 - Chunk is not spelled chunck.
 - Suggest foo accessors be named foo() and set_foo(), not get_foo() and set_foo().
   i.e. the more common "get" is implied.
- Allow free(0) w/o initialization.
2019-08-27 10:40:35 +02:00
Jay Krell 9f9e5f7803 [monodis] Combine adjacent identical cases. (#16501) 2019-08-27 10:36:59 +02:00
Zoltan Varga c7d785f329 Add SETUP_ICALL_FUNCTION. 2019-08-27 03:57:59 -04:00
Jay Krell 7ca15e5a42 mono_attach_load_agent: Fix leaks, alloc, copy. (#15950)
Extracted and slightly modified from https://github.com/mono/mono/pull/15868.
2019-08-27 09:46:35 +02:00
Stephen Toub a0307b1e34 Fix comment typo in ManualResetValueTaskSourceCore
Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Vladimir Sadov 88ac3856eb Do not use AllocateUninitializedArray in private array pools. (#26338)
User may have extra expectations for the privatly owned array pools.
For example there could be an expectation that array never contains negative numbers (since user never puts them there). Uninitialized allocations can break such expectations.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Stephen Toub 82f618023f Disable SA1001 for LocaleData.Unix.cs
Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Stephen Toub 39d62eb8c8 Separate Async*MethodBuilder types into their own files (#26364)
* Separate async method builder types into their own files

* Minor cleanup of AsyncTaskMethodBuilder types

* Split AsyncValueTaskMethodBuilder types into their own files

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Nikolay Sivov 515ec75154 [git] Set csharp diff pattern for *.cs sources. (#16495) 2019-08-27 01:24:30 +02:00
Jay Krell 67c97d30b4 Add missing const, particularly on functions "like" memcpy. (#16477)
But do not change the signatures of external functions.
2019-08-27 01:04:46 +02:00
Zoltan Varga 4f8c0b9b46 Merge pull request #16451 from vargaz/gc-handle-pin
[runtime] Make the local gc handles pin the object they point to by scanning the handle area conservatively.
2019-08-26 17:31:50 -04:00
Aleksey Kliger (λgeek) 78edafd6e2 [threadpool-io] Throw if backend won't be able to register a FD (#16396)
The poll+select i/o selector backend can't handle file descriptor ids greater
than FD_SETSIZE.  This can happen if too many files are open and we want to
wait on it.

Previously, mono would fail in the i/o selector thread by which point it was
too late to do anything.

With this change we will fail eagerly on the thread that calls IOSelector.Add
by throwing a NotSupportedException.

Addresses https://github.com/mono/mono/issues/15931
2019-08-26 17:13:45 -04:00
Zoltan Varga 65a71a2912 Fix review comments. 2019-08-26 14:17:09 -04:00
Egor Bogatov 044d1519bd Implement Lzcnt hw intrinsics 2019-08-26 21:08:53 +03:00
Egor Bogatov 85265d884c [netcore] Implement missing ThreadPool methods (for metrics) (#16076)
* Implement missing ThreadPool methods

* Address feedback

* ifdef for mono_threadpool_worker_get_completed_threads_count

* remove GetPendingUnmanagedWorkItemCount
2019-08-26 11:35:38 -04:00
Zoltan Varga a908b72037 [runtime] Convert mono_reflection_create_custom_attr_data_args () to use handles. 2019-08-26 11:17:15 -04:00
Johan Lorensson f9be7a67d3 Fix special case tramp assert in x86/amd64. (#16476)
https://github.com/mono/mono/pull/16408 temporary fixed a tramp issue on
x86 (also exist on amd64). This commit fixes the underlying issue and re-enable
the use of noreturn outside of netcore.

The problem hit by the use of OP_NOT_REACHED on x86/amd64 is due to a specific
case when the address to patch happens to fall at the start of a different
managed method. Since the use of OP_NOT_REACHED can end a method
with the call instruction on x86/amd64 in combination with how patch location
(using the return address from stack) is resolved, the implementation in
mono_arch_patch_callsite didn't take this case into account, reading incorrect
patch data, triggering assert oh x86. So, based on timing we could end up with
the following code:

025715B3 call 02570D98
025715B8 push ebp

since the patch target is 025715B8, but also the start of a completely different
method, method_start == origin_code. The fix is to detect this case on x86/amd64
(done on each arch since the way origin_code is detected is arch specific)
and then use a method_start of NULL, that is already a supported scenario.
2019-08-26 16:46:43 +02:00
monojenkins 80504b8c98 [bcl] Update BCL Linked Size (#16427) 2019-08-26 14:01:25 +02:00
Jan Kotas 501b9cc3dd Fix BinaryReader.ReadChars for fragmented Streams (dotnet/coreclr#26324)
BinaryReader.ReadChars incorrectly read more than necessary from the underlying Stream when multi-byte characters straddled the read chunks.

Fixes https://github.com/dotnet/corefx/issues/40455

Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Next Turn 366f2750cd Remove unnecessary !s for [DoesNotReturn] (dotnet/coreclr#26350)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub 37c6d154ef Contribute to IDE0055 (fix formatting)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub 2f11eff09b Contribute to StyleCop SA1002 (semicolon spacing)
However, I'm not enabling the rule permanently, as it disagrees with VS IDE0055 about the spacing around semicolons at the end of empty while and for loops.

Because this also impacts `for (;;)` loops, I also searched and replaced those with `while (true)`, to standardize on that across the codebase.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub a86fb8c847 Fix StyleCop warning SA1001 (spaces around commas)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Alexander Köplinger 2f2771fcfa [utils] Don't use MAP_32BIT on Apple platforms, fixes crash with XCode 11 beta6 (#16441)
As of XCode 11 beta6 the MacOSX SDK defines the MAP_32BIT symbol:

```diff
--- /Applications/Xcode11-beta5.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/mman.h    2019-07-25 17:43:49.000000000 -0400
+++ /Applications/Xcode11-beta6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/mman.h    2019-08-06 21:03:07.000000000 -0400
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2000-2002 Apple Computer, Inc. All rights reserved.
+ * Copyright (c) 2000-2019 Apple Computer, Inc. All rights reserved.
  *
  * @APPLE_OSREFERENCE_LICENSE_HEADER_START@
  *
@@ -145,6 +145,10 @@
 #define MAP_RESILIENT_CODESIGN  0x2000 /* no code-signing failures */
 #define MAP_RESILIENT_MEDIA     0x4000 /* no backing-store failures */

+#if !defined(CONFIG_EMBEDDED)
+#define MAP_32BIT       0x8000          /* Return virtual addresses <4G only: Requires entitlement */
+#endif  /* !defined(CONFIG_EMBEDDED) */
+
 #endif  /* (!_POSIX_C_SOURCE || _DARWIN_C_SOURCE) */

 /*
```

This causes the mono_valloc() function to try to use the MAP_32BIT flag for mmap().

However as mentioned in the comment for the symbol in mman.h it seems to require a special entitlement
which isn't available/documented anywhere yet.
This in turn causes the mmap call to fail presumably because we're missing that entitlement.

Instead we now skip setting this flag on Apple platforms to make mmap() behave like it did before.
2019-08-26 13:53:40 +02:00
Egor Bogatov 6aaec2fe9b Optimized LLVM IR (#16448)
LLVM JIT: dump IR after optimizations 

```csharp
[MethodImpl(MethodImplOptions.NoInlining)]
static int Test(int a, int b, int c) => a * b + a * c;
```
`MONO_VERBOSE_METHOD="Test"` now prints both unoptimized and optimized IR:
```llvm
*** Unoptimized LLVM IR for P:Test ***

; Function Attrs: noinline uwtable
define monocc i32 @"P:Test (int,int,int)"(i32 %arg_a, i32 %arg_b, i32 %arg_c) #0 {
BB0:
  br label %BB3

BB3:                                              ; preds = %BB0
  br label %BB2

BB2:                                              ; preds = %BB3
  %t22 = mul i32 %arg_a, %arg_b
  %t25 = mul i32 %arg_a, %arg_c
  %t27 = add i32 %t22, %t25
  br label %BB1

BB1:                                              ; preds = %BB2
  ret i32 %t27
}
***


*** Optimized LLVM IR for P:Test ***

; Function Attrs: noinline uwtable
define monocc i32 @"P:Test (int,int,int)"(i32 %arg_a, i32 %arg_b, i32 %arg_c) #0 {
BB0:
  %t251 = add i32 %arg_b, %arg_c
  %t27 = mul i32 %t251, %arg_a
  ret i32 %t27
}
***
```
2019-08-26 07:36:04 -04:00
Jay Krell 6671316e35 [interp] Cleanup stackval_to_data and stackval_to_data_addr. (#16458)
No functional change, except for debugging, look up a frame
for the prior value.
2019-08-26 10:26:07 +02:00
Bernhard Urban b6ca76c484 [android] remove text relocations in AOT/arm32 (#16432)
[android] remove text relocations in AOT/arm32

Remove support for "big assemblies" for AOT on Android running on ARM 32bit. This will remove relocations in `.text` which are not allowed in the upcoming Android Q release anymore.

_If_ a customer should run into this problem, the suggested workarounds are:
* Switch to ARM64, or
* Do not use AOT compilation.

Fixes https://github.com/mono/mono/issues/16369
2019-08-26 03:46:12 -04:00
Ryan Lucia 510f3c7d70 Detect additional invalid assembly names when parsing (#16446) 2019-08-25 23:13:39 -04:00
Jay Krell e95ef822a1 Add MONO_LLVM_INTERNAL to mono_jit_compile_method. (#16463) 2019-08-25 23:52:17 +02:00
Jay Krell 2224d6b55d [interp] Outline mono_interp_box_vt. (#16454)
Contributes to https://github.com/mono/mono/issues/16172.

Seen to reduce frame from 0xB8 to 0xA8 on Linux/amd64/gcc against master.
Possibly similar elsewhere.
(Possibly 8 added to a rounded 8.)

Add const, add braces, they might help compiler and/or programmer.
2019-08-25 19:11:02 +02:00
Zoltan Varga 7d43ffda1a [runtime] Make the local gc handles pin the object they point to by scanning
the handle area conservatively.
2019-08-24 15:46:33 -04:00
Jay Krell 9d1eec6bf8 [interp] Outline STRMFLD_VT. (#16355)
Without being entirely scientific about it, this case seems
to be one that uses more than typical locals and therefore
be on a sort of critical path to reduce frame size.

I have a change that reduces frame to 0x78 bytes (albeit
not yet working) and to achieve that, in this case I refetched
the locals after function calls.

This achieves similar but perhaps more elegantly.

If we rest at larger frame size then this case might not be
on the critical path.

https://github.com/mono/mono/issues/16172
2019-08-24 09:56:40 +02:00
Jay Krell 99e9d80642 [windows gc] Trim one syscall per thread in garbage collection. (#16003)
* Skip one syscall per thread in garbage collection.
Specifically do not query for guard pages.
There are better ways:
 1. Visual C++ specific, __try / __except, get the guard page exception,
    put back the guard page. This is a little slower (exception + two syscalls),
    but exceedinly rare. __try itself is cheap on x86, free on the rest.
    This was also not allowed on UWP. It will be. And is VirtualProtectFromApp.

 2. Record the TEB and get from it the maximum commited page and just use MAX, trivial.

The comments are also not completely clear as to what is going on here, but they were close.
Augment them.
2019-08-24 09:56:00 +02:00
Jay Krell ab6a09741d Remove dlmalloc from Win32. (#15773) 2019-08-24 09:52:30 +02:00
Alexander Köplinger 3a409e8a7f Pass --clr-memory-model in csc/vbc wrapper script (#16439)
Should help with random CSC crashes on ARM (we've been using it in the Mono build for a while).
2019-08-24 00:08:25 +02:00
Zoltan Varga ab9c58d76d Merge pull request #16414 from vargaz/llvm-gc-poll-wrapper
[llvm] Add a mono wrapper method for the gc poll cold cconv wrapper so the EH code can unwind through it.
2019-08-23 18:06:17 -04:00
Ryan Lucia 83cf007ae3 Return an empty string for assemblies loaded from stream (#16345)
* Return an empty string for assemblies loaded from memory

* Blind attempt to fix wasm
2019-08-23 15:58:10 -04:00
Ryan Lucia 7615ba3e45 [netcore] Use requesting assembly's context for InternalLoad calls 2019-08-23 15:30:04 -04:00
Ryan Lucia 0661bf0424 [netcore] Random loader cleanup 2019-08-23 15:30:04 -04:00
Nathan Ricci f6b2befe8d Removed trailing whitespace. 2019-08-23 13:57:01 -04:00
Nathan Ricci 707db63569 Added stub for boehm. 2019-08-23 13:45:23 -04:00
Nathan Ricci cdb77f1391 Fixed fomatting issues. 2019-08-23 13:42:34 -04:00
Tarek Mahmoud Sayed cb1f7c0677 Cleanup GregorianCalendarHelper leap month checks (dotnet/coreclr#26321)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-23 15:43:54 +02:00
Alexander Köplinger 278fd48274 Bump corefx to get https://github.com/mono/corefx/pull/336 2019-08-23 14:43:05 +02:00
Zoltan Varga f0cbb79b7f Merge pull request #16431 from kjpou1/wasm-maindir-cleanup
[wasm] Small cleanup of main wasm directory
2019-08-23 08:13:21 -04:00
Zoltan Varga 97d2d97615 [jit] Make mono_unwind_frame () ignore registers which are greater than NUM_DWARF_REGS.
These registers can be encountered in the unwind info for functions with a cold
cconv, like xmm0 on amd64. Since they are caller saved, they don't have to
be restored during EH.
2019-08-23 07:49:20 -04:00
Zoltan Varga fdd0d7f027 Merge pull request #16429 from vargaz/llvm-jit-module
[llvm] Don't pass a module argument to the code in llvm-jit.cpp, we create a new LLVM module for each method.
2019-08-23 06:40:09 -04:00
Zoltan Varga 81f736e774 Fix the bitcode build. 2019-08-23 06:28:40 -04:00
Zoltan Varga 4ed1c3335f [llvm] Add a cold wrapper and mono wrapper for gc poll for JITted code as well. 2019-08-23 05:34:52 -04:00
Kenneth Pouncey f5717d21bb Address review comments wasm/support to wasm/src 2019-08-23 11:11:57 +02:00
Zoltan Varga 0d603261a0 Fix the build. 2019-08-23 04:41:50 -04:00
Zoltan Varga f6b189ecab [llvm] Add a mono wrapper method for the gc poll cold cconv wrapper so the EH code can unwind through it. 2019-08-23 04:41:50 -04:00
Zoltan Varga 8ace375450 Merge pull request #16407 from vargaz/fix-llvm-simd
[llvm] Fix type issues in the llvm IR emitted for SIMD. Fix a warning.
2019-08-23 04:41:10 -04:00
Zoltan Varga 14a76b03c5 Merge pull request #16408 from vargaz/noreturn-methods
[jit] Special case the ThrowHelper methods only on netcore.
2019-08-23 04:40:28 -04:00
Egor Bogatov 47efe6bab0 Merge pull request #16412 from EgorBo/fix-ci4
[netcore] Fix CI, ignore failing tests
2019-08-23 11:17:31 +03:00
Kenneth Pouncey 8b338a9619 [wasm] Small cleanup of main wasm directory
- Move source support files for custom building the wasm runtimes to a new directory called `wasm/support`
- Modified `packager.exe` to build from this new subdir.
- Updated the README.md to document the change.
2019-08-23 09:59:31 +02:00
Zoltan Varga c357de87ec [llvm] Don't pass a module argument to the code in llvm-jit.cpp, we create a new LLVM module for each method. 2019-08-23 03:27:41 -04:00
Zoltan Varga c97bfe2167 Merge pull request #16379 from EgorBo/improve-dump
Improve MONO_VERBOSE_METHOD
2019-08-23 02:08:12 -04:00
Michal Strehovský 9bffb810ee Mark methods specially handled in JitInterface Intrinsic (#26209)
* Mark methods specially handled in JitInterface Intrinsic

For CPAOT compiler's sake.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-23 00:51:24 +02:00
coypoop f626f3e7aa If either sched_get_priority_{min,max} failed, don't try to set a priority (#15898)
netbsd and linux both require that if SCHED_OTHER is used, a fixed value is used for sched_param ([0 for linux](http://man7.org/linux/man-pages/man7/sched.7.html), -1 for netbsd)

the netbsd case can also be detected by the [POSIX-defined failure](https://pubs.opengroup.org/onlinepubs/007908799/xsh/sched_get_priority_min.html) return value for sched_get_priority_min, a return value of -1.
2019-08-23 00:46:08 +02:00
EgorBo 71dce6502a ignore FSW test 2019-08-23 01:12:47 +03:00
Martin Baulig 473d07a86c Fix issue in SafeHandle.DangerousReleaseInternal() that caused #16034. (#16244)
Using the CoreCLR version of this line fixed the problem and my investigations
also revealed why.

The `SafePipeHandle` contains a reference to both the Socket as well as the
`SafeSocketHandle`, taking ownership of them.  If there are no additional
references to either of them, so that both are collected in the same GC pass,
their finalizer may be invoked in any order.

If the Socket gets finalized first, it's `SafeSocketHandle` will have a `_mstate`
of 8 (two references) at the beginning of `DangerousReleaseInternal()` - which
will set it to 6 (one reference plus disposed flag).

Then when the `SafePipeHandle` gets collected, it will call `DangerousRelease ()`
on it's `SafeSocketHandle`.  As mentioned above, it's `_mstate` will be 6.
Now we own the handle, so we'll decrease it's reference count by one and set the
`Closed` flag - the new `_mstate` should thus be 3.

However, using

    new_state = (old_state & RefCount_Mask) - RefCount_One;

would set it to 1 - thus effectively clearing the `Disposed' flag.  And since it's
reference count is now also zero, the next finalize pass will then throw.

The CoreCLR version of this class uses

    new_state = old_state - RefCount_One;

which fixes this.
2019-08-22 23:46:31 +02:00
Nathan Ricci 93ff9dab7b Removed start/stop the world gc api. 2019-08-22 16:50:58 -04:00
EgorBo 692f70cb67 ignore more tests 2019-08-22 23:08:44 +03:00
EgorBo 40cf68aa98 Disable the whole System.ComponentModel.Composition namespace 2019-08-22 21:47:45 +03:00
Stephen Toub f5a1fa4744 Fix IDE0034 (simplify default expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f256a4525d Fix IDE0054 (use compound assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub e2c7b4200a Fix (mostly) IDE0055 (formatting)
Lots of various spacing issues.  I say "mostly" because I explicitly undid some changes that added to rather than reduced inconsistency, e.g. region indents.  We can revisit that later.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 9be0c2217c Fix StyleCop warning SA1114 (parameter list should follow declaration)
Fixed just be removing stale/unnecessary comments.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub d3817c9302 Fix StyleCop warning SA1006 (empty statements)
This is just running the rule and auto-fixing everything.  I undid changes that converted e.g. `while (...);` to `while (...) { }`.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 35d8ca894b Fix StyleCop warning SA1005 (single line comment spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 55bc2fee8a Fix (mostly) StyleCop warning SA1003 (operator symbol spacing)
The "mostly" here is that I undid changes where it was forcing formatting around a `: this` or `: base` call for a ctor, in particular where it was moving one up from the next line to be at the end of the current line.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub e23c35f730 Fix StyleCop warning SA1024 (colon spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 26656df25e Fix StyleCop warning SA1015 (closing generic brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f3b0e98aae Fix StyleCop warning SA1013 (closing brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub ac42e06fea Fix StyleCop warning SA1012 (opening brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 8c01fd5189 Fix StyleCop warning SA1008 (opening paren spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f83a8da0ce Fix StyleCop warning SA1002 (semicolon spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 46302d54aa Fix StyleCop warning SA1001 (commas followed by whitespace)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Nathan Ricci bfce92d9dd Added tracking for detached threads. 2019-08-22 14:14:39 -04:00
EgorBo 7678654781 fix macOS 2019-08-22 20:53:14 +03:00
Steve Pfister fbdad78101 Added a workaround for an issue with the compiler server on 32-bit linux. (#16405)
Currently, if the process is 32-bit, roslyn tries to determine how much
memory is available.  To do that, it tries to pinvoke into GlobalMemoryStatusEx,
which is not supported on mono.

Without it, the compiler server bombs on https://github.com/dotnet/roslyn/blob/0e63260c5afb3fb5b74c357dd250e500172bcd63/src/Compilers/Server/VBCSCompiler/NamedPipeClientConnection.cs#L55-L59
2019-08-22 18:22:22 +02:00
EgorBo 779fda5103 update rsp 2019-08-22 19:02:53 +03:00
EgorBo f809fe55a8 ignore some tests 2019-08-22 18:52:52 +03:00
Stephen Toub 4dd0a137a4 Avoid race condition in DiagnosticCounter (dotnet/coreclr#26260)
* Avoid race condition in DiagnosticCounter

If while a DiagnosticCounter instance is being created the timer fires to process counters, the timer may end up evaluating a DiagnosticCounter whose constructor or derived types' constructor hasn't finished configuring the instance's state yet.

* Remove unnecessary #if

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 16:59:43 +02:00
Nikolay Sivov 068660732f [corlib] Improve file:// url handling in AppDomainSetup. (#16161)
* [corlib] Improve file:// url handling in AppDomainSetup.

WiX 3 installer toolkit uses file:/// prefix when setting ApplicationBase for newly created domain.
Currently this leads to invalid path exception:

file:///C:\... -> /C:\... -> full path expansion with current workdir on different drive -> Y:\C:\...

Later it fails check for multiple ':' -> NotSupportedException ("The given path's format is not supported.").

This patch uses logic already present in AppDomain::Load().

* [Test] Add a test for file:/// application base.

* [Test] Fix test failure on Windows.

* [corlib] Simplify file: protocol check for AppDomainSetup.
2019-08-22 16:54:32 +02:00
Zoltan Varga 49f913ac8a [jit] Special case the ThrowHelper methods only on netcore. 2019-08-22 10:36:43 -04:00
Zoltan Varga f2f58e522b [llvm] Fix type issues in the llvm IR emitted for SIMD. Fix a warning. 2019-08-22 10:28:47 -04:00
Aleksey Kliger (λgeek) e74736a471 [android] mono_dl_open_file: use g_file_test only on absolute paths (#16387)
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is
the name of a shared library we want to open is to speed up probing for
non-existent libraries.

See https://github.com/mono/mono/pull/12074

The problem is that if file is just a simple "libdl.so" then `dlopen (file)`
doesn't just look for it in the current working directory, it will probe some
other paths too.  (For example on desktop linux you'd also look in all the
directories in LD_LIBRARY_PATH).  So the g_file_test() call is not a robust way
to avoid calling dlopen if the filename is relative.

But it actually broke more things: https://github.com/xamarin/xamarin-android/issues/3388

When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try
prepending some paths that it knows about and we end up calling
`dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security
restrictions on what code can dlopen something from /system/lib with an
absolute path.  Eventually mono_lookup_pinvoke_call will go back to trying the
bare "libdl.so" which hits `g_file_test` and returns NULL.

The new code only does the file test if we pass it an absolute path, which
gives Bionic's dlopen a chance to deal with relative paths however it needs to.
2019-08-22 10:10:09 -04:00
Zoltan Varga f6b0c9b772 Merge pull request #16300 from EgorBo/llvm-fpm
Use basic LLVM optimizations (PassManager) in JIT mode
2019-08-22 09:07:02 -04:00
Jay Krell 5368fef5ee [interp] Replace endsp with preexisting child_frame.retval which appears equivalent. (#16378)
This should conserve stack. I didn't verify it.
child_frame being address-taken, compiler would struggle to remove it.
So we have the same value in two places.

https://github.com/mono/mono/issues/16172
2019-08-22 14:57:38 +02:00
Zoltan Varga 87a6364d58 Merge pull request #16390 from EgorBo/intptr-intrinsic
Make IntPtr.Zero intrinsic
2019-08-22 08:43:07 -04:00
Zoltan Varga 341cc0dd83 Merge pull request #16386 from vargaz/llvm-inline-limit
[llvm] Use a different inline limit (30) for llvm+aot.
2019-08-22 08:26:10 -04:00
Zoltan Varga 3f21a2c8f3 Merge pull request #16384 from vargaz/fix-16381
[llvm] Fix an assert.
2019-08-22 07:26:36 -04:00
Zoltan Varga 254d6d3758 Merge pull request #16389 from vargaz/fix-16380
[aot] Simplify a check to avoid calling into the assembly loading code while loading the aot image for corlib.
2019-08-22 05:36:17 -04:00
Zoltan Varga b7ca80ed58 Add comments. 2019-08-22 00:34:32 -04:00
Egor Bogatov 279676f43e Update llvm-jit.cpp 2019-08-22 01:46:10 +03:00
Egor Bogatov d02695ef47 cleanup 2019-08-22 01:43:56 +03:00
Egor Bogatov 1822f2faab Merge remote-tracking branch 'origin/master' into llvm-fpm 2019-08-22 00:47:40 +03:00
Nathan Ricci 6caa530970 Removed unneeded declarations. 2019-08-21 17:31:34 -04:00
Bernhard Urban 5c9be2dc2b [netcore] Use Release configuration for default System.Private.CoreLib build (#16391)
It seems the Configuration set inside of the System.Private.CoreLib.csproj is just completely ignored, making it default to "Debug".

Maybe because we're using Microsoft.NET.Sdk ?
2019-08-21 23:16:03 +02:00
Nathan Ricci b65443f324 Added stubs in null, boehm. 2019-08-21 16:55:51 -04:00
Nathan Ricci f913945e76 Implemented precise. 2019-08-21 16:27:02 -04:00
Nathan Ricci dc8aeba593 stopped the world for precise. 2019-08-21 16:05:29 -04:00
Nathan Ricci 56f44379d0 Took changes from miguel's branch. 2019-08-21 15:54:07 -04:00
Nathan Ricci 65360da5d9 Working on precise. 2019-08-21 15:51:45 -04:00
Egor Bogatov cb34178cd6 use EMIT_NEW_PCONST 2019-08-21 21:48:14 +03:00
Egor Bogatov d3ecd70ead Make IntPtr.Zero intrinsic 2019-08-21 21:23:19 +03:00
Zoltan Varga c4de19ef09 [aot] Simplify a check to avoid calling into the assembly loading code while loading the aot image for corlib.
Fixes https://github.com/mono/mono/issues/16380.
2019-08-21 14:15:25 -04:00
Nathan Ricci 0754f5ebe5 Fixed alphabitization error. 2019-08-21 13:59:50 -04:00
Nathan Ricci cfb4afff43 Got rid of some debug code. 2019-08-21 13:55:20 -04:00
Zoltan Varga 433b9a7f70 [llvm] Use a different inline limit (30) for llvm+aot. 2019-08-21 12:20:50 -04:00
Zoltan Varga 0e2da3c2ab [llvm] Fix an assert.
Fixes https://github.com/mono/mono/issues/16381.
2019-08-21 12:03:20 -04:00
Nathan Ricci 53939d0790 Implemeneted imprecise. 2019-08-21 11:20:27 -04:00
EgorBo 26d73b94f6 cleanup 2019-08-21 14:34:00 +03:00
EgorBo 467a561dcb Merge remote-tracking branch 'origin/master' into ffast-math
# Conflicts:
#	mono/mini/llvm-jit.cpp
#	scripts/ci/pipeline-netcore-runtime.yml
2019-08-21 14:30:07 +03:00
EgorBo ad713a70ef Merge remote-tracking branch 'origin/master' into llvm-fpm 2019-08-21 14:20:57 +03:00
Egor Bogatov 50b57fe926 Update mini.c 2019-08-21 14:07:59 +03:00
Egor Bogatov 6bb1e5ffc4 Update mini-llvm.c 2019-08-21 14:07:02 +03:00
Egor Bogatov e5acc9be7a undo change in mini.c 2019-08-21 14:04:57 +03:00
Egor Bogatov b0b3743f36 Improve MONO_VERBOSE_METHOD 2019-08-21 13:35:40 +03:00
Kenneth Pouncey 935d0a54e7 [wasm] Make sure message pump is working when run via webworker. (#16325)
- Tests if we are running in ENVIRONMENT_IS_WORKER
   - run message pump if we are.

- This also fixes gc not running.

issue: https://github.com/mono/mono/issues/16319
2019-08-21 06:24:06 +02:00
Zoltan Varga 977ccf7cb2 Merge pull request #16363 from vargaz/tp-thread-name
[runtime] Avoid resetting the tp thread name on every call to worker_callback ().
2019-08-20 23:14:44 -04:00
Zoltan Varga 285f33efe6 Merge pull request #16286 from vargaz/netcore-simd
[netcore] Add a netcore version of simd-intrinsics.c.
2019-08-20 23:11:27 -04:00
Nathan Ricci 37086aa1c4 Put things in correct order. 2019-08-20 18:28:35 -04:00
Nathan Ricci bf04a1946f Modified sample project. 2019-08-20 17:38:23 -04:00
Nathan Ricci fb7895dddf Added ical to netcore. 2019-08-20 17:37:58 -04:00
Alexander Köplinger 38c524cc2a Fix Windows build 2019-08-20 22:55:39 +02:00
Alexander Köplinger 55f2c4d6ad [profiler] Rename helper functions to have common prefix
Unfortunately https://github.com/mono/mono/pull/16335 didn't fix the new symbols showing up in the log profiler module.
The reason is that we actually pass `--disable-visibility-hidden` for iOS builds in https://github.com/mono/mono/blob/0b6d95e8ce7c882e2db9e89fa0dec850c687f65e/sdks/builds/ios.mk#L98.

Rename the helper functions instead to have a common prefix so they can be ignored in xamarin-macios tests.
2019-08-20 22:55:39 +02:00
Aleksey Kliger ed0063b4d5 [w32socket] Translate some errno codes to WSA_ errors imprecisely.
These aren't a perfect match, but better than falling into the `default` case
where Mono asserts and crashes.

Addresses part of https://github.com/mono/mono/issues/16024
2019-08-20 16:13:41 -04:00
Aleksey Kliger c38044d74f [w32error] Add WSA_INVALID_PARAMETER and WSA_INVALID_HANDLE 2019-08-20 16:13:41 -04:00
Ryan Lucia fc5e3fa039 [netcore] Remove local copy of static alc resolve methods 2019-08-20 21:25:00 +02:00
Ryan Lucia 3b47d28e37 Move AssemblyLoadContext VM-invoked static resolve methods to shared (#26248)
Shared between CoreCLR and Mono. Mono began using a copy of these methods with https://github.com/mono/mono/pull/16256/

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 21:25:00 +02:00
Nathan Ricci c1d01679c5 Merge branch 'master' of github.com:naricc/mono into naricc/GetTotalAllocatedBytes 2019-08-20 15:22:18 -04:00
Alexander Köplinger a147402d07 [ci] FreeBSD: Make sure we have gmake in /usr/bin 2019-08-20 21:08:54 +02:00
Alexander Köplinger ac3aa7c1c7 Bump corefx to get https://github.com/mono/corefx/pull/332 2019-08-20 20:52:02 +02:00
Stephen Toub 5f38497e19 Remove unused usings from Corlib (dotnet/coreclr#26250)
* Fix IDE0065 (misplaced using)

* Fix IDE0005 (remove unnecessary usings)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 19:45:30 +02:00
madmir 4dd016004b Re-enable xunit warning 1019 (dotnet/corefx#40415)
Make MemberData reference IEnumerable<object[]>

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 19:45:30 +02:00
Ryan Lucia 0b6d95e8ce [netcore] Remove unnecessary calls to mono_domain_get (#16336)
Should use mono_alc_domain where possible
2019-08-20 12:55:54 -04:00
Alexander Köplinger 5dc4a86d9d [netcore] Consolidate running tests between AzDO pipeline and Makefile (#16348)
* [netcore] Consolidate running tests between AzDO pipeline and Makefile

There was some unnecessary duplication between the two that we can remove.

* Add timeout to test execution
2019-08-20 17:59:55 +02:00
Zoltan Varga 8a413fad71 Fix the build. 2019-08-20 11:32:46 -04:00
Zoltan Varga b5cfeb3b5b [runtime] Avoid resetting the tp thread name on every call to worker_callback (). 2019-08-20 11:31:12 -04:00
Phil Jaenke b7dc4f0a46 Fix Jenkins build failure due to missing dependencies (#16362)
Jenkins builds are failing due to missing gettext; also realized that graphics unit tests will fail without some additional packages that will not be pulled in automatically.

Added:
`gettext-runtime gettext-tools cairo libdrm mesa-dri mesa-libs`
2019-08-20 16:19:36 +02:00
Jo Shields 3070ab6a96 Pass -fvisibility=hidden to profiler modules. (#16335)
* Pass -fvisibility=hidden to profiler modules.

This appears to not break the profilers:

```
directhex@breakfast:/tmp$ nm -g /tmp/ponyponyponypony/lib/libmono-profiler-log.so | grep close_socket_fd
directhex@breakfast:/tmp$
directhex@breakfast:/tmp$ /tmp/ponyponyponypony/bin/mono --profile=log:report bottle.exe
99 bottles of beer on the wall, 99 bottles of beer.
Take one down and pass it around, 99 bottles of beer on the wall.

[..]

No more bottles of beer on the wall, no more bottles of beer.
Go to the store and buy some more, 99 bottles of beer on the wall.

Mono log profiler data
	Profiler version: 3.0
	Data version: 17
	Arguments: log:report
	Architecture: x86-64
	Operating system: linux
	Mean timer overhead: 86 nanoseconds
	Program startup: Mon Aug 19 14:18:30 2019
	Program ID: 17715
	Server listening on: 41539

JIT summary
	Compiled methods: 753
	Generated code size: 228046
	JIT helpers: 0
	JIT helpers code size: 0

GC summary
	GC resizes: 0
	Max heap size: 0
	Object moves: 0

Metadata summary
	Loaded images: 3
	Loaded assemblies: 3

Exception summary
	Throws: 0
	Executed finally clauses: 3

Thread summary
	Thread: 0x7fcd968c9700, name: "Finalizer"
	Thread: 0x7fcd966c8700, name: "Profiler Sampler"
	Thread: 0x7fcd9b036780, name: "Main"
	Thread: 0x7fcd964c7700, name: "Profiler Helper"
	Thread: 0x7fcd954c5700, name: "Profiler Dumper"
	Thread: 0x7fcd95cc6700, name: "Profiler Writer"

Domain summary
	Domain: (nil), friendly name: "bottle.exe"

Context summary
	Context: (nil), domain: (nil)

Counters:
	Mono System:
		User Time                      : 70.000ms
		System Time                    : 0.000ms
		Total Time                     : 140.000ms
		Working Set                    : 22032384
		Private Bytes                  : 63455232
		Virtual Bytes                  : 449630208
		Page Faults                    : 3004
		CPU Load Average - 1min        : 1.870000
		CPU Load Average - 5min        : 2.670000
		CPU Load Average - 15min       : 2.900000
	Mono JIT:
		Methods from AOT               : 536
		Methods JITted using mono JIT  : 154
		Methods JITted using LLVM      : 0

```

* Add missing annotations and includes
2019-08-20 16:17:26 +02:00
Calvin Buckley b57d52ce77 Fix minizip compile on Haiku (#16301)
it doesn't have f*64 funcs, it seems
2019-08-20 16:00:24 +02:00
Jay Krell 169205813b [interp] Outline box_nullable. (#16356)
Without being entirely scientific about it, this case seems
to be one that uses more than typical locals and therefore
be on a sort of critical path to reduce frame size.

I have a change that reduces frame to 0x78 bytes (albeit
not yet working) and to achieve that, in this case I refetched
the locals after function calls.

This achieves similar but perhaps more elegantly.
If we rest at larger frame size then this case might not be on the critical path.

https://github.com/mono/mono/issues/16172.
2019-08-20 15:53:25 +02:00
Jan Kotas 1592d2a8fa Change missed by the mirror (dotnet/coreclr#26251)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 15:37:10 +02:00
Phil Jaenke 317e741c02 Many updates to freebsd.sh (#16342)
* Many updates to make it better

Too much to list by many measures. This makes sure appropriate kernel modules, file systems, build packages, supplemental packages, and directories are installed or created as appropriate. This also addresses a known failure condition in System.Security-xunit due to a missing directory resulting in ownership problems.

* Make corrections for Azure

I don't have an Azure to test with, so I missed that the script is calling with `-e` and that Azure uses a non-`GENERIC` kernel with procfs baked in (likely required for Azure drivers.)
2019-08-20 15:35:18 +02:00
Egor Bogatov c231a357a7 Merge branch 'master' into ffast-math 2019-08-20 14:19:06 +03:00
Jay Krell ea29349f77 [interp] Reduce register/stack pressure esp. around vtable initialization. (#16349)
Part of #16172 though not necessarily any gain from this change, at this time.
2019-08-20 13:03:12 +02:00
Alexander Köplinger b205f1bd9c [interp] Reduce register/stack pressure slightly, i.e. so that field does not have (#16339)
to be preserved across `mono_get_special_static_data ()` if non-volatile registers
are all otherwise allocated.
2019-08-20 12:04:30 +02:00
Jay Krell 33153ddd2d [interp] Relieve register/stack pressure around mono_thread_internal_current. (#16341)
Part of #16172 though not necessarily any gain from this change, at this time.
We might want to expose ifdef and inline mono_thread_internal_current also or instead.
2019-08-20 12:03:48 +02:00
Alexander Köplinger 5ebce09c51 Bump bockbuild to get gtksharp 2019-08-20 11:54:59 +02:00
Jay Krell bc2accfc7e Fix warnings. (#16287)
debug-mono-ppdb.c ppdb_compressed_size may be used uninitialized
   ppdb_size may be used uninitialized
 image.c iinfo set but not used
 icall.c klass set but not used
2019-08-20 11:47:38 +02:00
Egor Bogatov 8800f7c166 Update pipeline-netcore-runtime.yml 2019-08-20 12:14:23 +03:00
Egor Bogatov 4a5e2a3f79 Update pipeline-netcore-runtime.yml 2019-08-20 12:13:59 +03:00
Alexander Köplinger 9a5ae23f7e [netcore] Use Release configuration for CI builds (#16329)
* [netcore] Use Release configuration for CI builds

Fixes https://github.com/mono/mono/issues/16308

* [ci] Ensure the artifacts/logs/Release folder exists

Some arcade steps need the path to exist and we only create artifacts/logs/Debug right now,
because that folder is created by eng/common/tools.sh used by init-tools.sh defaulting to Debug.
2019-08-19 23:52:01 +02:00
Jay Krell 31aed632ba [interp] Inline MINT_NEWOBJ_VTST_FAST (#16314)
For purposes of stack savings, only MINT_NEWOBJ_VT_FAST w/o the "ST"
needed to be outlined, and inlining also the ST form increased
its stack use, w/o savings elsewhere.

That is, the recursive cases should generally not be outlined.
We were outlining two. We still outline one.

This is partially undoing my recent change, where I outlined both of these, but only one was justified.
2019-08-19 23:48:20 +02:00
Egor Bogatov feac6c0d04 temp: run tests on CI 2019-08-19 23:30:41 +03:00
Jay Krell 2313f1a884 [interp] Reduce register/stack pressure slightly, i.e. so that field does not have
to be preserved across `mono_get_special_static_data ()` if non-volatile registers
are all otherwise allocated.
2019-08-19 13:02:49 -07:00
Ryan Lucia 7c3dfbcbdf [netcore] Update ALC test exclusions (#16332) 2019-08-19 14:50:15 -04:00
Aleksey Kliger (λgeek) 2f0cbb5aa5 [coop] Add a handle frame in construct_culture (#16299)
Otherwise we see warnings like these:

```
ves_icall_System_Globalization_CultureInfo_internal_get_cultures_raw USED 680 handles
```
2019-08-19 11:01:09 -04:00
Alexander Köplinger 0ad1f5cca1 [ci] Add CI VM provisioning script for FreeBSD to the repo (#16326)
We'll download it from GitHub instead of storing it in Jenkins admin config.

Trial run with FreeBSD for moving all the other provisioning scripts to the repo too.
2019-08-19 14:44:00 +02:00
Martin Baulig a3c31b9ef7 Add missing Dispose() call on error in MonoTlsStream.CreateStream(). (#16233)
* Add missing `Dispose()` call on error in `MonoTlsStream.CreateStream()`.

* Remove the `finally` block to make it more easy to read.
2019-08-19 14:23:28 +02:00
Vincent Povirk 9f790cd5e2 eglib: Remove broken executable test on Windows. (#16295)
The strcmp is wrong, and it doesn't test anything meaningful anyway.

This flag is unused since #16156 was merged.
2019-08-19 13:09:14 +02:00
Jan Kotas a723047f68 Fix CoreRT build breaks
Signed-off-by: dotnet-bot <[email protected]>
2019-08-19 13:07:13 +02:00
Stephen Toub ad63045907 Remove useless GetLongPathNameW call (dotnet/coreclr#26210)
Immediately after calling this we're going to loop around and call it again at the beginning of the while loop.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-19 13:07:13 +02:00
Egor Bogatov 77a9c82523 Introduce ffast-math 2019-08-19 13:07:43 +03:00
Jay Krell 8886f71b2c [interp] Remove InterpFrame.locals, replace with reasonably efficient frame_locals macro.o (#16317)
Contributes to #16172.
2019-08-19 11:50:20 +02:00
Alexander Köplinger e754bb1c43 [interp] Put finally data back in locals instead of frame to save 16 bytes of stack. (#16307)
* Put finally data back in locals instead of frame to save 16 bytes of stack.

Partial undo of f8fd60b5c0.

* Add comments explaining goto main_loop vs. MINT_IN_DISPATCH.
2019-08-19 11:24:44 +02:00
Jay Krell 60cd1b3556 [interp] Outline ves_array_element_address to save 16 bytes of stack on Linux/amd64/clang. (#16304)
Contributes to #16172.
2019-08-19 11:24:10 +02:00
Alexander Köplinger 9001c4fa83 thread names: coop, constants, usually ignore-error. (#16272)
- Convert setting thread name to be coop-compatible.

- For constant thread names, just retain the constant, not a copy.
  This includes producing and using constant unicode thread names on Windows.

- Make setting a thread name often non-fatal, except where mandated by public API.

- From earlier PR: mono_free (mono-publib.c) would no longer be referenced and therefore
  no longer exported. That broke profilers and maybe other externals.
  Choices:
     1 Use it instead of g_free randomly sometimes.
     2 Call it randomly sometimes.
     3 include it in external-only.c or object.c
     4 Mark it external only (breaks profiler).
     5 Use a .def file or Unix equivalent.

     5 is best, 2 is done here, the advantages/disadvantages
     among most choices are subtle. 3 is probably better than 2
     but is slightly bigger change, to put off -- you'd stop
     compiling mono-publib.c and move it to include/extra_redist.

     The advantage of not-.def file is perhaps that it is exposed
     by the compiler, so maybe easy to port and work with.
     People are more comfortable with obscure C extensions than
     any linker options, and there are multiple linkers to contend
     with. Arguably there are fewer compilers.
2019-08-19 11:23:23 +02:00
Jay Krell cf0ce4f55a Reduce size of internal MonoErrors by 3 pointers, while trying to be compatible with old size.(#16218)
There is an incompatibility here.

If Mono hands off an InternalError and it is accessed
like an ExternalError, like memset(sizeof()), bad.

Hopefully nobody does that.

There is also compatability -- embedders can continue to embed MonoError
in their own data. Its size and hopefully alignment is unchanged.
2019-08-19 11:20:37 +02:00
Jay Krell 047c7d35fa [interp] Replace frame.args with frame.stack, the same except for type (#16316)
Contributes to #16172.
2019-08-19 11:19:10 +02:00
Vlad Brezae 975b543e92 [interp] Make sure we allocate stack for MINT_LD_DELEGATE_INVOKE_IMPL (#16224)
* [interp] Make sure we allocate stack for MINT_LD_DELEGATE_INVOKE_IMPL

This opcode is pushing a function pointer from the delegate, so it needs to have stack reserved.

* [interp] Fix overflow of stack space
2019-08-19 11:59:21 +03:00
Jay Krell cecea63691 warning: mono-threads-wasm.c no previous prototype for function mono_background_exec (#16289) 2019-08-19 05:22:06 +02:00
Stephen Toub 04909573e7 Undo use of Index/Range in files compiled into netfx-compat assemblies
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 22:12:06 +02:00
Stephen Toub 424af0fb98 Fix DecoderExceptionFallbackBuffer message (dotnet/coreclr#26211)
When the number of unknown bytes is exactly 20, the message includes a misleading " ..." at the end.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 22:12:06 +02:00
Jay Krell d1f47e7b48 [interp] Reduce register/stack pressure slightly. (#16303)
No savings on Linux/amd64/gcc/clang.
2019-08-18 17:36:33 +03:00
Zoltan Varga e1b57c371a Fix the build. 2019-08-18 08:48:15 -04:00
Zoltan Varga 4503b67a0b Implement a subset of Intel BMI1/BMI2 intrinsics used by corelib. 2019-08-18 06:08:31 -04:00
Zoltan Varga 7337a3e405 Add support for Bmi1 instrinsics. 2019-08-18 05:25:04 -04:00
Zoltan Varga 157bb9f4e3 Make simd-methods-netcore.h less verbose. 2019-08-18 05:23:15 -04:00
Zoltan Varga 121d9ca4a9 [jit] Don't mark ThrowForUnsupportedVectorBaseType<T> as do-not-return. Assert that do-not-return methods actually don't return. 2019-08-18 03:18:24 -04:00
Zoltan Varga 7e76abb4ca [llvm] Add support for 32 byte AVX types, not enabled yet. 2019-08-18 02:01:10 -04:00
Zoltan Varga 1a9f9fb927 [llvm] Compute cpu features lazily. 2019-08-18 01:57:14 -04:00
Zoltan Varga 3c6e8de0da [jit] Enable alias analysis dce even if there are no ldaddr instructions eliminated, they could have been eliminated earlier. Enable only for netcore for now. 2019-08-18 01:55:25 -04:00
Zoltan Varga c0e03ef28f [llvm] Fix support for 32 byte simd values. 2019-08-18 01:40:28 -04:00
Stephen Toub c48dd0608d Fix IDE0025 (use expression body for properties)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 2e9134f1a6 Fix IDE0056 (use index operator)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 08b942ab68 Fix IDE0057 (use range operator)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 9595189a0a Fix IDE0054 (use compound assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub c8eb6972b5 Fix IDE0060 (unused parameter)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 64f4ba27f0 Fix IDE0059 (unnecessary assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 71fcdf53e3 Fix IDE0029 (use coalesce expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 828f89360f Fix IDE0031 (use null propagation)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 8af19427ad Fix IDE0028 (collection initializers)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 20adb69c73 Fix IDE0008 (use explicit type instead of var)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 8dd17900cc Fix IDE0065 (using directives should be outside of a namespace)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub c0191b2575 Fix IDE0019/20 (Use pattern matching)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Kenneth Pouncey 02126de318 Add regression test for issue https://github.com/mono/mono/issues/12917
- IL Linker not working correctly with IQueryable extensions
2019-08-17 17:50:39 -05:00
Jay Krell 9a24e9a389 Add comments explaining goto main_loop vs. MINT_IN_DISPATCH. 2019-08-17 15:45:21 -07:00
Zoltan Varga 9df02d7a2d Fix the build. 2019-08-17 17:22:41 -04:00
Stephen Toub 2e618f0e63 Fix IDE0066 across corefx (dotnet/corefx#40288)
* Fix IDE0066 across corefx

"Fix all", but then manual tweaking to accommodate places the rule doesn't work quite right.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-17 23:07:24 +02:00
Zoltan Varga 44cfea466f Fix the build. 2019-08-17 12:26:35 -04:00
Zoltan Varga 7418248835 Add support for the System.Runtime.Intrinsics.X86.Popcnt intrinsics. 2019-08-17 12:25:59 -04:00
Zoltan Varga bfe4155083 [llvm] Add support for querying the cpu attributes from the LLVM jit. 2019-08-17 12:25:58 -04:00
Zoltan Varga b39abe5365 Fix the build. 2019-08-17 12:25:31 -04:00
Zoltan Varga 9eb687f2b9 Fix the windows build. 2019-08-17 12:25:31 -04:00
Zoltan Varga 98a02059e8 [jit] Add a netcore version of simd-intrinsics.c.
Differences from the old versions:
- Only Vector<T> from corelib is supported.
- Only LLVM is supported.
- It uses more general opcodes like XBINOP instead of having one opcode
  for every op+type combination which is not useful for LLVM.
2019-08-17 12:25:30 -04:00
Zoltan Varga 2c32eb1fa8 [jit] Optimize away box+unbox combinations which are common in corefx code. 2019-08-17 12:25:30 -04:00
Zoltan Varga 13fb0134a0 Merge pull request #16210 from EgorBo/opt-llvm-jit
Auto-detect current cpu in LLVM JIT
2019-08-17 12:22:07 -04:00
Jay Krell 3fb7834afb Put finally data back in locals instead of frame to save 16 bytes of stack.
Partial undo of f8fd60b5c0.
2019-08-17 00:05:50 -07:00
Jay Krell 0c6cb9fbc3 thread names: coop, constants, usually ignore-error.
- Convert setting thread name to be coop-compatible.

- For constant thread names, just retain the constant, not a copy.
  This includes producing and using constant unicode thread names on Windows.

- Make setting a thread name often non-fatal, except where mandated by public API.

- From earlier PR: mono_free (mono-publib.c) would no longer be referenced and therefore
  no longer exported. That broke profilers and maybe other externals.
  Choices:
     1 Use it instead of g_free randomly sometimes.
     2 Call it randomly sometimes.
     3 include it in external-only.c or object.c
     4 Mark it external only (breaks profiler).
     5 Use a .def file or Unix equivalent.

     5 is best, 2 is done here, the advantages/disadvantages
     among most choices are subtle. 3 is probably better than 2
     but is slightly bigger change, to put off -- you'd stop
     compiling mono-publib.c and move it to include/extra_redist.

     The advantage of not-.def file is perhaps that it is exposed
     by the compiler, so maybe easy to port and work with.
     People are more comfortable with obscure C extensions than
     any linker options, and there are multiple linkers to contend
     with. Arguably there are fewer compilers.
2019-08-16 15:24:30 -07:00
Egor Bogatov 6847508d2a Update llvm-jit.cpp 2019-08-17 00:25:19 +03:00
Jay Krell 16a81cf520 [Coop] Convert mono_attach_load_agent. (#15954)
Based on https://github.com/mono/mono/pull/15868.
Most cleanup and leak fixes moved and expanded to https://github.com/mono/mono/pull/15950.
2019-08-16 17:16:03 -04:00
Ryan Lucia 1e457cdd1e [netcore] Enable netcore resolution logic for mono_assembly_byname_lookup (#16256)
* Use netcore_load_reference in mono_assembly_request_byname

* Add ALC static Resolve event code from CoreCLR

* Wire up the AssemblyResolve event

* Enable newly passing test

* Set requesting assembly in InternalLoad

* Remove infinite recursion

* Only call netcore_load_reference from mono_assembly_request_byname

* Remove test completely
2019-08-16 16:54:47 -04:00
Nathan Ricci 325bbf541b Changes GC.cs 2019-08-16 11:48:35 -04:00
Nathan Ricci 029467271e Implemented imprecise version of get total bytes allocated. 2019-08-16 11:26:39 -04:00
Jay Krell e54f03168d Ignore NULL in mono_marshal_free_dynamic_wrappers and mono_free_method. (#14150) 2019-08-16 09:33:04 -04:00
Jay Krell 66afab5803 Fill in more MONO_PATCH_INFO_NONE support. (#13990) 2019-08-16 09:32:48 -04:00
Jay Krell f52bfb94d5 [interp] Localize c, i32, o as suggested. (#16268)
This saves 16 stack bytes for clang, gcc is unmoved probably figured it out on its own.
2019-08-16 09:32:07 -04:00
Jay Krell 4d9e6ae20e [interp] Fix more MonoError reuse (mono_object_new_checked). (#16263)
* [interp] Fix more MonoError reuse (mono_object_new_checked).

* Inline mono_interp_new/mono_object_new_checked now that we are in outlined mono_interp_newobj.
2019-08-16 16:26:49 +03:00
Steve Pfister da2c0159e9 [corlib] Fix sharing mode validation rules (#16247)
Fixes https://github.com/mono/mono/issues/16032

When we switched to corefx File.Copy on unix, the behavior changed where it opened a source and destination stream in order to pass the safe handles to Interop.Sys.CopyFile. This exposed a flaw in our sharing validation rules in MonoIO.Open, which threw an exception in this scenario:

new FileStream("source.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite, 2048, FileOptions.DeleteOnClose);
new FileStream("source.txt", FileMode.Open, FileAccess.Read, FileShare.Read, 2048, FileOptions.None);

Simplified the FileShare check down to:

1. If the caller specifies FileShare.None, then it's an exclusive lock and no
   sharing is allowed.

2. If the caller specifies some other FileShare option, then FileShare.None
    cannot be specified.
2019-08-16 07:52:22 -04:00
Egor Bogatov 4dca4f6619 [netcore] Add patch-local-dotnet-aot-llvm rule (#16120)
* Add patch-local-dotnet-aot-llvm rule

* Update Makefile

* Add run-sample-local-dotnet-llvm
2019-08-16 12:43:08 +02:00
Jay Krell 0c637a73a9 intepreter: remove last element from various arrays (#16134)
add const
 combine lines for searchability
2019-08-16 11:37:20 +03:00
Jay Krell 35fc20b637 [interp] Fully outline enum_hasflag to save stack. (#16228)
Alternative to https://github.com/mono/mono/pull/16197.
If you are going outline, you sometimes might as well take it all with you
instead of bouncing back and forth for a partial inlining.
2019-08-16 10:13:22 +03:00
Zoltan Varga 5461fb78c7 Merge pull request #16261 from vargaz/tp-no-random
[runtime] Use a PRNG in the threadpool instead of the os random number generator.
2019-08-15 23:48:35 -04:00
Jo Shields b6c499631e Exclude Authenticode tests on Watch (#16278)
1002a4de2b added Authenticode cert support on non-WatchOS. The tests,
however, were not non-WatchOS-only. Fix this, so Watch doesn't try to
run Authenticode tests.
2019-08-15 22:07:18 -04:00
Zoltan Varga 764091677e Merge pull request #16257 from EgorBo/llvm-ptr
Align C# pointers with LLVM pointers for primitive types
2019-08-15 21:59:42 -04:00
Jay Krell 6c05cfab59 [mini] Cleanup alloca: call twice instead of once, and shrink allocation slightly. (#16158)
It is an actual function call at least on Windows.
2019-08-15 18:14:37 -04:00
Jay Krell b654aa1ca8 [interp] Fix more MonoError reuse. (#16262) 2019-08-16 00:06:50 +02:00
Jay Krell dfbf1a1bd3 [Coop] Convert emit_invoke_call. (#15964)
Removing calls to mono_string_new_checked.
Extracted from https://github.com/mono/mono/pull/15900.
2019-08-15 17:56:33 -04:00
Jay Krell 44ff0597b8 Wrap more external-only functions in GC mode transitions. (#15895)
mono_compile_method
mono_runtime_set_main_args
mono_runtime_run_main
mono_runtime_exec_managed_code
mono_load_remote_field
mono_load_remote_field_new
mono_store_remote_field
mono_store_remote_field_new
2019-08-15 17:48:18 -04:00
Zoltan Varga 870aec4aad Llvm optimizations (#16191)
* [jit] Disable inlining for ThrowHelper methods for non-netcore as well. Add an OP_NOT_REACHED after calls to these methods to let LLVM know they don't return.

* [llvm] Use the cold calling convention instead of the preserveall one, it has the same effect and its better supported.

* [llvm] Call the aot init function using the cold cconv.

* [llvm] Make the condition in the poll function unlikely.

* [llvm] Add a wrapper around the GC poll icall so it can be called with a cold calling convention.

* Move the noreturn check before the code which is shared with calli.
2019-08-15 21:46:00 +02:00
Stephen Toub 133bdb5fa2 Fix non-ASCII characters across corefx (dotnet/corefx#40318)
* Fix several non-visible characters in test files

* Replace Unicode surrogate pairs with escaped hex values

* Fix non-ASCII characters in most C# files

Signed-off-by: dotnet-bot <[email protected]>
2019-08-15 21:44:47 +02:00
Jo Shields 4ec4d7a3b3 Add missing FCONV_TO_U8 on ILP32 (WatchOS 5) (#16276) 2019-08-15 13:56:04 -04:00
Jay Krell 7b64f1cd00 Retain thread names as UTF8 instead of only UTF16. (#15945)
Windows can still have its Unicode thread names, since we have UTF16 initially, just don't retain it.

This does penalize ves_icall_System_Threading_Thread_GetName_internal (non-netcore only),
which has to convert back to UTF16 each time.

But seems like a net win overall.

There is also a little prep work here for constant thread names and some temporary measures as multiple changes head toward the same code.

Rename `mono_thread_set_name_internal` to `mono_thread_set_name`.
Close possible race conditions.

Extracted from https://github.com/mono/mono/pull/15859.
2019-08-15 19:37:34 +02:00
Jay Krell f35c755fea [interp] Outline newobj and newobj_vt. (#16267)
In this variation, the exception handling and resume are split between the main function and the helper function, and no address-taken occurs on ip/sp/vt_sp.

The order is a little tangled but convincing.
The helper function returns MonoException* or checks context->has_resume_state and then returns, and then the caller does the checks again.

Based on past analysis this will conserve stack, but that was not double checked here.
i.e. due to `&valuetype_this`, though it is conditional and the code could be split up into inline and outline.
2019-08-15 19:23:24 +02:00
Egor Bogatov 2f7a06d208 [netcore] Mark "Run Tests" step as failed if there are failures (#16230)
Currently "Run CoreFX tests" step is always green
2019-08-15 18:06:15 +02:00
Jo Shields dd6bd568a0 Fix failing OSX Android lane (#16274)
Linker order matters when using .a files, so zlib.a MUST come after
sgen.a if sgen.a was compiled with support for compressed ppdb
2019-08-15 10:37:04 -04:00
Vlad Brezae 6fc089bf8b [exceptions] Don't debug hang on ThreadAbortException (#16240)
It is pretty much useless.
2019-08-15 14:57:26 +03:00
Jay Krell 63e1828f7e Reduce redundant setting of thread name. (#16253)
Maintain a counter and only set if the counter has changed.
This cannot be fully thread safe, so is not.
Any thread can set any other thread name at any time.
You could be atomic to do better, and loop to do even better,
but the thread name could still be changed again right after the loop exists.

The previous code was also not reliable in this way, since again the name
could be changed right away.

#16248
2019-08-15 10:56:38 +02:00
Jay Krell 5ec452666a [interp] [wasm] p vs. o on newobj (#16264) 2019-08-15 10:42:11 +02:00
Jay Krell fd4956fbf2 [interp] Fix some, not all, of the error management. (#16232)
From #16190.
2019-08-15 10:41:49 +02:00
Jay Krell 7766f9b926 [interp] Short-circuit MINT_STELEM_REF on NULL. (#16229)
Presumably this is not a useful semantic change,
to skip those functions, given NULL.

(We will probably want to outline all of this also, later).
2019-08-15 10:39:45 +02:00
Jay Krell fc724e6cf0 [interp] Compute locals later to reduce register/stack pressure and conserve stack. (#16231)
i.e. do not def before a function call and use after, do both after.
2019-08-15 10:39:27 +02:00
Jay Krell dde3452690 [interp] Split leave into pieces, partially outlined, to conserve stack, (#16213)
but without changing how exception handling macros access labels and locals.

This saves 16 bytes of stack on Linux/amd64/gcc and likely similar everywhere,
as it takes an address-taken local out of interp_exec_method_full (stackval tmp_sp).

See
https://github.com/mono/mono/pull/16171
https://github.com/mono/mono/pull/16171/commits/a6a8f0d5ffa875da9cda63444e401c37810a690f for a
full outlining.
2019-08-15 10:39:03 +02:00
Nikolay Sivov 5b66d3d518 [cominterop] Fix copy-paste error in ComVisibleAttribute testing helper.
This type was introduced with 29a95bafd8.
2019-08-15 09:29:14 +02:00
Thays Grazia dcfec3966c Enable and document debug with sdb with netcore config. 2019-08-15 09:23:02 +02:00
Zoltan Varga 928aab1b84 [runtime] Use a PRNG in the threadpool instead of the os random number generator, its good enough, and its much faster. 2019-08-15 02:14:06 -04:00
Egor Bogatov 93eceec724 address feedback 2019-08-15 02:39:13 +03:00
Jay Krell 2463605f01 Remove ampersand from cfg->error per old agreement. (#16219) 2019-08-15 01:07:56 +02:00
Martin Baulig 934c4f4c87 X509CertificateImplBtls.PrivateKey should allow to actually set the key. (#16212)
The `X509CertificateImplBtls.PrivateKey` property setter was previously ignoring all values and always used `null`.  This should help with #16189.
2019-08-15 01:07:35 +02:00
Egor Bogatov ed4be03b79 handle primitive pointers in llvm 2019-08-15 01:38:58 +03:00
Vlad Brezae 4e793ad30c [interp] Fix icall signature (#16227) 2019-08-15 00:18:45 +03:00
Jo Shields f2d813c90d Fix linking of OS-provided zlib (#16177)
* Fix linking of OS-provided zlib

This is a reduced version of https://github.com/mono/mono/pull/16091
without the Z_PREFIX stuff

* Get rid of HAVE_ZLIB, be explicit about HAVE_SYS_ZLIB vs HAVE_STATIC_ZLIB

* Proposed fixes from Alex
2019-08-14 16:58:07 -04:00
Stephen Toub a103b78163 Delete dead _pauseTicks from TimerQueue (dotnet/coreclr#26160)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 21:19:04 +02:00
Jan Kotas e46b3efbd0 Undo readonly on _isLastFrameFromForeignExceptionStackTrace
This makes the field symmetric with other fields and fixes build break for CoreRT

Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 21:19:04 +02:00
Aleksey Kliger (λgeek) ce92c4376f [metadata] Create strongly typed versions of mono_assembly_request_prepare (#16094)
* [metadata] Create strongly typed versions of mono_assembly_request_prepare

Addresses the family of issues similar to Coverity #1443346.

This came up as part of the Coverity review, and Coverity notices that this is
writting beyond the end of the structure.  While this is correct, this requires
that the idiom be properly followed - that is, that the structure being passed
is of the given size.

Replaced instead with a method that eliminates the room for error with methods
that will initialize the whole structure, and the actual size computation is
done inside the method with the proper type sizes.

While I used methods, I feel that this should use macros.
2019-08-14 15:07:26 -04:00
Aleksey Kliger (λgeek) a6b8494f0a (mono.1) Update the MONO_THREADS_SUSPEND description (#16236) 2019-08-14 19:23:36 +02:00
lateralusX 2d06126dc9 OP_LLVM_OUTARG_VT doesn't always create temporary vt needed by some ABI's.
LLVM lowering passes vt as reference to managed methods. On platforms where
the value type ABI is passing an address to the value type, either in register
or on stack (Windows x64) we must make sure we always use a temporary of
the value type or we will be breaking the value type semantics.

In JIT lowering of OP_OUTARG_VT we always make sure we use a temporary
in cases where argument storage is set to ArgValuetypeAddrInIReg/ArgValuetypeAddrOnStack.
On LLVM lowering this was currently not the case, but it still worked in many scenarios.
This was because IL load of arguments before doing a call did a
local copy and then we pass that byref. However, there are cases where this
copy could get optimized away and when that happens, we will change the
local value type in the caller’s frame, breaking the value type semantics.

Added test case in winx64structs hits such a scenario that will generate IL
breaking the value type semantics before the fix:

...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %49, %winx64_vector3Struct* %3, %winx64_vector3Struct* %16)
...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %64, %winx64_vector3Struct* %3, %winx64_vector3Struct* %15)
...

The two add calls will use the same reference (to a local value type in caller frame)
on both calls, this will break the value type semantics since they should both
be temporaries, like second argument is.

After the fix, the same code will be lowered into:

...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %55, %winx64_vector3Struct* %19, %winx64_vector3Struct* %18)
...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %72, %winx64_vector3Struct* %16, %winx64_vector3Struct* %15)
...

Each argument gets its own temporary that, according to the ABI, gets passed
by address down to function.

There will be situations where we could end up with two temporaries, one from
the load and one from the lowering of OP_LLVM_OUTARG_VT (same as we get on JIT),
this will however be detected by LLVM optimization pass, removing one of the copies.
2019-08-14 19:23:32 +02:00
Egor Bogatov abb43f9361 ignore stacktrace tests 2019-08-14 14:33:44 +02:00
Jay Krell d1c0974e46 [interp] Reduce frame size by partially outlining load remote field. (#16196)
Extracted from https://github.com/mono/mono/pull/16171.
Ultimately over 100 bytes can be saved.

Tactically not outlined or indirected:
 vt_sp change
 ip advancement
 error handling

 Also partially fixes the error reuse from https://github.com/mono/mono/pull/16190.
2019-08-14 12:34:54 +02:00
Jay Krell 60ac98ee29 Make small_id_mutex not recursive. (#15485)
(Later non-recursive Windows mutexes should be SRWLOCK and only statically initialized.)
2019-08-14 12:33:58 +02:00
Jay Krell 8c5d0a790e Use ERROR_DECL. (#16220) 2019-08-14 12:28:00 +02:00
Jay Krell 74b0d01e07 [interp] Remove address-taking of vt_sp. (#16215)
Theoretically could help reduce frame size but it has not been seen to.
2019-08-14 12:27:22 +02:00
Zoltan Varga 4c71a3e892 [wasm] Disable running xunit test suites on wasm, its not working reliably yet. (#16223) 2019-08-14 12:26:31 +02:00
Jay Krell 4c17442dc3 [interp] Outline mono_interp_calli_nat_dynamic_pinvoke to reduce stack. (#16201)
Now that we have that function, also change from malloc to alloca.

This is part of #16171 and ultimately over 100 bytes were saved.
2019-08-14 12:25:44 +02:00
Jay Krell 136205ab45 Completely replace mono_error_ok with is_ok and make first external_only. (#16217) 2019-08-14 12:24:50 +02:00
Jay Krell 12de10007c Skip degenerate qsort: num < 2, size == 0, base == 0 (#16016)
Fixes https://github.com/mono/mono/issues/15994

Some is redundant for some qsort implementations.
2019-08-14 12:22:40 +02:00
Jay Krell 6873d5370d Fix warning LNK4221 warning in assembly-load-context.obj (#16004)
LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
2019-08-14 12:19:08 +02:00
Jay Krell 9123ac0197 [interp] Outline get_virtual_method_fast to save 16 bytes of stack. (#16198)
Ultimately over 100 bytes are savable in a few steps.
2019-08-14 12:17:23 +02:00
Jay Krell dbf13301c8 [interp] Outline set_resume_state to conserve stack in main interpreter loop (#16200)
This is not intuitive but I definitely saw it.
2019-08-14 12:17:01 +02:00
Marek Safar 9ee80c6d3b Bump linker 2019-08-14 10:17:54 +02:00
Stephen Toub 3803a00421 Fix StyleCop warnings for rules enabled in corefx
Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 10:15:38 +02:00
Thays Grazia f812d90fa1 [debugger] Debugger tries to execute a single step when the thread is already terminated (#16208)
* Debugger tries to execute a single step when the thread is already terminated.

* Fixing how to get thread tls.
2019-08-13 22:47:28 -03:00
Stephen Toub 50d3fc69d8 Convert several static readonly byte[]s to ReadOnlySpan<byte> (#26138)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 9e13a7e097 Fix IDE1005 (simplify delegate invocation)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub db4441bfe7 Fix IDE0062 (make local functions static)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 15ea431db7 Contribute to IDE0060 (unused argument)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub d927c98e03 Contribute to IDE0059 (unnecessary assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub d82bc555c9 Contribute to IDE0019 (use pattern matching) and IDE0066 (use switch expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub e19d95b6de Contribute to IDE0044 (make field readonly)
I used the auto-fix and then undid changes in areas where the fields were potentially mutated in native, e.g. I undid many of the changes in reflection.  We can revisit those individually later.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 6c95cbadf9 Fix StyleCop warning SA1400 (missing access modifiers)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 0814b9de4d Enable StyleCop rules 1205, 1411, 1026, and 1000 (dotnet/corefx#40254)
* Fix StyleCop warning SA1411 (attribute ctor shouldn't use unnecessary parens)

* Fix StyleCop warning SA1026 (new[] spacing)

* Fix StyleCop warning SA1000 (consistent spacing around if, for, switch, etc.)

* Fix StyleCop warning SA1205 (partial elements should declare an access modifier)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Egor Bogatov 6c51440538 auto-detect cpu in llvm jit 2019-08-13 22:24:54 +03:00
Aleksey Kliger (λgeek) 592de53a1f [runtime] Initialize thread suspend policy at startup, inline getter (#16149)
* [runtime] Initialize thread suspend policy at startup, inline getter

On some benchmarks we saw mono_threads_suspend_policy() using a lot of CPU
time.

Rather than doing an init-on-first-use, initialize the policy once upfront when
the JIT it starting up and make the getter method static inline.

* move mono_threads_suspend_policy_init into mono_thread_info_init

* [mini] Initialize suspend policy for `mono --version`

Need to initialize the suspend policy before printing its name.  Otherwise the
suspend policy is unset since no other runtime initialization is done for `--version`.
2019-08-13 15:16:05 -04:00
Alexander Köplinger ff77b427f4 [netcore] Cleanup System.Private.CoreLib.csproj (#16199)
- The eng/Versions.props was already imported by the arcade targets causing a double-import warning:

> warning MSB4011: "/Users/alexander/dev/mono/eng/Versions.props" cannot be imported again. It was already imported at "/Users/alexander/.nuget/packages/microsoft.dotnet.arcade.sdk/1.0.0-beta.19379.1/tools/DefaultVersions.props (17,3)". This is most likely a build authoring error. This subsequent import will be ignored.

- Remove BaseAddress and TRACE from the project file, this was done upstream at https://github.com/dotnet/coreclr/commit/bc430cdd2010bf7790854a60380d00c506c346a6

- Remove unecessary DebugSymbols/DebugType settings, we already set them in the global PropertyGroup
2019-08-13 16:41:03 +02:00
Aleksey Kliger (λgeek) 1f4ac5d767 [sre] Outline typebuilder_setup_fields loop body (#16184)
The loop body uses coop handles, so wrap it in HANDLE_FUNCTION_ENTER/HANDLE_FUNCTION_RETURN
2019-08-13 09:31:31 -04:00
Jay Krell ef199425a2 [interp] Replace mono_error_ok with is_ok. (#16193)
The locally unknown function, despite being small/fast, forces saving values in stack and nonvolatile registers
around calling it.
Both nonvolatile registers and stack are at a premium here, whereas usually stack is cheap.
The macro is just a read from offset 0 and checking for zero.
2019-08-13 15:30:28 +02:00
Alexander Köplinger b60aaea791 [pkg] Bump libgdiplus to 6.0.2 2019-08-13 12:12:22 +02:00
Egor Bogatov c36f0a126e [netcore] decimal should have alignment=8 (#15666)
* Fix System.Runtime.InteropServices tests

* Update marshal.c

* Update marshal.c

* Update marshal.c
2019-08-12 21:57:47 +02:00
Zoltan Varga 5bc4944bf0 Merge pull request #16151 from spouliot/ios-intr-enable-sre
[sdk] Enable the SRE runtime code inside iOS builds
2019-08-12 14:44:54 -04:00
Stephen Toub b25395bde3 Re-enable StyleCop warning SA1400 (elements should specify access) (#40240)
One of the more common things we "nit" in PRs.  Now the compiler will flag it for us.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Aaron Robinson 66f7436d6f Remove unnecessary error check for QueryUnbiasedInterruptTime call (#26120)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Stephen Toub 4f821c3e8d Enable FxCop and StyleCop analyzers (dotnet/coreclr#26100)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Aleksey Kliger (λgeek) ad64c781eb [interp] Fix stub_get_resume_state write to out parameter (#16145)
Addresses Coverity CID 1452245
2019-08-12 13:09:59 -04:00
Ryan Lucia cadd114f64 [netcore] Pass MONO_ASMCTX_LOADFROM on LoadFrom calls (#16075) 2019-08-12 11:46:12 -04:00
Ankit Jain c028454bef [msbuild] Bump to track xplat-master and update roslyn (#16155)
Prompted by mono/msbuild#126:

This includes:

- merge mono-2019-06
- SDK update to track dotnet release/3.0.100-preview8
- Roslyn updated to 3.3.0-beta2-19381-14
2019-08-12 17:03:13 +02:00
UnknownShadow200 69ebdd8b93 [WinForms] Fix NotifyIcon not showing in systray on 64bit linux (#16164)
https://bugzilla.xamarin.com/show_bug.cgi?id=14976

Credit goes to datnhanlz for noting that removing XChangeProperty(DisplayHandle, hwnd.whole_window, _XEMBED_INFO, _XEMBED_INFO, 32, PropertyMode.Replace, atoms, 2); would also fix the bug. 

Note that the type of Atom is actually unsigned long (see usr/include/X11/X.h), however the code used a C# int for the atoms. As such the code worked without a problem on 32 bit linux, but broke on 64 bit linux.

I switched to using IntPtr instead to make the code also work on 64-bit.
2019-08-12 16:09:08 +02:00
Aleksey Kliger (λgeek) dd3f6e0d31 [debugger] Cleanup MonoError on in isFixedSizeArray (#16146)
Cleanup MonoError on error paths.  Also change `return FALSE` to `return 0`.

Addresses Coverity CID 1452205
2019-08-12 09:50:58 -04:00
Nikolay Sivov 612e0644b6 [mono] Fix some compiler warnings (#16159)
* [mini] Remove unused variables.

* [mini] Remove unused function.

Single call site is conditionally compiled.

* [sgen] Remove unused variable.

* [utils] Fix misleading indentation warnings.
2019-08-12 12:42:59 +02:00
Nikolay Sivov f9052ed78e [System.Drawing] Use CoreFX implementation of BufferedGraphics on Windows (#16163)
The issue was made visible with .NET Core WinForms. Some controls draw text with DrawTextEx to intermediate Graphics based on Bitmap, that later it's moved back to main Graphics with DrawImage. This results in a lost of transparency component along the way, making antialiased glyph bitmaps draw and B/W. Corruption is visible with gray antialiasing, but is more severe in subpixel modes.

I verified that same happens with gdiplus on Windows, code from CoreFX works correctly.
2019-08-12 12:42:07 +02:00
Jay Krell c70b4cc3cc [remoting+interp] [remoting+debugger] Fix use of dangling local. (#16173) 2019-08-12 12:35:30 +02:00
Jay Krell 4e5544f02d [interp] Remove unnecessary loops, that optimizer might handle. (#16169) 2019-08-12 12:34:46 +02:00
Vincent Povirk 7413bb84ec [w32process-win32] Don't duplicate CreateProcess search logic. (#16156)
When given a null application name, CreateProcess already
searches the path and appends .exe in some cases. .NET's
Process class behaves in the same way (see
https://github.com/mono/mono/pull/16089#issuecomment-519984244).

The duplicated logic is unnecessary and broken in several ways.
Failures are ignored and the command line arguments are
executed. Relative paths are expected not to end in "exe".
Many corner cases work differently from .NET and CreateProcess.

This change makes it impossible for process_complete_path to fail,
so I removed the gboolean return and pulled out the output
parameter. Its caller wasn't handling the return value correctly
anyway.
2019-08-12 12:05:05 +02:00
Zoltan Varga 835282004c Merge pull request #16136 from EgorBo/fix-inlining
[netcore] Fix inlining
2019-08-11 22:32:01 -04:00
Zoltan Varga 7c511c9937 Merge pull request #16160 from vargaz/wasm-xunit-4
[wasm] Xunit testing work.
2019-08-10 22:56:05 -04:00
Alexander Köplinger ec73f9ba17 Bump corefx to fix build break 2019-08-10 23:45:11 +02:00
Zoltan Varga c6054d5aa1 [wasm] Run System.Core xunit tests on CI. 2019-08-10 15:18:26 -04:00
Zoltan Varga 95b705d8c1 [wasm] Add argument conversions to the xunit runner. 2019-08-10 15:18:26 -04:00
Alexander Köplinger 738b6988b5 Bump corefx to get https://github.com/mono/corefx/pull/327 2019-08-10 16:19:01 +02:00
Nikita Voronchev 000430002a [System.Drawing.Design] Allow context be a null in the ColorEditor.EditValue (#16141)
If one call `ColorEditor.EditValue(IServiceProvider, Object)` ([MSDN](https://docs.microsoft.com/ru-ru/dotnet/api/system.drawing.design.uitypeeditor.editvalue?view=netframework-4.8#System_Drawing_Design_UITypeEditor_EditValue_System_IServiceProvider_System_Object_)), then `context` is `null` and drop-down menu doesn't appear.
2019-08-10 16:01:37 +02:00
Sebastien Pouliot 360c1c44c7 [sdk] Enable the SRE runtime code inside iOS builds
This will allow the interpreter to use System.Reflection.Emit.

Note: There is a small size increase (about 50kb) for an application
built in release mode with the LLVM backend (even if the interpreter
is now used).
2019-08-09 13:57:54 -04:00
Vlad Brezae b52830cf11 [interp] Fix overflow of execution stack (#16115)
We were not tracking max_stack_height for a few opcodes.
2019-08-09 20:28:30 +03:00
Zoltan Varga 54c07051a8 Merge pull request #16139 from steveisok/run-single-ios-sdk-test
Added the option to run a single test in the iOS SDK
2019-08-09 12:58:01 -04:00
Ryan Lucia a1bb37cd52 [netcore] Make mono_assembly_request_prepare take an ALC (#16061)
* Make mono_assembly_request_prepare take an ALC

* Typo

* Fix pedump

* Use default ALC

* Make v1 preload hook external-only

* Convert pedump and monodis

* fixup: use default ALC in netcore preload hook

trusted platform assemblies should be loaded into the default ALC
2019-08-09 11:44:50 -04:00
Alexander Köplinger 0970996f75 Bump eng/Versions.props again
Try to fix the netcore build.
2019-08-09 17:03:12 +02:00
Alexander Köplinger 75c21692a0 Add mono-hang-watchdog.in to tarball
Remove custom distdir target and use standard EXTRA_DIST to do so.
2019-08-09 16:48:16 +02:00
Stephen Toub 86a0ad2f88 Fix FxCop warning CA1823 (unused fields)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub f39ac1d32d Fix FxCop warning CA2229 (private deserialization ctor)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 2d3a6ea2bb Fix FxCop warning CA1825 (Array.Empty)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub cce1583769 Fix StyleCop warning SA1620 (generic XML comment ordering)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 15963fe836 Fix StyleCop warning SA1617 (void return XML comment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 4ce53c845b Fix StyleCop warning SA1113 (comma position for parameters)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 442e9ad144 Fix StyleCop warning SA1206 (modifer ordering)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 01fa50943f Fix StyleCop warning SA1121 (use built-in types)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 9c3037fc59 Fix StyleCop warning SA1200 (cref prefixes)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 1fcf7a1ab4 Fix StyleCop warnings SA1028 and SA1518 (trailing whitespace and blank lines)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 441257339f Fix FxCop warning CA1018 (attributes should have AttributeUsage)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Alexander Köplinger 74338e0f2d Bump corefx to get https://github.com/mono/corefx/pull/323 2019-08-09 16:21:12 +02:00
Alexander Köplinger 09e7c4509c Update dependencies from corefx (#16137) 2019-08-09 15:19:16 +02:00
Steve Pfister 9e9697c978 Added the option to run a single test in the iOS SDK 2019-08-09 09:10:13 -04:00
Alexander Köplinger f15fe47075 [build] Remove ILDISASM variable from platform Makefiles
It was written by configure.ac to config.make and is unused anyway.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 652f6c64b6 [build] Remove PLATFORM_CORLIB from platform Makefiles
Changing the corlib assembly name isn't supported anywhere else in the build.
2019-08-09 12:53:14 +02:00
Alexander Köplinger e80f3449da [build] Remove MBAS from the platform Makefiles
It no longer exists.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 8697b31082 [build] Remove EXTERNAL_MCS setting in platform Makefiles
Setting an explicit mcs is no longer supported by the rest of the build scripts.
2019-08-09 12:53:14 +02:00
Alexander Köplinger d7174c9aa6 [build] Remove confusing commented out code 2019-08-09 12:53:14 +02:00
Alexander Köplinger ab1d61509d [build] Remove PLATFORM_MAKE_CORLIB_CMP
The corlib_cmp doesn't exist anymore.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 3e77e01071 [build] Replace PLATFORM_ ifdefs with HOST_PLATFORM checks
Building Linux BCL assemblies from a macOS machine didn't work since we
were enabling some Apple features if the build machine was macOS.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 175e345122 [tools] Add .gitignore for mono-hang-watchdog 2019-08-09 12:39:34 +02:00
Stephen Toub 67a89b3a6f Move CodeAnalysis.ruleset to shared (dotnet/corefx#40153)
So that the same settings can be applied to corelib.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 12:30:05 +02:00
Egor Bogatov a7e428d82f fix inlining 2019-08-09 12:55:50 +03:00
Egor Bogatov 7deb53514d Make System.Runtime.Intrinsics.X86.*.IsSupported intrinsic 2019-08-09 09:34:29 +02:00
Egor Bogatov 9442ed95d7 fix minor issues 2019-08-09 09:34:29 +02:00
Alexis Christoforides 2c4b60fe24 [merp] Use a separate program as the hang supervisor. (#15715)
* [merp] Use a separate program as the hang supervisor.

Fixes https://github.com/mono/mono/issues/15646

macOS does not like signals being sent from the forked supervisor process, possibly towards anywhere but definitely when sent to the parent process. The following message is currently spewed after the supervisor process attempting to send a SIGSEGV to a hanged Mono process:
"The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec()."

We follow that direction and introduce a new binary that, when available in the mono executable's binary directory, is used to abort the parent process for us.
2019-08-09 10:31:57 +03:00
Ryan Lucia 6a459c25d7 [netcore] Don't return REFERENCE_MISSING in netcore_load_reference 2019-08-09 09:23:10 +02:00
Jeffrey Zhao a49ecebd4e Catch InvalidCastException properly in Collection (dotnet/coreclr#25953)
The explicit implementations of IList.Add/Insert/Set methods catch
InvalidCastException for casting input argument to type T to throw
an ArgumentException. The issue here is they put the core collection
modification operation in the try block, which could be overridden
by sub-classes and produce an InvalidCastException from user logic,
even the argument passed to IList.Add/Insert/Set is valid.

To fix the issue, we just need to move the collection modification
outside the try block, and leave the casting in try block only. In
this way, we'll get an ArgumentException when the argument type is
invalid, but propagate the InvalidCastException thrown from user
code as it should be.

dotnet/corefx#39919

Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 09:22:28 +02:00
Zoltan Varga e71604d695 Merge pull request #16125 from vargaz/wasm-keep-config-cache
[wasm] Add a KEEP_CONFIG_CACHE option to Make.config to avoid deleting the autoconf cache file on make clean.
2019-08-09 03:20:03 -04:00
monojenkins 1369ebcfaf [bcl] Update BCL Linked Size 2019-08-09 09:19:32 +02:00
Alexander Köplinger fde0a5b628 [ci] Trigger PR builds for netcore only on master branch
The work is happening in master only for now.
2019-08-09 00:53:19 +02:00
Alexander Köplinger 6075417d88 Bump corefx so it points to correct branch 2019-08-09 00:14:18 +02:00
Alexander Köplinger ec35e14636 [tests] Fix accessing field of MarshalByRefObject in remoting1.cs (#16119)
We need to copy it to a local variable first.
This was exposed by a new Roslyn optimization for String.Concat (see https://github.com/dotnet/roslyn/issues/37830).
2019-08-09 00:07:41 +02:00
Stephen Toub da2c1cde97 Re-enable StyleCop warnings SA1028 and SA1518 (trailing whitespace and blank lines) (#40113)
Mechanical/automated change.  The only manual changes were a) removing the two rules from the CodeAnalysis.ruleset file, b) removing spaces from one line of a .targets file that is used to generate a .cs file, and c) reverting the changes in a few test files where the naive replacement happened in the middle of a verbatim string where the whitespace actually mattered.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 23:44:46 +02:00
Alexis Christoforides 1127974861 [System] Guard against attempting to creating out-of-range spans in macOS FileSystemWatcher 2019-08-09 00:24:06 +03:00
Zoltan Varga adeef6c0fc [wasm] Add a KEEP_CONFIG_CACHE option to Make.config to avoid deleting the autoconf cache file on make clean. 2019-08-08 17:08:51 -04:00
Aleksey Kliger (λgeek) 36af44e3be [perfcounter] Define mono_perfcounter_foreach even if perfcounters are disabled. (#16100)
Fixes netcore build on win32
2019-08-08 14:06:21 -04:00
monojenkins 18917b885c [csproj] Update project files 2019-08-08 19:10:03 +02:00
Marek Safar 2ef50f54ee Bump corefx for #15259 fix 2019-08-08 19:10:03 +02:00
Adam Sitnik b15518dd5e make DateTime.UtcNow 5% faster to minimize the leap second performance impact (#26046)
* make few methods used by DateTime.UtcNow inlinable to minimize the leap second performance regression impact, related to #25728

* apply suggested micro-optimizations

Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 18:55:19 +02:00
Jan Kotas 7c59cd14fe Cleanup ProjectN ifdefs (dotnet/corert#7675)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 18:55:19 +02:00
Alexander Köplinger dfa5a6ce4d [aprofutil] Install the tool correctly (#16112)
It needed a few tweaks to be usable when Mono is installed to the prefix.
2019-08-08 16:46:55 +02:00
Steve Pfister 64ea10fbcd Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements. (#16054)
https://github.com/mono/mono/issues/16010

The current behavior for GetConstructorCandidates when the name is null, "",
or "*" is to return the set of constructors for the type. For "", .NET Core and
.NET Framework return 0 elements.  This PR adds a check for "" to match that
behavior.
2019-08-08 12:14:39 +02:00
Stephen Toub c189afd4f5 Delete PROJECTN ifdefs from Corelib (dotnet/coreclr#26058)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 10:10:47 +02:00
Sung Yoon Whang 240202cd90 Improve ArgumentNullExceptions on DiagnosticCounter.DisplayName/Unit (#26035)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 10:10:47 +02:00
monojenkins 53b9c88795 [bcl] Update BCL Linked Size 2019-08-08 10:09:17 +02:00
imhameed fc4d234ef6 [sdb][interp] Use the interpreter's resume state to compute the IL offset in compute_frame_info. (#15936)
[sdb][interp] Use the interpreter's resume state to compute the IL offset in compute_frame_info.

This recovers the relevant bits of the "resume state" from the interpreter and uses that to recalculate the `il_offset` for the top frame.

Fixes https://github.com/mono/mono/issues/15687.
2019-08-07 20:05:55 -04:00
imhameed 4462f10fb6 [llvm] Explicitly check for errors when calling LegacyIRCompileLayer::addModule. (#16082)
[llvm] Explicitly check for errors when calling LegacyIRCompileLayer::addModule.

LLVM built with LLVM_ENABLE_ASSERTIONS=On will crash if errors are not
explicitly consumed.
2019-08-07 15:15:55 -04:00
Ryan Lucia aece3efc99 Relax locking (#15986)
* Remove usage of loader lock

Originally added in https://github.com/mono/mono/commit/c4cbd3b700eea52c84c68c9450d80357b3be681d but no longer relevant

* Make get_agent_domain_info lock-free and relax loader lock

* Syntax errors

* Undo first commit, other than the commit message

* Fixes
2019-08-07 14:17:19 -04:00
Marek Safar b3344fa0b2 Bump linker (#16014)
* Bump linker

* Bump api-doc-tools and add NEW_CECIL define to mdoc

* Bump linker

* Bump API snapshot submodule

* [csproj] Update project files
2019-08-07 19:37:19 +02:00
Sylvain 4780486237 Memcpy instead of memcpy1 in System.Buffer.MemoryCopy (void*, void*, long, long). (#16056)
In the same way it is done in System.Buffer.MemoryCopy (void*, void*, ulong, ulong).

Correct what appears to be a typo in System.MemoryCopy with "long, long" size arguments.

Consequently, copy is not done byte per byte for the last int.MaxValue bytes when MemoryCopy is called with long argument (most common case). This should provide noticeable performance increase.
2019-08-07 18:36:47 +02:00
Radek Doulik 1f2b37ed37 [aotprof-tool] Initial import of AOT profiler tool (#15384)
* [aotprof-tool] Initial import

Imported https://github.com/radekdoulik/aotprofile-tool and updated it
to use Mono.Profiler.Log

* [aotprof-tool] Added man page

* [aotprof-tool] Use the shortened name in README.md
2019-08-07 18:24:51 +02:00
Ryan Lucia dcd5715161 [netcore] Re-enable IsAssignableFrom test
This was fixed a while back but never removed from the exclusions
2019-08-07 17:27:35 +02:00
Zoltan Varga d18daefdbc Merge pull request #16085 from kjpou1/wasm-sdk-v0.2.0
[wasm][sdk] Bump SDK Version 0.2.0
2019-08-07 11:17:12 -04:00
Jay Krell e8d28feb1e There is no need to zero static data, and it is racy (#15699)
if there are multiple threads calling (unlikely).
This zeroing was made moot by 470354fa81 January 2019.
2019-08-07 10:40:57 -04:00
Thays Grazia e690fb0240 Allow generating DIM bitcode (#16070) 2019-08-07 11:06:56 -03:00
Kenneth Pouncey 4767c581c7 Update sdk version for tests 2019-08-07 15:57:45 +02:00
Alexander Köplinger 521d4c8395 [utils] Move MAP_ANONYMOUS check after sys/mman.h import
The flag is defined in that header on OSX. Followup to 2abe569ede
2019-08-07 15:54:09 +02:00
Aleksey Kliger (λgeek) aea457cdb3 [w32socket] Translate ELOOP and ENAMETOOLONG (#16039)
Translate to WSAELOOP and WSAENAMETOOLONG, respectively.

See https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2

Addresses part of https://github.com/mono/mono/issues/16024
2019-08-07 09:42:39 -04:00
Nikolay Sivov 10c5e6407e [trace] Improve enter/leave output formatting (#16036)
* [trace] Use specific integer types for return value traces.

* [trace] Improve argument list formatting for ENTER lines.

Return value changes are especially visible for bools, where currently 4 bytes integers are printed.
2019-08-07 15:05:44 +02:00
Kenneth Pouncey b0611a5ee9 Bump SDK Version 0.2.0 2019-08-07 14:59:53 +02:00
Nikolay Sivov d34381f72e [win32] Fix some compilation warnings (#15956)
* [mini] Remove unused variable.

* [mini] Mark win32 exception handling functions static.

* [mini] Remove unused variable.

* [mini] Suppress unused function warning.

* [metadata] Fix unused function warning for Win32 build.
2019-08-07 14:45:19 +02:00
iamphi 2abe569ede Fix compilation on OSX 10.9 (#14411)
Found solution from: 
https://github.com/eclipse/omr/issues/902#issuecomment-288559044
2019-08-07 14:42:57 +02:00
Alex Thibodeau 24bbb94b6b Swap bytes for tcp port number because windows API gives us port in network order. (#16030)
Addresses the FIXME in Win32IPGlobalProperties.cs :: FillTcpTable

Fixed Unity issue: issuetracker.unity3d.com/issues/ipglobalproperties-dot-getactivetcpconnections-return-different-port-numbers-when-compared-to-cli-netstat-a-output
2019-08-07 14:30:07 +02:00
Zoltan Varga ba1c949a24 [runtime] Fix an autoconf warning. (#16078)
[runtime] Fix an autoconf warning.

<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-08-07 06:06:00 -04:00
Vlad Brezae e702e8286f [interp] Fix DEBUG_INTERP build (#16057) 2019-08-07 12:47:41 +03:00
Marek Safar 25b84915e1 Remove code moved to shared partition 2019-08-07 09:48:03 +02:00
Stephen Toub c5027781de Remove some unused locals (dotnet/corefx#39956)
* Remove some unused locals

Trying out various analyzers, including one to flag unused locals.  There are more than this, but this is a first batch.  Some of these are just clean-up.  Others are saving unnecessary cost.  At least one appears to be a real functional bug.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Stephen Toub 824351f49c Fix CounterGroup to use built-in types
Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Aleksey Kliger (λgeek) b2ab9198d3 Move AssemblyLoadContext VM-invoked resolve methods to shared (#25989)
Share between CoreCLR and Mono

Mono began using a copy of these methods with
https://github.com/mono/mono/pull/15946

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Tanner Gooding 0378116ffc Updating Math.Round and MathF.Round to be IEEE compliant so that the intrinsic and managed form are deterministic. (#25901)
* Updating Math.Round and MathF.Round to be IEEE compliant so that the intrinsic and managed form are deterministic.

* Fixing the Math.Round and MathF.Round handling for values greater than 0.5 and less than 1.0

* Applying formatting patch.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
John SalemandNoah Falk a76f6fa9b5 Allow for interface implementations in EventSource.WriteEventVarArgs (#25844)
* Allow for interface implementations in EventSource.WriteEventVarArgs

* Also account null ref types and nullable types

* fix error in comment and simplify first part of boolean logic

* Update src/System.Private.CoreLib/shared/System/Diagnostics/Tracing/EventSource.cs

Co-Authored-By: Noah Falk <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Ryan Lucia 022fc4ea37 [netcore] Set flag for processor arch when parsing assembly names 2019-08-07 09:22:04 +02:00
Bernhard Urban fe0f311a84 [arm] respect I1 and I2 on gsharedvt out transition (#16000)
[arm] respect I1 and I2 on gsharedvt out transition

In order to trigger the problem, we must call into gsharedvt code ("in" transition) and then escape the gsharedvt world ("out" transition). The bug happens in the latter. Also some other stars need to align (I haven't fully understood it), thus the repro is quite long.

The provided repro only triggers with LLVM, which we don't have enabled on CI yet: https://github.com/mono/mono/issues/15999

Fixes https://github.com/mono/mono/issues/15261
Fixes https://github.com/mono/mono/issues/15262
Fixes https://github.com/mono/mono/issues/15263
Fixes https://github.com/mono/mono/issues/15307
2019-08-06 18:25:51 -04:00
Zoltan Varga add830f9cb Merge pull request #16071 from vargaz/wasm-xunit-3
[wasm] More xunit fixes.
2019-08-06 17:57:22 -04:00
Zoltan Varga 476af240d2 [wasm] Compile interp.c with -mllvm -join-liveintervals=false to fix compilation times. 2019-08-06 16:46:19 -05:00
Jay Krell 9bf703b031 [Coop] Convert mono_object_isinst_icall. (#15963) 2019-08-06 15:26:55 -04:00
Zoltan Varga 59a285a546 [wasm] More xunit fixes. 2019-08-06 15:24:32 -04:00
coypoop 5a4283d1b3 Update NetBSD support (#15938)
Based on the FreeBSD variation.
Needs some additional changes I'm still ironing out.
2019-08-06 20:32:38 +02:00
Ryan Lucia 64bf7ef9db [netcore] Update test exclusions to reflect ALC work (#16059) 2019-08-06 14:08:18 -04:00
Zoltan Varga 3c88e9f251 [netcore] Avoid calling load hooks during AOT.
Fixes https://github.com/mono/mono/issues/16046.
2019-08-06 19:40:23 +02:00
Zoltan Varga 662b72f913 Merge pull request #16058 from vargaz/wasm-requests-2
[wasm] Update WASM-REQUESTS.md.
2019-08-06 12:53:44 -04:00
Zoltan Varga dda05d0a09 [wasm] Update WASM-REQUESTS.md. 2019-08-06 10:48:42 -04:00
Egor Bogatov 0fbc5d8315 DeflateStream: check IsInvalid before disposing (#16048) 2019-08-06 14:53:24 +02:00
Ryan Lucia 85296ef1be [netcore] Make the load hook ALC-aware (#16012)
The meat of this PR is what the title suggests, and with this commit all the basic ALC functionality should be working properly. Probably worth checking if this makes any new tests pass.

Additionally, this PR includes a second commit that switches over the search hook to use loaded_assemblies from the ALCs. This may break some existing tests due to remaining unimplemented ALC behavior, notably the native loading mechanism needed for Openssl in some of the cryptography tests. My inclination is to include the commit regardless and just disable the failing tests so that any future ALC work can watch for regressions in the loader, but I would appreciate input from steveisok on how important those tests are. I've not disabled anything yet in order to see exactly what new failures are introduced.
2019-08-06 12:16:15 +02:00
Jay Krell 474092add6 UWP now allows LoadLibrary. (#16015) 2019-08-06 12:14:38 +02:00
Martin Baulig 1c13b30225 SocketAsyncEventArgs cleanups and preparations for the real thing. (#16001)
Cleaning up the `SocketAsyncEventArgs` class and preparing it for it's eventual
adaption from CoreFX.

There are a few instance fields and property accessors with `internal` visibility,
which the CoreFX version does not have.  Replaced these with accessor methods.

The `PolicyRestricted` property and it's related `SocketAsyncEventArgs (bool)`
constructor is not used anywhere and the CoreFX version has a constructor with the
same signature, but different semantics.

In the CoreFX version, `Complete ()` does not call `DoComplete (this)` - renamed it
into `Complete_internal ()` to avoid any ambiguities and help the transitioning.
2019-08-06 12:13:40 +02:00
Alexander Köplinger 7ab3811b93 Bump api-doc-tools to latest master (#16035)
* Bump api-doc-tools to latest master

Disable building mdoc.exe in mcs mode since it no longer builds because of new C# features.

* Fix mcs build by disabling more mdoc invocations

* Bump monodroid_tools to compile against v4.7 reference assemblies

mdoc needs ValueTuple which is not in v4.6.

* [csproj] Update project files
2019-08-06 10:40:24 +02:00
Zoltan Varga d57f3f55f0 Merge pull request #16043 from kjpou1/wasm-sdk-small-fix
[wasm][sdk] Fix misspelling and trailing comma
2019-08-06 02:29:20 -04:00
Kenneth Pouncey ca51e5b5cc Fix misspelling and trailing comma 2019-08-06 07:30:47 +02:00
Zoltan Varga 9b4bc613e4 Merge pull request #16006 from vargaz/as-gsharedvt
[jit] Avoid asserting on Unsafe.As<T> in gsharedvt methods.
2019-08-05 21:04:44 -04:00
Kenneth Pouncey 69aa5dcaa7 Fix indentation 2019-08-05 15:03:08 -05:00
Kenneth Pouncey 357a3aaec5 [wasm][sdk] Fix issue with WasmLinkAssemblies task wrongly using reference assemblies
The issue is that when executing the `WasmLinkAssemblies` task the reference assemblies are currently used as input to the linker step and output in the published application instead of the actual framework library file.

My findings so far on this is that the package references are resolved in the MSBuild [ResolvePackageAssets](https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/ResolvePackageAssets.cs#L205.)

Those are then remapped from [here](https://github.com/dotnet/sdk/blob/ec9f5fc673f798dd6a89c7d11a4e62727837757c/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.PackageDependencyResolution.targets#L270) to the `RuntimeCopyLocalItems`.

What I did was pass the `RuntimeCopyLocalItems` to the `WasmLinkAssemblies` task and do a remap to the runtime versions.  This works against everything I have thrown at it so far.

Of course this could fail if the nuget has somehow set `copy to local` false maybe but I do not see anything in the code referenced above checking for that.
2019-08-05 15:03:08 -05:00
Jo Shields c8c52f7e9f Bump version to 6.7 (#16031)
* Bump version to 6.7
2019-08-05 15:31:47 -04:00
Alexander Köplinger 807e4b3d6f [ci] Disable 32bit llvm sdks build
It was removed in https://github.com/mono/mono/commit/a6e6c72c8fc3702db37e1572694a49902f63195f
2019-08-05 21:29:51 +02:00
Kenneth Pouncey ac80d041e6 [wasm][sdk] Decoupling the WebAssembly runtime debug and release versions. (#16028)
* Update templates

- Add PropertyGroups for Debug and Release to set the DebugSymbols and DebugType by default.

* Allow the selection of the WebAssembly runtime version Debug | Release

- Decouple the reliance on `_DebugSymbolsProduced` to decide which runtime version to publish.
- Add support for <MonoWasmRuntime>Debug | Release</MonoWasmRuntime>
- Add validation target of the property to Mono.WebAssembly.Build.targets.

* Indentation
2019-08-05 19:03:35 +02:00
Zoltan Varga 93a6bc3151 [netcore] ifdef out unused icalls/files. (#16009)
* [netcore] ifdef out unused icalls/files.

* Add MONO_EMPTY_SOURCE_FILE.
2019-08-05 17:13:40 +02:00
Zoltan Varga 60bfa99043 [jit] Avoid asserting on Unsafe.As<T> in gsharedvt methods.
Fixes https://github.com/mono/mono/issues/15992.
2019-08-03 15:06:58 -04:00
2778 changed files with 105237 additions and 132534 deletions

No files matched your search

+7 -1
View File
@@ -1,3 +1,6 @@
# diff settings
*.cs diff=csharp
# ensure LF endings on all checkouts
configure.ac crlf=input
config.rpath crlf=input
@@ -7,6 +10,7 @@ mkinstalldirs crlf=input
*.sources crlf=input
.gitattributes crlf=input
*akefile* crlf=input
m4/mono-output.m4 crlf=input
# ensure native line endings on checkout
*.c crlf
@@ -16,8 +20,10 @@ mkinstalldirs crlf=input
*.sln crlf
*.*proj* crlf
*.bat text eol=crlf
*.cmd text eol=crlf
# don't do anything to line-endings. Let CRLFs/LFs go into the repo, and CRLF/LFs on checkout
*.bat -crlf
*.xml -crlf
# CRLF Handling
+18 -10
View File
@@ -15,7 +15,7 @@
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger @kg
/acceptance-tests/profiler-stress* @alexrp @akoeplinger
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/data @directhex @akoeplinger
/data/gdb @lewurm @vargaz
@@ -23,7 +23,7 @@
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp @lewurm
/docs/sources/mono-api-profiler.html @alexrp
/docs/sources/mono-api-profiler.html @BrzVlad
/external @akoeplinger @marek-safar
/external/api-snapshot @marek-safar @akoeplinger
@@ -34,16 +34,16 @@
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/linker @marek-safar
/external/llvm @SamMonoRT @vargaz
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
/libgc @BrzVlad @vargaz @naricc
/llvm @vargaz @SamMonoRT
/llvm @vargaz @SamMonoRT @imhameed
/man @marek-safar @miguel
/man/*prof* @alexrp
/man/*prof* @BrzVlad
/mcs @marek-safar @akoeplinger @egorbo
@@ -92,23 +92,32 @@
/mono/metadata/thread* @lateralusX @lambdageek
/mono/metadata/w32* @lateralusX @lambdageek
/mono/metadata/*-win* @lateralusX @lambdageek
/mono/metadata/appdomain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/assembly* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/domain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/loader* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lewurm @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz @BrzVlad
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @alexrp @lambdageek
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
/mono/native @baulig @egorbo @alexischr @marek-safar
/mono/profiler @alexrp @lambdageek
/mono/profiler @BrzVlad @lambdageek
/mono/sgen @BrzVlad @lambdageek
/mono/tools/offsets-tool @lewurm @rolfbjarne
/mono/utils @vargaz @lewurm @lambdageek
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
@@ -129,7 +138,7 @@
/runtime @akoeplinger @marek-safar
/samples @lambdageek @lewurm
/samples/profiler @alexrp @lambdageek
/samples/profiler @BrzVlad @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
@@ -149,6 +158,5 @@
/tools @lambdageek @vargaz
/tools/locale-builder @marek-safar @egorbo
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/sgen @brzvlad @vargaz @naricc
-1
View File
@@ -140,7 +140,6 @@ sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
##############################################################################
# Arcade output directories
##############################################################################
+7 -4
View File
@@ -58,13 +58,16 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/llvm"]
path = external/llvm
url = git://github.com/mono/llvm.git
branch = release_60
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+15 -51
View File
@@ -2,33 +2,11 @@ ACLOCAL_AMFLAGS = -I m4
AM_CFLAGS = $(WERROR_CFLAGS)
MONOTOUCH_SUBDIRS = $(libgc_dir) mono
SUBDIRS = @MONO_SUBDIRS@
noinst_SUBDIRS = @MONO_NOINST_SUBDIRS@
DIST_SUBDIRS = $(SUBDIRS) m4 netcore
# Some tools might not build when cross-compiling
if CROSS_COMPILING
tools_dir =
else
tools_dir = tools
endif
if ENABLE_MSVC
build_with_msvc = msvc
build_without_msvc =
else
build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
SUBDIRS = mk llvm mono runtime netcore
else
update_submodules = update_submodules
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
endif
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore
if !ENABLE_NETCORE
all: $(update_submodules)
@@ -46,9 +24,10 @@ EXTRA_DIST= \
winconfig.h \
code_of_conduct.md \
external \
netcore/Makefile \
mcs/class/referencesource
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_MCS=false EXTERNAL_RUNTIME=false
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_RUNTIME=false
# Distribute the 'mcs' tree too
GIT_DIR ?= $(srcdir)/.git
@@ -66,6 +45,9 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
@@ -112,30 +94,10 @@ mcs-do-compiler-tests:
bootstrap-world: compiler-tests
$(MAKE) install
if INSTALL_MONOTOUCH
monotouch-do-build: config.h
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-build";; \
* ) target="all";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-build)
monotouch-do-clean:
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-clean";; \
* ) target="clean";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-clean)
endif
install:
for mydir in $(filter-out $(noinst_SUBDIRS),$(SUBDIRS)); do \
(cd $${mydir} && ${MAKE} install) \
done
update-csproj:
-rm msvc/scripts/order
@@ -179,3 +141,5 @@ update-solution-files:
update-solution-files-with-tests:
$(MAKE) "GENPROJ_ARGS=2012 true true" update-solution-files
endif
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "7a840bd5697ef6ea41f554a09d837636fba9d88d",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+30 -9
View File
@@ -104,12 +104,10 @@ if grep "^AM_PROG_LIBTOOL" configure.ac >/dev/null; then
fi
fi
#
# Plug in the extension module
#
# Parse parameters
has_ext_mod=false
ext_mod_args=''
has_disable_boehm=false
for PARAM; do
if [[ $PARAM =~ "--enable-extension-module" ]] ; then
has_ext_mod=true
@@ -117,8 +115,14 @@ for PARAM; do
ext_mod_args=`echo $PARAM | cut -d= -f2`
fi
fi
if [[ $PARAM =~ "--disable-boehm" ]] ; then
has_disable_boehm=true
fi
done
#
# Plug in the extension module
#
if test x$has_ext_mod = xtrue; then
pushd $top_srcdir../mono-extensions/scripts
sh ./prepare-repo.sh $ext_mod_args || exit 1
@@ -150,17 +154,34 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
# Update all submodules recursively to ensure everything is checked out
if test -e $srcdir/scripts/update_submodules.sh; then
(cd $srcdir && scripts/update_submodules.sh)
fi
if test x$has_disable_boehm = xfalse -a -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
echo "MONO_EXTRA_CONFIGURE_FLAGS is $MONO_EXTRA_CONFIGURE_FLAGS"
fi
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings" #--enable-iso-c
host_conf_flag=
build_uname_all=`(uname -a) 2>/dev/null`
case "$build_uname_all" in
CYGWIN*)
if [[ "$@" != *"--host="* ]]; then
echo "Missing --host parameter, configure using ./configure --host=i686-w64-mingw32 or --host=x86_64-w64-mingw32"
echo "Falling back using --host=x86_64-w64-mingw32 as default."
host_conf_flag="--host=x86_64-w64-mingw32"
fi
;;
esac
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings $host_conf_flag" #--enable-iso-c
if test x$NOCONFIGURE = x; then
echo Running $srcdir/configure $conf_flags "$@" ...
+60 -18
View File
@@ -574,6 +574,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ilasm", "mcs/ilasm/ilasm.cs
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "al", "mcs/tools/al/al.csproj", "{0A20671F-CCA9-457D-886C-0828C7D64128}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "aprofutil", "mcs/tools/aprofutil/aprofutil.csproj", "{658B566B-CED1-4C42-9926-C1AA930BA368}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "monolinker", "mcs/tools/linker/monolinker.csproj", "{D60432F1-3C12-4235-A1CC-4CA5575403BD}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "culevel", "mcs/tools/culevel/culevel.csproj", "{E8E246BD-CD0C-4734-A3C2-7F44796EC47B}"
@@ -4012,24 +4014,42 @@ Global
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|net_4_x.Build.0 = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.Build.0 = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.Build.0 = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.Build.0 = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.Build.0 = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.Build.0 = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.Build.0 = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.Build.0 = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.Build.0 = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.Build.0 = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.Build.0 = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.Build.0 = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.Build.0 = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.Build.0 = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.Build.0 = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.Build.0 = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.Build.0 = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.Build.0 = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.Build.0 = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.Build.0 = Release|xammac
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.Build.0 = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|net_4_x.ActiveCfg = Release|net_4_x
@@ -10100,6 +10120,28 @@ Global
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|winaot.ActiveCfg = Release|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Debug|xammac.ActiveCfg = Debug|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|xammac.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.Build.0 = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.Build.0 = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monodroid.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|orbis.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|orbis.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|unreal.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|unreal.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|wasm.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|wasm.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|winaot.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|winaot.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|xammac.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|xammac.ActiveCfg = Release|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.Build.0 = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Release|net_4_x.ActiveCfg = Release|net_4_x
+461 -418
View File
File diff suppressed because it is too large. Load diff
+13 -13
View File
@@ -1,8 +1,9 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
print '%c thread #%i' % ('*' if current_thread else ' ', thread.idx)
print('%c thread #%i' % ('*' if current_thread else ' ', thread.idx))
if current_thread:
selected_frame = thread.GetSelectedFrame()
@@ -14,17 +15,16 @@ def print_frames(thread, num_frames, current_thread):
if function_name == "interp_exec_method_full":
try:
s = 'frame->imethod->method'
klassname = frame.EvaluateExpression('(char*) ' + s + '->klass->name').summary[1:-1]
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
methoddesc = frame.EvaluateExpression('(char*) mono_method_full_name (' + s + ', 1)').summary[1:-1]
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
var = '%s::%s @ %d %s || %s' % (klassname, methodname, ipoffset, insn, frame)
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
except Exception as e:
print "DBG: execfail:" + str(e)
traceback.print_exc()
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -32,7 +32,7 @@ def print_frames(thread, num_frames, current_thread):
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
var = 'transforming %s::%s || %s' % (klassname, methodname, frame)
except Exception as e:
print "DBG: transformfail:" + str(e)
print("DBG: transformfail:" + str(e))
elif pc[0] == '0':
try:
framestr = frame.EvaluateExpression('(char*)mono_pmip((void*)%s)' % pc).summary[1:-1]
@@ -40,7 +40,7 @@ def print_frames(thread, num_frames, current_thread):
except:
pass
print ' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var)
print(' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var))
def monobt(debugger, command, result, dict):
opts = {'all_bt': False, 'num_frames': None}
@@ -50,20 +50,20 @@ def monobt(debugger, command, result, dict):
elif command.isdigit():
opts['num_frames'] = int(command)
elif command != '':
print 'error: monobt [<number>|all]'
print('error: monobt [<number>|all]')
return
target = debugger.GetSelectedTarget()
process = target.process
if not process.IsValid():
print 'error: invalid process'
print('error: invalid process')
return
if opts['all_bt']:
for thread in process.threads:
print_frames(thread, len(thread), process.selected_thread == thread)
print ''
print('')
else:
thread = process.selected_thread
num_frames = len(thread) if opts['num_frames'] is None else opts['num_frames']
@@ -76,4 +76,4 @@ def __lldb_init_module (debugger, dict):
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f monobt.monobt monobt')
debugger.HandleCommand('command alias mbt monobt')
print '"monobt" command installed'
print('"monobt" command installed')
@@ -242,7 +242,7 @@ namespace HtmlAgilityPack
}
/// <summary>
/// Checks for existance of attribute with given name
/// Checks for existence of attribute with given name
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
+23
View File
@@ -18,6 +18,27 @@ convert.exe: $(srcdir)/convert.cs AgilityPack.dll
AgilityPack.dll:
$(CSCOMPILE) -r:$(topdir)/class/lib/$(PROFILE)/mscorlib.dll -r:$(topdir)/class/lib/$(PROFILE)/System.dll -r:$(topdir)/class/lib/$(PROFILE)/System.Xml.dll -target:library -out:$@ $(srcdir)/HtmlAgilityPack/*.cs
ifdef MCS_MODE
monoapi.zip:
touch $@
monoapi.tree: monoapi.zip
touch $@
mono-tools.zip:
touch $@
mono-tools.tree: mono-tools.zip
touch $@
mono-file-formats.zip:
touch $@
mono-file-formats.tree: mono-file-formats.zip
touch $@
else
monoapi.zip: monoapi.tree
@test -f $@ || { rm -f $< && $(MAKE) $<; }
@@ -36,5 +57,7 @@ mono-file-formats.zip: mono-file-formats.tree
mono-file-formats.tree: $(srcdir)/mono-file-formats.config $(srcdir)/docs.make
$(MDOC) assemble -o mono-file-formats -f man $<
endif
.doc-stamp:
+61 -39
View File
@@ -2,36 +2,58 @@
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documents
## Useful RISC-V documentation
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications/)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa/)
* [RISC-V Debug Specification Standard](https://github.com/riscv/riscv-debug-spec/blob/master/riscv-debug-spec.pdf)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual/blob/master/riscv-asm.md)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md)
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V Linux](https://github.com/riscv/riscv-linux)
* [RISC-V LLD](https://github.com/riscv/riscv-lld)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V QEMU](https://github.com/riscv/riscv-qemu)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
* [lowRISC Organization](https://github.com/lowrisc)
* [RISC-V LLVM](https://github.com/lowrisc/riscv-llvm)
## Setting up a cross environment
Setting up a cross environment with a Linux toolchain and QEMU is quite easy.
### Debian/Ubuntu packages
First, add these to your `.bashrc` (or some other script that you run):
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
@@ -39,47 +61,42 @@ export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Install some dependencies needed to build the toolchain:
Next, install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev libusb-1.0-0-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev device-tree-compiler pkg-config libexpat-dev libglib2.0-dev libpixman-1-dev
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
```
Now you can build the toolchain:
Finally, build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-tools.git
$ cd riscv-tools
$ ./build.sh
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-multilib
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ make linux
$ cd ../..
$ git clone --recursive [email protected]:riscv/riscv-qemu.git
$ cd riscv-qemu
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --disable-werror
$ make
$ make install
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
@@ -89,19 +106,24 @@ Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 $RISCV/sysroot/bin/mono hello.exe
$ qemu-riscv64 ./mono/mini/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono-sgen --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono-sgen
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
```
## Things to be done
In no particular order:
Things that need to be done beyond the basic 64-bit port, in no particular
order:
* Complete the soft float port.
* Complete the 32-bit port.
+6
View File
@@ -109,6 +109,12 @@ typedef struct {
<h3>Write Barriers</h3>
<p>SGen is a concurrent and generational GC, features which require
tracking changes to the state of the heap. This is achieved through
write barriers. Whenever native code is changing the state of the
heap by storing references into another managed object, it needs to
do it using this write barrier API.
<h4><a name="api:mono_gc_wbarrier_arrayref_copy">mono_gc_wbarrier_arrayref_copy</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_nostore">mono_gc_wbarrier_generic_nostore</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_store">mono_gc_wbarrier_generic_store</a></h4>
+10 -10
View File
@@ -1,27 +1,27 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha1.19375.11">
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>97d5957fc71853961082e22873d926269b291b92</Sha>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha1.19375.9">
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>2977763fd35b8dc120f4eac94314aba7b05ae9ee</Sha>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.0-beta2-19381-14">
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>ef3a7a3863ae53b610a4b0c3682a35cad0829583</Sha>
<Sha>39d545dc14bc4d5a546e5e2f58d1ebaca72fcd61</Sha>
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19379.1">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.20109.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>193793488c727c2d12a04b4abe5e6ea63ad20526</Sha>
<Sha>b0e8d3944155f94f83deea8afe025debe369e69f</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19379.1">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.20109.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>193793488c727c2d12a04b4abe5e6ea63ad20526</Sha>
<Sha>b0e8d3944155f94f83deea8afe025debe369e69f</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
+4 -4
View File
@@ -1,14 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>1.0.0</VersionPrefix>
<VersionPrefix>5.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha1.19375.11</MicrosoftPrivateCoreFxNETCoreAppVersion>
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha1.19375.9</MicrosoftNETCoreAppVersion>
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.0-beta2-19374-02</MicrosoftNetCompilersVersion>
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
</Project>
+27 -26
View File
@@ -5,11 +5,12 @@ param(
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
. $PSScriptRoot\pipeline-logging-functions.ps1
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $TargetServerParam, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $TargetServerParameter, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $SymbolsPath
)
@@ -21,36 +22,36 @@ function FirstMatchingSymbolDescriptionOrDefault {
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
$SymbolPath = $SymbolsPath + '\' + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
$PdbPath = $SymbolPath.Replace($Extension, '.pdb')
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
$NGenPdb = $SymbolPath.Replace($Extension, '.ni.pdb')
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
$SODbg = $SymbolPath.Replace($Extension, '.so.dbg')
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$DylibDwarf = $SymbolPath.Replace($Extension, '.dylib.dwarf')
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParameter $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
return 'PDB'
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
return 'NGen PDB'
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
return 'DBG for SO'
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
return 'Dwarf for Dylib'
}
elseif (Test-Path $SymbolPath) {
return "Module"
return 'Module'
}
else {
return $null
@@ -68,7 +69,7 @@ function CountMissingSymbols {
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
$RelevantExtensions = @('.dll', '.exe', '.so', '.dylib')
# How many files are missing symbol information
$MissingSymbols = 0
@@ -76,7 +77,7 @@ function CountMissingSymbols {
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath 'Symbols'
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
@@ -86,31 +87,31 @@ function CountMissingSymbols {
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
if ($_.FullName -Match '\\ref\\') {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault -FullPath $_.FullName -TargetServerParameter '--microsoft-symbol-server' -SymbolsPath $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault -FullPath $_.FullName -TargetServerParameter '--internal-server' -SymbolsPath $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
Write-Host "Symbols found on MSDL (${$SymbolsOnMSDL}) and SymWeb (${$SymbolsOnSymWeb})"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
Write-Host 'No symbols found on MSDL or SymWeb!'
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
Write-Host 'No symbols found on MSDL!'
}
else {
Write-Host "No symbols found on SymWeb!"
Write-Host 'No symbols found on SymWeb!'
}
}
}
@@ -129,26 +130,26 @@ function CheckSymbolsAvailable {
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
if ($FileName -Match 'Microsoft\.DotNet\.Darc\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
elseif ($FileName -Match 'Microsoft\.DotNet\.Maestro\.Tasks\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-Error "Missing symbols for $Status modules in the package $FileName"
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message "Missing symbols for $Status modules in the package $FileName"
}
Write-Host
+4 -2
View File
@@ -37,6 +37,8 @@
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymWeb Condition="'$(PublishToSymWeb)' == ''">true</PublishToSymWeb>
<PublishToMSDL Condition="'$(PublishToMSDL)' == ''">true</PublishToMSDL>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
@@ -56,7 +58,7 @@
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
Condition="$(PublishToSymbolServer) and $(PublishToMSDL)"/>
<!--
Symbol Uploader: SymWeb
@@ -73,7 +75,7 @@
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
Condition="$(PublishToSymbolServer) and $(PublishToSymWeb)"/>
</Target>
<ItemGroup>
+141
View File
@@ -0,0 +1,141 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile $(Build.SourcesDirectory)/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
if (!$Password) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Please supply a valid PAT'
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
$dotnet3Source = $sources.SelectSingleNode("add[@key='dotnet3']")
if ($dotnet3Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$dotnet31Source = $sources.SelectSingleNode("add[@key='dotnet3.1']")
if ($dotnet31Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$doc.Save($filename)
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $(Build.SourcesDirectory)/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
Write-PipelineTelemetryError -Category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [ -z "$CredToken" ]; then
Write-PipelineTelemetryError -category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Please supply a valid PAT"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
PackageSources=()
# Ensure dotnet3-internal and dotnet3-internal-transport are in the packageSources if the public dotnet3 feeds are present
grep -i "<add key=\"dotnet3\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal')
grep -i "<add key=\"dotnet3-internal-transport\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal-transport')
fi
# Ensure dotnet3.1-internal and dotnet3.1-internal-transport are in the packageSources if the public dotnet3.1 feeds are present
grep -i "<add key=\"dotnet3.1\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3.1-internal\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal')
grep -i "<add key=\"dotnet3.1-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal-transport')
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources+="$IFS"
PackageSources+=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" $ConfigFile
fi
done
+140 -129
View File
@@ -6,140 +6,15 @@ param(
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
Write-PipelineTelemetryError -Category "Build" -Message "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
Write-PipelineTelemetryError -Category "Build" -Message "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
@@ -160,7 +35,7 @@ function ValidateSourceLinkLinks {
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
Write-PipelineTelemetryError -Category "Build" -Message "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
@@ -181,4 +56,140 @@ function ValidateSourceLinkLinks {
}
}
Measure-Command { ValidateSourceLinkLinks }
try {
. $PSScriptRoot\pipeline-logging-functions.ps1
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
Measure-Command { ValidateSourceLinkLinks }
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'SourceLink' -Message $_
ExitWithExitCode 1
}
+51 -44
View File
@@ -18,6 +18,7 @@ Param(
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
@@ -25,49 +26,50 @@ Param(
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
. $PSScriptRoot\tools.ps1
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot "restore-toolset.ps1"
$script = Join-Path $EngRoot 'restore-toolset.ps1'
if (Test-Path $script) {
. $script
@@ -78,13 +80,17 @@ function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'Build.binlog') } else { '' }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { '' }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
@@ -109,7 +115,15 @@ function Build {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($clean) {
if (Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host 'Artifacts directory deleted.'
}
exit 0
}
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
@@ -119,14 +133,7 @@ try {
$nodeReuse = $false
}
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
if ($restore) {
InitializeNativeTools
}
@@ -134,7 +141,7 @@ try {
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+12 -12
View File
@@ -26,6 +26,7 @@ usage()
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
@@ -62,6 +63,7 @@ publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
@@ -82,6 +84,9 @@ while [[ $# > 0 ]]; do
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
@@ -196,20 +201,15 @@ function Build {
ExitWithExitCode 0
}
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
if [[ "$restore" == true ]]; then
InitializeNativeTools
fi
+16 -5
View File
@@ -25,8 +25,9 @@ __UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
__AlpinePackagesEdgeTesting=" lldb-dev"
__AlpinePackagesEdgeMain=" llvm9-libs"
__AlpinePackagesEdgeMain+=" python3"
# symlinks fixer
__UbuntuPackages+=" symlinks"
@@ -193,19 +194,29 @@ fi
if [[ "$__LinuxCodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.7
__AlpineVersion=3.9
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeMain
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeTesting
rm -r $__ApkToolsDir
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
+15 -34
View File
@@ -31,6 +31,10 @@ else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
@@ -39,48 +43,25 @@ if(TARGET_ARCH_NAME STREQUAL "armel")
endif()
endif()
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
endif()
endforeach()
endmacro()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_param(CROSS_LINK_FLAGS "-m32")
add_link_options(-m32)
endif()
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
@@ -103,7 +84,7 @@ elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths
# Set LLDB include and library paths for builds that need lldb.
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
@@ -131,7 +112,7 @@ if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+21 -7
View File
@@ -1,13 +1,14 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
$versionEndpoint = 'https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16',
$verbosity = 'minimal',
$toolpath = $null
)
$verbosity = "m"
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
$darcCliPackageName = 'microsoft.dotnet.darc'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
@@ -23,11 +24,24 @@ function InstallDarcCli ($darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
if (-not $toolpath) {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g --tool-path '$toolpath'"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
}
InstallDarcCli $darcVersion
try {
InstallDarcCli $darcVersion
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Darc' -Message $_
ExitWithExitCode 1
}
+26 -8
View File
@@ -2,7 +2,8 @@
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
versionEndpoint='https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16'
verbosity='minimal'
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
@@ -15,6 +16,14 @@ while [[ $# > 0 ]]; do
versionEndpoint=$2
shift
;;
--verbosity)
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
@@ -34,7 +43,6 @@ while [[ -h "$source" ]]; do
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity=m
. "$scriptroot/tools.sh"
@@ -48,17 +56,27 @@ function InstallDarcCli {
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
fi
local arcadeServicesSource="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json"
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
}
InstallDarcCli
+12 -11
View File
@@ -1,26 +1,27 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet"
[string] $verbosity = 'minimal',
[string] $architecture = '',
[string] $version = 'Latest',
[string] $runtime = 'dotnet',
[string] $RuntimeSourceFeed = '',
[string] $RuntimeSourceFeedKey = ''
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
if ($architecture -and $architecture.Trim() -eq 'x86') {
$installdir = Join-Path $installdir 'x86'
}
InstallDotNet $installdir $version $architecture $runtime $true
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+45 -5
View File
@@ -11,9 +11,13 @@ while [[ -h "$source" ]]; do
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
@@ -29,20 +33,56 @@ while [[ $# > 0 ]]; do
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
echo "Invalid argument: $1"
usage
Write-PipelineTelemetryError -Category 'Build' -Message "Invalid argument: $1"
exit 1
;;
esac
shift
done
. "$scriptroot/tools.sh"
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv7l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
dotnetRoot="$repo_root/.dotnet"
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
@@ -0,0 +1,13 @@
param(
[string] $token
)
. $PSScriptRoot\pipeline-logging-functions.ps1
# Write-PipelineSetVariable will no-op if a variable named $ci is not defined
# Since this script is only ever called in AzDO builds, just universally set it
$ci = $true
Write-PipelineSetVariable -Name 'VSS_NUGET_ACCESSTOKEN' -Value $token -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'VSS_NUGET_URI_PREFIXES' -Value 'https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/' -IsMultiJobVariable $false
+25 -26
View File
@@ -3,39 +3,39 @@ Param(
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $darcVersion, # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
Write-PipelineTelemetryError -Category 'Arcade' -Message "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
$ErrorActionPreference = 'Stop'
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
Write-Host 'Installing darc...'
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
CheckExitCode 'Running darc-init'
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$engCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
$nativeToolBaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
$installBin = Join-Path $graphvizInstallDir 'bin'
Write-Host "Installing dot..."
Write-Host 'Installing dot...'
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
@@ -51,37 +51,36 @@ try {
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = @( "--github-pat", "$gitHubPat", "--azdev-pat", "$azdoPat", "--password", "$barToken" )
$baseOptions = @( '--github-pat', "$gitHubPat", '--azdev-pat', "$azdoPat", '--password', "$barToken" )
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += @( "--include-toolset" )
Write-Host 'Toolsets will be included in the graph...'
$baseOptions += @( '--include-toolset' )
}
Write-Host "Generating standard dependency graph..."
Write-Host 'Generating standard dependency graph...'
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
CheckExitCode "Generating normal dependency graph"
CheckExitCode 'Generating normal dependency graph'
Write-Host "Generating flat dependency graph and graphviz file..."
Write-Host 'Generating flat dependency graph and graphviz file...'
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
CheckExitCode "Generating flat and graphviz dependency graph"
CheckExitCode 'Generating flat and graphviz dependency graph'
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
CheckExitCode "Generating graphviz image"
CheckExitCode 'Generating graphviz image'
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
Write-Host 'This might be a toolset repo which includes only toolset dependencies. ' -NoNewline -ForegroundColor Yellow
Write-Host 'Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again...' -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
} finally {
Pop-Location
Pop-Location
}
+21 -16
View File
@@ -35,7 +35,7 @@ File path to global.json file
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $BaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external',
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
@@ -45,26 +45,27 @@ Param (
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName 'global.json'
}
Set-StrictMode -version 2.0
$ErrorActionPreference="Stop"
$ErrorActionPreference='Stop'
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
. $PSScriptRoot\pipeline-logging-functions.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$Verbose = $VerbosePreference -Eq 'Continue'
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$EngCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
$InstallBin = Join-Path $NativeBaseDir 'bin'
$InstallerPath = Join-Path $EngCommonBaseDir 'install-tool.ps1'
# Process tools list
Write-Host "Processing $GlobalJsonFile"
@@ -74,7 +75,7 @@ try {
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
Select-Object -Expand 'native-tools' -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
@@ -112,18 +113,21 @@ try {
}
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host 'Native tools bootstrap failed'
exit 1
}
}
else {
Write-Host "No native tools defined in global.json"
Write-Host 'No native tools defined in global.json'
exit 0
}
@@ -131,17 +135,18 @@ try {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host 'Native tools are available from ' (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message 'Native tools install directory does not exist, installation failed'
exit 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message $_
ExitWithExitCode 1
}
+5 -4
View File
@@ -12,6 +12,7 @@ retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/pipeline-logging-functions.sh
. $scriptroot/native/common-library.sh
while (($# > 0)); do
@@ -70,8 +71,7 @@ function ReadGlobalJsonNativeTools {
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[[:space:]]')"
native_tools_list=$( echo "$native_tools_list" | sed 's/\s//g' | sed 's/,/\n/g' )
local old_IFS=$IFS
while read -r line; do
@@ -108,6 +108,7 @@ else
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
echo $installer_command
if [[ $force = true ]]; then
installer_command+=" --force"
@@ -120,7 +121,7 @@ else
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
exit 1
fi
done
@@ -134,7 +135,7 @@ if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
echo "Native tools install directory does not exist, installation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Native tools install directory does not exist, installation failed"
exit 1
fi
+13 -15
View File
@@ -6,9 +6,8 @@ param(
[switch] $IsFeedPrivate
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
@@ -21,7 +20,7 @@ function SetupCredProvider {
)
# Install the Cred Provider NuGet plugin
Write-Host "Setting up Cred Provider NuGet plugin in the agent..."
Write-Host 'Setting up Cred Provider NuGet plugin in the agent...'
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
@@ -29,18 +28,18 @@ function SetupCredProvider {
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host "Installing plugin..."
Write-Host 'Installing plugin...'
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-Host "CredProvider plugin was not installed correctly!"
Write-PipelineTelemetryError -Category 'Arcade' -Message 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
}
else {
Write-Host "CredProvider plugin was installed correctly!"
Write-Host 'CredProvider plugin was installed correctly!'
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
@@ -49,7 +48,7 @@ function SetupCredProvider {
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-Host "NuGet.config file not found in repo's root!"
Write-PipelineTelemetryError -Category 'Build' -Message 'NuGet.config file not found in repo root!'
ExitWithExitCode 1
}
@@ -81,7 +80,7 @@ function SetupCredProvider {
}
else
{
Write-Host "No internal endpoints found in NuGet.config"
Write-Host 'No internal endpoints found in NuGet.config'
}
}
@@ -99,7 +98,7 @@ function InstallDotNetSdkAndRestoreArcade {
& $dotnet restore $restoreProjPath
Write-Host "Arcade SDK restored!"
Write-Host 'Arcade SDK restored!'
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
@@ -113,23 +112,22 @@ function InstallDotNetSdkAndRestoreArcade {
try {
Push-Location $PSScriptRoot
if ($Operation -like "setup") {
if ($Operation -like 'setup') {
SetupCredProvider $AuthToken
}
elseif ($Operation -like "install-restore") {
elseif ($Operation -like 'install-restore') {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-Host "Unknown operation '$Operation'!"
Write-PipelineTelemetryError -Category 'Arcade' -Message "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
}
finally {
Pop-Location
Pop-Location
}
+2 -2
View File
@@ -30,7 +30,7 @@ function SetupCredProvider {
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
echo "CredProvider plugin was not installed correctly!"
Write-PipelineTelemetryError -category 'Build' 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
@@ -42,7 +42,7 @@ function SetupCredProvider {
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
echo "NuGet.config file not found in repo's root!"
Write-PipelineTelemetryError -category 'Build' "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
+2 -3
View File
@@ -1,6 +1,6 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
@@ -18,9 +18,8 @@ try {
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+2
View File
@@ -152,6 +152,8 @@ function Get-File {
}
else {
Write-Verbose "Downloading $Uri"
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
while($Attempt -Lt $DownloadRetries)
{
try {
+5 -5
View File
@@ -34,7 +34,7 @@ function ExpandZip {
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
echo Unable to remove '$output_directory'>&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to remove '$output_directory'"
return 1
fi
fi
@@ -45,7 +45,7 @@ function ExpandZip {
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
echo "Unable to extract '$zip_path'" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to extract '$zip_path'"
return 1
fi
@@ -117,7 +117,7 @@ function DownloadAndExtract {
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to download '$uri' to '$temp_tool_path'."
return 1
fi
@@ -125,7 +125,7 @@ function DownloadAndExtract {
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to extract '$temp_tool_path' to '$installDir'."
return 1
fi
@@ -148,7 +148,7 @@ function NewScriptShim {
fi
if [[ ! -f $tool_file_path ]]; then
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Specified tool file path:'$tool_file_path' does not exist"
return 1
fi
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake-test"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
exit 0
+3 -3
View File
@@ -69,7 +69,7 @@ tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/cmake/$tool_name_moniker.tar.gz"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
@@ -101,7 +101,7 @@ fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
@@ -110,7 +110,7 @@ fi
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
+7 -5
View File
@@ -46,6 +46,8 @@ Param (
[int] $RetryWaitTimeInSeconds = 30
)
. $PSScriptRoot\..\pipeline-logging-functions.ps1
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
@@ -93,7 +95,7 @@ try {
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-Error "Installation failed"
Write-PipelineTelemetryError "Installation failed" -Category "NativeToolsetBootstrapping"
exit 1
}
}
@@ -103,7 +105,7 @@ try {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Error "$ToolName was not found in $ToolFilePath."
Write-Host "$ToolName was not found in $ToolFilePath."
exit 1
}
@@ -117,14 +119,14 @@ try {
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-Error "Generate shim failed"
Write-PipelineTelemetryError "Generate shim failed" -Category "NativeToolsetBootstrapping"
return 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "NativeToolsetBootstrapping" -Message $_
exit 1
}
+27 -2
View File
@@ -5,8 +5,14 @@
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
<DotnetExe>%HELIX_CORRELATION_PAYLOAD%\performance\tools\dotnet\$(Architecture)\dotnet.exe</DotnetExe>
<Percent>%25%25</Percent>
<XMLResults>%HELIX_WORKITEM_ROOT%\testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
@@ -24,14 +30,24 @@
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<BaselineCoreRun>$(BaseDirectory)/Baseline_Core_Root/corerun</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>$(PerformanceDirectory)/src/tools/ResultsComparer/ResultsComparer.csproj</ResultsComparer>
<DotnetExe>$(PerformanceDirectory)/tools/dotnet/$(Architecture)/dotnet</DotnetExe>
<Percent>%25</Percent>
<XMLResults>$HELIX_WORKITEM_ROOT/testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(UseBaselineCoreRun)' == 'true'">
<BaselineCoreRunArgument>--corerun $(BaselineCoreRun)</BaselineCoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>
@@ -57,20 +73,29 @@
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>
<PropertyGroup Condition="'$(Compare)' == 'true'">
<FailOnTestFailure>false</FailOnTestFailure>
</PropertyGroup>
<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --artifacts $(ArtifactsDirectory) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults);$(FinalCommand)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --artifacts $(ArtifactsDirectory)"</Command>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
+23 -7
View File
@@ -1,22 +1,25 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $BaselineCoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp3.0",
[string] $Framework="netcoreapp5.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="coreclr corefx",
[string] $RunCategories="Libraries Runtime",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[string] $Configurations="CompilationMode=$CompilationMode"
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode RunKind=$Kind"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance")
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)
$UseBaselineCoreRun = ($BaselineCoreRootDirectory -ne [string]::Empty)
$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
@@ -29,18 +32,25 @@ $HelixSourcePrefix = "pr"
$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"
if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS4.Open"
$Queue = "Windows.10.Amd64.ClientRS5.Open"
}
if ($Compare) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf.Open"
$PerfLabArguments = ""
$ExtraBenchmarkDotNetArguments = ""
}
if ($Internal) {
$Queue = "Windows.10.Amd64.ClientRS5.Perf"
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}
$CommonSetupArguments="--frameworks $Framework --queue $Queue --build-number $BuildNumber --build-configs $Configurations"
# FIX ME: This is a workaround until we get this from the actual pipeline
$CommonSetupArguments="--channel master --queue $Queue --build-number $BuildNumber --build-configs $Configurations --architecture $Architecture"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
if ($RunFromPerformanceRepo) {
@@ -56,6 +66,10 @@ if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}
if ($UseBaselineCoreRun) {
$NewBaselineCoreRoot = (Join-Path $PayloadDirectory "Baseline_Core_Root")
Move-Item -Path $BaselineCoreRootDirectory -Destination $NewBaselineCoreRoot
}
$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory
@@ -80,7 +94,9 @@ Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVaria
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseBaselineCoreRun' -Value "$UseBaselineCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Compare' -Value "$Compare" -IsMultiJobVariable $false
# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
+63 -23
View File
@@ -2,20 +2,23 @@
source_directory=$BUILD_SOURCESDIRECTORY
core_root_directory=
baseline_core_root_directory=
architecture=x64
framework=netcoreapp3.0
framework=netcoreapp5.0
compilation_mode=tiered
repository=$BUILD_REPOSITORY_NAME
branch=$BUILD_SOURCEBRANCH
commit_sha=$BUILD_SOURCEVERSION
build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="coreclr corefx"
run_categories="Libraries Runtime"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations=
configurations="CompliationMode=$compilation_mode RunKind=$kind"
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
@@ -28,6 +31,10 @@ while (($# > 0)); do
core_root_directory=$2
shift 2
;;
--baselinecorerootdirectory)
baseline_core_root_directory=$2
shift 2
;;
--architecture)
architecture=$2
shift 2
@@ -72,6 +79,10 @@ while (($# > 0)); do
internal=true
shift 1
;;
--compare)
compare=true
shift 1
;;
--configurations)
configurations=$2
shift 2
@@ -102,7 +113,7 @@ while (($# > 0)); do
esac
done
if [[ "$repository" == "dotnet/performance" ]]; then
if [ "$repository" == "dotnet/performance" ] || [ "$repository" == "dotnet-performance" ]; then
run_from_perf_repo=true
fi
@@ -114,6 +125,10 @@ if [ -z "$core_root_directory" ]; then
use_core_run=false
fi
if [ -z "$baseline_core_root_directory" ]; then
use_baseline_core_run=false
fi
payload_directory=$source_directory/Payload
performance_directory=$payload_directory/performance
workitem_directory=$source_directory/workitem
@@ -123,6 +138,19 @@ queue=Ubuntu.1804.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"
if [[ "$compare" == true ]]; then
extra_benchmark_dotnet_arguments=
perflab_arguments=
# No open queues for arm64
if [[ "$architecture" = "arm64" ]]; then
echo "Compare not available for arm64"
exit 1
fi
queue=Ubuntu.1804.Amd64.Tiger.Perf.Open
fi
if [[ "$internal" == true ]]; then
perflab_arguments="--upload-to-perflab-container"
helix_source_prefix="official"
@@ -132,11 +160,11 @@ if [[ "$internal" == true ]]; then
if [[ "$architecture" = "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
queue=Ubuntu.1804.Amd64.Perf
queue=Ubuntu.1804.Amd64.Tiger.Perf
fi
fi
common_setup_arguments="--frameworks $framework --queue $queue --build-number $build_number --build-configs $configurations"
common_setup_arguments="--channel master --queue $queue --build-number $build_number --build-configs $configurations --architecture $architecture"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
@@ -156,21 +184,33 @@ if [[ "$use_core_run" = true ]]; then
mv $core_root_directory $new_core_root
fi
if [[ "$use_baseline_core_run" = true ]]; then
new_baseline_core_root=$payload_directory/Baseline_Core_Root
mv $baseline_core_root_directory $new_baseline_core_root
fi
ci=true
_script_dir=$(pwd)/eng/common
. "$_script_dir/pipeline-logging-functions.sh"
# Make sure all of our variables are available for future steps
echo "##vso[task.setvariable variable=UseCoreRun]$use_core_run"
echo "##vso[task.setvariable variable=Architecture]$architecture"
echo "##vso[task.setvariable variable=PayloadDirectory]$payload_directory"
echo "##vso[task.setvariable variable=PerformanceDirectory]$performance_directory"
echo "##vso[task.setvariable variable=WorkItemDirectory]$workitem_directory"
echo "##vso[task.setvariable variable=Queue]$queue"
echo "##vso[task.setvariable variable=SetupArguments]$setup_arguments"
echo "##vso[task.setvariable variable=Python]python3"
echo "##vso[task.setvariable variable=PerfLabArguments]$perflab_arguments"
echo "##vso[task.setvariable variable=ExtraBenchmarkDotNetArguments]$extra_benchmark_dotnet_arguments"
echo "##vso[task.setvariable variable=BDNCategories]$run_categories"
echo "##vso[task.setvariable variable=TargetCsproj]$csproj"
echo "##vso[task.setvariable variable=RunFromPerfRepo]$run_from_perf_repo"
echo "##vso[task.setvariable variable=Creator]$creator"
echo "##vso[task.setvariable variable=HelixSourcePrefix]$helix_source_prefix"
echo "##vso[task.setvariable variable=Kind]$kind"
echo "##vso[task.setvariable variable=_BuildConfig]$architecture.$kind.$framework"
Write-PipelineSetVariable -name "UseCoreRun" -value "$use_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "UseBaselineCoreRun" -value "$use_baseline_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "Architecture" -value "$architecture" -is_multi_job_variable false
Write-PipelineSetVariable -name "PayloadDirectory" -value "$payload_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerformanceDirectory" -value "$performance_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "WorkItemDirectory" -value "$workitem_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "Queue" -value "$queue" -is_multi_job_variable false
Write-PipelineSetVariable -name "SetupArguments" -value "$setup_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "Python" -value "$python3" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerfLabArguments" -value "$perflab_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "ExtraBenchmarkDotNetArguments" -value "$extra_benchmark_dotnet_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "BDNCategories" -value "$run_categories" -is_multi_job_variable false
Write-PipelineSetVariable -name "TargetCsproj" -value "$csproj" -is_multi_job_variable false
Write-PipelineSetVariable -name "RunFromPerfRepo" -value "$run_from_perf_repo" -is_multi_job_variable false
Write-PipelineSetVariable -name "Creator" -value "$creator" -is_multi_job_variable false
Write-PipelineSetVariable -name "HelixSourcePrefix" -value "$helix_source_prefix" -is_multi_job_variable false
Write-PipelineSetVariable -name "Kind" -value "$kind" -is_multi_job_variable false
Write-PipelineSetVariable -name "_BuildConfig" -value "$architecture.$kind.$framework" -is_multi_job_variable false
Write-PipelineSetVariable -name "Compare" -value "$compare" -is_multi_job_variable false
+39 -33
View File
@@ -12,6 +12,7 @@ $script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
@@ -25,49 +26,53 @@ function Write-PipelineTelemetryError {
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
[switch]$AsOutput,
[switch]$Force)
$PSBoundParameters.Remove("Category") | Out-Null
$PSBoundParameters.Remove('Category') | Out-Null
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
$PSBoundParameters.Remove("Message") | Out-Null
$PSBoundParameters.Add("Message", $Message)
$PSBoundParameters.Remove('Message') | Out-Null
$PSBoundParameters.Add('Message', $Message)
Write-PipelineTaskError @PSBoundParameters
}
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
if(!$ci) {
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force
)
if(!$Force -And (-Not (Test-Path variable:ci) -Or !$ci)) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
$PSBoundParameters.Remove('Force') | Out-Null
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
@@ -80,7 +85,7 @@ function Write-PipelineTaskError {
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if($ci) {
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
@@ -95,7 +100,8 @@ function Write-PipelineTaskError {
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if($ci) {
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
@@ -231,4 +237,4 @@ function Write-LogIssue {
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
}
+9 -2
View File
@@ -2,6 +2,7 @@
function Write-PipelineTelemetryError {
local telemetry_category=''
local force=false
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
@@ -11,6 +12,9 @@ function Write-PipelineTelemetryError {
telemetry_category=$2
shift
;;
-force|-f)
force=true
;;
-*)
function_args+=("$1 $2")
shift
@@ -22,19 +26,22 @@ function Write-PipelineTelemetryError {
shift
done
if [[ "$ci" != true ]]; then
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
if [[ $force == true ]]; then
function_args+=("-force")
fi
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ "$ci" != true ]]; then
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
@@ -2,26 +2,26 @@ param(
[Parameter(Mandatory=$true)][int] $BuildId,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
. $PSScriptRoot\post-build-utils.ps1
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that the channel we are going to promote the build to exist
$channelInfo = Get-MaestroChannel -ChannelId $ChannelId
if (!$channelInfo) {
Write-Host "Channel with BAR ID $ChannelId was not found in BAR!"
Write-PipelineTelemetryCategory -Category 'PromoteBuild' -Message "Channel with BAR ID $ChannelId was not found in BAR!"
ExitWithExitCode 1
}
# Get info about which channels the build has already been promoted to
# Get info about which channel(s) the build has already been promoted to
$buildInfo = Get-MaestroBuild -BuildId $BuildId
if (!$buildInfo) {
Write-Host "Build with BAR ID $BuildId was not found in BAR!"
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "Build with BAR ID $BuildId was not found in BAR!"
ExitWithExitCode 1
}
@@ -39,10 +39,10 @@ try {
Assign-BuildToChannel -BuildId $BuildId -ChannelId $ChannelId
Write-Host "done."
Write-Host 'done.'
}
catch {
Write-Host "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
Write-Host $_
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
ExitWithExitCode 1
}
+17 -8
View File
@@ -2,24 +2,34 @@ param(
[Parameter(Mandatory=$true)][int] $BarBuildId, # ID of the build which assets should be downloaded
[Parameter(Mandatory=$true)][string] $DropLocation, # Where the assets should be downloaded to
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken, # Token used to access Maestro API
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com", # Maestro API URL
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16" # Version of Maestro API to use
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com', # Maestro API URL
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16' # Version of Maestro API to use
)
. $PSScriptRoot\post-build-utils.ps1
try {
Write-Host "Installing DARC ..."
. $PSScriptRoot\post-build-utils.ps1
Write-Host 'Installing DARC ...'
. $PSScriptRoot\..\darc-init.ps1
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-PipelineTaskError "Something failed while running 'darc-init.ps1'. Check for errors above. Exiting now..."
Write-PipelineTelemetryError -Category "Darc" -Message "Something failed while running 'darc-init.ps1'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
# For now, only use a dry run.
# Ideally we would change darc to enable a quick request that
# would check whether the file exists that you can download it,
# and that it won't conflict with other files.
# https://github.com/dotnet/arcade/issues/3674
# Right now we can't remove continue-on-error because we ocassionally will have
# dependencies that have no associated builds (e.g. an old dependency).
# We need to add an option to baseline specific dependencies away, or add them manually
# to the BAR.
darc gather-drop --non-shipping `
--dry-run `
--continue-on-error `
--id $BarBuildId `
--output-dir $DropLocation `
@@ -28,8 +38,7 @@ try {
--latest-location
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "Darc" -Message $_
ExitWithExitCode 1
}
+5 -6
View File
@@ -6,20 +6,19 @@ param(
[Parameter(Mandatory=$true)][string] $ToolDestinationPath # Where the validation tool should be downloaded to
)
. $PSScriptRoot\post-build-utils.ps1
try {
$url = "https://raw.githubusercontent.com/NuGet/NuGetGallery/jver-verify/src/VerifyMicrosoftPackage/verify.ps1"
. $PSScriptRoot\post-build-utils.ps1
New-Item -ItemType "directory" -Path ${ToolDestinationPath} -Force
$url = 'https://raw.githubusercontent.com/NuGet/NuGetGallery/jver-verify/src/VerifyMicrosoftPackage/verify.ps1'
New-Item -ItemType 'directory' -Path ${ToolDestinationPath} -Force
Invoke-WebRequest $url -OutFile ${ToolDestinationPath}\verify.ps1
& ${ToolDestinationPath}\verify.ps1 ${PackagesPath}\*.nupkg
}
catch {
Write-PipelineTaskError "NuGet package validation failed. Please check error logs."
Write-Host $_
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NuGetValidation' -Message $_
ExitWithExitCode 1
}
+16 -15
View File
@@ -1,16 +1,17 @@
# Most of the functions in this file require the variables `MaestroApiEndPoint`,
# `MaestroApiVersion` and `MaestroApiAccessToken` to be globally available.
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
. $PSScriptRoot\..\tools.ps1
function Create-MaestroApiRequestHeaders([string]$ContentType = "application/json") {
function Create-MaestroApiRequestHeaders([string]$ContentType = 'application/json') {
Validate-MaestroVars
$headers = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
@@ -50,14 +51,6 @@ function Get-MaestroSubscriptions([string]$SourceRepository, [int]$ChannelId) {
return $result
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Validate-MaestroVars
@@ -66,24 +59,32 @@ function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Invoke-WebRequest -Method Post -Uri $apiEndpoint -Headers $apiHeaders | Out-Null
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Validate-MaestroVars {
try {
Get-Variable MaestroApiEndPoint -Scope Global | Out-Null
Get-Variable MaestroApiVersion -Scope Global | Out-Null
Get-Variable MaestroApiAccessToken -Scope Global | Out-Null
if (!($MaestroApiEndPoint -Match "^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$")) {
Write-PipelineTaskError "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
if (!($MaestroApiEndPoint -Match '^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
ExitWithExitCode 1
}
if (!($MaestroApiVersion -Match "^[0-9]{4}-[0-9]{2}-[0-9]{2}$")) {
Write-PipelineTaskError "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
if (!($MaestroApiVersion -Match '^[0-9]{4}-[0-9]{2}-[0-9]{2}$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
ExitWithExitCode 1
}
}
catch {
Write-PipelineTaskError "Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script."
Write-PipelineTelemetryError -Category 'MaestroVars' -Message 'Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script.'
Write-Host $_
ExitWithExitCode 1
}
@@ -1,26 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $ReleaseConfigsPath # Full path to ReleaseConfigs.txt asset
)
. $PSScriptRoot\post-build-utils.ps1
try {
$Content = Get-Content $ReleaseConfigsPath
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
Write-PipelineSetVariable -Name 'BARBuildId' -Value $BarId
Write-PipelineSetVariable -Name 'InitialChannels' -Value "$Channels"
Write-PipelineSetVariable -Name 'IsStableBuild' -Value $IsStableBuild
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
+66 -36
View File
@@ -1,8 +1,8 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit, # GitHub commit SHA used to build the packages
[Parameter(Mandatory=$false)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$false)][string] $GHCommit, # GitHub commit SHA used to build the packages
[Parameter(Mandatory=$true)][string] $SourcelinkCliVersion # Version of SourceLink CLI to use
)
@@ -13,6 +13,12 @@ param(
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
# Maximum number of jobs to run in parallel
$MaxParallelJobs = 6
# Wait time between check for system load
$SecondsBetweenLoadChecks = 10
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
@@ -22,15 +28,15 @@ $ValidatePackage = {
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
Write-Host "Input file does not exist: $PackagePath"
return 1
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
Write-Host -NoNewLine 'Validating ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
@@ -38,7 +44,7 @@ $ValidatePackage = {
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
[System.IO.Directory]::CreateDirectory($ExtractPath) | Out-Null
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
@@ -52,7 +58,7 @@ $ValidatePackage = {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
if ($FileName.EndsWith('.resources.dll')) {
return
}
@@ -90,7 +96,7 @@ $ValidatePackage = {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match "github" -or $Uri.Host -match "githubusercontent")) {
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match 'github' -or $Uri.Host -match 'githubusercontent')) {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
@@ -137,17 +143,19 @@ $ValidatePackage = {
}
if ($FailedFiles -eq 0) {
Write-Host "Passed."
Write-Host 'Passed.'
return 0
}
else {
Write-PipelineTaskError "$PackagePath has broken SourceLink links."
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$PackagePath has broken SourceLink links."
return 1
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
if (!($GHRepoName -Match "^[^\s-]+-[^\s]+$")) {
Write-PipelineTaskError "GHRepoName should be in the format <org>/<repo> or <org>-<repo>"
if ($GHRepoName -ne '' -and !($GHRepoName -Match '^[^\s\/]+/[^\s\/]+$')) {
if (!($GHRepoName -Match '^[^\s-]+-[^\s]+$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
ExitWithExitCode 1
}
else {
@@ -155,30 +163,33 @@ function ValidateSourceLinkLinks {
}
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-PipelineTaskError "GHCommit should be a 40 chars hexadecimal string"
if ($GHCommit -ne '' -and !($GHCommit -Match '^[0-9a-fA-F]{40}$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
ExitWithExitCode 1
}
$RepoTreeURL = -Join("http://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
if ($GHRepoName -ne '' -and $GHCommit -ne '') {
$RepoTreeURL = -Join('http://api.github.com/repos/', $GHRepoName, '/git/trees/', $GHCommit, '?recursive=1')
$CodeExtensions = @('.cs', '.vb', '.fs', '.fsi', '.fsx', '.fsscript')
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL -UseBasicParsing | ConvertFrom-Json | Select-Object -ExpandProperty tree
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL -UseBasicParsing | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL . Execution will proceed without caching."
}
}
catch {
Write-PipelineTaskError "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
Write-Host $_
ExitWithExitCode 1
elseif ($GHRepoName -ne '' -or $GHCommit -ne '') {
Write-Host 'For using the http caching mechanism both GHRepoName and GHCommit should be informed.'
}
if (Test-Path $ExtractPath) {
@@ -186,19 +197,38 @@ function ValidateSourceLinkLinks {
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName | Out-Null
$NumJobs = @(Get-Job -State 'Running').Count
while ($NumJobs -ge $MaxParallelJobs) {
Write-Host "There are $NumJobs validation jobs running right now. Waiting $SecondsBetweenLoadChecks seconds to check again."
sleep $SecondsBetweenLoadChecks
$NumJobs = @(Get-Job -State 'Running').Count
}
foreach ($Job in @(Get-Job -State 'Completed')) {
Receive-Job -Id $Job.Id
Remove-Job -Id $Job.Id
}
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
$ValidationFailures = 0
foreach ($Job in @(Get-Job)) {
$jobResult = Wait-Job -Id $Job.Id | Receive-Job
if ($jobResult -ne '0') {
$ValidationFailures++
}
}
if ($ValidationFailures -gt 0) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$ValidationFailures package(s) failed validation."
ExitWithExitCode 1
}
}
function InstallSourcelinkCli {
$sourcelinkCliPackageName = "sourcelink"
$sourcelinkCliPackageName = 'sourcelink'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
@@ -209,7 +239,7 @@ function InstallSourcelinkCli {
}
else {
Write-Host "Installing SourceLink CLI version $sourcelinkCliVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $sourcelinkCliPackageName --version $sourcelinkCliVersion --verbosity "minimal" --global
}
}
@@ -220,8 +250,8 @@ try {
ValidateSourceLinkLinks
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'SourceLink' -Message $_
ExitWithExitCode 1
}
+44 -45
View File
@@ -4,10 +4,6 @@ param(
[Parameter(Mandatory=$true)][string] $DotnetSymbolVersion # Version of dotnet symbol to use
)
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
@@ -23,39 +19,39 @@ function FirstMatchingSymbolDescriptionOrDefault {
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
$SymbolPath = $SymbolsPath + '\' + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
$PdbPath = $SymbolPath.Replace($Extension, '.pdb')
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
$NGenPdb = $SymbolPath.Replace($Extension, '.ni.pdb')
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
$SODbg = $SymbolPath.Replace($Extension, '.so.dbg')
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$DylibDwarf = $SymbolPath.Replace($Extension, '.dylib.dwarf')
$dotnetsymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetsymbolExe = Resolve-Path "$dotnetsymbolExe\dotnet-symbol.exe"
$dotnetSymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetSymbolExe = Resolve-Path "$dotnetSymbolExe\dotnet-symbol.exe"
& $dotnetsymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
& $dotnetSymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
return 'PDB'
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
return 'NGen PDB'
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
return 'DBG for SO'
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
return 'Dwarf for Dylib'
}
elseif (Test-Path $SymbolPath) {
return "Module"
return 'Module'
}
else {
return $null
@@ -74,7 +70,7 @@ function CountMissingSymbols {
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
$RelevantExtensions = @('.dll', '.exe', '.so', '.dylib')
# How many files are missing symbol information
$MissingSymbols = 0
@@ -82,38 +78,38 @@ function CountMissingSymbols {
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath 'Symbols'
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
if ($_.FullName -Match '\\ref\\') {
Write-Host "`t Ignoring reference assembly file " $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--microsoft-symbol-server' $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--internal-server' $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
Write-Host -NoNewLine "`t Checking file " $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
Write-Host 'No symbols found on MSDL or SymWeb!'
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
Write-Host 'No symbols found on MSDL!'
}
else {
Write-Host "No symbols found on SymWeb!"
Write-Host 'No symbols found on SymWeb!'
}
}
}
@@ -132,58 +128,61 @@ function CheckSymbolsAvailable {
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
if ($FileName -Match 'Microsoft\.DotNet\.Darc\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
elseif ($FileName -Match 'Microsoft\.DotNet\.Maestro\.Tasks\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-PipelineTaskError "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
}
Write-Host
}
}
function Installdotnetsymbol {
$dotnetsymbolPackageName = "dotnet-symbol"
function InstallDotnetSymbol {
$dotnetSymbolPackageName = 'dotnet-symbol'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$dotnetsymbolPackageName*") -and ($toolList -like "*$dotnetsymbolVersion*")) {
Write-Host "dotnet-symbol version $dotnetsymbolVersion is already installed."
if (($toolList -like "*$dotnetSymbolPackageName*") -and ($toolList -like "*$dotnetSymbolVersion*")) {
Write-Host "dotnet-symbol version $dotnetSymbolVersion is already installed."
}
else {
Write-Host "Installing dotnet-symbol version $dotnetsymbolVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
& "$dotnet" tool install $dotnetsymbolPackageName --version $dotnetsymbolVersion --verbosity "minimal" --global
Write-Host "Installing dotnet-symbol version $dotnetSymbolVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $dotnetSymbolPackageName --version $dotnetSymbolVersion --verbosity "minimal" --global
}
}
try {
Installdotnetsymbol
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
InstallDotnetSymbol
CheckSymbolsAvailable
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message $_
ExitWithExitCode 1
}
+50 -43
View File
@@ -2,56 +2,63 @@ param(
[Parameter(Mandatory=$true)][string] $SourceRepo,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
. $PSScriptRoot\post-build-utils.ps1
try {
. $PSScriptRoot\post-build-utils.ps1
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
if (!$subscriptions) {
Write-Host "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
if (!$subscriptions) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: $subscription.id"
[void]$subscriptionsToTrigger.Add($subscription.id)
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: ${$subscription.id}"
[void]$subscriptionsToTrigger.Add($subscription.id)
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host 'done.'
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message 'At least one subscription failed to be triggered...'
ExitWithExitCode 1
}
else {
Write-Host 'All subscriptions were triggered successfully!'
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host "done."
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-Host "At least one subscription failed to be triggered..."
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message $_
ExitWithExitCode 1
}
else {
Write-Host "All subscriptions were triggered successfully!"
}
+9 -10
View File
@@ -1,8 +1,8 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $configuration = "Debug",
[string] $configuration = 'Debug',
[string] $task,
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[string] $msbuildEngine = $null,
[switch] $restore,
[switch] $prepareMachine,
@@ -32,7 +32,7 @@ function Print-Usage() {
}
function Build([string]$target) {
$logSuffix = if ($target -eq "Execute") { "" } else { ".$target" }
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
@@ -46,33 +46,32 @@ function Build([string]$target) {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
if ($task -eq "") {
Write-Host "Missing required parameter '-task <value>'" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Missing required parameter '-task <value>'" -ForegroundColor Red
Print-Usage
ExitWithExitCode 1
}
$taskProject = GetSdkTaskProject $task
if (!(Test-Path $taskProject)) {
Write-Host "Unknown task: $task" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Unknown task: $task" -ForegroundColor Red
ExitWithExitCode 1
}
if ($restore) {
Build "Restore"
Build 'Restore'
}
Build "Execute"
Build 'Execute'
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+103 -92
View File
@@ -1,99 +1,110 @@
Param(
[string] $GuardianPackageName, # Required: the name of guardian CLI package (not needed if GuardianCliLocation is specified)
[string] $NugetPackageDirectory, # Required: directory where NuGet packages are installed (not needed if GuardianCliLocation is specified)
[string] $GuardianCliLocation, # Optional: Direct location of Guardian CLI executable if GuardianPackageName & NugetPackageDirectory are not specified
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_SOURCESDIRECTORY ("artifacts")), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
[bool] $TsaPublish=$False, # Optional: true will publish results to TSA; only set to true after onboarding to TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaBranchName=$env:BUILD_SOURCEBRANCH, # Optional: required for TSA publish; defaults to $(Build.SourceBranchName); TSA is the automated framework used to upload test results as bugs.
[string] $TsaRepositoryName=$env:BUILD_REPOSITORY_NAME, # Optional: TSA repository name; will be generated automatically if not submitted; TSA is the automated framework used to upload test results as bugs.
[string] $BuildNumber=$env:BUILD_BUILDNUMBER, # Optional: required for TSA publish; defaults to $(Build.BuildNumber)
[bool] $UpdateBaseline=$False, # Optional: if true, will update the baseline in the repository; should only be run after fixing any issues which need to be fixed
[bool] $TsaOnboard=$False, # Optional: if true, will onboard the repository to TSA; should only be run once; TSA is the automated framework used to upload test results as bugs.
[string] $TsaInstanceUrl, # Optional: only needed if TsaOnboard or TsaPublish is true; the instance-url registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the codebase registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaProjectName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the project registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaNotificationEmail, # Optional: only needed if TsaOnboard is true; the email(s) which will receive notifications of TSA bug filings (e.g. [email protected]); TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel="Standard", # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
[string] $GuardianPackageName, # Required: the name of guardian CLI package (not needed if GuardianCliLocation is specified)
[string] $NugetPackageDirectory, # Required: directory where NuGet packages are installed (not needed if GuardianCliLocation is specified)
[string] $GuardianCliLocation, # Optional: Direct location of Guardian CLI executable if GuardianPackageName & NugetPackageDirectory are not specified
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ('artifacts')), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
[bool] $TsaPublish=$False, # Optional: true will publish results to TSA; only set to true after onboarding to TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaBranchName=$env:BUILD_SOURCEBRANCH, # Optional: required for TSA publish; defaults to $(Build.SourceBranchName); TSA is the automated framework used to upload test results as bugs.
[string] $TsaRepositoryName=$env:BUILD_REPOSITORY_NAME, # Optional: TSA repository name; will be generated automatically if not submitted; TSA is the automated framework used to upload test results as bugs.
[string] $BuildNumber=$env:BUILD_BUILDNUMBER, # Optional: required for TSA publish; defaults to $(Build.BuildNumber)
[bool] $UpdateBaseline=$False, # Optional: if true, will update the baseline in the repository; should only be run after fixing any issues which need to be fixed
[bool] $TsaOnboard=$False, # Optional: if true, will onboard the repository to TSA; should only be run once; TSA is the automated framework used to upload test results as bugs.
[string] $TsaInstanceUrl, # Optional: only needed if TsaOnboard or TsaPublish is true; the instance-url registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the codebase registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaProjectName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the project registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaNotificationEmail, # Optional: only needed if TsaOnboard is true; the email(s) which will receive notifications of TSA bug filings (e.g. [email protected]); TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel='Standard', # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
try {
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
. $PSScriptRoot\..\tools.ps1
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path 'tools' 'guardian.cmd'))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Invalid Guardian CLI Location.'
ExitWithExitCode 1
}
& $(Join-Path $PSScriptRoot 'init-sdl.ps1') -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory '.gdn'
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not onboard to TSA -- not all required values ($TsaCodebaseName, $TsaNotificationEmail, $TsaCodebaseAdmin, $TsaBugAreaPath) were specified.'
ExitWithExitCode 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Update baseline'
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-publish failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not publish to TSA -- not all required values ($TsaBranchName, $BuildNumber) were specified.'
ExitWithExitCode 1
}
}
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path "tools" "guardian.cmd"))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
exit 1
}
& $(Join-Path $PSScriptRoot "init-sdl.ps1") -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $ArtifactsDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $ArtifactsDirectory ".gdn"
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $ArtifactsDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $ArtifactsDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not onboard to TSA -- not all required values ($$TsaCodebaseName, $$TsaNotificationEmail, $$TsaCodebaseAdmin, $$TsaBugAreaPath) were specified."
exit 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $ArtifactsDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $ArtifactsDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Update baseline"
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $SourceDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $ArtifactsDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-publish failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not publish to TSA -- not all required values ($$TsaBranchName, $$BuildNumber) were specified."
exit 1
}
}
+50 -40
View File
@@ -3,47 +3,16 @@ param(
[Parameter(Mandatory=$true)][string] $ExtractPath # Full path to directory where the packages will be extracted
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Extracting" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
}
finally {
$zip.Dispose()
}
}
function ExtractArtifacts {
function ExtractArtifacts {
if (!(Test-Path $InputPath)) {
Write-Host "Input Path does not exist: $InputPath"
ExitWithExitCode 0
@@ -60,11 +29,52 @@ $ExtractPackage = {
}
try {
. $PSScriptRoot\..\tools.ps1
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Extracting ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
finally {
$zip.Dispose()
}
}
Measure-Command { ExtractArtifacts }
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+28 -13
View File
@@ -1,16 +1,22 @@
Param(
[string] $GuardianCliLocation,
[string] $Repository,
[string] $BranchName="master",
[string] $BranchName='master',
[string] $WorkingDirectory,
[string] $AzureDevOpsAccessToken,
[string] $GuardianLoggerLevel="Standard"
[string] $GuardianLoggerLevel='Standard'
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
. $PSScriptRoot\..\tools.ps1
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
# Construct basic auth from AzDO access token; construct URI to the repository's gdn folder stored in that repository; construct location of zip file
$encodedPat = [Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$AzureDevOpsAccessToken"))
$escapedRepository = [Uri]::EscapeDataString("/$Repository/$BranchName/.gdn")
@@ -18,11 +24,10 @@ $uri = "https://dev.azure.com/dnceng/internal/_apis/git/repositories/sdl-tool-cf
$zipFile = "$WorkingDirectory/gdn.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
$gdnFolder = (Join-Path $WorkingDirectory ".gdn")
Try
{
$gdnFolder = (Join-Path $WorkingDirectory '.gdn')
try {
# We try to download the zip; if the request fails (e.g. the file doesn't exist), we catch it and init guardian instead
Write-Host "Downloading gdn folder from internal config repostiory..."
Write-Host 'Downloading gdn folder from internal config repostiory...'
Invoke-WebRequest -Headers @{ "Accept"="application/zip"; "Authorization"="Basic $encodedPat" } -Uri $uri -OutFile $zipFile
if (Test-Path $gdnFolder) {
# Remove the gdn folder if it exists (it shouldn't unless there's too much caching; this is just in case)
@@ -30,19 +35,29 @@ Try
}
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipFile, $WorkingDirectory)
Write-Host $gdnFolder
} Catch [System.Net.WebException] {
ExitWithExitCode 0
} catch [System.Net.WebException] { } # Catch and ignore webexception
try {
# if the folder does not exist, we'll do a guardian init and push it to the remote repository
Write-Host "Initializing Guardian..."
Write-Host 'Initializing Guardian...'
Write-Host "$GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel"
& $GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian init failed with exit code $LASTEXITCODE."
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian init failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We create the mainbaseline so it can be edited later
Write-Host "$GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline"
& $GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian baseline failed with exit code $LASTEXITCODE."
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian baseline failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
& $(Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Initialize gdn folder"
}
& $(Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Initialize gdn folder'
ExitWithExitCode 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+2 -2
View File
@@ -1,4 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Guardian.Cli" version="0.6.0"/>
<package id="Microsoft.Guardian.Cli" version="0.7.2"/>
</packages>
+53 -39
View File
@@ -1,51 +1,65 @@
Param(
[string] $Repository,
[string] $BranchName="master",
[string] $BranchName='master',
[string] $GdnFolder,
[string] $AzureDevOpsAccessToken,
[string] $PushReason
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP "sdl"
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
try {
. $PSScriptRoot\..\tools.ps1
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-Error "Git clone failed with exit code $LASTEXITCODE."
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) ".gdn"
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host "git add ."
git add .
if ($LASTEXITCODE -ne 0) {
Write-Error "Git add failed with exit code $LASTEXITCODE."
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-Error "Git commit failed with exit code $LASTEXITCODE."
}
Write-Host "git push"
git push
if ($LASTEXITCODE -ne 0) {
Write-Error "Git push failed with exit code $LASTEXITCODE."
}
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP 'sdl'
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
# Return to the original directory
Pop-Location
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git clone failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) '.gdn'
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host 'git add .'
git add .
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git add failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git commit failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host 'git push'
git push
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git push failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# Return to the original directory
Pop-Location
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+47 -45
View File
@@ -5,63 +5,65 @@ Param(
[string] $GdnFolder,
[string[]] $ToolsList,
[string] $UpdateBaseline,
[string] $GuardianLoggerLevel="Standard",
[string] $GuardianLoggerLevel='Standard',
[string[]] $CrScanAdditionalRunConfigParams,
[string[]] $PoliCheckAdditionalRunConfigParams
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder "r"
$ValidPath = Test-Path $GuardianCliLocation
try {
. $PSScriptRoot\..\tools.ps1
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
exit 1
}
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder 'r'
$ValidPath = Test-Path $GuardianCliLocation
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
$config = $False
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq "credscan") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory : $TargetDirectory `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory : $TargetDirectory " $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
$config = $True
}
if ($tool -eq "policheck") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target : $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target : $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
$config = $True
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Invalid Guardian CLI Location."
ExitWithExitCode 1
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel --config $gdnConfigFile $config"
if ($config) {
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel --config $gdnConfigFile
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian run for $tool using $gdnConfigFile failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
$configParam = @('--config')
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq 'credscan') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
} else {
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian run for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
if ($tool -eq 'policheck') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
$configParam+=$gdnConfigFile
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+52 -17
View File
@@ -1,35 +1,70 @@
parameters:
enable: 'false' # Whether the SDL validation job should execute or not
overrideParameters: '' # Optional: to override values for parameters.
additionalParameters: '' # Optional: parameters that need user specific values eg: '-SourceToolsList @("abc","def") -ArtifactToolsList @("ghi","jkl")'
continueOnError: false # optional: determines whether to continue the build if the step errors;
# There is some sort of bug (has been reported) in Azure DevOps where if this parameter is named
# 'continueOnError', the parameter value is not correctly picked up.
# This can also be remedied by the caller (post-build.yml) if it does not use a nested parameter
sdlContinueOnError: false # optional: determines whether to continue the build if the step errors;
dependsOn: '' # Optional: dependencies of the job
artifactNames: '' # Optional: patterns supplied to DownloadBuildArtifacts
# Usage:
# artifactNames:
# - 'BlobArtifacts'
# - 'Artifacts_Windows_NT_Release'
jobs:
- job: Run_SDL
dependsOn: ${{ parameters.dependsOn }}
displayName: Run SDL tool
condition: eq( ${{ parameters.enable }}, 'true')
variables:
- group: DotNet-VSTS-Bot
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
name: Hosted VS2017
steps:
- checkout: self
clean: true
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: current
downloadType: specific files
matchingPattern: "**"
downloadPath: $(Build.SourcesDirectory)\artifacts
- ${{ if ne(parameters.artifactNames, '') }}:
- ${{ each artifactName in parameters.artifactNames }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: ${{ artifactName }}
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if eq(parameters.artifactNames, '') }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: specific files
itemPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.SourcesDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.SourcesDirectory)\artifacts\BlobArtifacts
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
displayName: Extract Blob Artifacts
continueOnError: ${{ parameters.continueOnError }}
continueOnError: ${{ parameters.sdlContinueOnError }}
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.SourcesDirectory)\artifacts\PackageArtifacts
-ExtractPath $(Build.SourcesDirectory)\artifacts\PackageArtifacts
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
displayName: Extract Package Artifacts
continueOnError: ${{ parameters.continueOnError }}
continueOnError: ${{ parameters.sdlContinueOnError }}
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
- task: NuGetCommand@2
@@ -43,12 +78,12 @@ jobs:
- ${{ if ne(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1 ${{ parameters.overrideParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.continueOnError }}
continueOnError: ${{ parameters.sdlContinueOnError }}
- ${{ if eq(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1
-GuardianPackageName Microsoft.Guardian.Cli.0.6.0
-GuardianPackageName Microsoft.Guardian.Cli.0.7.2
-NugetPackageDirectory $(Build.SourcesDirectory)\.packages
-AzureDevOpsAccessToken $(dn-bot-dotnet-build-rw-code-rw)
${{ parameters.additionalParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.continueOnError }}
continueOnError: ${{ parameters.sdlContinueOnError }}
+75 -64
View File
@@ -1,64 +1,33 @@
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
parameters:
# Job schema parameters - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
cancelTimeoutInMinutes: ''
condition: ''
continueOnError: false
container: ''
continueOnError: false
dependsOn: ''
displayName: ''
steps: []
pool: ''
steps: []
strategy: ''
timeoutInMinutes: ''
variables: []
workspace: ''
# Job base template specific parameters
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
# See schema documentation - https://github.com/dotnet/arcade/blob/master/Documentation/AzureDevOps/TemplateSchema.md
artifacts: ''
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
enableTelemetry: false
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: define the helix type for telemetry (example: 'build/product/')
helixType: ''
# Required: name of the job
enablePublishUsingPipelines: false
name: ''
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
preSteps: []
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- job: ${{ parameters.name }}
@@ -90,7 +59,7 @@ jobs:
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- ${{ if ne(parameters.enableTelemetry, 'false') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- ${{ each variable in parameters.variables }}:
@@ -122,21 +91,12 @@ jobs:
workspace: ${{ parameters.workspace }}
steps:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendStartTelemetry@0
displayName: 'Send Helix Start Telemetry'
inputs:
helixRepo: ${{ parameters.helixRepo }}
${{ if ne(parameters.helixType, '') }}:
helixType: ${{ parameters.helixType }}
buildConfig: $(_BuildConfig)
runAsPublic: ${{ parameters.runAsPublic }}
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if ne(parameters.preSteps, '') }}:
- ${{ each preStep in parameters.preSteps }}:
- ${{ preStep }}
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
@@ -148,6 +108,16 @@ jobs:
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- task: NuGetAuthenticate@0
- ${{ if or(eq(parameters.artifacts.download, 'true'), ne(parameters.artifacts.download, '')) }}:
- task: DownloadPipelineArtifact@2
inputs:
buildType: current
artifactName: ${{ coalesce(parameters.artifacts.download.name, 'Artifacts_$(Agent.OS)_$(_BuildConfig)') }}
targetPath: ${{ coalesce(parameters.artifacts.download.path, 'artifacts') }}
itemPattern: ${{ coalesce(parameters.artifacts.download.pattern, '**') }}
- ${{ each step in parameters.steps }}:
- ${{ step }}
@@ -160,20 +130,60 @@ jobs:
env:
TeamName: $(_TeamName)
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendEndTelemetry@0
displayName: 'Send Helix End Telemetry'
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if ne(parameters.artifacts.publish, '') }}:
- ${{ if or(eq(parameters.artifacts.publish.artifacts, 'true'), ne(parameters.artifacts.publish.artifacts, '')) }}:
- task: CopyFiles@2
displayName: Gather binaries for publish to artifacts
inputs:
SourceFolder: 'artifacts/bin'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/bin'
- task: CopyFiles@2
displayName: Gather packages for publish to artifacts
inputs:
SourceFolder: 'artifacts/packages'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/packages'
- task: PublishBuildArtifacts@1
displayName: Publish pipeline artifacts
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.logs, 'true'), ne(parameters.artifacts.publish.logs, '')) }}:
- publish: artifacts/log
artifact: ${{ coalesce(parameters.artifacts.publish.logs.name, 'Logs_Build_$(Agent.Os)_$(_BuildConfig)') }}
displayName: Publish logs
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- ${{ if and(ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.ArtifactStagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if ne(parameters.enablePublishBuildArtifacts, 'false') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_$(Agent.JobName)
ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
continueOnError: true
condition: always()
@@ -187,7 +197,7 @@ jobs:
continueOnError: true
condition: always()
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(variables['_PublishUsingPipelines'], 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
@@ -195,6 +205,7 @@ jobs:
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
+4 -2
View File
@@ -5,6 +5,7 @@ parameters:
displayName: '' # optional -- display name for the job. Will use jobName if not passed
pool: '' # required -- name of the Build pool
container: '' # required -- name of the container
osGroup: '' # required -- operating system for the job
extraSetupParameters: '' # optional -- extra arguments to pass to the setup script
frameworks: ['netcoreapp3.0'] # optional -- list of frameworks to run against
continueOnError: 'false' # optional -- determines whether to continue the build if the step errors
@@ -44,12 +45,13 @@ jobs:
- HelixPreCommand: ''
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(variables['Agent.Os'], 'Windows_NT') }}:
- ${{ if eq( parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'set "PERFLAB_UPLOAD_TOKEN=$(PerfCommandUploadToken)"'
- IsInternal: -Internal
- ${{ if ne(variables['Agent.Os'], 'Windows_NT') }}:
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'export PERFLAB_UPLOAD_TOKEN="$(PerfCommandUploadTokenLinux)"'
- IsInternal: --internal
- group: DotNet-HelixApi-Access
- group: dotnet-benchview
@@ -47,6 +47,10 @@ jobs:
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: NuGetAuthenticate@0
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
@@ -59,6 +63,7 @@ jobs:
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: powershell@2
displayName: Create ReleaseConfigs Artifact
inputs:
@@ -67,12 +72,14 @@ jobs:
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(BARBuildId)
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value "$(DefaultChannels)"
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(IsStableBuild)
- task: PublishBuildArtifacts@1
displayName: Publish ReleaseConfigs Artifact
inputs:
PathtoPublish: '$(Build.StagingDirectory)/ReleaseConfigs.txt'
PublishLocation: Container
ArtifactName: ReleaseConfigs
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
+28 -46
View File
@@ -1,19 +1,10 @@
parameters:
# Optional: 'true' if failures in job.yml job should not fail the job
# See schema documentation in /Documentation/AzureDevOps/TemplateSchema.md
continueOnError: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Enable publishing using release pipelines
enablePublishUsingPipelines: false
@@ -23,19 +14,9 @@ parameters:
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
# if enabled then the 'helixRepo' parameter should also be specified
enableTelemetry: false
# Required: A collection of jobs to run - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
jobs: []
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: Override automatically derived dependsOn value for "publish build assets" job
publishBuildAssetsDependsOn: ''
@@ -62,29 +43,30 @@ jobs:
name: ${{ job.job }}
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if and(eq(parameters.graphFileGeneration.enabled, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if eq(parameters.graphFileGeneration.enabled, true) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
@@ -0,0 +1,171 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
/p:PublishToMSDL=false
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=true
/p:ChecksumsTargetStaticFeed=$(InternalChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(InternalChecksumsBlobFeedKey)
/p:InstallersTargetStaticFeed=$(InternalInstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(InternalInstallersBlobFeedKey)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:PublishToMSDL=false
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -0,0 +1,170 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,147 +0,0 @@
parameters:
enableSymbolValidation: true
stages:
- stage: IS_Publish
dependsOn: validate
variables:
- template: ../common-variables.yml
displayName: Internal Servicing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.InternalServicing_30_Channel_Id)
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Artifacts
inputs:
downloadType: specific files
matchingPattern: "*Artifacts*"
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:Configuration=Release
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.InternalServicing_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: PowerShell@2
displayName: Add Assets Location
env:
AZURE_DEVOPS_EXT_PAT: $(dn-bot-dnceng-unviersal-packages-rw)
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ChannelId=$(InternalServicing_30_Channel_Id)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:AzureStorageAccountName=$(ProxyBackedFeedsAccountName)
/p:AzureStorageAccountKey=$(dotnetfeed-storage-access-key-1)
/p:AzureDevOpsFeedsBaseUrl=$(dotnetfeed-internal-private-feed-url)
/p:StaticInternalFeed=$(dotnetfeed-internal-nonstable-feed-url)
/p:NugetPath=$(Agent.BuildDirectory)\Nuget\NuGet.exe
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)\BlobArtifacts'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)\PackageArtifacts'
/p:Configuration=Release
- task: NuGetCommand@2
displayName: Publish Packages to AzDO Feed
condition: contains(variables['TargetAzDOFeed'], 'pkgs.visualstudio.com')
inputs:
command: push
vstsFeed: $(AzDoFeedName)
packagesToPush: $(Build.ArtifactStagingDirectory)\PackageArtifacts\*.nupkg
publishVstsFeed: $(AzDoFeedName)
- task: PowerShell@2
displayName: Publish Blobs to AzDO Feed
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/publish-blobs-to-azdo.ps1
arguments: -FeedName $(AzDoFeedName)
-SourceFolderCollection $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-PersonalAccessToken $(dn-bot-dnceng-unviersal-packages-rw)
enabled: false
- template: ../trigger-subscription.yml
parameters:
ChannelId: ${{ variables.InternalServicing_30_Channel_Id }}
- stage: IS_PublishValidation
displayName: Publish Validation
variables:
- template: ../common-variables.yml
jobs:
- template: ../setup-maestro-vars.yml
- ${{ if eq(parameters.enableSymbolValidation, 'true') }}:
- job:
displayName: Symbol Availability
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.InternalServicing_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Check Symbol Availability
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/symbols-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/ -ExtractPath $(Agent.BuildDirectory)/Temp/ -DotnetSymbolVersion $(SymbolToolVersion)
- template: ../promote-build.yml
parameters:
ChannelId: ${{ variables.InternalServicing_30_Channel_Id }}
@@ -1,148 +0,0 @@
parameters:
enableSymbolValidation: true
stages:
- stage: Publish
dependsOn: validate
variables:
- template: ../common-variables.yml
displayName: Developer Channel
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicDevRelease_30_Channel_Id)
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Artifacts
inputs:
downloadType: specific files
matchingPattern: "*Artifacts*"
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:Configuration=Release
- job:
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicDevRelease_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: PowerShell@2
displayName: Add Assets Location
env:
AZURE_DEVOPS_EXT_PAT: $(dn-bot-dnceng-unviersal-packages-rw)
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ChannelId=$(PublicDevRelease_30_Channel_Id)
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(Agent.BuildDirectory)\Nuget\NuGet.exe
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-unviersal-packages-rw)'
/p:TargetFeedPAT='$(dn-bot-dnceng-unviersal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
- task: NuGetCommand@2
displayName: Publish Packages to AzDO Feed
condition: contains(variables['TargetAzDOFeed'], 'pkgs.visualstudio.com')
inputs:
command: push
vstsFeed: $(AzDoFeedName)
packagesToPush: $(Build.ArtifactStagingDirectory)\PackageArtifacts\*.nupkg
publishVstsFeed: $(AzDoFeedName)
- task: PowerShell@2
displayName: Publish Blobs to AzDO Feed
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/publish-blobs-to-azdo.ps1
arguments: -FeedName $(AzDoFeedName)
-SourceFolderCollection $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-PersonalAccessToken $(dn-bot-dnceng-unviersal-packages-rw)
enabled: false
- stage: PublishValidation
displayName: Publish Validation
variables:
- template: ../common-variables.yml
jobs:
- template: ../setup-maestro-vars.yml
- ${{ if eq(parameters.enableSymbolValidation, 'true') }}:
- job:
displayName: Symbol Availability
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicDevRelease_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Check Symbol Availability
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/symbols-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/ -ExtractPath $(Agent.BuildDirectory)/Temp/ -DotnetSymbolVersion $(SymbolToolVersion)
- template: ../darc-gather-drop.yml
parameters:
ChannelId: ${{ variables.PublicDevRelease_30_Channel_Id }}
- template: ../promote-build.yml
parameters:
ChannelId: ${{ variables.PublicDevRelease_30_Channel_Id }}
@@ -1,147 +0,0 @@
parameters:
enableSymbolValidation: true
stages:
- stage: PubRel_Publish
dependsOn: validate
variables:
- template: ../common-variables.yml
displayName: Public Release
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicRelease_30_Channel_Id)
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Artifacts
inputs:
downloadType: specific files
matchingPattern: "*Artifacts*"
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:Configuration=Release
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicRelease_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: PowerShell@2
displayName: Publish
env:
AZURE_DEVOPS_EXT_PAT: $(dn-bot-dnceng-unviersal-packages-rw)
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ChannelId=$(PublicRelease_30_Channel_Id)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:AzureStorageAccountName=$(ProxyBackedFeedsAccountName)
/p:AzureStorageAccountKey=$(dotnetfeed-storage-access-key-1)
/p:AzureDevOpsFeedsBaseUrl=$(dotnetfeed-internal-private-feed-url)
/p:StaticInternalFeed=$(dotnetfeed-internal-nonstable-feed-url)
/p:NugetPath=$(Agent.BuildDirectory)\Nuget\NuGet.exe
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)\BlobArtifacts'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)\PackageArtifacts'
/p:Configuration=Release
- task: NuGetCommand@2
displayName: Publish Packages to AzDO Feed
condition: contains(variables['TargetAzDOFeed'], 'pkgs.visualstudio.com')
inputs:
command: push
vstsFeed: $(AzDoFeedName)
packagesToPush: $(Build.ArtifactStagingDirectory)\PackageArtifacts\*.nupkg
publishVstsFeed: $(AzDoFeedName)
- task: PowerShell@2
displayName: Publish Blobs to AzDO Feed
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/publish-blobs-to-azdo.ps1
arguments: -FeedName $(AzDoFeedName)
-SourceFolderCollection $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-PersonalAccessToken $(dn-bot-dnceng-unviersal-packages-rw)
enabled: false
- template: ../trigger-subscription.yml
parameters:
ChannelId: ${{ variables.PublicRelease_30_Channel_Id }}
- stage: PubRel_PublishValidation
displayName: Publish Validation
variables:
- template: ../common-variables.yml
jobs:
- template: ../setup-maestro-vars.yml
- ${{ if eq(parameters.enableSymbolValidation, 'true') }}:
- job:
displayName: Symbol Availability
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicRelease_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Check Symbol Availability
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/symbols-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/ -ExtractPath $(Agent.BuildDirectory)/Temp/ -DotnetSymbolVersion $(SymbolToolVersion)
- template: ../promote-build.yml
parameters:
ChannelId: ${{ variables.PublicRelease_30_Channel_Id }}
@@ -1,99 +0,0 @@
stages:
- stage: PVR_Publish
dependsOn: validate
variables:
- template: ../common-variables.yml
displayName: Validation Channel
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], variables.PublicValidationRelease_30_Channel_Id)
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: PowerShell@2
displayName: Add Assets Location
env:
AZURE_DEVOPS_EXT_PAT: $(dn-bot-dnceng-unviersal-packages-rw)
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ChannelId=$(PublicValidationRelease_30_Channel_Id)
/p:ArtifactsCategory=$(_DotNetValidationArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(Agent.BuildDirectory)\Nuget\NuGet.exe
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-unviersal-packages-rw)'
/p:TargetFeedPAT='$(dn-bot-dnceng-unviersal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)\BlobArtifacts'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)\PackageArtifacts'
/p:Configuration=Release
- task: NuGetCommand@2
displayName: Publish Packages to AzDO Feed
condition: contains(variables['TargetAzDOFeed'], 'pkgs.visualstudio.com')
inputs:
command: push
vstsFeed: $(AzDoFeedName)
packagesToPush: $(Build.ArtifactStagingDirectory)\PackageArtifacts\*.nupkg
publishVstsFeed: $(AzDoFeedName)
- task: PowerShell@2
displayName: Publish Blobs to AzDO Feed
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/publish-blobs-to-azdo.ps1
arguments: -FeedName $(AzDoFeedName)
-SourceFolderCollection $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-PersonalAccessToken $(dn-bot-dnceng-unviersal-packages-rw)
enabled: false
- stage: PVR_PublishValidation
displayName: Publish Validation
variables:
- template: ../common-variables.yml
jobs:
- template: ../setup-maestro-vars.yml
- template: ../darc-gather-drop.yml
parameters:
ChannelId: ${{ variables.PublicValidationRelease_30_Channel_Id }}
- template: ../promote-build.yml
parameters:
ChannelId: ${{ variables.PublicValidationRelease_30_Channel_Id }}
@@ -1,14 +1,34 @@
variables:
- group: AzureDevOps-Artifact-Feeds-Pats
- group: DotNet-Blob-Feed
- group: DotNet-DotNetCli-Storage
- group: DotNet-MSRC-Storage
- group: Publish-Build-Assets
# .NET Core 3.1 Dev
- name: PublicDevRelease_31_Channel_Id
value: 128
# .NET Core 3 Dev
- name: PublicDevRelease_30_Channel_Id
value: 3
# .NET Core 5 Dev
- name: NetCore_5_Dev_Channel_Id
value: 131
# .NET Tools - Validation
- name: PublicValidationRelease_30_Channel_Id
# .NET Eng - Validation
- name: Net_Eng_Validation_Channel_Id
value: 9
# .NET Eng - Latest
- name: Net_Eng_Latest_Channel_Id
value: 2
# .NET 3 Eng - Validation
- name: NET_3_Eng_Validation_Channel_Id
value: 390
# .NET 3 Eng
- name: NetCore_3_Tools_Channel_Id
value: 344
# .NET Core 3.0 Internal Servicing
- name: InternalServicing_30_Channel_Id
value: 184
@@ -17,14 +37,26 @@ variables:
- name: PublicRelease_30_Channel_Id
value: 19
# .NET Core 3.1 Release
- name: PublicRelease_31_Channel_Id
value: 129
# General Testing
- name: GeneralTesting_Channel_Id
value: 529
# .NET Core 3.1 Blazor Features
- name: NetCore_31_Blazor_Features_Channel_Id
value: 531
# .NET Core Experimental
- name: NetCore_Experimental_Channel_Id
value: 562
# Whether the build is internal or not
- name: IsInternalBuild
value: ${{ and(ne(variables['System.TeamProject'], 'public'), contains(variables['Build.SourceBranch'], 'internal')) }}
# Storage account name for proxy-backed feeds
- name: ProxyBackedFeedsAccountName
value: dotnetfeed
# Default Maestro++ API Endpoint and API Version
- name: MaestroApiEndPoint
value: "https://maestro-prod.westus2.cloudapp.azure.com"
@@ -37,3 +69,24 @@ variables:
value: 3.0.0
- name: SymbolToolVersion
value: 1.0.1
# Feed Configurations
# These should include the suffix "/index.json"
# Default locations for Installers and checksums
# Public Locations
- name: ChecksumsBlobFeedUrl
value: https://dotnetclichecksums.blob.core.windows.net/dotnet/index.json
- name: InstallersBlobFeedUrl
value: https://dotnetcli.blob.core.windows.net/dotnet/index.json
# Private Locations
- name: InternalChecksumsBlobFeedUrl
value: https://dotnetclichecksumsmsrc.blob.core.windows.net/dotnet/index.json
- name: InternalChecksumsBlobFeedKey
value: $(dotnetclichecksumsmsrc-storage-key)
- name: InternalInstallersBlobFeedUrl
value: https://dotnetclimsrc.blob.core.windows.net/dotnet/index.json
- name: InternalInstallersBlobFeedKey
value: $(dotnetclimsrc-access-key)
@@ -5,7 +5,7 @@ jobs:
- job: gatherDrop
displayName: Gather Drop
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], ${{ parameters.ChannelId }})
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.ChannelId }}))
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
+265 -71
View File
@@ -1,94 +1,288 @@
parameters:
enableSourceLinkValidation: true
enableSourceLinkValidation: false
enableSigningValidation: true
enableSymbolValidation: true
enableSymbolValidation: false
enableNugetValidation: true
publishInstallersAndChecksums: false
SDLValidationParameters:
enable: false
continueOnError: false
params: ''
artifactNames: ''
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
symbolPublishingAdditionalParameters: ''
artifactsPublishingAdditionalParameters: ''
signingValidationAdditionalParameters: ''
# Which stages should finish execution before post-build stages start
dependsOn: [build]
validateDependsOn:
- build
publishDependsOn:
- Validate
stages:
- stage: validate
dependsOn: ${{ parameters.dependsOn }}
- stage: Validate
dependsOn: ${{ parameters.validateDependsOn }}
displayName: Validate
variables:
- template: common-variables.yml
jobs:
- ${{ if eq(parameters.enableNugetValidation, 'true') }}:
- job:
displayName: NuGet Validation
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- template: setup-maestro-vars.yml
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- job:
displayName: NuGet Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableNugetValidation }}, 'true')
pool:
vmImage: 'windows-2019'
variables:
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
- ${{ if eq(parameters.enableSigningValidation, 'true') }}:
- job:
displayName: Signing Validation
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:Configuration=Release
- job:
displayName: Signing Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSigningValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
- ${{ if eq(parameters.enableSourceLinkValidation, 'true') }}:
- job:
displayName: SourceLink Validation
variables:
- template: common-variables.yml
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- ${{ if eq(parameters.SDLValidationParameters.enable, 'true') }}:
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
${{ parameters.signingValidationAdditionalParameters }}
- template: \eng\common\templates\post-build\channels\public-dev-release.yml
- template: ../steps/publish-logs.yml
parameters:
StageLabel: 'Validation'
JobLabel: 'Signing'
- job:
displayName: SourceLink Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSourceLinkValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
enable: ${{ parameters.SDLValidationParameters.enable }}
dependsOn: setupMaestroVars
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
artifactNames: ${{ parameters.SDLValidationParameters.artifactNames }}
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
enableSymbolValidation: ${{ parameters.enableSymbolValidation }}
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev5_Publish'
channelName: '.NET Core 5 Dev'
channelId: 131
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\public-validation-release.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Latest_Publish'
channelName: '.NET Eng - Latest'
channelId: 2
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\public-release.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Validation_Publish'
channelName: '.NET Eng - Validation'
channelId: 9
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\internal-servicing.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'General_Testing_Publish'
channelName: 'General Testing'
channelId: 529
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Dev_Publishing'
channelName: '.NET Core Tooling Dev'
channelId: 548
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Release_Publishing'
channelName: '.NET Core Tooling Release'
channelId: 549
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NET_Internal_Tooling_Publishing'
channelName: '.NET Internal Tooling'
channelId: 551
transportFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Experimental_Publishing'
channelName: '.NET Core Experimental'
channelId: 562
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Int_Publish'
channelName: '.NET Eng Services - Int'
channelId: 678
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Prod_Publish'
channelName: '.NET Eng Services - Prod'
channelId: 679
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
@@ -1,25 +0,0 @@
parameters:
ChannelId: 0
jobs:
- job:
displayName: Promote Build
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], ${{ parameters.ChannelId }})
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: ChannelId
value: ${{ parameters.ChannelId }}
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/promote-build.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId $(ChannelId)
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,11 +1,20 @@
jobs:
- job: setupMaestroVars
displayName: Setup Maestro Vars
variables:
- template: common-variables.yml
- name: BuildId
value: $[ coalesce(variables.BARBuildId, 0) ]
- name: PromoteToChannelId
value: $[ coalesce(variables.PromoteToMaestroChannelId, 0) ]
pool:
vmImage: 'windows-2019'
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
displayName: Download Release Configs
condition: eq(variables.PromoteToChannelId, 0)
inputs:
buildType: current
artifactName: ReleaseConfigs
@@ -14,5 +23,59 @@ jobs:
name: setReleaseVars
displayName: Set Release Configs Vars
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/setup-maestro-vars.ps1
arguments: -ReleaseConfigsPath '$(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt'
targetType: inline
script: |
try {
if ($Env:PromoteToChannelId -eq 0) {
$Content = Get-Content $(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
$AzureDevOpsProject = $Env:System_TeamProject
$AzureDevOpsBuildDefinitionId = $Env:System_DefinitionId
$AzureDevOpsBuildId = $Env:Build_BuildId
$PromoteToMaestroChannelId = 0
}
else {
$buildApiEndpoint = "${Env:MaestroApiEndPoint}/api/builds/${Env:BARBuildId}?api-version=${Env:MaestroApiVersion}"
$apiHeaders = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$apiHeaders.Add('Accept', 'application/json')
$apiHeaders.Add('Authorization',"Bearer ${Env:MAESTRO_API_TOKEN}")
$buildInfo = try { Invoke-WebRequest -Method Get -Uri $buildApiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
$BarId = $Env:BARBuildId
$Channels = 'None'
#TODO: Fix this once this issue is done: https://github.com/dotnet/arcade/issues/3834
$IsStableBuild = 'False'
$AzureDevOpsProject = $buildInfo.azureDevOpsProject
$AzureDevOpsBuildDefinitionId = $buildInfo.azureDevOpsBuildDefinitionId
$AzureDevOpsBuildId = $buildInfo.azureDevOpsBuildId
$PromoteToMaestroChannelId = $Env:PromoteToMaestroChannelId
}
Write-Host "##vso[task.setvariable variable=BARBuildId;isOutput=true]$BarId"
Write-Host "##vso[task.setvariable variable=InitialChannels;isOutput=true]$Channels"
Write-Host "##vso[task.setvariable variable=IsStableBuild;isOutput=true]$IsStableBuild"
Write-Host "##vso[task.setvariable variable=AzDOProjectName;isOutput=true]$AzureDevOpsProject"
Write-Host "##vso[task.setvariable variable=AzDOPipelineId;isOutput=true]$AzureDevOpsBuildDefinitionId"
Write-Host "##vso[task.setvariable variable=AzDOBuildId;isOutput=true]$AzureDevOpsBuildId"
Write-Host "##vso[task.setvariable variable=PromoteToMaestroChannelId;isOutput=true]$PromoteToMaestroChannelId"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
exit 1
}
env:
MAESTRO_API_TOKEN: $(MaestroApiAccessToken)
@@ -0,0 +1,13 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/add-build-to-channel.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -0,0 +1,23 @@
parameters:
StageLabel: ''
JobLabel: ''
steps:
- task: Powershell@2
displayName: Prepare Binlogs to Upload
inputs:
targetType: inline
script: |
New-Item -ItemType Directory $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
Move-Item -Path $(Build.SourcesDirectory)/artifacts/log/Debug/* $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
continueOnError: true
condition: always()
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/PostBuildLogs'
PublishLocation: Container
ArtifactName: PostBuildLogs
continueOnError: true
condition: always()
@@ -23,6 +23,7 @@ parameters:
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
HelixBaseUri: 'https://helix.dot.net/' # optional -- sets the Helix API base URI (allows targeting int)
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
@@ -55,6 +56,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
@@ -85,6 +87,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
+187 -70
View File
@@ -5,7 +5,7 @@
[bool]$ci = if (Test-Path variable:ci) { $ci } else { $false }
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { "Debug" }
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { 'Debug' }
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
@@ -24,7 +24,7 @@
[bool]$restore = if (Test-Path variable:restore) { $restore } else { $true }
# Adjusts msbuild verbosity level.
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { "minimal" }
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { 'minimal' }
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
[bool]$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { !$ci }
@@ -39,19 +39,25 @@
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.ps1
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { 'v1' }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @('msbuild', 'dotnet', 'vbcscompiler') }
$disableConfigureToolsetImport = if (Test-Path variable:disableConfigureToolsetImport) { $disableConfigureToolsetImport } else { $null }
set-strictmode -version 2.0
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
function Create-Directory([string[]] $path) {
if (!(Test-Path $path)) {
New-Item -path $path -force -itemType "Directory" | Out-Null
New-Item -path $path -force -itemType 'Directory' | Out-Null
}
}
@@ -92,7 +98,10 @@ function Exec-Process([string]$command, [string]$commandArgs) {
}
}
function InitializeDotNetCli([bool]$install) {
# createSdkLocationFile parameter enables a file being generated under the toolset directory
# which writes the sdk's location into. This is only necessary for cmd --> powershell invocations
# as dot sourcing isn't possible.
function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
if (Test-Path variable:global:_DotNetInstallDir) {
return $global:_DotNetInstallDir
}
@@ -115,7 +124,7 @@ function InitializeDotNetCli([bool]$install) {
# Find the first path on %PATH% that contains the dotnet.exe
if ($useInstalledDotNetCli -and (-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -eq $null)) {
$dotnetCmd = Get-Command "dotnet.exe" -ErrorAction SilentlyContinue
$dotnetCmd = Get-Command 'dotnet.exe' -ErrorAction SilentlyContinue
if ($dotnetCmd -ne $null) {
$env:DOTNET_INSTALL_DIR = Split-Path $dotnetCmd.Path -Parent
}
@@ -128,13 +137,13 @@ function InitializeDotNetCli([bool]$install) {
if ((-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -ne $null) -and (Test-Path(Join-Path $env:DOTNET_INSTALL_DIR "sdk\$dotnetSdkVersion"))) {
$dotnetRoot = $env:DOTNET_INSTALL_DIR
} else {
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
if (-not (Test-Path(Join-Path $dotnetRoot "sdk\$dotnetSdkVersion"))) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
ExitWithExitCode 1
}
}
@@ -142,6 +151,24 @@ function InitializeDotNetCli([bool]$install) {
$env:DOTNET_INSTALL_DIR = $dotnetRoot
}
# Creates a temporary file under the toolset dir.
# The following code block is protecting against concurrent access so that this function can
# be called in parallel.
if ($createSdkLocationFile) {
do {
$sdkCacheFileTemp = Join-Path $ToolsetDir $([System.IO.Path]::GetRandomFileName())
}
until (!(Test-Path $sdkCacheFileTemp))
Set-Content -Path $sdkCacheFileTemp -Value $dotnetRoot
try {
Rename-Item -Force -Path $sdkCacheFileTemp 'sdk.txt'
} catch {
# Somebody beat us
Remove-Item -Path $sdkCacheFileTemp
}
}
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
# It also ensures that VS msbuild will use the downloaded sdk targets.
@@ -149,6 +176,7 @@ function InitializeDotNetCli([bool]$install) {
# Make Sure that our bootstrapped dotnet cli is available in future steps of the Azure Pipelines build
Write-PipelinePrependPath -Path $dotnetRoot
Write-PipelineSetVariable -Name 'DOTNET_MULTILEVEL_LOOKUP' -Value '0'
Write-PipelineSetVariable -Name 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE' -Value '1'
@@ -156,20 +184,54 @@ function InitializeDotNetCli([bool]$install) {
}
function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
$installScript = Join-Path $dotnetRoot 'dotnet-install.ps1'
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
Invoke-WebRequest "https://dot.net/v1/dotnet-install.ps1" -OutFile $installScript
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
$maxRetries = 5
$retries = 1
$uri = "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1"
while($true) {
try {
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
break
}
catch {
Write-Host "Failed to download '$uri'"
Write-Error $_.Exception.Message -ErrorAction Continue
}
if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
throw "Unable to download file in $maxRetries attempts."
}
}
}
return $installScript
}
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = "") {
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = '') {
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) {
function InstallDotNet([string] $dotnetRoot,
[string] $version,
[string] $architecture = '',
[string] $runtime = '',
[bool] $skipNonVersionedFiles = $false,
[string] $runtimeSourceFeed = '',
[string] $runtimeSourceFeedKey = '') {
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
@@ -180,10 +242,32 @@ function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $archit
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Failed to install dotnet cli (exit code '$lastExitCode')."
ExitWithExitCode $lastExitCode
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from public location."
# Only the runtime can be installed from a custom [private] location.
if ($runtime -and ($runtimeSourceFeed -or $runtimeSourceFeedKey)) {
if ($runtimeSourceFeed) { $installParameters.AzureFeed = $runtimeSourceFeed }
if ($runtimeSourceFeedKey) {
$decodedBytes = [System.Convert]::FromBase64String($runtimeSourceFeedKey)
$decodedString = [System.Text.Encoding]::UTF8.GetString($decodedBytes)
$installParameters.FeedCredential = $decodedString
}
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from custom location '$runtimeSourceFeed'."
ExitWithExitCode 1
}
} else {
ExitWithExitCode 1
}
}
}
@@ -204,16 +288,16 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { '15.9' }
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
$msbuildCmd = Get-Command "msbuild.exe" -ErrorAction SilentlyContinue
$msbuildCmd = Get-Command 'msbuild.exe' -ErrorAction SilentlyContinue
if ($msbuildCmd -ne $null) {
# Workaround for https://github.com/dotnet/roslyn/issues/35793
# Due to this issue $msbuildCmd.Version returns 0.0.0.0 for msbuild.exe 16.2+
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split(@('-', '+'))[0])
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split([char[]]@('-', '+'))[0])
if ($msbuildVersion -ge $vsMinVersion) {
return $global:_MSBuildExe = $msbuildCmd.Path
@@ -233,7 +317,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
InitializeVisualStudioEnvironmentVariables $vsInstallDir $vsMajorVersion
} else {
if (Get-Member -InputObject $GlobalJson.tools -Name "xcopy-msbuild") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'xcopy-msbuild') {
$xcopyMSBuildVersion = $GlobalJson.tools.'xcopy-msbuild'
$vsMajorVersion = $xcopyMSBuildVersion.Split('.')[0]
} else {
@@ -241,9 +325,12 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
$xcopyMSBuildVersion = "$vsMajorVersion.$($vsMinVersion.Minor).0-alpha"
}
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
$vsInstallDir = $null
if ($xcopyMSBuildVersion.Trim() -ine "none") {
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
}
if ($vsInstallDir -eq $null) {
throw "Unable to find Visual Studio that has required version and components installed"
throw 'Unable to find Visual Studio that has required version and components installed'
}
}
@@ -267,7 +354,7 @@ function InstallXCopyMSBuild([string]$packageVersion) {
}
function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
$packageName = "RoslynTools.MSBuild"
$packageName = 'RoslynTools.MSBuild'
$packageDir = Join-Path $ToolsDir "msbuild\$packageVersion"
$packagePath = Join-Path $packageDir "$packageName.$packageVersion.nupkg"
@@ -278,11 +365,12 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
Create-Directory $packageDir
Write-Host "Downloading $packageName $packageVersion"
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dotnet.myget.org/F/roslyn-tools/api/v2/package/$packageName/$packageVersion/" -OutFile $packagePath
Unzip $packagePath $packageDir
}
return Join-Path $packageDir "tools"
return Join-Path $packageDir 'tools'
}
#
@@ -299,32 +387,32 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
if (Get-Member -InputObject $GlobalJson.tools -Name "vswhere") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'vswhere') {
$vswhereVersion = $GlobalJson.tools.vswhere
} else {
$vswhereVersion = "2.5.2"
$vswhereVersion = '2.5.2'
}
$vsWhereDir = Join-Path $ToolsDir "vswhere\$vswhereVersion"
$vsWhereExe = Join-Path $vsWhereDir "vswhere.exe"
$vsWhereExe = Join-Path $vsWhereDir 'vswhere.exe'
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host "Downloading vswhere"
Write-Host 'Downloading vswhere'
Invoke-WebRequest "https://github.com/Microsoft/vswhere/releases/download/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild", "-products", "*")
$args = @('-latest', '-prerelease', '-format', 'json', '-requires', 'Microsoft.Component.MSBuild', '-products', '*')
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
if (Get-Member -InputObject $vsRequirements -Name 'version') {
$args += '-version'
$args += $vsRequirements.version
}
if (Get-Member -InputObject $vsRequirements -Name "components") {
if (Get-Member -InputObject $vsRequirements -Name 'components') {
foreach ($component in $vsRequirements.components) {
$args += "-requires"
$args += '-requires'
$args += $component
}
}
@@ -350,28 +438,27 @@ function InitializeBuildTool() {
# Initialize dotnet cli if listed in 'tools'
$dotnetRoot = $null
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
$dotnetRoot = InitializeDotNetCli -install:$restore
}
if ($msbuildEngine -eq "dotnet") {
if ($msbuildEngine -eq 'dotnet') {
if (!$dotnetRoot) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "/global.json must specify 'tools.dotnet'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "/global.json must specify 'tools.dotnet'."
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
$buildTool = @{ Path = Join-Path $dotnetRoot 'dotnet.exe'; Command = 'msbuild'; Tool = 'dotnet'; Framework = 'netcoreapp2.1' }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
ExitWithExitCode 1
}
@@ -380,15 +467,15 @@ function InitializeBuildTool() {
function GetDefaultMSBuildEngine() {
# Presence of tools.vs indicates the repo needs to build using VS msbuild on Windows.
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
if (Get-Member -InputObject $GlobalJson.tools -Name 'vs') {
return 'vs'
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
return 'dotnet'
}
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
ExitWithExitCode 1
}
@@ -397,9 +484,9 @@ function GetNuGetPackageCachePath() {
# Use local cache on CI to ensure deterministic build,
# use global cache in dev builds to avoid cost of downloading packages.
if ($useGlobalNuGetCache) {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile ".nuget\packages"
$env:NUGET_PACKAGES = Join-Path $env:UserProfile '.nuget\packages'
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot ".packages"
$env:NUGET_PACKAGES = Join-Path $RepoRoot '.packages'
}
}
@@ -412,7 +499,7 @@ function GetSdkTaskProject([string]$taskName) {
}
function InitializeNativeTools() {
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
if (-Not (Test-Path variable:DisableNativeToolsetInstalls) -And (Get-Member -InputObject $GlobalJson -Name "native-tools")) {
$nativeArgs= @{}
if ($ci) {
$nativeArgs = @{
@@ -441,14 +528,14 @@ function InitializeToolset() {
}
if (-not $restore) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Toolset version $toolsetVersion has not been restored."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 1
}
$buildTool = InitializeBuildTool
$proj = Join-Path $ToolsetDir "restore.proj"
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "ToolsetRestore.binlog") } else { "" }
$proj = Join-Path $ToolsetDir 'restore.proj'
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'ToolsetRestore.binlog') } else { '' }
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Set-Content $proj
@@ -470,7 +557,7 @@ function ExitWithExitCode([int] $exitCode) {
}
function Stop-Processes() {
Write-Host "Killing running build processes..."
Write-Host 'Killing running build processes...'
foreach ($processName in $processesToStopOnExit) {
Get-Process -Name $processName -ErrorAction SilentlyContinue | Stop-Process
}
@@ -484,9 +571,21 @@ function Stop-Processes() {
function MSBuild() {
if ($pipelinesLog) {
$buildTool = InitializeBuildTool
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if ($ci -and $buildTool.Tool -eq 'dotnet') {
dotnet nuget locals http-cache -c
$env:NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS = 20
$env:NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS = 20
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS' -Value '20'
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS' -Value '20'
}
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
$path = Join-Path $path (Join-Path $buildTool.Framework 'Microsoft.DotNet.Arcade.Sdk.dll')
$args += "/logger:$path"
}
@@ -501,12 +600,12 @@ function MSBuild() {
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
Write-PipelineTaskError -Message "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Binary log must be enabled in CI build.'
ExitWithExitCode 1
}
if ($nodeReuse) {
Write-PipelineTaskError -Message "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Node reuse must be disabled in CI build.'
ExitWithExitCode 1
}
}
@@ -516,7 +615,10 @@ function MSBuild-Core() {
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
$cmdArgs += ' /warnaserror /p:TreatWarningsAsErrors=true'
}
else {
$cmdArgs += ' /p:TreatWarningsAsErrors=false'
}
foreach ($arg in $args) {
@@ -528,7 +630,7 @@ function MSBuild-Core() {
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-PipelineTaskError -Message "Build failed."
Write-PipelineTelemetryError Category 'Build' -Message 'Build failed.'
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
@@ -543,12 +645,12 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
if ($arg.StartsWith('/bl:', "OrdinalIgnoreCase")) {
return $arg.Substring('/bl:'.Length)
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
if ($arg.StartsWith('/binaryLogger:', 'OrdinalIgnoreCase')) {
return $arg.Substring('/binaryLogger:'.Length)
}
}
}
@@ -558,14 +660,14 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
. $PSScriptRoot\pipeline-logging-functions.ps1
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$ToolsetDir = Join-Path $ArtifactsDir "toolset"
$ToolsDir = Join-Path $RepoRoot ".tools"
$LogDir = Join-Path (Join-Path $ArtifactsDir "log") $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir "tmp") $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot "global.json") | ConvertFrom-Json
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$ArtifactsDir = Join-Path $RepoRoot 'artifacts'
$ToolsetDir = Join-Path $ArtifactsDir 'toolset'
$ToolsDir = Join-Path $RepoRoot '.tools'
$LogDir = Join-Path (Join-Path $ArtifactsDir 'log') $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir 'tmp') $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot 'global.json') | ConvertFrom-Json
# true if global.json contains a "runtimes" section
$globalJsonHasRuntimes = if ($GlobalJson.tools.PSObject.Properties.Name -Match 'runtimes') { $true } else { $false }
@@ -578,3 +680,18 @@ Write-PipelineSetVariable -Name 'Artifacts.Toolset' -Value $ToolsetDir
Write-PipelineSetVariable -Name 'Artifacts.Log' -Value $LogDir
Write-PipelineSetVariable -Name 'TEMP' -Value $TempDir
Write-PipelineSetVariable -Name 'TMP' -Value $TempDir
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
if (!$disableConfigureToolsetImport) {
$configureToolsetScript = Join-Path $EngRoot 'configure-toolset.ps1'
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
if ((Test-Path variable:failOnConfigureToolsetError) -And $failOnConfigureToolsetError) {
if ((Test-Path variable:LastExitCode) -And ($LastExitCode -ne 0)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'configure-toolset.ps1 returned a non-zero exit code'
ExitWithExitCode $LastExitCode
}
}
}
}
+101 -16
View File
@@ -41,10 +41,14 @@ fi
# Configures warning treatment in msbuild.
warn_as_error=${warn_as_error:-true}
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.sh
dotnetInstallScriptVersion=${dotnetInstallScriptVersion:-'v1'}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
if [[ "$ci" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
@@ -77,7 +81,7 @@ function ReadGlobalVersion {
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Error: Cannot find \"$key\" in $global_json_file"
Write-PipelineTelemetryError -category 'Build' "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
@@ -168,7 +172,7 @@ function InstallDotNetSdk {
function InstallDotNet {
local root=$1
local version=$2
GetDotNetInstallScript "$root"
local install_script=$_GetDotNetInstallScript
@@ -187,15 +191,59 @@ function InstallDotNet {
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from public location (exit code '$exit_code')."
if [[ -n "$runtimeArg" ]]; then
local runtimeSourceFeed=''
if [[ -n "${6:-}" ]]; then
runtimeSourceFeed="--azure-feed $6"
fi
local runtimeSourceFeedKey=''
if [[ -n "${7:-}" ]]; then
decodedFeedKey=`echo $7 | base64 --decode`
runtimeSourceFeedKey="--feed-credential $decodedFeedKey"
fi
if [[ -n "$runtimeSourceFeed" || -n "$runtimeSourceFeedKey" ]]; then
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from custom location '$runtimeSourceFeed' (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
ExitWithExitCode $exit_code
fi
fi
}
}
function with_retries {
local maxRetries=5
local retries=1
echo "Trying to run '$@' for maximum of $maxRetries attempts."
while [[ $((retries++)) -le $maxRetries ]]; do
"$@"
if [[ $? == 0 ]]; then
echo "Ran '$@' successfully."
return 0
fi
timeout=$((2**$retries-1))
echo "Failed to execute '$@'. Waiting $timeout seconds before next attempt ($retries out of $maxRetries)." 1>&2
sleep $timeout
done
echo "Failed to execute '$@' for $maxRetries times." 1>&2
return 1
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
local install_script_url="https://dot.net/v1/dotnet-install.sh"
local install_script_url="https://dot.net/$dotnetInstallScriptVersion/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
@@ -204,12 +252,19 @@ function GetDotNetInstallScript {
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script"
with_retries curl "$install_script_url" -isSLv --retry 10 --create-dirs -o "$install_script" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
wget -q -O "$install_script" "$install_script_url"
with_retries wget -v -O "$install_script" "$install_script_url" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
fi
fi
# return value
_GetDotNetInstallScript="$install_script"
}
@@ -218,11 +273,11 @@ function InitializeBuildTool {
if [[ -n "${_InitializeBuildTool:-}" ]]; then
return
fi
InitializeDotNetCli $restore
# return values
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildToolCommand="msbuild"
_InitializeBuildToolFramework="netcoreapp2.1"
}
@@ -241,6 +296,9 @@ function GetNuGetPackageCachePath {
}
function InitializeNativeTools() {
if [[ -n "${DisableNativeToolsetInstalls:-}" ]]; then
return
fi
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
@@ -283,14 +341,14 @@ function InitializeToolset {
if [[ "$binary_log" == true ]]; then
bl="/bl:$log_dir/ToolsetRestore.binlog"
fi
echo '<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' > "$proj"
MSBuild-Core "$proj" $bl /t:__WriteToolsetLocation /clp:ErrorsOnly\;NoSummary /p:__ToolsetLocationOutputFile="$toolset_location_file"
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Invalid toolset path: $toolset_build_proj"
Write-PipelineTelemetryError -category 'Build' "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
@@ -317,6 +375,18 @@ function MSBuild {
if [[ "$pipelines_log" == true ]]; then
InitializeBuildTool
InitializeToolset
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if [[ "$ci" == true ]]; then
"$_InitializeBuildTool" nuget locals http-cache -c
export NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS=20
export NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS=20
Write-PipelineSetVariable -name "NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS" -value "20"
Write-PipelineSetVariable -name "NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS" -value "20"
fi
local toolset_dir="${_InitializeToolset%/*}"
local logger_path="$toolset_dir/$_InitializeBuildToolFramework/Microsoft.DotNet.Arcade.Sdk.dll"
args=( "${args[@]}" "-logger:$logger_path" )
@@ -328,12 +398,12 @@ function MSBuild {
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
Write-PipelineTaskError "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
Write-PipelineTaskError "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
@@ -347,7 +417,7 @@ function MSBuild-Core {
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
Write-PipelineTaskError "Build failed (exit code '$exit_code')."
Write-PipelineTelemetryError -category 'Build' "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
@@ -388,3 +458,18 @@ Write-PipelineSetVariable -name "Artifacts.Toolset" -value "$toolset_dir"
Write-PipelineSetVariable -name "Artifacts.Log" -value "$log_dir"
Write-PipelineSetVariable -name "Temp" -value "$temp_dir"
Write-PipelineSetVariable -name "TMP" -value "$temp_dir"
# Import custom tools configuration, if present in the repo.
if [ -z "${disable_configure_toolset_import:-}" ]; then
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
View File
File renamed without changes.
-51
View File
@@ -1,51 +0,0 @@
<Project DefaultTargets="PublishPackages">
<!-- TODO: move properties imported from here into a common props file -->
<!-- <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dir.props))\dir.props" /> -->
<!-- Use an explicit SDK import so that arcade uses the build
configuration computed in dir.common.props. -->
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
<Import Project="$(MSBuildThisFileDirectory)\..\.packages\microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<PropertyGroup>
<!-- Set the TargetFramework just to make the SDK happy -->
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<Target Name="PublishPackages">
<ItemGroup>
<PackagesToPublish Include="$(MSBuildThisFileDirectory)\..\artifacts\*.nupkg" />
</ItemGroup>
<ItemGroup>
<ItemsToPushToBlobFeed Include="@(PackagesToPublish)" />
</ItemGroup>
<Error Condition=" '$(AzureFeedUrl)' == '' " Text="AzureFeedUrl must be set" />
<Error Condition=" '$(AzureAccountKey)' == '' " Text="AzureAccountKey must be set" />
<Error Condition=" '$(BUILD_REPOSITORY_URI)' == '' " Text="BUILD_REPOSITORY_URI must be set" />
<Error Condition=" '$(BUILD_SOURCEBRANCH)' == '' " Text="BUILD_SOURCEBRANCH must be set" />
<Error Condition=" '$(BUILD_BUILDNUMBER)' == '' " Text="BUILD_BUILDNUMBER must be set" />
<Error Condition=" '$(BUILD_SOURCEVERSION)' == '' " Text="BUILD_SOURCEVERSION must be set" />
<PropertyGroup>
<AssetManifestFilePath>$(ArtifactsLogDir)AssetManifest\$(AssetManifest)</AssetManifestFilePath>
</PropertyGroup>
<Message Importance="Low" Text="BUILD_REPOSITORY_URI: $(BUILD_REPOSITORY_URI)" />
<PushToBlobFeed ExpectedFeedUrl="$(AzureFeedUrl)"
AccountKey="$(AzureAccountKey)"
ItemsToPush="@(ItemsToPushToBlobFeed)"
ManifestBuildData="Location=$(AzureFeedUrl)"
ManifestRepoUri="$(BUILD_REPOSITORY_URI)"
ManifestBranch="$(BUILD_SOURCEBRANCH)"
ManifestBuildId="$(BUILD_BUILDNUMBER)"
ManifestCommit="$(BUILD_SOURCEVERSION)"
AssetManifestPath="$(AssetManifestFilePath)" />
</Target>
</Project>
Vendored Submodule
+1
Submodule external/bdwgc added at 6e8db07c8d.
+1 -1
+1 -1
+1 -1
+1 -1
-1
Submodule external/llvm deleted from 29f36fd315.
Vendored Submodule
+1
Submodule external/llvm-project added at 0a3f49c40d.
+3 -3
View File
@@ -1,9 +1,9 @@
{
"tools": {
"dotnet": "5.0.100-alpha1-013628"
"dotnet": "5.0.100-alpha1-015772"
},
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "1.0.0-beta.19379.1",
"Microsoft.DotNet.Helix.Sdk": "2.0.0-beta.19379.1"
"Microsoft.DotNet.Arcade.Sdk": "5.0.0-beta.20109.1",
"Microsoft.DotNet.Helix.Sdk": "5.0.0-beta.20109.1"
}
}
-4
View File
@@ -1,4 +0,0 @@
ltconfig crlf=input
install-sh crlf=input
/BCC_MAKEFILE -crlf
/digimars.mak -crlf
Loaded 100 of 2778 files, more files were not shown because too many files have changed in this diff. Show more