Compare commits

...
1569 Commits
Author SHA1 Message Date
Bepis 7328415ac5 Marshal extension is finally working 2020-09-04 22:41:37 +10:00
Bepis 20a03cdb8e Separate mono_delegate_to_ftnptr_impl into new method 2020-09-04 21:45:16 +10:00
Bepis 5f805b6b03 Fix extension parameter placement and convention overwriting 2020-09-04 20:07:17 +10:00
Bepis 4205769da9 Implement custom GetFunctionPointerForDelegateInternal extension 2020-09-04 19:34:03 +10:00
Zoltan Varga 5756e65a5c [runtime] Allocate code memory using MAP_JIT on apple silicon, patching code randomly crashes without it. (#20342) 2020-09-03 17:56:24 -04:00
monojenkinsandvargaz dfd7ba8ff6 Add ios support to the cmake build system. (#20333)
Co-authored-by: vargaz <[email protected]>
2020-09-03 19:57:08 +02:00
Vlad Brezae 623fc84874 [interp] Optimize out conditional branches (#20330)
* [interp] Remove unop conditional branch applied to constant

Convert it to unconditional branch

* [interp] Enable more optimization

If we failed to emit stloc.np we no longer track stack contents or try other optimizations

* [interp] Fix emit_store_value_as_local

mint_type for a native type is i4/i8 while we were loading the address of this local using LDLOC_VT. This was wrong. Passing a valuetype this is done with LDLOCA.

* [interp] Fix incorrect cond branch opcodes

We didn't emit correct conditional branch opcode, because we didn't check the new stack types, after the addition of the implicit conversion opcodes.

* [interp] Remove binop conditional branch applied to constant

Convert it to unconditional branch
2020-09-02 00:01:38 +03:00
monojenkinsandCoffeeFlux f6d04a53f9 [mono] Add mono_image_open_from_data_alc to unstable embedding API (#20303)
Xamarin-Android will need this for the net6 migration, so might as well add it before I forget about it. They currently use `mono_image_open_from_data_with_name`. cc: @grendello

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-31 14:53:45 -04:00
monojenkinsandradical e5cb8c4e34 [wasm][debugger] Disable an extraneous debug message (#20317)
Recent debugger test runs were showing lot of
`CWL: Failed to lookup sequence point` messages.

These are being shown for cases like:

`CWL: list_frames: Failed to lookup sequence point. method: runtime_invoke_direct_void, native_offset: 56`

This is a warning, and doesn't need to be emitted by default.

Co-authored-by: radical <[email protected]>
2020-08-31 17:10:47 +02:00
csanchezdll 59ee4edb6e Ignore EINVAL errors on ioctl TIOCMGET/TIOCMSET so (#20219)
pseudo-ttys are usable. (#20218)
2020-08-31 15:15:09 +02:00
Stefanos Kornilios Mitsis Poiitidis c58000534f mono: result of regmask should be stored in regmask_t, not int (#20322) 2020-08-31 15:13:09 +02:00
jeffgoku 30689421ab Fix potential crash for Encoder.Convert (#20271)
GetByteCount will not return the right number of bytes needed, so GetBytes in Encoder.Convert will crash for some input
2020-08-31 15:08:58 +02:00
Eberhard BeilharzandMarkS 728b1e9258 [resgen] Honour absence of -usesourcepath (#20247)
* Only set basepath to the source directory if useSourcePath is set.
* Fixes a regression from mono 4 that happened when changing the code
  to not use reflection.

Co-authored-by: MarkS <[email protected]>
2020-08-31 11:38:39 +02:00
Leslie Zhai d71eff1541 [MIPS32] Fix mono_arch_get_cie_program and ELF_MACHINE undefined issue (#20323) 2020-08-30 12:17:16 -04:00
imhameed 8e75f5a28e [mono] Squelch Mach exception handlers (#20321)
Work around LLDB's unconditionally-installed and inflexible Mach
exception handler by installing a no-op task-wide Mach exception
handler.

See also:
- https://github.com/openjdk/jdk/blob/50cc54c730bb0f130991648d1547d42483edddc1/src/hotspot/os/bsd/os_bsd.cpp#L2810-L2817
- https://github.com/mono/mono/issues/10180#issuecomment-683119507
2020-08-29 11:23:09 -07:00
monojenkinsandvargaz cd226d8452 [runtime] Fix signature of mini_cleanup (). (#20320)
Fixes https://github.com/dotnet/runtime/issues/41470.

Co-authored-by: vargaz <[email protected]>
2020-08-29 04:05:06 -04:00
Alexander Köplinger 481e856031 Revert "[mono] Convert Environment.CurrentManagedThreadId to an icall (#20313)" (#20319)
This reverts commit 5d30ccbafc.
2020-08-28 11:05:29 +02:00
monojenkinsandEgorBo 5d30ccbafc [mono] Convert Environment.CurrentManagedThreadId to an icall (#20313)
In order to avoid the static Thread initialization (System.Thread.CurrentThread inits a static field) see https://github.com/dotnet/runtime/pull/41360

Benchmark:
```csharp
[Benchmark]
public int CurrentThreadId() => Environment.CurrentManagedThreadId;
```

Mono JIT (benchmarked on a slow machine):
```
       |          Method |     Mean |     Error |    StdDev |
       |---------------- |---------:|----------:|----------:|
master | CurrentThreadId | 7.191 ns | 0.0128 ns | 0.0107 ns |
PR     | CurrentThreadId | 5.034 ns | 0.0395 ns | 0.0350 ns |
```

Co-authored-by: EgorBo <[email protected]>
2020-08-28 10:09:43 +03:00
HinTakandRyan Lucia b1b58ccc0d Issue18826 fix (#19580)
* Look for "xmonkeysloveplay" also at the end of LC_SYMTAB section, for signed Mac OS X binaries

This extends Miguel de Icaza's previous work on allowing mono to load embedded
assemblies / shared libraries, which was in turn a workaround on recent
Mac OS X's increased security (and stopping LD_LIBRARY_PATH and friends from
working). Miguel de Icaza came up with the idea of appending all of those
embedded assemblies / shared libraries to the end of mono's main executable,
and getting it to look at its own file and unpacking resources from an offset
number + "xmonkeysloveplay" written to the end of file.

Even more recent Mac OS X starts to prefer /require binaries to be code-signed.
Code signing appends a "LC_CODE_SIGNATURE" section to the binary, thus breaking
mono's embedded resource-loading. Also code-signing requires that the binary
has no unknown parts.

To make mkbundle / mono embedded resource-loading work in a code-signing
situation, two parts are required:

- adjust mkbundle to extend the LC_SYMTAB section to cover the appended
assemblies / shared libraries, so code-signing can happen.
( addresses https://github.com/mono/mono/issues/17881 )

- adjust mono's main executable to also look at the end of the LC_SYMTAB
section for the magic offset +  "xmonkeysloveplay" token to find the embedded
resources.
( addresses https://github.com/mono/mono/issues/18826 )

This change addresses the 2nd of the above. I also tried looking for the
presence and beginnig of the LC_CODE_SIGNATURE section, but that proves
unreliable: Code-signing can pad a few null bytes after "xmonkeysloveplay"
to align the LC_CODE_SIGNATURE section to an 8(?)-byte boundary.

* saving offset before read, so later lseek does not need to backtrack on the read

This is a small simplification of the previous commit on this part of code.

* fixes wrong drop-through from mis-positioning of #endif

* Update mono/mini/main.c

Adding spaces for clarity

Co-authored-by: Ryan Lucia <[email protected]>

* Update mono/mini/main.c

Adding spaces for clarity

Co-authored-by: Ryan Lucia <[email protected]>

* Update mono/mini/main.c

Adding spaces for clarity

Co-authored-by: Ryan Lucia <[email protected]>

* Update mono/mini/main.c

Adding spaces for clarity

Co-authored-by: Ryan Lucia <[email protected]>

* Update mono/mini/main.c

Added comment about what's going on.

Co-authored-by: Ryan Lucia <[email protected]>

* Adding comments for clarity

* rename variable "h" to "bin_header", for clarity

* Limit Apple code-signing work around to only the desktop OS (vs iDevices/TV)

* remove stylistic "else"; no change in functionality

Co-authored-by: Ryan Lucia <[email protected]>
2020-08-27 11:38:38 -04:00
monojenkinsandCoffeeFlux 123ac44eb3 [mono] ALC function restructuring for unloadability (#20306)
We previously had a weird split between the domain and loader headers/files. This makes fewer things public, moves the useful ALC functions to loader-internals.h with the implementation in assembly-load-context.c, and cleans up the flow for some unloading behavior.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-27 10:56:34 -04:00
monojenkinsandvargaz 6f23092e7c Add support for using cmake in the mono runtime build. (#20309)
Not enabled by default. Ninja is used for building if available, else we fall
back to make.

Co-authored-by: vargaz <[email protected]>
2020-08-27 10:14:06 -04:00
Thays Grazia 26ae896ecb [wasm][debugger] Merge dotnet/runtime to mono/mono (#20307)
* Merge dotnet/runtime.

* Missing file.
2020-08-26 14:14:07 -03:00
monojenkinsandCoffeeFlux 990f516a4b [mono] Add mono_assembly_decref, return value on both decref and addref (#20305)
Needed for unloadability

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-26 10:00:03 -04:00
monojenkinsandCoffeeFlux ad95c02c2a [mono] Enable additional warnings as errors (#20293)
This PR:
* Enables `-Wc++-compat` on all platforms: we support building the runtime as C++, so this helps catch errors before they're found in CI and will help keep the netcore build C++-compatible
* Sets `-Werror` on netcore CI for OSX

It also fixes various warnings currently present in the build.

`-Werror` should probably be extended to cover all CI platforms eventually. Adding a new platform is as simple as adding to the line in mono.proj whitelisting platforms and fixing the warnings.

Not sure if there's a better way to handle the cpuidex function, but if we're using that from mini it should definitely be in a header somewhere.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-25 17:55:37 -04:00
Alexander Köplinger d543f5e540 Update README.md 2020-08-25 22:13:18 +02:00
Alexander Köplinger d724f744bc Replace gitter links with discord 2020-08-25 22:00:57 +02:00
imhameedandAlexis Christoforides 1f3fbedc39 [runtime] Catch attempts to create enum types with an underlying type that is itself an incomplete enum type. (#20282)
* Catch attempts to create enum types with an underlying type that is itself an incomplete enum type.

Previously, only non-enum types containing fields with incomplete enum
types were forbidden.

* Permit static constant fields with incomplete types if the incomplete type is the type currently being defined

Co-authored-by: Alexis Christoforides <[email protected]>
2020-08-25 10:42:44 -07:00
monojenkinsandlateralusX 45d996b7ba Port EventPipe CoreCLR changes to Mono. (#20278)
Port changes done in the following CoreCLR PR's affecting corresponding Mono runtime EventPipe library sources:

https://github.com/dotnet/runtime/pull/36720
https://github.com/dotnet/runtime/pull/37002
https://github.com/dotnet/runtime/pull/38967
https://github.com/dotnet/runtime/pull/40191
https://github.com/dotnet/runtime/pull/40332
https://github.com/dotnet/runtime/pull/40499

EventPipe native code changes track by https://github.com/dotnet/runtime/issues/36820.

Co-authored-by: lateralusX <[email protected]>
2020-08-25 08:43:03 +02:00
imhameed 8aeab70720 Avoid following invalid pointers in mono_w32process_get_modules on Darwin. (#20277)
* Avoid following invalid pointers in mono_w32process_get_modules on Darwin.

`_dyld_get_image_name` and `_dyld_get_image_header` can return pointers
to data that will be freed or unmapped by concurrent calls to `dlclose`.
Avoid this by using `_dyld_register_func_for_add_image` and
`_dyld_register_func_for_remove_image` to install callbacks that
maintain a runtime-internal set of loaded dynamic libraries.

Some notes:

- dyld doesn't provide a public "app store friendly" API (like
`dl_iterate_phdr`) to traverse a snapshot of all loaded dynamic
libraries in a process. There are two alternative private APIs that do
this: one uses `_dyld_process_info_create` and friends and is present in
macOS 10.12 and above, and the other depends on
`_dyld_get_all_image_infos`, which is documented by Apple (... in a WWDC
video) to "go away" at some point in dyld3.

- `dladdr` linearly scans the dyld loaded image array (and each image's
segment set) for a matching address, so `image_added` will get slower as
more dynamic libraries are loaded in the process. But dyld3's
implementation of dlopen already scales linearly in time with the number
of loaded libraries. The use of `dladdr` can be avoided by using
`_dyld_register_for_image_loads` or
`_dyld_register_for_bulk_image_loads`, but those are both private APIs.

- The callbacks registered with `_dyld_register_func_for_add_image` and
`_dyld_register_func_for_remove_image` can never be unregistered, so
this is yet another thing that makes it impossible to safely unload an
embedded mono.

- `MonoW32ProcessModule::inode` only seems to be used to avoid dupicates
in other implementations of w32process; sorting the returned module list
in `mono_w32process_get_modules` wouldn't be necessary if the
`mono_dyld_image_info`s were also kept in an intrusive circular doubly
linked list.

- It looks like `mono_w32process_get_modules` in `w32process-unix-bsd.c`
has a similar race condition.

Also see: https://github.com/mono/mono/issues/20107

* Formatting.

* Make `image_added` and `image_removed` non-blocking.

It is theoretically possible for `mono_w32process_get_modules` to cause
a deadlock if either `image_added` or `image_removed` is waiting for the
image table mutex to be released and if `g_new0`, `g_strdup`,
`g_hash_table_iter_next`, or any of the symbols they depend on are
lazily bound and haven't already been resolved.

Only one thread may iterate over the image table and only one thread may
add/remove entries to/from the image table.

The writer thread attemps to lock a mutex to block readers from
iterating over the table. If the writer successfully acquires this lock,
then no readers are concurrently iterating over the image table and it
is safe to eagerly free stale data.

If the writer doesn't successfully acquire this lock, then the reader
thread is running concurrently with the writer; if this happens, the
writer pushes stale data into a limbo list. Readers opportunistically
clear this list.

Use `mono_mutex_t` instead of `MonoCoopMutex`. Hand-roll the necessary
GC state transitions inside `mono_w32process_get_modules`.

Add `mono_conc_hashtable_foreach_snapshot` to mono-conc-hashtable.{h,c}.

`MonoConcurrentHashTable` depends on the hazard pointer table
(`hazard_table` inside hazard-pointer.c), which is initialized lazily on
the first call to `mono_thread_small_id_alloc`. Perhaps this should be
explicitly initialized early during `mono_init_internal`.

* Revert "Make `image_added` and `image_removed` non-blocking."

This reverts commit a22a4440df5f0f13da29b1b90e80a3420223c2b9.

* Use mono_mutex_t instead of MonoCoopMutex

* Explicitly load symbols not used by `image_added` before `mono_w32process_get_modules` runs
2020-08-24 19:06:37 -04:00
monojenkinsandBrzVlad ced76ae314 [coop] Add configuration option to disable stack scan (#20264)
When this experiment is enabled, the GC doesn't scan stack or registers. This mode is meant to disable scanning of native frames. This implementation only makes sense in full interpreter mode, where the interp stack is scanned separately from the thread stack. Longer term we could use this mode to scan only the managed stack, also in jit mode.

The point of this mode is to be able to stress test the GC in a wasm like environment, where stack can't be scanned. We can more easily reproduce and investigate GC crashes in this configuration, that would also happen on wasm.

Co-authored-by: BrzVlad <[email protected]>
2020-08-24 23:03:37 +03:00
Zoltan Varga b854e882b4 [wasm] Bump emscripten to 2.0.1. (#20287) 2020-08-24 11:02:27 -04:00
monojenkinsandCoffeeFlux fbfc91536f [mono] Thread DllImportSearchPath flags down to LoadLibraryEx (#20292)
This mostly completes support for the attribute. Our algorithm differs a bit from CoreCLR right now, so we don't support the legacy behavior flag since it makes less since in the context of our algorithm, but this should be good enough for most cases.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-24 10:40:49 -04:00
monojenkinsandCoffeeFlux 44d5fa37ad [mono] Add more detailed logs for the PInvoke resolution process (#20291)
Similar logging was added to the managed loading algorithm when it proved extremely useful for debugging a Blazor crash, so the equivalent is probably useful for the unmanaged library algorithm as well

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-24 00:24:56 -04:00
monojenkinsandCoffeeFlux eb2267f30b [mono] Don't leak coop handles, fix type in managed ALC struct (#20281)
The important change here is not leaking the coop handles, the other is because I'm too lazy to create two separate PRs for this.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-22 14:27:52 -04:00
monojenkinsandCoffeeFlux 6f0cc47bbf [mono] Disable XML file loading entirely on netcore (#20280)
Co-authored-by: CoffeeFlux <[email protected]>
2020-08-22 07:09:22 -04:00
monojenkinsandCoffeeFlux 3cb098653b [mono] Add g_ptr_array_find to eglib (#20279)
Co-authored-by: CoffeeFlux <[email protected]>
2020-08-22 07:09:02 -04:00
Zoltan Varga 8afa37c663 [runtime] Make the code which handles pinvoke lookup retries netcore only. (#20289) 2020-08-22 05:19:39 -04:00
monojenkinsandvargaz 5521167e5d [runtime] Ongoing work on mono cmake build (#20268)
Co-authored-by: vargaz <[email protected]>
2020-08-22 01:30:44 -04:00
monojenkinsandEgorBo 84d66c6b35 [mono][llvm] Revert noopt modifier (#20286)
For some reason it leads to a crash on Windows x64+FullAOT+LLVM config, investigating...

Co-authored-by: EgorBo <[email protected]>
2020-08-21 22:31:26 -04:00
monojenkinsandBrzVlad 39138f1ed5 [coop] Add handles for some SRE scenarios (#20259)
Co-authored-by: BrzVlad <[email protected]>
2020-08-22 01:38:59 +03:00
monojenkinsandfanyang-mono dc1bd5bde1 Add missing exception checkpoint (#20274)
Fixes dotnet/runtime#38826

Co-authored-by: fanyang-mono <[email protected]>
2020-08-21 19:35:46 +02:00
Alexander Köplinger e88d9ced3b Bump corefx to disable DateTime test
See https://github.com/mono/mono/issues/16623
2020-08-21 17:17:06 +02:00
monojenkinsandEgorBo f564c1d808 [mono] Add MethodImplOptions.NoOptimization support (#20265)
Mono currently just ignores `MethodImplOptions.NoOptimization` flag, this PR adds support for it.

```csharp
static int Add(int a, int b) => a + b;

[MethodImpl(MethodImplOptions.NoOptimization)]
static int Test(int a, int b) => Add(a, b) * Add(a, b); // NoOpt means - don't inline Add here, don't do opts

static void Main(string[] args)
{
    Test(1, 2);
}
```
Codegen for Test:
```asm
0000000000000000 <gram_Test__int_int_>:
<BB>:2
   0:   48 83 ec 18             sub    $0x18,%rsp
   4:   4c 89 34 24             mov    %r14,(%rsp)
   8:   4c 89 7c 24 08          mov    %r15,0x8(%rsp)
   d:   4c 8b f7                mov    %rdi,%r14
  10:   4c 8b fe                mov    %rsi,%r15
  13:   e8 29 00 00 00          callq  41 <gram_Test__int_int_+0x41>
  18:   48 89 44 24 10          mov    %rax,0x10(%rsp)
  1d:   49 8b fe                mov    %r14,%rdi
  20:   49 8b f7                mov    %r15,%rsi
  23:   e8 19 00 00 00          callq  41 <gram_Test__int_int_+0x41>
  28:   48 8b c8                mov    %rax,%rcx
  2b:   48 8b 44 24 10          mov    0x10(%rsp),%rax
  30:   0f af c1                imul   %ecx,%eax
<BB>:1
  33:   4c 8b 34 24             mov    (%rsp),%r14
  37:   4c 8b 7c 24 08          mov    0x8(%rsp),%r15
  3c:   48 83 c4 18             add    $0x18,%rsp
  40:   c3                      retq
```
with llvm backend (emits `noopt` attribute for llvm function):
```asm
0000000000000000 <gram_Test__int_int_>:
<BB>:1
   0:   55                      push   %rbp
   1:   41 57                   push   %r15
   3:   41 56                   push   %r14
   5:   53                      push   %rbx
   6:   50                      push   %rax
   7:   41 89 f6                mov    %esi,%r14d
   a:   89 fd                   mov    %edi,%ebp
   c:   eb 00                   jmp    e <gram_Test__int_int_+0xe>
   e:   eb 00                   jmp    10 <gram_Test__int_int_+0x10>
  10:   49 bf c0 b6 7c 53 37    movabs $0x5637537cb6c0,%r15
  17:   56 00 00
  1a:   49 8b 07                mov    (%r15),%rax
  1d:   89 ef                   mov    %ebp,%edi
  1f:   44 89 f6                mov    %r14d,%esi
  22:   ff d0                   callq  *%rax
  24:   89 c3                   mov    %eax,%ebx
  26:   49 8b 07                mov    (%r15),%rax
  29:   89 ef                   mov    %ebp,%edi
  2b:   44 89 f6                mov    %r14d,%esi
  2e:   ff d0                   callq  *%rax
  30:   0f af d8                imul   %eax,%ebx
  33:   89 d8                   mov    %ebx,%eax
  35:   48 83 c4 08             add    $0x8,%rsp
  39:   5b                      pop    %rbx
  3a:   41 5e                   pop    %r14
  3c:   41 5f                   pop    %r15
  3e:   5d                      pop    %rbp
  3f:   c3                      retq
```
If I replace `LLVM_ATTR_OPTIMIZE_NONE` with `LLVM_ATTR_OPTIMIZE_FOR_SIZE` it becomes https://www.diffchecker.com/S02HEp0w for llvm.

Co-authored-by: EgorBo <[email protected]>
2020-08-20 13:12:54 +02:00
monojenkinsandYoussef1313 ef1589a6cd Fix trailing whitespaces (#20254)
Related to dotnet/runtime#40887 and dotnet/runtime#40884

Co-authored-by: Youssef1313 <[email protected]>
2020-08-20 12:14:59 +02:00
Alexander Köplinger 07e2ed7708 Add --sdk_root to sdkmanager 2020-08-19 02:24:13 +02:00
Alexander Köplinger c73aa699d6 Support overriding SDK_DIR/NDK_DIR in sdks/android/Makefile
We need this for the new Jenkins Android emulator plugin since it uses a different adb version than what we have in $(ANDROID_TOOLCHAIN_DIR)/sdk.
2020-08-19 01:25:30 +02:00
Zoltan Varga 746ece2e0f [aot] Fix the lookup of jit info for WASM. On wasm, methods have no size, so (#20262)
only the method address can be looked up, jinfo->code_size is set to 1.

Fixes https://github.com/mono/mono/issues/20256.
2020-08-18 14:23:56 -04:00
monojenkinsandlateralusX 1d85cba972 Enable tests failing due to missing array ctor and lbound index checks. (#20100)
Fix https://github.com/mono/mono/pull/19494 adds missing array ctor needed by several tests currently failing as described by https://github.com/dotnet/runtime/issues/34068.

This PR fixes an additional problem hit by the IL array tests using an lbound != 0. Current Mono codegen for Set/Get/Address array methods didn't adjust index with regards to lbound causing failures in several of the array tests. NOTE, this scenario should only be hit when directly calling Set/Get/Address using IL on an array with an lbound != 0. It shouldn't affect regular szarray's at all since that code paths should never be used and the codegen currently looks up the lbounds unconditionally in the array, so if that code path ever gets hit by a szarray, it will imdediate crash, so shouldn't be causing any performance impact on szarrays. There is also an additional check on the array type as well as an additional precaution, so only arrays of type MONO_TYPE_ARRAY, hitting codegen for Set/Get/Address array method calls will adjust index based on lbound.

This PR fix the lbound index codegen for that specific code path and re-eanbles the tests currently listed in https://github.com/dotnet/runtime/issues/34068.

Co-authored-by: lateralusX <[email protected]>
2020-08-18 18:49:26 +02:00
monojenkinsandlateralusX 6d26f8ba24 Fix EventActivityIdControl test on MonoVM. (#20106)
Make EventSource call static version of EventActivityIdControl in EventPipeEventProvider instead of direct calling low level
EventPipeInternal (now only called from EventPipe* sources). Doing it that way will also keep EventPipeEventProvider.EventActivityIdControl in S.P.C meaning that tests like eventactivityidcontrol won't risk
of using methods that has been trimmed away by linker.

Added missing case into Mono's EventPipeInternal::EventActivityIdControl implementations.

Enabled eventactivityidcontrol test on Mono runtime. Adjusted test to use static EventPipeEventProvider.EventActivityIdControl version.

Co-authored-by: lateralusX <[email protected]>
2020-08-18 13:02:40 +02:00
monojenkinsandCoffeeFlux 669ed125e3 [mono] Loader improvements: bundle optimization and logging (#20246)
See individual commits and description.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-17 18:53:59 -04:00
monojenkinsandimhameed f51cfe74e3 [mono] Add X86Base support. (#20253)
Implement `BitScanForward` and `BitScanReverse` as intrinsics.

Implement `__cpuidex` as a netcore-only icall. Rename `cpuid` (in
mono-hwcap-x86.c) to `mono_cpuidex`, give it extern linkage, and add
support for extended CPUID.

Related: https://github.com/dotnet/runtime/pull/40167

Co-authored-by: imhameed <[email protected]>
2020-08-17 15:21:35 -07:00
Zoltan Varga 1ebc801381 [aot] Avoid emitting generic instances with vtype arguments on wasm to save space. (#20252)
* [aot] Avoid emitting generic instances with vtype arguments on wasm to save space.

* [aot] Avoid dedup-ing gsharedvt methods.
2020-08-17 17:15:27 -04:00
Thays Grazia 70219cc2b5 [wasm] Merge dotnet/runtime changes to mono/mono. (#20249)
* Reformatting files.
Merging dotnet/runtime to mono/mono

* Removing testharnessstartup from browserdebughost.
Fix format of EvaluateOnCallFrameTests.

* Fix package.
2020-08-15 14:03:15 -03:00
monojenkinsandCoffeeFlux 0161714a5a [mono] Make bundle filename changes netcore-only (#20251)
cc: @thaystg

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-15 01:18:40 -03:00
Caleb Cornett 442aeedc5e [wasm] Add extra-emcc-flags parameter to packager (#20220)
* Add extra-emcc-flags parameter to packager

* Style fix
2020-08-14 20:52:36 -07:00
monojenkinsandvargaz 2d845f0711 [runtime] Update cmake build. (#20243)
Co-authored-by: vargaz <[email protected]>
2020-08-14 10:56:35 -04:00
monojenkinsandCoffeeFlux 02b9be1282 [mono] Improve MonoImage filename handling for bundled images (#20245)
This change means we will handle bundles consistently across platforms and ensures that the MonoImageStorage is keyed by the in-memory location rather than the assembly name, which seems less error-prone.

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-14 09:42:34 -04:00
Thays Grazia 2d567b395e [wasm] Renaming debugger files. (#20244)
* Renaming projects and files as we did on dotnet/runtime repo.

* Fixing library-mono.js to run debugger tests.
Fixing typo on DebuggerTestSuite.
2020-08-14 08:29:47 -03:00
monojenkinsandakoeplinger 5f44f27a0d Cleanup old corefx/coreclr GitHub issue links (#20201)
Replace them with the current URL after the redirect.

Similar to @stephentoub's PR: https://github.com/dotnet/runtime/pull/2063.

This is a mechanical find/replace, I did not check whether the references are still accurate or issues are already fixed.

Co-authored-by: akoeplinger <[email protected]>
2020-08-13 23:14:08 +02:00
monojenkinsandthaystg 5b33599427 [wasm] Remove unnecessary console messages. (#20236)
Changing some console.log to console.debug to not pollute the console.
Changing printf to DEBUG_PRINTF to follow the debug level chosen by user.
Remove unnecessary console.log.
Fix dotnet/runtime#40721

Co-authored-by: thaystg <[email protected]>
2020-08-13 08:46:09 +02:00
Zoltan Varga 5e05c87d3d [aot] Fix an assert which is hit for generic instances with a lot of arguments. (#20238)
The c# compiler generates these for anonymous types.

Fixes https://github.com/xamarin/xamarin-macios/issues/9289.
2020-08-12 20:02:19 -04:00
monojenkinsandBrzVlad 89e7e5c210 [interp] Rework OBJREF and fix a GC issue (#20234)
Co-authored-by: BrzVlad <[email protected]>
2020-08-13 01:39:55 +03:00
monojenkinsandCoffeeFlux e3b3a6f864 [mono] Fix configure to actually disable dllmap on non-mobile platforms (#20230)
`$enable_monodroid` and `$enable_monotouch` are set to no by default, not an empty string.

Accidentally enabled globally here: https://github.com/dotnet/runtime/commit/a54d391d9daebf7b53e7f1efc1690c1c35177570#diff-294e5f7514d4059e724506770913e1d7R1386

Fixes https://github.com/dotnet/runtime/issues/40147

Co-authored-by: CoffeeFlux <[email protected]>
2020-08-12 12:35:26 -04:00
monojenkinsandBrzVlad 74243d9b93 [coop] Use handles for mono_runtime_try_invoke_array (#20225)
Co-authored-by: BrzVlad <[email protected]>
2020-08-12 17:25:32 +03:00
Zoltan Varga 92ce33daba [aot] Avoid dedup-ing non-public instances. (#20226)
This will decrease the size of the dedup assembly which can get
large with lots of instances.

Hopefully fixes https://github.com/mono/mono/issues/20224.
2020-08-11 23:35:09 -04:00
monojenkinsandbenaadams 0cd363fb98 Type.IsAssignableTo (#20206)
Resolves https://github.com/dotnet/runtime/issues/30714

Co-authored-by: benaadams <[email protected]>
2020-08-11 11:56:35 +02:00
monojenkinsandvargaz a7aa383fc6 [runtime] If the initial pinvoke lookup fails, emit some code inside the managed-to-native wrapper to retry to lookup on every call until it succeeds. (#20187)
Fixes dotnet/runtime#35219

Co-authored-by: vargaz <[email protected]>
2020-08-10 19:54:40 -04:00
Alex Thibodeau 45f2b54612 Fixing debugger hang that would occur when inspecting a field that has a Debugger.Break in it's accessor. (#20211)
Fixing TypeInfo debugger test. I added a new property which broke the num properties check.
2020-08-10 18:18:10 -03:00
Zoltan Varga 9d9ec219c1 [aot] Fix the handling of r4/r8 parameter types with attributes during generic sharing. (#20217)
The attributes need to be ignored as with the other types, otherwise gsharedvt wrappers for signatures with
parameters like double f = default will not be found.

Fixes https://github.com/mono/mono/issues/20195.
2020-08-10 08:30:49 -04:00
monojenkinsandradical 4835e29470 [wasm][debugger] Add support for invoking getters on ValueTypes (#20222)
- this allows IDEs to let users invoke property getters for ValueTypes

Co-authored-by: radical <[email protected]>
2020-08-09 21:11:58 +02:00
8a2dc9dc32 [wasm] [debugger] Support Exception Break on Debugger (handled and unhandled) (#20215)
* [wasm] [debugger] Support Exception Break on Debugger (handled and unhandled)

Based on https://github.com/mono/mono/pull/20142

* Add missing function to library_mono.js

Co-authored-by: thaystg <[email protected]>
Co-authored-by: Larry Ewing <[email protected]>
2020-08-08 12:32:02 +02:00
monojenkinsandimhameed 61ba224a6e [mono] Allow creating TypedReference * and TypedReference & via reflection. (#20221)
CoreCLR gained support for creating`TypedReference *` and `TypedReference &` via `MakePointerType` and `MakeByRefType` in https://github.com/dotnet/coreclr/pull/25817.

Reenable byref System.Void-related reflection tests.

Fixes https://github.com/dotnet/runtime/issues/31713.
Fixes https://github.com/dotnet/runtime/issues/37489.

Co-authored-by: imhameed <[email protected]>
2020-08-07 14:55:58 -07:00
monojenkinsandsteveisok 350828e54c [Wasm] Ensure monovm_initialize is called on startup and address other startup items (#20129)
This PR addresses:

- Calling monovm_initialize matches how other mono platforms init themselves (Fixes https://github.com/dotnet/runtime/issues/39074)
- Set the entry assembly (Fixes https://github.com/dotnet/runtime/issues/38994)
- Make sure the assembly location is set to empty for assemblies that are loaded from memory (Fixes https://github.com/dotnet/runtime/issues/39650)

Co-authored-by: steveisok <[email protected]>
2020-08-07 09:30:48 -04:00
monojenkins 743ebf9c09 [mono][interp] Optimize box+unbox to no-op (#20174) 2020-08-07 16:08:26 +03:00
monojenkinsandvargaz 1ae5414a2c [wasm] Fix GC support in ves_icall_get_trace (). (#20210)
Co-authored-by: vargaz <[email protected]>
2020-08-06 10:53:01 -04:00
monojenkinsandimhameed 9d4e3ba229 [mono] Don't save byref System.Void into MonoDomain::typeof_void. (#20212)
`mono_type_get_object_checked` will return the `MonoReflectionType`
referred to by `MonoDomain::typeof_void` if present. If a byref
`System.Void` is stored in this field, then
`get_ContainsGenericParameters` applied to `System.Void` will loop
indefinitely, because our implementation of
`get_ContainsGenericParameters` expects `GetElementType` to yield a
`MonoReflectionType` with one layer of "type function application"
removed.

Fixes https://github.com/dotnet/runtime/issues/37489.

Co-authored-by: imhameed <[email protected]>
2020-08-05 17:49:13 -07:00
monojenkinsandakoeplinger 9f129a6b3a Mono: Ensure tasklets are removed in netcore build (#20203)
Tasklets were an old mono-specific extension for continuations that is no longer relevant.

Co-authored-by: akoeplinger <[email protected]>
2020-08-04 22:26:08 +02:00
monojenkinsandakoeplinger 462971120c Fix warning in debugger-agent.c (#20204)
```
  CC       libmono_dbg_la-debugger-agent.lo
/Users/runner/work/1/s/src/mono/mono/mini/debugger-agent.c:1088:9: warning: unused variable 'argv' [-Wunused-variable]
                char *argv [ ] = {
```

Co-authored-by: akoeplinger <[email protected]>
2020-08-04 21:51:16 +02:00
monojenkinsandvargaz ce43eb0beb [runtime] Handle null byref returns in runtime invokes. (#20179)
Fixes https://github.com/dotnet/runtime/issues/39380.

Co-authored-by: vargaz <[email protected]>
2020-08-04 20:02:35 +02:00
monojenkinsandfanyang-mono 4ca4a459ad Check if a method is dynamic when transforming internal calls (#20199)
Fixes dotnet/runtime#38897 , by allowing transforming internal calls when method is dynamic.

Co-authored-by: fanyang-mono <[email protected]>
2020-08-03 22:11:45 -04:00
monojenkinsandakoeplinger 601a2d6e8a Fix ICU_CFLAGS in mono configure.ac (#20198)
We need to set these defines so we don't use obsolete ICU functions and cause warnings, just like we do in the CMakeLists.txt version..

Co-authored-by: akoeplinger <[email protected]>
2020-08-03 16:05:11 +02:00
Ankit Jain d09e9e21b3 [master] Bump msbuild to track xplat-master (#20189) 2020-08-02 15:43:57 +02:00
Zoltan Varga 92f54ee4c2 [runtime] Add a configure workaround for apple silicon + older config.guess versions. (#20192) 2020-08-02 15:42:47 +02:00
Neale Ferguson 71307187cb Fix __asm__ specifications for Clang (#20190)
* Fix __asm__ specs to work with clang

* 1. No need for USE_STRING_INLINES anymore
2. Code around clang bug where %r0 is being rejected from BASR instruction

* Fix a couple of more __asm__ specifications for clang
2020-07-31 15:24:58 +02:00
Zoltan Varga f0229da768 Bump emscripten to 1.40.0. (#20188) 2020-07-31 01:32:08 -04:00
Bernhard Urban-Forster 2921130fa8 [RISCV] Build fixes and simple exception handling (#20184)
* [riscv] add ELF machine definition

* [riscv] add mono_arch_get_cie_program to fix the build

* [interp] remove unused pointer

* [riscv] add offsets for MonoContext members

* [riscv] add pattern to nop_stub, so it is easier to spot in the debugger what caused the hang

* [riscv] minimal exception handling support
2020-07-30 19:20:45 -04:00
Zoltan Varga a57e7e4d5f [jit] Avoid calling mono_marshal_get_native_wrapper () too early, it needs to be called when the pinvoke method is first called. (#20177) 2020-07-30 10:04:44 -04:00
monojenkinsandakoeplinger b258af8b7f [wasm] Remove unused icalls for Environment properties (#20180)
They are no longer needed after https://github.com/dotnet/runtime/pull/38996 since we return values in managed code now.

Co-authored-by: akoeplinger <[email protected]>
2020-07-29 17:34:09 +02:00
Glenn Slayden c16f6e0955 jay.exe (ILASM): Incorrect loop indexing, fixes #14241 (#20144)
Fix incorrect loop indexing.

Fixes #14241
2020-07-29 14:30:24 +02:00
EsmeandVincent Povirk 9a66d68d96 [System] Mark BTLS callbacks as cdecl. (#19872)
Fixes crashes when BTLS is compiled for Windows x86 with Clang.

Co-authored-by: Vincent Povirk <[email protected]>
2020-07-29 14:29:48 +02:00
Karl 957d1942d1 Fix Line problem with wrapping and tabs (#20112)
Also adds a minor improvement to line wrapping, by not updating the wrap
position when it can only move it earlier. This means when processing
end-of-wrapped-line whitespace that it doesn't get adjusted every time.
Also doesn't update the previous wrap position, but that shouldn't be
used in this situation as it is only for wrapping after dashes.

The real problem that this fixes is that wrapping a line at a point such
that there was whitespace including a tab after the wrap position could
cause an infinite loop. Without the fix, with a wrap_pos > 0 (normal),
it only split the line after reaching non-whitespace. In most situations
that was OK as we norwally wrap after whitespace. However a tab is
whitespace we wrap before. Once pos was equal to wrap_pos it stopped
incrementing pos as it was now at the split position and therefore it
shouldn't change. But it didn't set split_tag, so didn't split, and
therefore didn't exit the outer loop.
2020-07-29 14:28:16 +02:00
monojenkinsandEgorBo 4c7c1181cf [mono][interp] Intrinsify MemoryMarshal.GetArrayDataReference (#20163)
it's used under the hood in String, Span, Memory, etc.
It's also a public API now.

Co-authored-by: EgorBo <[email protected]>
2020-07-29 14:11:24 +02:00
Aleksey Kliger (λgeek) b3ce259316 Make install-culture-table write to mono/culture/ (#20124)
As part of https://github.com/mono/mono/pull/19912 we moved the locale files to the mono/culture/ subdirectory.
2020-07-29 12:01:31 +02:00
monojenkinsandCoffeeFlux b6d95e093b [mono] Disable cfgdir config on Windows netcore builds (#20173)
This was already disabled with autotools, but I forgot to update winconfig.h as well.

Co-authored-by: CoffeeFlux <[email protected]>
2020-07-29 11:19:17 +02:00
monojenkinsandvargaz cb2d589e24 [interp] Add null checks to STIND opcodes. (#20176)
Co-authored-by: vargaz <[email protected]>
2020-07-29 11:04:59 +02:00
monojenkinsandkg cb630c292c Update threadpool callback error messages to fix dotnet/runtime#38803 (#20171)
EDIT: Simple test case to trigger an error:
```csharp
        ThreadPool.QueueUserWorkItem((_) => {
            throw new Exception("test");
        });
        var t = typeof(System.Threading.ThreadPool);
        var m = t.GetMethod("PumpThreadPool", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static);
        m.Invoke(null, null);
```

Co-authored-by: kg <[email protected]>
2020-07-29 10:46:04 +02:00
Thays Grazia cf9935e136 Revert "[debugger] Removing unhandled_exception which was used for android. (#20122)" (#20175)
This reverts commit 60643c36f5.
2020-07-28 19:35:45 -03:00
monojenkinsandCoffeeFlux be199e71b5 [mono] Fix warnings on OSX netcore (#20170)
This commit brings us back to zero warnings for the default build configuration.

Co-authored-by: CoffeeFlux <[email protected]>
2020-07-28 15:57:16 -04:00
Zoltan Varga ae983fe58f Apple Silicon support (#20166)
* Fix configure check for arm64+osx.

* Add mono_codeman_enable_write/disable_write functions.

* Use clang --shared for linking aot images.

* Fix eglib remapping.

* Make code memory writable during code generation.

* Enable/disable writable code around dlmalloc operations, dlmalloc allocates from code memory.

* Fix some defines.

* Fix set/clear_breakpoint.

* Fix llvm jit support.

* Fix gsharedvt.

* Force float32 to be enabled on osx/arm64.

* Add codegen macros for arm64e.

* Disable some crash tests if --disable-crash-reporting is used.

* Disable btls. Disable boehm by default.

* Fix a warning.

* Fix vararg calls, the vararg icalls assume every argument is in in own stack slot, but the default osx/arm64 calling code is the same as ios which allows small arguments to share a stack slot.

* Fix formatting.

* Fix the build.

* Use aarch64-darwin-ios as the target triple when cross compiling to ios so configure can distinguish between osx/arm64 and ios.

* Assume ios when cross compiling to aarch64-darwin.
2020-07-28 08:34:07 -04:00
monojenkinsandalexischr fbe6a9618a [mono] Use DefaultDllSearchPaths attribute for PInvokes (#20090)
So far we have been ignoring the flags contained in the attribute.

Attribute can be applied to a pinvoke method or an assembly. See https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.defaultdllimportsearchpathsattribute.-ctor?view=netcore-3.1#System_Runtime_InteropServices_DefaultDllImportSearchPathsAttribute__ctor_System_Runtime_InteropServices_DllImportSearchPath_

Co-authored-by: alexischr <[email protected]>
2020-07-28 01:22:27 -04:00
Zoltan Varga 218419e77b [wasm] Use interp-only instead of interp-llvmonly as the default execution mode. (#20159)
* [wasm] Use interp-only instead of interp-llvmonly as the default execution mode.

interp-llvmonly caused mono_llvm_only to be set, causing bad behavior like
stack walks not working.

* Address comments.
2020-07-27 21:59:29 -04:00
monojenkinsandvargaz 5d7d1b12be [jit] Add a intrinsic for ThrowForUnsupportedVectorBaseType<T>. (#20167)
Part of https://github.com/dotnet/runtime/issues/38718.

Co-authored-by: vargaz <[email protected]>
2020-07-27 20:21:23 -04:00
monojenkinsandakoeplinger f87a9d7baf [mono] Fix building multiple os/arch combinations in the same working directory (#20168)
After adding the ICU shim code to the runtime we hit an issue when e.g. compiling for "desktop" mono in a working directory that already had Browser wasm artifacts in it.
This is due to automake putting the intermediate compilation artifacts at the same location where a file was referenced from.

This means that e.g. when using `../../../libraries/Native/Unix/System.Globalization.Native/pal_icushim.c` in the Makefile.am we'd get the intermediaries in `artifacts/obj/mono/libraries/` instead of somewhere in `artifacts/obj/mono/OSX.x64.Debug`.

Later on we'd get the following error because it was reusing the existing .o file from another architecture:

```
Undefined symbols for architecture x86_64:
  "_gPalGlobalizationNative", referenced from:
    _c_qcalls in libmonoruntimesgen.a(libmonoruntimesgen_la-native-library-qcall.o)
```

Fix this by symlinking the source files into the build directory.

Co-authored-by: akoeplinger <[email protected]>
2020-07-28 01:00:33 +02:00
Thays Grazia 60643c36f5 [debugger] Removing unhandled_exception which was used for android. (#20122)
* Removing unhandled_exception that was used for android.

* Removing unhandled_exception that was used for android.
2020-07-24 11:42:15 -03:00
Alexander Köplinger c0aa77d2ea Fix Cairo issue on macOS Big Sur
Bumps bockbuild to get the changes from mono/bockbuild#162

BigSur's CoreGraphics seems to need a copy of the data to stick around longer than the surface,
so we provide it a copy of the data instead.

Backport of https://github.com/mono/mono/pull/20154 to master
2020-07-24 13:54:57 +02:00
monojenkinsandvargaz aafaa6d93f [runtime] Fix some more gc tracking problems in create_cattr_named/typed_arg. (#20155)
Co-authored-by: vargaz <[email protected]>
2020-07-24 11:23:49 +02:00
Zoltan Varga 83083ef905 [aot] Avoid a crash in generic sharing for invalid generic instances. (#20157)
Fixes https://github.com/mono/mono/issues/20138.
2020-07-23 20:58:34 -04:00
monojenkinsandvargaz 5d3116f4ff [runtime] Make sure newly created objects are pinned during a runtime invoke in create_cattr_typed/named_arg (). (#20148)
Hopefully helps with https://github.com/dotnet/runtime/issues/39473.

Co-authored-by: vargaz <[email protected]>
2020-07-22 23:05:35 +02:00
monojenkinsandvargaz 850af3ba40 [runtime] Initialize the class if needed in type_is_blittable (). (#20145)
Fixes https://github.com/dotnet/runtime/issues/39100.

Co-authored-by: vargaz <[email protected]>
2020-07-21 18:08:55 -04:00
Zoltan Varga a2fcf072e8 Bump emscripten to 1.39.20. (#20143) 2020-07-21 07:31:37 -04:00
monojenkinsandeerhardt f8d27a4847 Allow Debugger attributes to be trimmed in Mono (#20132)
* Changed debugger-agent to allow for the Debugger Attributes to not exist.
* Removed the Debugger Attributes entries from the Descriptors.xml file.

Fix mono/linker#1355

Co-authored-by: eerhardt <[email protected]>
2020-07-20 23:20:44 +02:00
monojenkinsandBrzVlad 0372a381ad [wasm] Trim down size of dotnet.wasm (#20139)
This disables several debugging features in sgen, helps the compiler remove some unused code related to concurrent collector and disables more assertion messages, but also some other logs. We might have to reenable some of them if this is problematic.

Saves about 60k on wasm

Co-authored-by: BrzVlad <[email protected]>
2020-07-19 00:40:17 +03:00
Ganbarukamo41 2058deb70f Remove some non-unicode characters from source code (#20137)
* Remove ZWS from the source code

* Remove smart quotes from source code
2020-07-18 12:45:25 +02:00
monojenkinsandvargaz 2b6531e9b4 [mono] Fix an assert which happens when marshalling ftnptr types. (#20134)
MonoClass-es created for ftnptr types have an implicit pointer field so their size is not
the same as the computed size.

Co-authored-by: vargaz <[email protected]>
2020-07-17 23:54:15 +02:00
monojenkinsandvargaz e9560636b9 Add a --enable-minimal=eventpipe configure option. Disable eventpipe on wasm. (#20095)
Co-authored-by: vargaz <[email protected]>
2020-07-17 12:28:03 -04:00
Matthew Leibowitz d8dda25c27 [Wasm] Additional signatures for SkiaSharp interop + More (#20133)
* [Wasm] Additional signatures for SkiaSharp interop

* Update InterpToNativeGenerator.cs

* Generate more APIs
Include the bits in https://github.com/dotnet/runtime/commit/aa667de1e3fff0f951d5136a45fbe8acefad9be9
2020-07-17 16:47:40 +02:00
Zoltan Varga e1ba14ff82 [wasm] Fix handling of enum types in the pinvoke generator. (#20125)
Fixes https://github.com/mono/mono/issues/20123.
2020-07-17 02:45:46 -04:00
monojenkinsandkg 28315c66b7 ICU integration and asset loading overhaul (#20029)
Co-authored-by: kg <[email protected]>
2020-07-17 02:28:26 +02:00
monojenkinsandEgorBo f026b9a31c [mono] fix bound checks for Spans in LLVM-jit mode (add extra SROA pass) (#20054)
I noticed that the following code leaves bound checks when compiled using LLVM-JIT:
```csharp
static void Test(Span<int> span)
{
    for (int i = 0; i < span.Length; i++)
        x[i] = 0;
}
```
#### Current codegen:
```asm
Test:
0000000000000000	pushq	%rax
0000000000000001	testl	%esi, %esi
0000000000000003	jle	0x3f
0000000000000005	xorl	%eax, %eax
0000000000000007	movl	%esi, %ecx
0000000000000009	cmpq	%rcx, %rax
000000000000000c	jae	0x2c
000000000000000e	nop
0000000000000010	movl	$0x0, (%rdi)
0000000000000016	incq	%rax
0000000000000019	movslq	%esi, %rcx
000000000000001c	cmpq	%rcx, %rax
000000000000001f	jge	0x3f
0000000000000021	addq	$0x4, %rdi
0000000000000025	movl	%esi, %ecx
0000000000000027	cmpq	%rcx, %rax
000000000000002a	jb	0x10
000000000000002c	movabsq	$0x7fc330531710, %rax
0000000000000036	movl	$0x106, %edi
000000000000003b	callq	*(%rax) ;; <---- bound check
000000000000003d	ud2
000000000000003f	popq	%rax
0000000000000040	retq
```
looks like an additional -sroa pass after lower-expect and instcombine fixes it:
#### New codegen:
```asm
Test:
0000000000000000	testl	%esi, %esi
0000000000000002	jle	0x1f
0000000000000004	movl	%esi, %eax
0000000000000006	nopw	%cs:(%rax,%rax)
0000000000000010	movl	$0x0, (%rdi)
0000000000000016	addq	$0x4, %rdi
000000000000001a	decq	%rax
000000000000001d	jne	0x10
000000000000001f	retq
```
Here is the original LLVM IR before optimizations: https://godbolt.org/z/PLfR6x

Co-authored-by: EgorBo <[email protected]>
2020-07-16 20:08:51 +03:00
Zoltan Varga 3856ef71f8 [wasm] Fix handling of void types in the pinvoke generator. (#20121)
Fixes https://github.com/mono/mono/issues/20120.
2020-07-15 17:56:08 -04:00
monojenkinsandBrzVlad a463d28a41 [interp] Intrinsify span ctor (#20108)
Co-authored-by: BrzVlad <[email protected]>
2020-07-15 18:37:32 +02:00
Zoltan Varga a64bca988d Bump ikvm to pick up unmanaged cconv changes. (#20119) 2020-07-15 11:48:23 -04:00
Zoltan Varga b601abd311 [wasm] Fix the propagation of exceptions through runtime invokes. (#20117)
The previous code would convert exceptions into errors which would
be rethrown later, clearing the resume state.

Hopefully fixes https://github.com/dotnet/runtime/issues/38337.
2020-07-15 02:22:33 -04:00
monojenkinsandimhameed 77ae877253 Update Mono's implementation of Vector<T> intrinsics. (#20118)
Recognize `get_AllBitsSet` instead of `get_AllOnes`; this internal
intrinsic was renamed in https://github.com/dotnet/runtime/pull/36579
(1c66ad1b71bb84a16b71bb5efcef1fa999c1c8bc).

Implement `Vector<T>.One` as an intrinsic.

Co-authored-by: imhameed <[email protected]>
2020-07-14 21:40:32 -07:00
Zoltan Varga b923fc1651 [wasm] Restrict the changes in #20113 to netcore only. (#20114) 2020-07-14 15:50:36 -04:00
monojenkinsandakoeplinger 74782ab634 WASM: Fix System.Diagnostics.Tracing (#20115)
It was missing a few signatures in wasm_m2n_invoke.g.h that caused it to fail with `WASM-ERR: CANNOT HANDLE INTERP ICALL SIG IIILIIII`

Co-authored-by: akoeplinger <[email protected]>
2020-07-14 18:07:05 +02:00
monojenkinsandlateralusX 0bced44f8f Keep weak gc handle for EventPipe delegate callback in provider. (#20110)
Delegate callback can be held in native event pipe provider instance and called multiple times during provider lifetime (enable/disable sessions). Store a weak GC reference to delegate to correctly handle case where delegate gets moved by GC. It should not be collected since delegate should be held alive by managed event pipe class.

Co-authored-by: lateralusX <[email protected]>
2020-07-14 15:09:14 +02:00
monojenkinsandvargaz 4647328f76 [wasm] Fail right away in interp_create_method_pointer () when trying to create a function pointer to a native-to-managed wrapper. (#20113)
The previous method of returning interp_no_native_to_managed doesn't work:
* The runtime expects different function pointers for different methods, returning
  the same pointer leads to all kinds of weird errors.
* On wasm, the native caller expects the function to have the right signature, so
  calling interp_no_native_managed () will leads a to signature mismatch error which is
  hard to debug.

Co-authored-by: vargaz <[email protected]>
2020-07-14 08:49:05 -04:00
monojenkins 3059051ddd Run CoreCLR runtime tests using Mono with LLVM AOT. (#20077)
- coreclr/build-test.sh now has a new subcommand: 'mono_aot', which builds a
new target, named 'MonoAotCompileTests', added to
coreclr/tests/src/runtest.proj. This target compiles the runtime tests using
Mono LLVM AOT in a simple configuration where the host platform is identical to
the target platform. Parallel compilation happens via a hack: actual
compilation happens in mono/msbuild/aot-compile.proj, a single-target msbuild
file, and runtest.proj invokes this single-purpose project and target using
batching to create multiple parallelizable instances of this project. Future
work: use the MonoAOTCompiler custom task currently used to build the iOS
sample program.

- Avoid using the runtimeVariant string when defining
coreClrProductArtifactName in mono/templates/xplat-pipeline-job.yml. There are
no "runtime variants" of CoreCLR configured with this parameter; instead,
depend on a shared non-runtime-variant build of CoreCLR.

- Mark function DISubprograms as local definitions--this is an LLVM 9
compatibility fix.

- Use --tag=CXX when linking libmonosgen-2.0.so via libtool when LLVM is linked
into Mono.

  This makes libtool use the C++ compiler driver when linking Mono--which uses
whatever platform-specific flags are necessary to link against the C++ stdlib.
Previously, libtool would use the C compiler driver, which didn't do this and
would produce shared objects with no explicit dependency on libstdc++.

  This problem is normally masked because of the very lax dynamic linking
semantics on ELF, but Mono on our CI setup is built in a CentOS 7 image (which
does not contain a C++11 libstdc++) that has a GCC 7 compatibility package
installed, along with a clang 9 installation that detects headers from the GCC
7 compatibility package. This compatibility package includes a libstdc++ linker
script that links C++11 libstdc++ components statically into the target while
dynamically linking against components present in pre-C++11 libstdc++. The end
result of all of this is that Mono built with this configuration will
dynamically depend on C++11 libstdc++ symbols that should have been statically
linked into the library, and will outright fail to run on machines without a
newer version of libstdc++ available.

- Add tests that fail after LLVM AOT compilation to issues.targets.
2020-07-13 21:05:55 -07:00
monojenkinsandhughbe c94f94bb17 Add some edge case tests for Array.CreateInstance and fix Mono handling (#19796)
Co-authored-by: hughbe <[email protected]>
2020-07-13 17:16:23 +02:00
monojenkinsandBrzVlad 047ed897dd [interp] Add intrinsics for some common operations (#20071)
These are mostly around string operations and are meant to be a quick boost in performance for some common blazor scenarios, around rendering and json parsing, before interp optimizations catch up. Even if the interpreter will have more advanced optimization passes, it is likely that it would still be much slower on these low level operations. Long term we would have methods like these AOT-ed, but, depending on how easy to maintain and how effective these turn out to be, we could also stick to intrinsifying some bcl code in the future.

Co-authored-by: BrzVlad <[email protected]>
2020-07-13 12:05:13 +03:00
monojenkinsandCoffeeFlux 4456328a3e [mono] Disable config dir lookups on netcore (#20074)
Fixes https://github.com/dotnet/runtime/issues/33079

Co-authored-by: CoffeeFlux <[email protected]>
2020-07-13 10:53:07 +02:00
Ankit Jain ee3a669dc3 [wasm][debugger] Fix callFunctionOn+silent arg behavior (#20044)
`silent` argument for `callFunctionOn` should not cause errors to be
suppressed. According to the protocol docs:

`In silent mode exceptions thrown during evaluation are not reported and do not pause execution. Overrides setPauseOnException state.`

- so, the result returned should have the error, if any.
- Added corresponding JS test, so that we know we are matching the
correct behavior.
2020-07-10 18:54:22 -04:00
monojenkinsandBrzVlad 9ecb588fc7 [interp] Add debug option which reports the hottest methods (#20097)
This tracks how many interpreter opcodes were executed as part of each method and reports them during runtime shutdown. Example output :
```
Total executed opcodes 305590200
11% Opcounts 34261110, calls 81900, Method System.Text.Json.JsonWriterHelper:ToUtf8 (System.ReadOnlySpan`1<byte>,System.Span`1<byte>,int&,int&), imethod ptr 0x55f2d503d870
22% Opcounts 33980080, calls 849502, Method System.Text.ASCIIUtility:WidenFourAsciiBytesToUtf16AndWriteToBuffer (char&,uint), imethod ptr 0x55f2d4d9ce08
29% Opcounts 23040810, calls 81900, Method System.Text.Encodings.Web.DefaultJavaScriptEncoderBasicLatin:FindFirstCharacterToEncode (char*,int), imethod ptr 0x55f2d5038218
36% Opcounts 20590344, calls 1715862, Method System.ReadOnlySpan`1<byte>:.ctor (void*,int), imethod ptr 0x55f2d4d3fee0
```

Co-authored-by: BrzVlad <[email protected]>
2020-07-10 15:00:25 +03:00
monojenkins 47ef5a8d6a [bcl] Update BCL Linked Size (#19446) 2020-07-09 20:39:05 -05:00
24be27d5e7 [mono] Add support for UnmanagedCallersOnlyAttribute (#20060)
* [marshal] Add mono_method_has_unmanaged_callers_only_attribute

* [marshal] Allow calls to mono_marshal_get_managed_wrapped without a delegate class

   In that case, create a wrapper based on the signature of the method itself.

* [aot] Allow decode_method_ref to decode NATIVE_TO_MANAGED wrappers without a delegate class

   Bump the AOT file format

* [interp] ldftn will return a native-to-managed wrapper to UnmanagedCallersOnly methods

* aot: emit byte when we don't expect a class

* jit: create wrapper creation for ldftn in method-to-ir

   Do it at IR generation of the caller, not every time the ldftn is executed

* jit: don't create a jump trampoline for ldftn of a native-to-managed wrapper

   The wrapper might be called from a thread that's not attached to the runtime, and the jump trampoline will look at TLS vars that are not initialized

* interp: transform LDFTN into LDC of a create_method_pointer for UnmanagedCallersOnly method

* marshal: throw invalid program exception for instance and generic methods

* Emit IPE throw instead of aborting JIT or interp compilation for bad UnmanagedCallersOnly methods

   Instead of throwing while JITing (or transforming), throw when the LDFTN is executed.

* disallow delegate constructor calls on UnmanagedCallersOnly methods

* throw IPE if UnmanagedCallersOnly method has non-blittable args

* disallow DllImport and UnmanagedCallersOnly together

   throw NotSupportedException

Co-authored-by: Ryan Lucia <[email protected]>
Co-authored-by: lambdageek <[email protected]>
2020-07-09 20:37:38 -04:00
monojenkinsandvargaz 47777a57c0 [wasm] Add beginnings of support for building with ICU. (#20091)
Co-authored-by: vargaz <[email protected]>
2020-07-10 01:12:51 +02:00
monojenkinsandpi1024e f963132ac8 Change double NOT operation to comparison to 0 (#19960)
If strstr does not return 0, found is true, so instead of using !!, let's use != 0.

Co-authored-by: pi1024e <[email protected]>
2020-07-10 01:11:59 +02:00
monojenkinsandstephentoub 1e27513c9b Improve terminology used in code (#20089)
Co-authored-by: stephentoub <[email protected]>
2020-07-09 20:48:28 +02:00
monojenkinsandlateralusX c9664e6c40 Use mono_runtime_delegate_invoke in EventPipeInternal::CreateProvider. (#20093)
Due to issues on some platforms related to mono_delegate_to_ftnptr_impl, switch to use mono_runtime_delegate_invoke without need for delegate wrapper generation currently not supported on all platforms.

Co-authored-by: lateralusX <[email protected]>
2020-07-09 19:16:09 +02:00
monojenkinsandstephentoub 0edd8d4102 Fix remaining license headers (#20088)
After https://github.com/dotnet/runtime/pull/38793 tweaked the license header on ~30K files, this cleans up after it, handling another ~1300, and adding license headers where they were missing in .cs files under libraries.  It also adds a default license header to the .editorconfig.

cc: @jkotas

Co-authored-by: stephentoub <[email protected]>
2020-07-09 11:08:03 +02:00
monojenkinsandlateralusX b35ae56d96 Fix case where EventPipeInternal::Enable is called with NULL output file. (#20084)
Co-authored-by: lateralusX <[email protected]>
2020-07-08 22:01:51 -04:00
monojenkinsandlambdageek ffea1753d3 [metadata] Move rarely used bits out of MonoMethod and give 1 bit back to MonoMethod:slot (#20061)
The `MonoMethod:slot` width was accidentally changed in https://github.com/dotnet/runtime/commit/8465be73eb40a9c409dc8974b9d6b235a1b5158f

To compensate, we move 3 bits out to image properties
   * `verification_success` - rarely used nowadays, protected by the image lock (like all image properties)
   * `is_reabstracted`, `is_covariant_override_impl` - infrequently used features of newer C# language features (default interface methods and covariant returns).  Setting these assumes the loader lock is held.

Co-authored-by: lambdageek <[email protected]>
2020-07-08 21:22:17 -04:00
Zoltan Varga 1bf3add26d [wasm] Fix the build. (#20085) 2020-07-08 17:22:24 -05:00
Pablo Martínez 9c0121161c Init a timer on TransactionScope initialization to be able to timeout (#20080)
Fixes #19955
2020-07-08 21:05:42 +02:00
Alexander Köplinger 5bd9f53dc0 Update license headers 2020-07-08 17:05:37 +02:00
Thays Grazia 1d94e306fb [wasm][debugger] DebugDirectoryEntryType = Reproducible means that has debug information (#20083)
* Considering DebugDirectoryEntryType = Reproducible as an assembly that has debug information.
Fix #20075

* Adding test which Ankit wrote.
2020-07-08 11:39:50 -03:00
Zoltan Vargaandmonojenkins f047b0b299 [wasm] Implement support for pinvoke callbacks in the interpeter. (#20076)
* [wasm] Refactor the wasm tuner.

Move the pinvoke table generation code into a new PInvokeTableGenerator.cs file,
to be similar to the corresponding code in WasmAppBuilder in dotnet/runtime. NFC.

* [wasm] Implement support for pinvoke callbacks in the interpeter.

* Modify the tuner to generate a table of trampoline functions which
have the same signature as the pinvoke callback methods.
* These functions transfer control to the interpreter.
* Modify the interpreter/wasm runtime to look up the functions
corresponding to the pinvoke callback methods.

* [csproj] Update project files

* Fix license.

* Remove leftover debug code.

Co-authored-by: monojenkins <[email protected]>
2020-07-08 09:47:32 -04:00
monojenkinsandBrzVlad dc7448f15f [interp] Remove usage of some gotos (#20073)
Their usage resulted in worse code on wasm.

Co-authored-by: BrzVlad <[email protected]>
2020-07-08 14:04:08 +03:00
Ankit Jain 1da3a20c8b [wasm][debugger] Add support for deref'ing pointers (#19760)
* [wasm][debugger] Send var names also, to `mono_wasm_get_variables`

- this will be useful in upcoming commits, which will need to use the
names for local pointer vars

* [wasm][debugger] Add support for deref'ing pointers

* [wasm][debugger] Dereference the pointer in js, instead of debugger.c

Based on suggestion from @vargaz

* [wasm][debugger][tests] Fix setting async methods when setting

.. breakpoint by method name. Async methods themselves don't have the
debug information. Instead, that can be found on the generated async
implementation type/method. Here, we just look for a type name
starting with `{original_type_name}/<{async_method_name}>`, and use the
method `MoveNext` from that.

* [wasm][debugger][tests] Update to set bp by method name+lineoffset

* Address review comments from Katelyn Gadd (@kg)

* Add test to check deref'ing invalid pointers

- We should really be handling invalid object ids, and report errors for
those, where appropriate. This will be done in a future PR
- For this PR, I'm adding tests that just check that invalid objects, or
trying to deref regular objects, doesn't blow up

* Remove old unused test code

* [wasm][debugger] Use JsonConvert instead of manually building the json string

- addresses review comment from @lewing
2020-07-07 13:39:12 -04:00
Andrii Vitiv 618cdf6ff7 Fix typo in docs/README (#20070) 2020-07-07 19:17:23 +02:00
Neale Ferguson 6cb3c5bdbb Add eventpipe to dist files (#20079) 2020-07-07 14:10:47 +02:00
Vlad Brezae 6de06a3636 [interp] Access frame arguments as locals (#20040)
* [interp] Make args part of called method stack frame

This also compacts stack usage. A new frame starts at the location of the first pushed argument.

* [interp] Remove ldarg/starg opcodes

We replace them with opcodes that operate on locals instead. This also enables additional optimizations that were applying to locals.

* [interp] Kill stack_args

It is now redundant

* [interp] Actually allocate filter frame in separate space

By mistake, the filter frame was using the same stack space as the original frame. This means that the filter clause could corrupt frames from which we still need to run finally clauses. This also refactors the code a little bit, moving frame copying outside interp_exec_method, which is already overly complex.
2020-07-07 10:46:16 +03:00
Thays Grazia 8b8ee716f4 [wasm][debugger] Don't download unnecessary assemblies to DebugProxy (#20032)
* Identifying if an assembly has debug info and only download it to debug proxy if it has, this will speed up a lot the debugger initialisation.
2020-07-06 09:04:47 -03:00
Martin Baulig b6ef72c244 Fix sdk/wasm/.gitignore to not-ignore the DebuggerTestSuite directory. (#20058)
The top-level .gitignore has `[Dd]ebug*/` - which matches `sdk/wasm/DebuggerTestSuite/*`;
to override, we need to add `!DebuggerTestSuite` to sdk/wasm/.gitignore
2020-07-02 15:05:21 +02:00
Martin Bauligandmonojenkins b70c794f70 [wasm][System.IO.Compression]: Disable BrotliStream on wasm. (#19961)
* [wasm][System.IO.Compression]: Disable Brotli on wasm.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-07-02 07:54:04 -05:00
monojenkinsandakoeplinger 38850478fd WASM: Add an internal method to pump the threadpool (#20056)
This can by used by the xharness xunit runner to make sure async tasks are executed.

We also need to tweak how `ThreadPoolTaskScheduler` handles `TaskCreationOptions.LongRunning` since the usual mode of starting a new thread doesn't work, instead we treat it like the option wasn't set and queue the task on the threadpool.

Co-authored-by: akoeplinger <[email protected]>
2020-07-02 14:25:37 +02:00
monojenkinsandalexischr 793020489a [mono] Implement getting the type modifiers for DynamicMethod parameters (#20052)
Fixes https://github.com/dotnet/runtime/issues/36271

Co-authored-by: alexischr <[email protected]>
2020-07-01 16:15:03 -04:00
Thays Grazia 96927e3532 Adding qcalls-def.h to Makefile.am 2020-06-30 19:19:06 -03:00
monojenkinsandBrzVlad 163ab2747d [interp] Small improvements around ByReference ctor (#20041)
Co-authored-by: BrzVlad <[email protected]>
2020-06-30 16:04:18 +03:00
monojenkinsandlateralusX ffb6ee3c0a Port EventPipeBuffer, EventPipeBufferList, EventPipeBufferManager to Mono. (#19958)
Continuation of https://github.com/dotnet/runtime/pull/34600, adding BufferManager into the library including unit tests, merging source files and additional library restructuring.

- Port EventPipeBuffer.
- Port EventPipeBufferList.
- Port EventPipeManager.
- Merge internals source files. Changed GETTER_SETTER to only mandate use of inlined functions when calling betweeen source files for different types. Upgraded all use within each source file for direct struct access.
- Changed function entry error checkin strategy. Only apply active checks on outer library API, ep.h, and use EP_ASSERT for all other functions.
- Dropped requires_lock_held in function naming, switch to comment in header or in forward declare for functions that validates that lock is held.
- Removed currently unused GETTER/SETTERS.
- Moved more comments into sources from corresponding CoreCLR EventPipe sources.
- Add EventPipeBuffer/EventPipeBufferManager unit tests.

Next step after this PR will be to enable a file session during runtime and add more unit tests and then start getting diagnostic server component over to Mono.

Co-authored-by: lateralusX <[email protected]>
2020-06-30 14:31:19 +02:00
monojenkinsandBrzVlad f50ea76a08 [interp] Intrinsify ClearWithReferences (#20042)
This greatly impacts Array.Clear performance, which is commonly used in blazor rendering.

Co-authored-by: BrzVlad <[email protected]>
2020-06-30 14:55:34 +03:00
mathieubourgeois 51dfcfe5c0 Bring back previous debug_abbrev offset behavior on Apple platforms (#20031)
https://github.com/mono/mono/pull/19794 broke on iOS. Analyzing the result showed that, while the assembly was generated as expected, the linked result was different. Analyzing the LLVM code (which triggered the original fix) seems to imply that, on Apple platforms, they will not generate a label for it, only doing the 0-offset as we were doing before. Therefore, match the LLVM behavior by bringing back our previous logic when targetting Mach, otherwise use the new way of doing things.

Fixes https://github.com/mono/mono/issues/8806 (again)
2020-06-30 12:01:52 +02:00
Zoltan Varga 174aeaa31c [wasm] Optimize interp->native transitions. (#20034)
* [wasm] Optimize interp->native transitions.

* Generate tables of signatures/transition functions.
* Add mono_wasm_get_interp_to_native_trampoline () function to look up the transition function
for a signature.
* Cache the transition function in a imethod->method_data slot.

Previously, we were calling a mono_wasm_interp_to_native_trampoline () function and a generated
icall_trampoline_dispatch () function which added a lot of overhead.

* Avoid sharing cache entries.
2020-06-30 04:13:18 -04:00
Neale Ferguson 1d3e43ec3a Fix #20022 - changing the signature of mono_handle_native_crash() for s390x (#20028)
* - Update exception signature as per other archs
- Make roslyn the default for s390x

* Put the compiler check in the right place
2020-06-29 21:41:28 -04:00
Martin Baulig b873e91b11 Add mono_exception_try_get_managed_backtrace(). (#20024)
This is an enhanced version of mono_exception_get_managed_backtrace()
that takes an optional `const char *prefix` argument (to indent the
stack-trace) and returns FALSE with a NULL backtrace on error.
2020-06-29 14:10:19 +02:00
Zebediah Figura a79bf1902d [cominterop] Support ClassInterfaceType and auto-dual class interfaces (#19768)
* [cominterop] Don't leak the MonoMethodBuilder in cominterop_get_ccw_checked().

* [cominterop] Support ClassInterfaceAttribute.

* [cominterop] Factor out cominterop_get_ccw_method().

* [cominterop] Implement auto-dual class interfaces.

* [tests] Add tests for ClassInterfaceType and auto-dual class interfaces.
2020-06-27 12:19:47 -04:00
monojenkinsandlambdageek 336bbb079c [mono][mini] Do a non-virtual call for bound delegates (#20030)
When we call a delegate that was bound to a specific function, don't do a virtual call.

Fixes mono/mono#19964

Co-authored-by: lambdageek <[email protected]>
2020-06-27 11:34:24 -04:00
Safia Abdalla 380d60dc70 Re-enable verbose and trace logging in debug proxy (#19926)
* Re-enable verbose and trace logging in debug proxy

* Update LogLevel in ProxyDriver files

* Revert to use LogDebug for verbose messages
2020-06-26 20:31:31 -05:00
Alexis Christoforides 80a9ecba2e [merp] Remove dead code (#20015)
Entry to crash reporting is now more properly handled by mono_dump_start()
2020-06-26 16:42:14 -04:00
Tammy Qiu b3110cd291 [wasm] Fix package update scripts for local builds
Fix package update scripts for local builds
2020-06-26 15:13:51 -04:00
monojenkins c511814182 [metadata] Implement PreserveBaseOverridesAttribute and covariant returns support (#19927)
Covariant return implementation for Mono.

---

There are also a couple of general Mono fixes in here, too:

1. Interpreter support for `MONO_TYPE_FNPTR`  in `mint_type()`
2. a "signature assignment" mode for `mono_class_is_assignable_from` that doesn't treat `IntPtr[]` and `long[]` (or `int[]`) as interchangeable
3. support for unmanaged pointer comparisons in `mono_class_is_assignable_from`
4. Factor `mono_byref_type_is_assignable_from` out of `ves_icall_RuntimeTypeHandle_type_is_assignable_from` and make it generally usable.

---

For covariant returns, there are a couple of pieces here:
1. Method overrides come in two flavors: implicit (matching by name and sig) and explicit (using a `.override` directive).
   * for explicit overrides, we have to check that the override has a signature that is subsumed by (its return type is a subtype of the return type of) the previous override, if any, and the declared method.
   * for implicit overrides, we have to check that the override has a signature that is subsumed by the previous most derived override (not just the one that we happened to find with a precisely matching signature).
2. If any override is marked by `[PreserveBaseOverridesAttribute]` then we need to find all the other slots that may have methods from the override chain and update them.  (The attribute is expected to be applied to a `newslot virtual` method that is an explicit `.override` of some other method).

There's an interesting test case where a class has _both_ an implicit and and explicit override for the same slot, arrived via different methods - in this case, the implicit one is supposed to be ignored.  We implement this by doing the implicit slot filling first, then the explicit, and saving the covariant return checking until after both are done.

---

Because the method (`MethodImpl`) is both an override of another virtual method,
and is itself a `newslot`, it will occupy (at least) 2 slots in the vtable -
its own newslot and the `MethodDecl` that it is overriding.

If a third class subclasses the class with the `MethodImpl` or one of its
descendents and itself overrides either the `MethodDecl` or (one of the)
`MethodImpl` (either implicitly or with another explicit .override)  the
`PreserveBaseOverridesAttribute` is a signal to apply the new override that it
should apply to all the slots.

The way this works is when processing the overrides, we walk up the class
hierarchy looking at the contents of the impl slot and if any of them have the
attribute, we make sure to add a mapping to the "`override_map`" to replace any
occurrences of a previous impl with the newest (most derived) impl.

An (existing) later pass goes through the vtable and applies the override_map.

Contributes to dotnet/runtime#37509


* [metadata] Implement support for PreserveBaseOverridesAttribute

The attribute is expected to be applied to a `newslot virtual` method that
is an explicit .override of some other method.

Becuase the method (MethodImpl) is both an override of another virtual method,
and is itself a `newslot`, it will occupy (at least) 2 slots in the vtable -
its own newslot and the MethodDecl that it is overriding.

If a third class subclasses the class with the MethodImpl or one of its
descendents and itself overrides either the MethodDecl or (one of the)
MethodImpl (either implicitly or with another explicit .override)  the
PreserveBaseOverridesAttribute is a signal to apply the new override that it
should apply to all the slots.

The way this works is when processing the overrides, we walk up the class
hierarchy looking at the contents of the impl slot and if any of them have the
attribute, we make sure to add a mapping to the "override_map" to replace any
occurrences of a previous impl with the newest (most derived) impl.

An (existing) later pass goes through the vtable and applies the override_map.

* add FIXME for dynamic images

* scan entire chain of inheritance, don't stop at the class with the attribute

   Makes OverrideMoreDerivedReturn.il pass

* UnitTestDelegates is also passing

* [metadata] Add mono_metadata_signature_equal_no_ret

* inflate signature->ret when doing covariant checking

* [class] Add mono_class_signature_is_assignable_from_checked

   In ECMA I.8.7.1 "assignment compatability for signature types" we need to
   distinguish IntPtr[] from int32[] and int64[].  The ordinary cast_class for an
   array type in Mono implements the "intermediate type" relation from I.8.7.3
   "general assignment compatability" - IntPtr[] gets treated as inter-castable
   with int32[] or int64[], depending on platform pointer size.

   The new mono_class_signature_is_assignable_from_checked function does the finer
   relation for signatures, while the old mono_class_signature_is_assignable_from_checked
   preserves the existing behavior used elsewhere in the runtime.

* Check every method previously in the slot against the new impl

* UnitTest_GVM passes

* [metadata] Don't infinite loop in mono_class_get_flags for a FNPTR

   When a MonoClass is a MONO_TYPE_FNPTR, the element_class is itself.  We should
   probably not do that, but meanwhile, make `mono_class_get_flags` return
   something else for function pointer types.

   Makes mono_class_is_interface not overflow the stack

* [metadata] Set MonoClass:cast_class for pointer types to the intermediate type

   Ignore signedness differences and set cast_class to the storage type, same as
   for arrays.  This is used by mono_class_is_assignable_from when comparing
   IntPtr* vs ulong*, for example.

* [metadata] Make mono_class_is_assignable_from work for unmanaged pointer types

   The issue is that we need to compare intermediate type or the reduced type of
   the element type, depending on whether we're doing compatability or general
   assignment compatability (upshot: ulong* and long* are always inter-assignable,
   but IntPtr* is not in signatures, but is for general storage)

* [metadata] Pull mono_type_byref_is_assignable_from out of ves_icall_RuntimeTypeHandle_type_is_assignable_from

   Make it a separate internal function

* [metadata] mono_type_byref_is_assignable_from for valuetypes and class types should check for identity

   X& and Y& where both are either a reference types or valuetypes are only
   assignable if X and Y are identical.  The old code was checking for identity
   for valuetypes, but for reference types it would allow any two reference types
   at all, which is incorrect.

* [class-init] Check for byref types in covariant return signatures

Lets us pass Loader/classloader/MethodImpl/CovariantReturns/UnitTest/CompatibleWithTest.il from the CoreCLR testsuite

* [interp] handle MONO_TYPE_FNPTR in mint_type

   It's just a pointer-sized integer, as far as the interpreter's type discipline is concerned

* [class-init] Use a bit for covariant return method impls

   Allows us to check implicit overrides against previous impl's sig when the previous impl potentially had a covariant return type.

   Also use the bit to decide if we need to check signatures of the entire impl chain for explicit instantiations

* ReturnTypeValidation unit tests work on Mono now

* [class-init] check covariant vtable slots after overloads are applied

   The check has to be done after both implicit and explicit overrides have been
   applied because some implicit overrides must be ignored.  If the check is done
   as soon as we detect that a covariant return method override is needed, we will
   incorrectly check implicit overrides that are to be ignored because a later
   explicit override applied to the same slot.

* [metadata] Fix byref IsAssignableFrom

   The reflection API has an interesting behavior, whereas the ECMA definition is
   stricter.

   ```
   public interface I {}
   
   public class B {}
   public class D : B, I {}
   
   public struct S : I { }
   
   class Program {
           static void Main(string[] args)
           {
   	    var tb = typeof (B).MakeByRefType ();
   	    var td = typeof (D).MakeByRefType ();
   	    var ti = typeof (I).MakeByRefType ();
   	    var ts = typeof (S).MakeByRefType ();
   
   	    Console.WriteLine ($"{tb.FullName} is assignable from {td.FullName} ? {tb.IsAssignableFrom (td) }");
   	    Console.WriteLine ($"{td.FullName} is assignable from {tb.FullName} ? {td.IsAssignableFrom (tb) }");
   
   	    Console.WriteLine ($"{ti.FullName} is assignable from {td.FullName} ? {ti.IsAssignableFrom (td) }");
   	    Console.WriteLine ($"{td.FullName} is assignable from {ti.FullName} ? {td.IsAssignableFrom (ti) }");
   
   	    Console.WriteLine ($"{ti.FullName} is assignable from {ts.FullName} ? {ti.IsAssignableFrom (ts) }");
   	    Console.WriteLine ($"{ts.FullName} is assignable from {ti.FullName} ? {ts.IsAssignableFrom (ti) }");
   
           }
   }
   
   // Expected Output:
   // B& is assignable from D& ? True
   // D& is assignable from B& ? False
   // I& is assignable from D& ? True
   // D& is assignable from I& ? False
   // I& is assignable from S& ? False
   // S& is assignable from I& ? False
   ```

* rename mono_class_signature_is_assignable

   No need for _checked suffix

* rename mono_byref_type_is_assignable_from

* [class-init] Move vtable setup to a separate file

   The code is getting quite long, move it to a separate file
2020-06-26 14:16:03 -04:00
Zoltan Varga f562eb8869 [wasm] Use -emit-llvm instead of the obsolete -s WASM_OBJECT_FILES=0 option. (#20026) 2020-06-25 17:32:45 -04:00
Zoltan Varga 28ba000f84 [wasm] Disable the checked build for release builds. (#20020) 2020-06-25 03:28:11 -04:00
Zoltan Varga 0d655a5eb6 [wasm] Remove obsolete --llvm-lto 1 argument, lto is already enabled when -s WASM_OBJECT_FILES=0 is used. (#20021) 2020-06-24 23:46:32 -04:00
Zoltan Varga a654692162 [jit] Add a fastpath to fill_runtime_generic_context (). (#19923)
* [jit] Add a fastpath to fill_runtime_generic_context ().

This is needed in llvmonly mode, since that mode doesn't use the
fastpath lazy rgctx fetch trampolines.

* Fix method_rgctx_array_size ().
2020-06-24 18:21:05 -04:00
Steffen Kieß 82d2277a08 [mcs] Fix help text for default value of -sdk: option (#20011)
The `mcs` help text says that it will use the value `4.5` by default for the option `-sdk:`, but it actually uses `4.x`. Fix the help text.
2020-06-24 19:58:37 +02:00
Alexis Christoforides 7a0425e81f [crashing] Improve crash chaining (#19973)
With this change, `mono_post_native_crash_handler` chains the crashing signal directly, instead of that having to happen after `mono_handle_native_crash` returns (which created a fragile repetitive code pattern). Finally, this way `mono_post_native_crash_handler` can also re-raise the signal right after chaining it. Since we remove our crashing signal handlers in `mono_handle_native_crash`, this should allow the OS to handle the signal as the final action.

Fixes https://github.com/mono/mono/issues/19860
2020-06-24 11:01:48 -04:00
Ankit Jain caf00ee3d6 [master] Bump msbuild to track xplat-master (#19972)
Also, bumps roslyn to `3.7.0-1.20262.2`
2020-06-24 16:29:40 +02:00
Jay Krell 872139a3e0 [wasm][interp] Fix warning about unprototyped mono_wasm_interp_to_native_trampoline. (#19183) 2020-06-24 16:19:32 +02:00
Alexander Köplinger a0231c5fc3 [wasm] Missing signatures for Microsoft.Extensions.EventSource (#20017)
Fixes tests in dotnet/runtime that currently fail with `WASM-ERR: CANNOT HANDLE COOKIE VL`.
2020-06-24 15:37:23 +02:00
David Ávila 9c62b13be4 [WASM] Missing signatures for OpenGLES (#20001)
* Added missing signatures for OpenGLES/WebGL

* Remove duplicated signatures

* Update Wasm_m2n_invoke.g.h to include new signatures
2020-06-24 00:20:33 -04:00
monojenkinsandvargaz d9ce741562 [wasm] Make some libc stubs return an error instead of asserting. (#20010)
Co-authored-by: vargaz <[email protected]>
2020-06-23 23:13:57 +02:00
Tammy Qiu 21e1d30efc Remove xml from download-packages.ps1 2020-06-23 10:31:10 -04:00
Zoltan Varga 09c44c9323 [wasm] Use wasm-opt --strip-dwarf instead of wasm-strip which is missing from recent emscripten versions. (#20006)
Fixes https://github.com/mono/mono/issues/20002.
2020-06-22 18:55:44 -04:00
Tammy Qiu cec9dee254 Fix package update scripts for local builds 2020-06-22 17:56:27 -04:00
monojenkinsandvargaz dba7d43c7e Add threadpool support for webassembly. (#19981)
Register a JS callback when a work item is queued, the callback will call into the runtime/bcl to execute the work item.

Co-authored-by: vargaz <[email protected]>
2020-06-22 20:18:16 +02:00
monojenkinsandam11 32c3b17e50 Enable mono cross-build on SunOS-like OS (#19932)
Summary:
* wire `$(Compiler)` in `mono.proj`, that comes from `eng/build.sh`.
* add configurations for illumos cross-compilation.
* set `-Werror=strict-prototypes` for `madvise` and `posix_madvise` introspection, which is used during the compilation.
* implement `mono-threads-sunos.c`.
* add some solaris-specific thread priority levels.

With this set of changes, `dotnet hwapp.dll` works with mono flavored `System.Private.CoreLib.dll` and `libcoreclr.so` on SmartOS 2020.

Contributes to: dotnet/runtime#34944.

Co-authored-by: am11 <[email protected]>
2020-06-22 09:42:30 -04:00
Vlad Brezae 2c2a1718a9 [interp] Rework frame data storage (#19985)
* [interp] Allocate a single continous stack region

Makes things easier, slightly improving call perf, and enables us to statically know the exact offset of any local / instruction result. We allocate localloc sections separately for this reason.

* [interp] Add allocator for localloc sections

It mostly reuses the original stack fragment allocator. When allocating memory for an interp frame, we remember the original state of the allocator before we started allocating for this frame. When we finish executing the frame code and we pop back the stack, we do the same with this allocator, restoring the previous allocator state.

* [interp] Remove unreachable code that makes no sense

Probably, in ancient times, we had a separate InterpFrame that we were executing for pinvoke / icall methods.

* [interp] Add separate opcodes for returning from methods with localloc

Just to avoid the slight overhead of calls for the common scenario.

* [interp] Add separate memory accounting for interp stack
2020-06-21 19:59:45 +03:00
Vlad Brezae ab3cc22dfe [interp] Fix some warnings that show up on wasm (#19991)
Do overflow checks the same way we do it in the jit.
2020-06-20 14:10:31 +03:00
Vlad Brezae d919597d26 [interp] Add unaligned version for ldfld.vt opcodes (#19993)
Regressed recently with the addition of specialized ldfld.vt opcodes
2020-06-20 14:09:40 +03:00
Zoltan Varga a1b885f1fc [jit] Fix the Activator.CreateInstance () intrinsic for primitive types. (#19989)
Fixes https://github.com/xamarin/xamarin-android/issues/4817.
2020-06-19 20:01:02 -04:00
Larry Ewing 9954695c4c assume double numbers (#19714) 2020-06-19 12:14:51 -05:00
monojenkinsandthaystg 99794efa11 [mono] Add QCalls to the Runtime (#19945)
- Implementing QCalls on mono
- Use QCalls on ICU Shim implementation

Fixes https://github.com/dotnet/runtime/issues/36449

Co-authored-by: thaystg <[email protected]>
2020-06-19 11:43:46 -03:00
Thays Grazia ea8f106187 Fix readme and sample.html. (#19988) 2020-06-18 22:38:07 -03:00
monojenkinsandEgorBo 386ae0405d [mono] Don't contribute to inline_const if ctor has AggressiveInlining (#19930)
Mono's inliner is quite conservative (see https://github.com/dotnet/runtime/issues/34286) so we have to workaround some limitations, e.g.
```csharp
static ReadOnlySpan<byte> Arr => new byte[] {1, 2, 3, 4};

// access example:
byte GetByte(int i) => Arr[i];
```
#### Current codegen for `GetByte` (with https://github.com/dotnet/runtime/pull/37458 included)
```asm
0000000000000000 <gram_GetByte__int_>:
<BB>:1
   0:   48 83 ec 18             sub    $0x18,%rsp
   4:   c5 f8 57 c0             vxorps %xmm0,%xmm0,%xmm0
   8:   c5 f8 29 04 24          vmovaps %xmm0,(%rsp)
   d:   48 b8 50 22 73 d2 6e    movabs $0x556ed2732250,%rax
  14:   55 00 00
  17:   48 89 e7                mov    %rsp,%rdi
  1a:   ff 10                   callq  *(%rax)   ;;;; get_Arr() is not inlined!
  1c:   83 7c 24 08 00          cmpl   $0x0,0x8(%rsp)
  21:   74 0b                   je     2e <gram_GetByte__int_+0x2e>
  23:   48 8b 04 24             mov    (%rsp),%rax
  27:   8a 00                   mov    (%rax),%al
  29:   48 83 c4 18             add    $0x18,%rsp
  2d:   c3                      retq
  2e:   48 b8 68 22 73 d2 6e    movabs $0x556ed2732268,%rax
  35:   55 00 00
  38:   bf 02 01 00 00          mov    $0x102,%edi
  3d:   ff 10                   callq  *(%rax)
```

As you can see, `get_Arr()` is not inlined - it happened because the calculated size is 60 and the major contributor is `ReadOnlySpan<>.ctor` which is marked with [AggressiveInlining], so Mono inlined that `.ctor` into `get_Arr` and the total cost for `get_Arr` increased to 60.

#### Codegen with this PR:
```asm
0000000000000000 <gram_GetByte__int_>:
<BB>:1
   0:   48 b8 00 6e 1f 91 11    movabs $0x5611911f6e00,%rax
   7:   56 00 00
   a:   8a 00                   mov    (%rax),%al
   c:   c3                      retq
```
Thus, the fix helped to inline more and even reduced function size.

There are other places where we append method call costs to `inline_costs` but I decided to start with this one for constructors and only for LLVM to minimize possible regressions.

Possible regressions match current CoreCLR behavior, bad case:  [sharplab.io](https://sharplab.io/#v2:EYLgxg9gTgpgtADwGwBYA0AXEBDAzgWwB8ABAJgEYBYAKGIAYACY8gOgCUBXAOwwEt8YLAMIR8AB14AbGFADKMgG68wMXAG4aNYgGYm5JE1IMhNAN4MLNCxYDaAWRgYAFhAAmASXGSAFA+dvPMUkAeTE+CC5cFgBBAHNY2FxcXgUYdy5JXi4s2IBKAF0rax09A2IUBgAhXli/F1dvXKKLU2brC2YATm8AIgAJHtyNanb2rt6YQeHR4vJunskptrG53sWh5dn5iCWRmb1ujb2Z8Z6AdV39g96do6vTqEv90/Xp59We1yeTj4BCb4YAF9NMcmLpmGUKtVao56mcoNgxGIZI02q1Qe0APSYqo1OpuRoMXi4IkZLIwVwMPAMGAIZFgDAUhhwBgQADWmws0PxDTu1mB1DaJQhTAqABVVBhUaD0VdsQx3FT8AwuBAMDS6TAGTlcTD/JSMBAGMAYKTMlwmVlDQwJbgpU0MdZ5cAOOreG6SSzVer2ZzdTz4YjkVBGm8LALAUA===)

Co-authored-by: EgorBo <[email protected]>
2020-06-18 15:24:32 +03:00
Zoltan Varga 4ac081d6df Bump emscripten to 1.39.18. (#19980) 2020-06-17 19:07:44 -04:00
Vlad Brezae 61808b8f27 [interp] Switch locals and execution stack order (#19975)
* [interp] Switch locals and execution stack order

The order before was execution stack, vtstack, locals. The order now is locals, vtstack, execution stack. The reason for this is that in the future we would like to access arguments as locals, so the args pushed by the caller will end up being locals in the new frame. Additionally, this enables us to further compact the stack use in the future since a new frame will be allocated immediately following the current stack top. This also provides a small reduction in call overhead.

* [interp] Add stricter assertions for debug build

* [interp] Don't inline vararg calls

They have some weird call semantics

* [interp] Fix failure to align vt result to the vt alignment

* [interp] Remove pointless check

We already know we have a managed pointer on the stack, since we have the constrained prefix. We were actually losing the STACK_TYPE_MP state when storing/loading from a local, which can only be MINT_TYPE_I, and could underflow the vtstack.

* [interp] Replace warnings with asserts
2020-06-17 21:20:47 +03:00
Alexander Köplinger 74f6d2ce2d [netcore] Disable Interpreter CI runs (#19979)
We have those over in dotnet/runtime now.
2020-06-17 14:19:40 +02:00
monojenkinsandvargaz ac71dd3a3a Move the Buffer.Memmove<T> implementation into the runtime specific files since it requires a runtime specific BulkMoveWithWriteBarrier icall. (#19974)
The mono version has an extra type argument since it uses copy functions which are type specific.
Also add an elementCount > 0 check to avoid icall overhead if possible.

Co-authored-by: vargaz <[email protected]>
2020-06-16 14:07:52 -04:00
monojenkinsandvargaz 3a5d91fee2 [netcore] Implement timers for wasm. (#19967)
Co-authored-by: vargaz <[email protected]>
2020-06-16 02:05:41 +02:00
monojenkinsandvargaz bab1cde432 [wasm] Disable aot support in release builds. (#19959)
Co-authored-by: vargaz <[email protected]>
2020-06-15 14:15:23 +02:00
Vlad Brezae 35153475a2 [interp] Refactor ldfld from vt (#19963)
Add separate faster opcode and remove use of MINT_VTRESULT
2020-06-13 01:07:57 +03:00
Alexis Christoforides 540da38191 [merp] Add API methods for getting hashcode/reason of last crash (#19901)
* [merp] Add API methods for getting hashcode/reason of last crash

* Add test

* Review feedback and cleanup
2020-06-11 22:45:12 -04:00
Zoltan Varga e51b4af9cd [llvm] Emit a descriptive name for MONO_PATCH_INFO_JIT_ICALL_ADDR_NOCALL as well. (#19947) 2020-06-10 15:13:35 -04:00
Zoltan Varga 4966fc22c4 [wasm] Fix marshalling of pointer types. (#19946)
Fixes https://github.com/mono/mono/issues/19943.
2020-06-10 04:00:41 -04:00
Vlad Brezae 07e6f1e605 [interp] Don't save ip twice in InterpFrame (#19909)
We had frame->ip used by EH and debugger and frame->state.ip used to save the ip of the next instruction to be executed when we return from the call. Since both represent more or less the same thing, unify them and avoid duplicate storing inside InterpFrame for each call. Because we use this ip with exception handling, we need to subtract 1 (similar to the jit) so the ip ends up being in the call instruction and not the next.
2020-06-09 22:00:48 +03:00
monojenkinsandakoeplinger c99dcebb15 [mono] Add back sys/types.h include for clockid_t (#19941)
Looks like it was accidentally removed in https://github.com/dotnet/runtime/pull/37457

Co-authored-by: akoeplinger <[email protected]>
2020-06-09 19:06:41 +02:00
monojenkinsandfanyang-mono 363e89b403 Check the existence of clocked_t (#19915)
Fixes https://github.com/mono/mono/issues/19733

Co-authored-by: fanyang-mono <[email protected]>
2020-06-09 11:08:16 -04:00
Tammy Qiu e19ce7b433 Merge pull request #19938 from tqiu8/update-pkg-script
[wasm] Update package update scripts to include debug proxy
2020-06-09 10:31:16 -04:00
Zoltan Varga aa46acadd4 [wasm] Throw PlatformNotSupportedException instead of NotSupportedException when starting threads/waiting on monitors. (#19937)
* [wasm] Throw PlatformNotSupportedException instead of NotSupportedException when starting threads/waiting on monitors.

* Preseve PlatformNotSupportedException.
2020-06-08 19:41:40 -04:00
tqiu8 a0658ed9b4 [wasm] comment out steps updating local mono repo 2020-06-08 16:45:24 -04:00
tqiu8 965b2eb778 [wasm] incorporate pr feedback & fix powershell script 2020-06-08 16:31:58 -04:00
Tammy Qiu f37c1c91dc [wasm] Update package update scripts to include debug proxy 2020-06-08 13:39:59 -04:00
Rickard df2f9b01d2 Avoid setting PTHREAD_PRIO_INHERIT on Alpine, since this causes deadlocks (#19914)
Fixes #7167

The PR 3e8108ea65 introduced a deadlock on Alpine, making it impossible to even build mono. This PR reverts the setting of PTHREAD_PRIO_INHERIT when on Alpine.
Since GCC on Alpine doesn't provide a builtin macro to detect Alpine, I also added a #define in configure.ac.
2020-06-08 18:57:04 +02:00
Mike Gorse faee046f95 Use new ucontext handling for glibc 2.26 (#19935) 2020-06-08 10:36:26 -04:00
Zoltan Varga 5ec4240614 [wasm] Bump emscripten to 1.39.17. (#19929)
* [wasm] Bump emscripten to 1.39.17.

* [interp] Optimize MINT_DIV_I4/DIV_I8/REM_I4/REM_I8 to work around an llvm problem.
2020-06-08 08:24:33 -04:00
monojenkinsandEgorBo 497152e37c [mono] Append -sroa and -instcombine to LLVM-JIT pass manager (#19916)
Partially fixes https://github.com/dotnet/runtime/issues/37449
```csharp
static ReadOnlySpan<byte> Arr
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    get => new byte[] {1, 2, 3, 4};
}

[MethodImpl(MethodImplOptions.NoInlining)]
public static byte GetByte(int i) => Arr[0];
```
#### Codegen for GetByte() in LLVM-JIT mode:
```asm
0000000000000000 <gram_GetByte__int_>:
<BB>:1
   0:   48 83 ec 28             sub    $0x28,%rsp
   4:   c5 f8 57 c0             vxorps %xmm0,%xmm0,%xmm0
   8:   c5 f8 29 04 24          vmovaps %xmm0,(%rsp)
   d:   48 b8 b0 bd 55 d2 3f    movabs $0x563fd255bdb0,%rax
  14:   56 00 00
  17:   48 89 04 24             mov    %rax,(%rsp)
  1b:   c7 44 24 08 04 00 00    movl   $0x4,0x8(%rsp)
  22:   00
  23:   48 8b 04 24             mov    (%rsp),%rax
  27:   48 89 44 24 10          mov    %rax,0x10(%rsp)
  2c:   8b 44 24 08             mov    0x8(%rsp),%eax
  30:   89 44 24 18             mov    %eax,0x18(%rsp)
  34:   8b 44 24 0c             mov    0xc(%rsp),%eax
  38:   89 44 24 1c             mov    %eax,0x1c(%rsp)
  3c:   83 7c 24 18 00          cmpl   $0x0,0x18(%rsp)
  41:   74 0c                   je     4f <gram_GetByte__int_+0x4f>
  43:   48 8b 44 24 10          mov    0x10(%rsp),%rax
  48:   8a 00                   mov    (%rax),%al
  4a:   48 83 c4 28             add    $0x28,%rsp
  4e:   c3                      retq
  4f:   48 b8 d0 ae 42 d2 3f    movabs $0x563fd242aed0,%rax
  56:   56 00 00
  59:   bf 02 01 00 00          mov    $0x102,%edi
  5e:   ff 10                   callq  *(%rax)
```

Codegen after I appended `-sroa -instcombine` to the end of LLVM passes list:
```asm
0000000000000000 <gram_GetByte__int_>:
<BB>:1
   0:   48 b8 d0 ef a6 ba 31    movabs $0x5631baa6efd0,%rax
   7:   56 00 00
   a:   8a 00                   mov    (%rax),%al
   c:   c3                      retq
```

This https://godbolt.org/z/YKjzsV link explains motivation (on the left is the "final" LLVM IR our llvm-jit produces after the default optimizations). Zoltan noticed that LLVM-AOT where we use `opt -O2` instead of custom pass order optimized that code perfectly.

The other issue remains: for some reason we don't inline `get_Arr()` without AggressiveInlining, coreclr does inline it:
```
Successfully inlined Program:get_Arr():System.ReadOnlySpan`1[Byte] (12 IL bytes) (depth 1) [below ALWAYS_INLINE size]
```

Co-authored-by: EgorBo <[email protected]>
2020-06-08 09:36:38 +03:00
monojenkinsandCoffeeFlux b4092e2011 [runtime] Add remaining runtime events and fix some existing ones on netcore (#19799)
Fixes https://github.com/mono/mono/issues/16246

Contributes to https://github.com/mono/mono/issues/14788

The commits here should be reasonably chunked up, but a summary of what this accomplishes:

* Wires up OnResourceResolve
* Fixes our corlib satellite assembly check in netcore_load_reference (we were previously comparing "System.Private.Corelib" to "System.Private.Corelib.resources", which does not work)
* Adds some useful classes to mono_defaults
* Removes legacy events from MonoDomain and prevents calling them on netcore
* Fixes UnhandledException to call the correct event
* Wires up FirstChanceException

`AppDomainTests.AssemblyResolve_FirstChanceException` is now failing because of the exception type, so it should work once I sort out https://github.com/dotnet/runtime/issues/34030. Until then, keep it disabled.

Co-authored-by: CoffeeFlux <[email protected]>
2020-06-08 02:17:49 -04:00
Aleksey Kliger (λgeek) ba14e69474 [mono] Move some culture-dependent source files to a separate directory (#19912)
* [mono] Move culture-info-tables.h to a separate directory

We'd like to remove culture-info-tables.h from the dotnet/runtime copy of mono since we use ICU
there for globalization like the rest of .NET 5.

The plan is:
1. We move the file (and some others) to a separate directory (this PR)
2. We tell the sync bot to skip `culture/` in the future
3. We delete the files (or make them empty) in dotnet/runtime
* Also move locale.c and culture-info.h

* Also move mono_w32process_ver_language_name

* [msvc] don't compile locales.c if it doesn't exist

   It exists for now, but won't after we stop syncing culture/ to dotnet/runtime
2020-06-06 14:48:11 -04:00
imhameed ad0b2b8e3d Start unifying unaligned memory access codepaths. (#19913)
The intrinsic expansions for Unsafe.ReadUnaligned and
Unsafe.WriteUnaligned now use mini_emit_memory_load and
mini_emit_memory_store.

stind is now implemented in terms of mini_emit_memory_store; when used
during compilation of the the sgen nursery check wrapper,
mini_emit_memory_store will no longer longer emit a release fence when
storing a reference under the CLR memory model.

mini_emit_memory_load and mini_emit_memory_store also no longer
decompose unaligned loads/stores to explicit byte-wise memcpy move
sequences; the LLVM IR backend already converts
MONO_INST_UNALIGNED-flagged OP_LOAD*/OP_STORE* operations to LLVM memory
movement operations with alignment 1.

Should fix https://github.com/mono/mono/issues/19142.
2020-06-05 13:44:22 -07:00
Zoltan Varga fdb5e61a5b [aot] Fix the binary search in the llvmonly unbox tramp table. (#19922)
Fixes https://github.com/mono/mono/issues/19879.
2020-06-05 12:20:01 -04:00
Zoltan Varga 2321bba344 [wasm] Fix icall linking with netcore. (#19917) 2020-06-05 10:29:52 -04:00
Zoltan Varga 0250810bc9 [interp] Optimize do_jit_call (). (#19904)
* [interp] Optimize do_jit_call ().

* Move the JIT call related fields to a separate JitCallInfo structure.
* Compute calling convention information and store it into JitCallInfo
  to avoid most type processing at call time.

* Address review comments.
2020-06-05 08:38:35 -04:00
Zoltan Varga 5246aaf492 [wasm] Link out the SR resource string code on netcore. (#19919) 2020-06-05 08:21:40 -04:00
Zoltan Varga f27ccd0432 [wasm] Print out the real stack trace when load_runtime () fails. (#19918) 2020-06-05 08:00:46 -04:00
Zoltan Varga dda38c4d17 [wasm] Make aot-profile/runtime.js work with different executables. (#19921) 2020-06-05 07:58:01 -04:00
Zoltan Varga 9dc089f334 [wasm] Fix the print-table.py script. (#19920) 2020-06-05 07:57:46 -04:00
monojenkinsandvargaz 335a5949ad Fix mono_asmctx_get_name () after dotnet/runtime#37264. (#19903)
Co-authored-by: vargaz <[email protected]>
2020-06-03 13:28:17 -04:00
Zoltan Varga eaa32d1365 [runtime] Add a MONO_ASMCTX_INTERNAL assembly context. (#19892)
Use it to avoid triggering managed load events when loading the AOT container image.
2020-06-02 15:53:20 -04:00
Vlad Brezae 3b2e8e1102 [interp] Fix tracing (#19898)
Regressed recently with the change in return value handling.
2020-06-02 21:30:40 +03:00
Alexis Christoforides 51b5fa6421 [runtime] Fix a case where crash chaining was broken (#19891)
We must always have a call to mono_chain_signal() after mono_handle_native_crash(), in case signal chaining and crash chaining are both enabled.
2020-06-02 14:26:37 -04:00
monojenkinsandEgorBo 335750b3ca [mono] Rename libruntime-android.so to libmonodroid.so (#19889)
`libruntime-android.so` was just a random name I used for our test runner but since we need some name for `dlopen` (we can't use dlopen(NULL)) and to be shared between the test runner and Xamarin.Android let's switch to the XA name.

We could add an API for mono embedding to set desired name but 'm not sure if it's worth an effort now.

Co-authored-by: EgorBo <[email protected]>
2020-06-02 13:19:56 +03:00
Bernhard Urban-Forster 3f5ade5246 [github] remove me from codeowners 2020-06-02 10:24:42 +02:00
monojenkinsandlambdageek 5bf43dd0d0 [interp] Switch to GC Unsafe mode before calling get_interp_to_native_trampoline (#19871)
For PInvokes, `ves_pinvoke_method` is called when we're interpreting the
pinvoke wrapper, after it already executed a transition from GC Unsafe to GC
Safe mode.  At that point, calling back into the runtime we have to temporarily
switch back to GC Unsafe mode.

Example stack trace where we don't:

```
     0x10dbdd7d6 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_dump_native_crash_info
      0x10db7f665 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_handle_native_crash
      0x10dbdcf3f - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : sigabrt_signal_handler
      0x7fff6886af5a - /usr/lib/system/libsystem_platform.dylib : _sigtramp
      0x0 - Unknown
      0x7fff686081ae - /usr/lib/system/libsystem_c.dylib : abort
      0x10dd9e887 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : monoeg_assert_abort
      0x10dd75257 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_log_write_logfile
      0x10dd9ed15 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : monoeg_g_logv_nofree
      0x10dd9edd2 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : monoeg_g_log
      0x10dd8b4fc - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_threads_transition_do_blocking
      0x10dd8cc63 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_threads_enter_gc_safe_region_unbalanced_with_info
      0x10dc40a39 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_coop_mutex_lock
      0x10dc41e05 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_domain_alloc0
      0x10dadba6f - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_tramp_info_register_internal
      0x10dbece91 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : get_interp_to_native_trampoline
      0x10dbebaf9 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : ves_pinvoke_method
      0x10dbe0e58 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : interp_exec_method
      0x10dbdf17b - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : interp_runtime_invoke
      0x10dcbae58 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_runtime_invoke_checked
      0x10dcbacf3 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_runtime_object_init_handle
      0x10dc6367f - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_exception_new_by_name_domain
      0x10dc63523 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_exception_from_name_domain
      0x10dbe81c6 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : interp_exec_method
      0x10dbdf17b - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : interp_runtime_invoke
      0x10dcbae58 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : mono_runtime_invoke_checked
      0x10dcd6e69 - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : start_wrapper_internal
      0x10dcd6b9e - /private/tmp/helix/working/B38709D8/p/libcoreclr.dylib : start_wrapper
      0x7fff68874661 - /usr/lib/system/libsystem_pthread.dylib : _pthread_body
      0x7fff6887450d - /usr/lib/system/libsystem_pthread.dylib : _pthread_body
      0x7fff68873bf9 - /usr/lib/system/libsystem_pthread.dylib : thread_start
```

Co-authored-by: lambdageek <[email protected]>
2020-06-01 17:28:43 -04:00
monojenkinsandlambdageek 2d187d76ae [mono][metadata] Add a method custom attribute scanning function (#19880)
Our normal custom attribute lookup method may trigger loading.  This is undesirable during class initialization, for example.  For this reason, we have `mono_class_metadata_foreach_custom_attr` that scans a class for custom attributes without triggering loading.

This PR adds `mono_method_metadata_foreach_custom_attr` that does the same thing - looks for custom attributes on methods without causing any loader activity.

This PR also reorganizes the code in `class-init.c` that uses these metadata custom attribute lookups to be a bit easier to use.

Finally we add a convenience method (unused so far) to look for `PreserveBaseOverridesAttribute` which is defined in dotnet/runtime#35315 and which will be necessary to implement covariant returns.

Co-authored-by: lambdageek <[email protected]>
2020-06-01 14:29:20 -04:00
Vlad Brezae d5ed68483c [interp] Remove saving of clause_args from interp state (#19857)
* [interp] Remove saving of clause_args from interp state

clause_args can be set only from EH. This means that clause_args can be set only in the first executed frame, so there is no need to save it in the interp state. Also make sure we never check for it in the main_loop block, for normal calls and returns. Added exec_frame to the clause_args so we can detect whether a frame is the first frame invoked from EH. We can no longer use clause_args for this purpose because we no longer save it in the interp state.

* [interp] Remove error argument to interp_exec_method

We were doing this to avoid having a MonoError local in interp_exec_method, which is no longer relevant. Avoid reiniting the error with each interp call.
2020-06-01 10:58:52 +03:00
Zoltan Varga b7a100c036 [wasm] Fix netcore+aot support. (#19883)
* Add more substitutions for SIMD get_IsSupported () methods.

* [wasm] Fix netcore+aot support.
2020-05-31 11:13:34 -04:00
Zoltan Varga 9c0403bc46 [wasm] Print a warning in the pinvoke generator if multiple incompatible declarations would be emitted, and error (#19873)
out at runtime.

Fixes https://github.com/mono/mono/issues/19869.
2020-05-30 13:00:47 -04:00
Zoltan Varga d35dc0bcf4 [wasm] Fix netcore support in the packager. (#19881) 2020-05-30 06:35:12 -04:00
monojenkinsandlateralusX 233077f452 Fix Mono C++ build. (#19875)
Co-authored-by: lateralusX <[email protected]>
2020-05-30 02:53:10 +02:00
monojenkinsandBrzVlad a42f4fea6b [interp] Remove assertions from release and add more to debug builds (#19878)
Co-authored-by: BrzVlad <[email protected]>
2020-05-30 01:42:45 +03:00
Alexander Köplinger e163c5b6a2 Add back whitespace at end of sre-save.c 2020-05-29 11:12:01 +02:00
monojenkinsandalexischr da0a7f2352 [mono] Implement AssemblyBuilder.GetReferencedAssemblies (#19818)
Some info from https://docs.microsoft.com/en-us/dotnet/api/system.reflection.emit.assemblybuilder.getreferencedassemblies?view=netcore-3.1:

> This method does not return a complete list of referenced assemblies. For example, if you apply a custom attribute to the AssemblyBuilder, the assembly in which the attribute was defined is included in the list returned by this method. However, if you use a Type object to specify the type of a method parameter, that type is not included.

Contributes to https://github.com/mono/mono/issues/14788

Co-authored-by: alexischr <[email protected]>
2020-05-29 10:34:48 +02:00
monojenkinsandalexischr d163874806 [mono] Implement AssemblyBuilder.GetSatelliteAssembly (#19870)
Contributes to https://github.com/mono/mono/issues/14788

Co-authored-by: alexischr <[email protected]>
2020-05-29 10:32:40 +02:00
monojenkinsandCoffeeFlux bea273e99c [runtime] Fix Mono BStr marshaling, cleanup IL generation (#19701)
This PR checks in recent string marshaling work and enables a subset of tests that now run successfully. It's possible that this fixes or changes the failure on other tests, but I've only checked the StringMarshalling ones.

Status of string marshaling of this PR:
* AnsiBStr - tests pass
* BStr - tests pass
* LPStr - failure with StringBuilder
* LPTStr - immediate failure
* UTF8Str - crash with invalid malloc in StringBuilder code (??)
* LPWStr - no tests
* TBStr - no tests

Followup work:
- [ ] Fix UTF8 crash (probably corruption, the crash used to be in a free call)
- [ ] Clean up StringBuilder code (lots of issues, some caught by the tests, in progress)
- [ ] Add TBStr and LPWStr tests (in progress)
- [ ] Fix LPTStr to properly handle both ANSI and UTF16 text (or maybe change the test to align with our historical behavior? currently the test uses LPWStr on all platforms)
- [ ] Add a wall of text on string marshaling behavior for classic and netcore Mono at the start of `conv_to_icall`

Co-authored-by: CoffeeFlux <[email protected]>
2020-05-28 15:01:11 -04:00
monojenkinsandvargaz dcfa0ec293 [runtime] Implement the many argument version of the Vector128.Create () intrinsic. (#19851)
Co-authored-by: vargaz <[email protected]>
2020-05-28 09:17:39 -07:00
Zoltan Varga 5c204cfb10 [wasm] Fix the handling of i8/u8 in get_wrapper_shared_type_full (). (#19859)
Previously, these were returned verbatim, which caused sharing issues when the type had
attributes.

Fixes https://github.com/mono/mono/issues/19841.
2020-05-28 10:44:14 -04:00
Jérôme Laban 5c9b63da2e [wasm] Fix packager not emitting debug symbols when ninja is enabled (#19863)
Fix the packager not emitting debug symbols when ninja is enabled (e.g. when statically linking a bitcode file, still using the interpreter).
2020-05-28 11:55:27 +02:00
monojenkinsandsafern 6489563f28 Add ICU App-Local support on Mono (#19852)
This adds AppLocalIcu switch support on Mono. I also refactored GlobalizationMode to share the sources in between Mono and CoreCLR to avoid duplicating all the AppLocal and UseNls switches logic.

cc: @jkotas @thaystg @akoeplinger @tarekgh @ericstj

Co-authored-by: safern <[email protected]>
2020-05-28 10:37:43 +02:00
Zoltan Varga da59b3acab [wasm] Include the pinvoke table into a separate file, so the generated declarations don't conflict with the ones in the header files included by driver.c. (#19853)
Fixes https://github.com/mono/mono/issues/19837.
2020-05-27 09:53:53 -04:00
monojenkinsandBrzVlad 834bf5ede3 [interp] Remove return value indirection (#19845)
When returning from a method we used to save the return value into frame->retval. We would then have separate opcodes for normal and void call and, depending on the type of call, we would access the retval and push it on the stack. In case of valuetype returns, the value is not returned in the proper space so we have an additional MINT_VTRESULT that copies the return around in the caller frame. In addition to this we had retval, is_void locals, saved as part of the current frame state which just make things confusing and are not needed.

This commit simplifies the call procedure by completely moving the responsability of the return to the called method. Simply put, once a method is ready for the call, it will leave its stack and vtstack to the state after pop-ing the arguments. The called method will then directly push the result to the parent frame's stack/vtstack. This means the calling code doesn't care about the type of the return anymore, the IL instructions after the call will just expect the return on the stack, if any, and use it naturally.

We still make use of frame->retval for interp entry frames (since we don't have a parent frame to push to), for pinvoke and jit_call frames out of convenience, but we should consider removing this variable to further trim down InterpFrame.

Co-authored-by: BrzVlad <[email protected]>
2020-05-27 11:58:16 +03:00
Zoltan Varga 14ff5bd29a [wasm] Add more runtime debugging docs. (#19850) 2020-05-26 22:12:57 -04:00
Johan Lorensson 540e8d1d7b Merge pull request #19472 from monojenkins/sync-pr-34600-from-runtime
Initial work porting native diagnostic event pipe library to C library potentially shared between runtimes.
2020-05-26 15:29:18 +02:00
Marek Habersack 53d0017701 [reflection] Check whether a pointer is valid before dereferencing (#19839)
`Xamarin.Android` native runtime calls `mono_reflection_type_from_name`
and passes `NULL` as the `image` parameter.  The parameter is then
propagated all the way to `_mono_reflection_get_type_from_info` where,
in case the assembly isn't loaded yet, it is used to obtain base
directory of the assembly.  However, since the `image` parameter is
`NULL` in our case, attempt to dereference it causes a segfault:

    libc    : Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4c0 in tid 11029 (ompanyname.app3), pid 11029 (ompanyname.app3)
    crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone
    /system/bin/tombstoned: received crash request for pid 11029
    crash_dump64: performing dump of process 11029 (target tid = 11029)
    DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    DEBUG   : Build fingerprint: 'google/sdk_gphone_x86_64/generic_x86_64:10/QSR1.190920.001/5891938:user/release-keys'
    DEBUG   : Revision: '0'
    DEBUG   : ABI: 'x86_64'
    DEBUG   : Timestamp: 2020-05-25 14:45:29+0200
    DEBUG   : pid: 11029, tid: 11029, name: ompanyname.app3  >>> com.companyname.app3 <<<
    DEBUG   : uid: 10134
    DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4c0
    DEBUG   : Cause: null pointer dereference
    DEBUG   :     rax 000000000000002f  rbx 0000000000000001  rcx 0000000000000000  rdx 0000000000000030
    DEBUG   :     r8  0000000000000003  r9  000000000013e2e2  r10 0173eed800000000  r11 0000000000000206
    DEBUG   :     r12 0000000000000000  r13 00007478530343c0  r14 00007478075eda33  r15 000074780763efb0
    DEBUG   :     rdi 0000000000000000  rsi 00007478e2cb14d0
    DEBUG   :     rbp 00007ffef3a35680  rsp 00007ffef3a355d0  rip 0000747807a4066a
    DEBUG   :
    DEBUG   : backtrace:
    DEBUG   :       #00 pc 00000000003ba66a  /data/app/com.companyname.app3-aQUF6Ge6_v-WaLb5i8Q7vw==/lib/x86_64/libmonosgen-2.0.so (_mono_reflection_get_type_from_info+474)
    DEBUG   :       #01 pc 00000000003ba3d1  /data/app/com.companyname.app3-aQUF6Ge6_v-WaLb5i8Q7vw==/lib/x86_64/libmonosgen-2.0.so (mono_reflection_type_from_name_checked+321)
    DEBUG   :       #02 pc 00000000003ba26d  /data/app/com.companyname.app3-aQUF6Ge6_v-WaLb5i8Q7vw==/lib/x86_64/libmonosgen-2.0.so (mono_reflection_type_from_name+125)
    DEBUG   :       #03 pc 000000000000ddb5  /data/app/com.companyname.app3-aQUF6Ge6_v-WaLb5i8Q7vw==/lib/x86_64/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(char const*)+389) (BuildId: 9952f1cfe0d910ae631abc73479f88eef34fd71d)
    DEBUG   :       #04 pc 000000000000def3  /data/app/com.companyname.app3-aQUF6Ge6_v-WaLb5i8Q7vw==/lib/x86_64/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(_MonoString*)+99) (BuildId: 9952f1cfe0d910ae631abc73479f88eef34fd71d)
    DEBUG   :       #05 pc 0000000000069532  <anonymous:5ad25000>

Even though this happens in `Xamarin.Android`, the error may occur for
any embedding application which passes `NULL` for the `image` parameter
in situation when the assembly isn't in memory yet.
2020-05-26 09:25:33 -04:00
Zoltan Varga 38356be989 [wasm] Remove obsolete emscripten options. (#19844) 2020-05-26 07:33:45 -04:00
Johan Lorensson df8f3c3c3d Create Makefile 2020-05-25 21:08:08 +02:00
lateralusX 6bb89469e8 Initial work porting native diagnostic event pipe library to C library potentially shared between runtimes.
PR handling port of EventPipe C++ library from CoreCLR into a C library that can be shared between Mono as well as CoreCLR runtime.

Due to dependencies and current time line it has been decided to initially target Mono runtime (since it doesn't have event pipe functionality at the moment), but library is still designed to integrate into other runtimes, replacing current C++ library implementation, when time permits.

Until that happens, we will need to make sure changes done in C++ code base gets ported over to C library. The amount of changes going into the native library should however be small and if major changes needs to be done, we should first make sure to move over to use the shared version of event pipe C-library in CoreCLR and then only do changes into shared version of the library.

Port is currently mapping existing C++ code base, no focus on rewrite or change current implementation (unless needed to port code over to C).

Library source files are currently placed into mono/eventpipe folder and build as a separate library, linked into runtime, but will be moved when ready to be shared between runtimes. We might also move all code into a loadable library and p/invokes, depending on size, at least for Mono runtime targeting mobile devices.

There is also a unit test runner under mono/eventpipe/tests using eglib test runner, running through a set of unit tests covering the eventpipe library implementation.

Library is split up over a couple of source files closely mapping corresponding coreclr source files. Most code could be shared between runtimes and artifacts that are runtime specific, like locks/lists/maps/strings/pal layer etc, a shim API is defined in `ep-rt.h` and `ep-rt-types.h` that gets implemented by the targeted runtime consuming the event pipe C library, like `ep-rt-mono.h` and `ep-rt-types-mono.h`. All function in the shim API is small and should be inlinable. For example, locking the configuration lock is declared like this in `ep-rt.h`:

```
bool
ep_rt_config_aquire (void);
```

and in `ep-rt-mono.h` it is implemented like this:

```
static
inline
bool
ep_rt_config_aquire (void)
{
	ep_rt_spin_lock_aquire (ep_rt_mono_config_lock_get ());
	return true;
}
```

and CoreCLR should implement the function using its corresponding locking primitives. NOTE, since we primarily target Mono in initial port there might smaller adjustments needed to the shim API once bringing it over to CoreCLR, but those changes/adjustments can be done when needed.

Since library could be shared, it tries to standardize on common C99 data types when available. If there are runtime specific types needed, they are declared in `ep-rt-types.h` and implemented by each runtime in corresponding file. Most of library is currently following Mono coding guidelines.

For shared types naming is done like this, `EventPipe[Type]`, like `EventPipeProvider`. This naming follow naming of other types in Mono as well as CoreCLR. There is one exception to this rule, types that are runtime specific implementation, naming is following function naming with a _t prefix, like `ep_rt_session_provider_list_t`, that is a list specific type, implemented by each runtime. This is done to make it easier to see difference of types implemented in the runtime specific shim and types implemented in the shared library API surface.

Since library shim implemented by targeted runtime will be called by C code, it can't throw exceptions, the C library should however compile under C++, meaning that CoreCLR's shim should be able to use C++ constructions as long as it don't throw any exceptions.

Library function naming follows the following pattern, for "public" functions included in `ep-*.h` , ep_ + object + action, so for example, `ep_config_create_provider`, corresponds to `EventPipe::Configuration::CreateProvider` in C++ library. For "private/internal" functions not, ep_ prefix is dropped, but object + action is still preserved, for example `config_create_provider` and function is marked as being static when possible. If a function assumes a lock is held when getting called, it is using _lock_held (similar to SAL annotations) postfix to indicate that function doesn't claim lock and if function needs lock to be held by caller, it will assert that condition on enter (similar to `PRECONDITION(EventPipe::IsLockOwnedByCurrentThread())`).

Library uses asserts internally to validate state and conditions. This is a debug/checked build assert that should be excluded in release builds. It is defined in `EP_ASSERT` and should be redefined by targeted runtime. Private/internal functions asserts incorrect input parameters and state, while "public" functions return back error to caller when detecting incorrect input parameters. Incorrect state can still be assert in all kinds of functions.

Getters/Setters are setup for accessing type members. All functions except a type's own alloc/init/fini/free implemented in `ep-*.c` are not allowed to use direct type member access and should go through available inlinable getter/setter functions. There is also a way to build the library that catches incorrect use of members at compile time (not in `ep-*-internals.c` since it include alloc/init/fini/free implementations, but in other source files).

Types that can be stack allocated offers a init/fini function pairs. Types that should only be heap allocated, alloc/free, and types supporting both stack/heap, implements alloc/init/fini/free. Since C can't prevent incorrect usage patterns, the implementation needs to follow correct life time management patterns. Types allocated using alloc should be freed with corresponding free function. For example, `ep_event_alloc` should be freed by `ep_event_free`, the init/fini methods are then called by the alloc/free implementations. For types supporting stack allocation and allocated on stack or aggregated within other types, init must be called before first use and fini before going out of scope.

Ownership of allocated objects is only considered complete on successful function calls. If a function fails, caller still owns resource and need to take actions.

Library uses UTF8 strings internally, mainly since Mono runtime already uses that and it is more portable. Number of strings in the library is limited, and in cases where stream protocol uses UTF16 strings, a prealloc string is stored on the type, reducing need to do multiple UTF8 -> UTF16 conversions over the lifetime of the type.

NOTE, this is first PR setting up most of the direction of the library and (as part of doing that) ported a great deal of eventpipe code and added ~40 native unittests. It is however not complete, or fully working, so there will be several follow up PR's building on top of this work in order to get complete library. Next component to integrate into the library, once this PR is merged, is BufferManager.
2020-05-25 19:06:54 +00:00
w-flo dd9bc98945 configure.ac: check for sockaddr_in6 on Win32 (#19812)
HAVE_STRUCT_SOCKADDR_IN6 needs to be defined to enable IPv6 features in
mono/metadata/w32socket.c.

Fixes TcpListener issues in wine-mono.
2020-05-25 17:08:05 +02:00
Zoltan Varga 02e83f0ddc [wasm] Avoid overwriting the pinvoke table if not changed. (#19833) 2020-05-23 00:25:39 -04:00
Zoltan Varga 8446996cab [jit] Return the compiled method and an unbox trampoline from ldvirtftn when calling ldvirtftn on a valuetype method. (#19809)
* [jit] Return the compiled method and an unbox trampoline from ldvirtftn when calling ldvirtftn on a valuetype method.

Fixes https://github.com/dotnet/runtime/issues/34379.

* [runtime] Add some caching to mono_aot_get_unbox_trampoline (). Register unwind info for trampolines.

* Add a 'method' field to MonoTrampInfo to be used by unbox trampolines.

* Refactor delegate initialization code.

Also handle unbox trampolines during the addr->method lookup.

* Merge mono_delegate_ctor () and mono_delegate_ctor_with_method ().
* Move the actual logic to mini_init_delegate ().
* Remove callbacks.create_jump_trampoline, its not used any more.

* Fix the build.

* Remove an interpreter workaround.

* Avoid leaking the amodule->unbox_tramp_per_method array.
2020-05-23 00:17:50 -04:00
Thays Grazia 8f0fbfcd95 Fixing make dist after implementation of static ICU Shim. (#19819) 2020-05-22 17:23:15 -04:00
monojenkinsandCoffeeFlux 8e071069af Add MONO_LOADER_LIBRARY_NAME define (#19830)
Co-authored-by: CoffeeFlux <[email protected]>
2020-05-22 17:21:12 -04:00
Zoltan Varga d2da900426 [wasm] Preserve UnmanagedFunctionPointerAttribute. (#19829)
Fixes https://github.com/mono/mono/issues/19826.
2020-05-22 03:41:19 -04:00
Ankit Jain a1b68d97c3 [wasm][debugger] Allow invoking property getters for objects (#19726)
* [wasm][debugger][tests] Move the test files to `tests/debugger`

- Also, rename `class Math` to `class SampleMath` in `sample.cs`,
to allow building sample.dll, with all the tests added.

(cherry picked from commit dc265710139d7c3a63f714e4b19bdb60014b5bcb)

* [wasm][debugger][tests] Add back missing dependency

(cherry picked from commit 2bc6482bb3049b34da918e2dc91df459774873c9)

* [wasm][debugger][tests] move *datetime-test.cs to `tests/debugger`

(cherry picked from commit 2b43270a42a599fe2bdb460018460e32571bbc79)

* [wasm][debugger][tests] Move CFO test to debugger-cfo-test.cs

(cherry picked from commit aba95ffddad94a613a9a12eaf0dddcc9dd661b58)

* [wasm][debugger] Allow invoking property getters for objects

- These can be invoked interactively by the user from the IDE/chrome
debugger
- Getters on valuetypes are not supported right now

* [wasm][debugger] Fix warnings

* [wasm][debugger][tests] Fix DateTimeLocale test to await getting

.. properties for a DateTime object. Required after rebasing on master.
2020-05-21 22:38:16 -04:00
Alex Thibodeau 7490cc0e2b Adding check to bypass decoding pdbs that have no sequence points. (#19815)
Unity has a pdb with zero sequence points somehow. If a user added a reference to the library and then attempted to set a managed breakpoint mono would hit an assert and crash when it would attempt to decode this pdb.

Call stack would look like:
```
>	mono-2.0-boehm.dll!mono_ppdb_get_seq_points(_MonoDebugMethodInfo * minfo, char * * source_file, _GPtrArray * * source_file_list, int * * source_files, MonoSymSeqPoint * * seq_points, int * n_seq_points) Line 508	C
 	mono-2.0-boehm.dll!mono_debug_get_seq_points(_MonoDebugMethodInfo * minfo, char * * source_file, _GPtrArray * * source_file_list, int * * source_files, MonoSymSeqPoint * * seq_points, int * n_seq_points) Line 1093	C
 	mono-2.0-boehm.dll!get_source_files_for_type(_MonoClass * klass) Line 6920	C
 	mono-2.0-boehm.dll!get_types_for_source_file(void * key, void * value, void * user_data) Line 7003	C
 	mono-2.0-boehm.dll!monoeg_g_hash_table_foreach(_GHashTable * hash, void(*)(void *, void *, void *) func, void * user_data) Line 364	C
 	mono-2.0-boehm.dll!mono_de_foreach_domain(void(*)(void *, void *, void *) func, void * user_data) Line 95	C
 	mono-2.0-boehm.dll!vm_commands(int command, int id, unsigned char * p, unsigned char * end, Buffer * buf) Line 7341	C
 	mono-2.0-boehm.dll!debugger_thread(void * arg) Line 10323	C
 	mono-2.0-boehm.dll!start_wrapper_internal(StartInfo * start_info, unsigned __int64 * stack_ptr) Line 1241	C
 	mono-2.0-boehm.dll!start_wrapper(void * data) Line 1315	C
 	kernel32.dll!00007ffc12017bd4()	Unknown
 	ntdll.dll!00007ffc121ece51()	Unknown
```

Adding a check to ignore pdbs like this prevents the crash and debugging can continue normally.

Related unity issue: https://issuetracker.unity3d.com/issues/macos-editor-crashes-on-mono-log-write-logfile-when-attaching-a-debugger-and-then-setting-a-breakpoint
2020-05-21 14:24:12 -04:00
monojenkinsandthaystg 2f56fc19dc Fixing tests on android that uses Icu Shim Globalization (#19817)
The linker was removing the icu shim functions even exporting them. The unique solution that we found until now is to force the linking using the -u flag.
This is a temporary fix until we don't implement QCalls.
Fixes https://github.com/dotnet/runtime/issues/36685

Co-authored-by: thaystg <[email protected]>
2020-05-21 15:28:13 +02:00
monojenkinsandalexischr 78dbf5b308 [mono] Implement AsAny marshalling for simple arrays (#19683)
Simple here means of rank 1, and with a blittable value type as element type.

Contributes to https://github.com/dotnet/runtime/issues/35225

Co-authored-by: alexischr <[email protected]>
2020-05-21 10:05:56 +02:00
Vlad Brezae 783b96aa4c [interp] Add separate opcode for pop vt (#19814)
* [interp] Add separate opcode for pop vt

Before this change, pop-ing a vt from the stack was done in 2 opcodes, a MINT_POP (decrementing the stack) and the weird MINT_VTRESULT (decrementing the vtstack). However, optimizations could have removed both the initial loading of the value type on the stack as well as the MINT_POP, leaving MINT_VTRESULT underflowing the vtstack. Fix this and cleanup the code by pop-ing a value type from the stack in a single instruction.

* [interp] Re-enable optimizing away loading of unused value type on the stack
2020-05-21 00:35:51 +03:00
Zoltan Varga cec64d9b7f [interp] Fix interp entry for methods with lots of arguments in llvmonly+interp mode. (#19802)
Fixes https://github.com/mono/mono/issues/19801.
2020-05-20 11:20:19 -04:00
Vlad Brezae 0c87593336 [interp] Small cleanups (#19805)
* [interp] Refactor calli code

Previous code was very confusing.

* [interp] Extract string creation into separate opcode

For speed and simplification of code.
2020-05-20 13:51:43 +03:00
Jo Shields e2b1e065d0 Always include Unicode charinfo, so tar made in csc mode works in mcs mode (#19807)
As-is, a tarball created in Roslyn mode is missing the file charinfo.nlp, which mcs needs to build the classlib.
2020-05-20 11:45:20 +02:00
Andy Savage 2918655063 Added hard limit to sub-make jobs. (#19798)
Fixes #19784
2020-05-20 11:40:35 +02:00
Ankit Jain 98320ecbde [wasm][debugger][tests] Allow overriding which browser to use (#19769)
* [wasm][debugger] Reduce debugger-tests timeout to 40m

- And run that before the system tests since the debugger is more likely
to fail at this point.

Based on @lewing's suggestion

* [wasm][tests] Allow overriding which browser to use

.. via the envvar `WASM_TESTS_ChromePath`.

This can also be set in an `appsettings.json` file next to the test
binaries, like:
```
{ "ChromePath": "</path/to/chrome/binary>" }
```

Also, prefer stable Chrome over Canary.

Co-authored by @lewing
2020-05-18 15:02:38 -04:00
Zoltan Varga 4490662bc0 [wasm] Bump emscripten to 1.39.16. (#19793) 2020-05-18 10:36:30 -04:00
monojenkinsandthaystg 01be43b9cb [mono] Linking statically ICU shim on mono (#19699)
Linking statically ICU shim on mono for windows, linux, macOs and android.

Co-authored-by: thaystg <[email protected]>
2020-05-18 11:02:21 -03:00
w-flo 669ba26333 [corlib] Improve RuntimeThread stub (#19782)
The only call site of RuntimeThread.InitializeThreadPoolThread() is in
ThreadPoolCallbackWrapper, and it needs to return a valid RuntimeThread
reference to prevent a NullReferenceException later on. So return
`new RuntimeThread (null)` instead of a null reference.

This fixes a wine-mono crash I have encountered in "ZUSI 3 - Aerosoft
Edition" (in ZusiDisplay), and it might fix mono bug #17994.
2020-05-18 11:52:53 +02:00
monojenkinsandakoeplinger e35988adc0 Fixes for iOS x86 build (#19795)
Needed for the iOS Simulator for 32bit iOS devices.

Co-authored-by: akoeplinger <[email protected]>
2020-05-18 11:28:11 +02:00
Vlad Brezae b552da81ff [interp] Don't share interp_in signatures for different valuetypes (#19788)
* [interp] Don't share interp_in signatures for different valuetypes

We share interp_in wrappers for different types of signatures if the corresponding params are equivalent. This was added in https://github.com/mono/mono/commit/5cbe93884798684efbb81abd79e0e2a170544b75. This was reusing some sharing mechanism used by gsharedvt. Those wrappers are shared with regard to managed->managed transitions so it takes additional freedoms, converting all valuetypes to ValueTuples instances. These can end up being marshalled differently from the initial struct so we can't use this valuetype sharing infrastructure in the interp_in_wrappers which can operate on native structs.

Fixes test_0_marshal_struct_delegate from pinvoke3.cs

* [interp] Keep sharing interp_in signatures with VT if not pinvoke

As suggested by zoltan
2020-05-16 23:47:28 +03:00
Alexander Köplinger 38584afad0 Bump bockbuild to get https://github.com/mono/bockbuild/pull/159 2020-05-15 14:30:41 +02:00
monojenkinsandlambdageek 8c8497e79f [Mono] Don't set thread name of main thread on Linux (#19742)
Fixes https://github.com/dotnet/runtime/issues/35908

This adds a new function `mono_native_thread_main_thread_known` which returns `TRUE` when the `MonoNativeThreadId` (`pthread_t` on Linux) of the main thread is known.  On Linux the main thread is the one where `gettid () == getpid ()`.  There are two complications:
1. `pthread_t` and `pid_t` are distinct tokens, so knowing the `pid_t` ("os id" in Mono) of the main thread doesn't tell us its `pthread_t` (`MonoNativeThreadId`)
2. In an embedding scenario, the main thread may never interact with Mono (for example all the calls to `mono_jit_init`, etc might be happening on a helper thread).
So the solution is to augment `register_thread` (responsible for initializing a `MonoThreadInfo`) to check for `getpid () == gettid()` and to save `pthread_self()` as the "main thread" when it becomes available.

Co-authored-by: lambdageek <[email protected]>
2020-05-14 15:57:03 -04:00
Ankit Jain 1a022edfcb [wasm][debugger] Clear valuetypes cache on Debugger.resume (#19778)
* [wasm][debugger] Clear valuetypes cache on Debugger.resume

- This fixes:
	1. Check valuetype properties
	2. Debugger.resume, over some code that updates that valuetype
	3. That valuetype's properties show older values

* [wasm][debugger] Send mono_wasm_debugger_resume only if stopped on

.. a managed frame.
2020-05-14 11:48:51 -04:00
Thays Grazia 13c5c4129e [debugger] Removing some asserts (#19758)
* Removing some asserts and returning err_invalid_argument with an error message when it's possible.

* if we don't find method get_ObjectIdForDebugger we try to find the property Task to continue async debug.
2020-05-14 09:16:53 -03:00
monojenkinsandakoeplinger f24a85069b Fix mono runtime build warnings when building iOS config (#19779)
```
  /Users/alexander/dev/runtime/src/mono/mono/mini/aot-runtime.c:5647:13: warning: unused variable 'image' [-Wunused-variable]
```
```
  /Users/alexander/dev/runtime/src/mono/mono/mini/simd-intrinsics-netcore.c:11:1: warning: no previous prototype for function 'mono_simd_intrinsics_init' [-Wmissing-prototypes]
```
```
  /Users/alexander/dev/runtime/src/mono/mono/utils/mono-state.c:1230:1: warning: no previous prototype for function 'mono_crash_save_failfast_msg' [-Wmissing-prototypes]
  /Users/alexander/dev/runtime/src/mono/mono/utils/mono-state.c:1236:1: warning: no previous prototype for function 'mono_crash_get_failfast_msg' [-Wmissing-prototypes]
```

Co-authored-by: akoeplinger <[email protected]>
2020-05-14 12:13:04 +02:00
Ankit Jain bab72f364a [wasm][debugger][tests] Rework SetBreakpointInMethod, to get the (#19771)
* [wasm][debugger][tests] Rework SetBreakpointInMethod, to get the

.. location info from the proxy, and then use `setBreakpointByUrl` to
set the breakpoint.

* [wasm][debugger] Bump timeout for debugger tests, since we are not running

.. tests in parallel now.
2020-05-14 02:15:04 -04:00
Maxim Lipnin b9d0edb407 [wasm] Re-enable some corlib tests (#19642) 2020-05-14 00:16:37 -05:00
monojenkinsandimhameed 3905678ac5 Improve Sse41.Multiply compatibility with LLVM 9. (#19777)
The llvm.x86.sse41.pmuldq intrinsic is no longer available as of
https://github.com/dotnet/llvm-project/commit/254ed028a4bd4fa81d0049d90e6ab23d704dd366.

This also adds a convenience wrapper for creating constant LLVM IR
vectors, up to 16 elements long, of arbitrary integral type.

Finally, fix a latent bug in init_function_pass_manager;
llvm::StringRef::size returns a value of type size_t, but the printf
precision specifier requires a value of type int.

Co-authored-by: imhameed <[email protected]>
2020-05-13 20:17:12 -07:00
mathieubourgeois b66f812449 Emit DWARF debug_abbrev offset for compile units as a label instead of 0 (#19770)
When outputting DWARF code to start a compilation unit in .debug_info, the standard expect a 4-byte offset from the .debug_abbrev code. Mono has always output an offset of 0.

However, this doesn't work in every cases. When we have linux+fullaot, we link two object files (one from Mono, one from LLVM). Both have their .debug_abbrev section. If we use 0 as an offset, it seems possible that the linker will keep thinking that our offset is 0, no matter the circumstances. Since the offset is always 0, it can be using the wrong abbreviation table (i.e. the one from the LLVM assembly instead of the one from the Mono assembly). The consequence of this is that the linked file is not valid DWARF (dwarfdump and objdump will complain about invalid offsets). At best, some tools will be able to work with a part of what we have, but any program requiring entirely valid DWARF will fail.

To fix this, we generate a label for the start of our debug_abbrev section and we instead generate it by generating a long with that label. This matches existing behavior seen in the LLVM generated code, and makes dwarfdump and objdump react properly to the linked product.

Fixes https://github.com/mono/mono/issues/8806
2020-05-13 14:22:51 +02:00
Zoltan Varga f265d9a783 [runtime] Move delegate_bound_static_invoke_cache to MonoWrapperCaches. (#19766) 2020-05-13 08:01:40 -04:00
Alexander Köplinger bcbb4cf5a6 Revert accidental api-snapshot commit 2020-05-13 10:53:46 +02:00
monojenkinsandtqiu8 b7274d8a27 [master] [wasm][debugger][tests] Add locale tests for datetime (#19759)
* [wasm][debugger][tests] Add locale tests for datetime

* incorporate PR feedback & reference issue number for disabled locales

Co-authored-by: tqiu8 <[email protected]>
2020-05-12 13:57:35 -04:00
Thays Graziaandmonojenkins 33b0653c2c [debugger] Merge debugger-libs/Mono.Debugger.Soft (#19725)
Trying to make mono/debugger-libs/Mono.Debugger.Soft be the same as mono/mono/mcs/class/Mono.Debugger.Soft

Relates to https://github.com/mono/debugger-libs/pull/323
* Bump API snapshot submodule

Co-authored-by: monojenkins <[email protected]>
2020-05-12 18:18:49 +02:00
Zoltan Varga 430570e18b [wasm] Bump emscripten to 1.39.15. (#19744) 2020-05-11 13:33:27 -04:00
Calvin Buckley 169a842f4e Fix musl PowerPC build (#19747)
PT_NIP and friends are in this kernel header, but glibc includes it
somewhere in the chain implicitly. musl doesn't. Fix that.
2020-05-11 10:34:16 +02:00
Vlad Brezae 866914eeff [tests] Adjustments to interp disabled tests (#19748)
* [interp] Disable fullaotmixed suite on x86

* [interp] Small cleanup of disabled tests

Enable a bunch of tests on arm (which I thought I enabled 2 years ago). Avoid duplication by moving some arch specific disabled tests to all platforms. Enable a few tests that seem to have been fixed in the meantime.
2020-05-10 22:39:56 +03:00
Vlad Brezae 8e1f242bdb [interp] Fix sdks build (#19745)
* [interp] Fix sdks build

* [interp] Fix build
2020-05-09 11:26:30 +03:00
Vlad Brezae 798f22399e [interp] Add x86 support (#19730)
* [interp] Aling data to 8 bytes

On x86 linux, double doesn't guarantee 8 byte alignment, which we need since all objects on the vt stack need to be 8 byte aligned. Use padding instead to achieve this.

* [interp] Add x86 support

This mainly consists of a trampoline that is used when calling from interp into native code, as well as the call convention logic that initializes the CallContext structure (which is passed to the trampoline). On other platforms we also have a single trampoline used when entering interpreter code. That trampoline works in a similar way and its purpose is to avoid compilation of all possible interp entry signatures, which is problematic on full aot scenarios. Given the interp x86 support is currently provided for android, we can avoid using that trampoline for now and use the per signature jit compiled interp_in_wrappers instead.

The few disabled interp tests are the same as on amd64.

We only support pinvoke into cdecl functions.
2020-05-08 17:44:21 +03:00
Zoltan Varga 049de7513a [interp] Fix memory overwrite during pinvoke r4 parameter passing. (#19709)
* [interp] Fix memory overwrite during pinvoke r4 parameter passing.

Fixes https://github.com/mono/mono/issues/19684.

* [interp] Allocate 1 slot for every pinvoke float arg only.
2020-05-08 05:35:04 -04:00
Larry Ewing 680481793a [wasm][debugger] Only send resolved events on load (#19736)
* [wasm][debugger] Only send resolved events on load

* Add the request to the context after potentially loading the store

* improve the resolution logic
2020-05-07 23:27:59 -05:00
Larry Ewing 8d774f90bf [wasm][debugger] Get loaded files without having a round trip (#19735) 2020-05-07 21:37:38 -05:00
Zoltan Varga ae1ba25fb1 [wasm] Use the official emsdk repo. Use the same version of emsdk as emscripten. (#19737)
* [wasm] Use the official emsdk repo. Use the same version of emsdk as emscripten.

* [wasm] Reclone the wasm repo when the version changes.
2020-05-07 22:30:45 -04:00
monojenkinsandBrzVlad cc7c5855c3 [interp] Remove unnecessary code (#19729)
param_count is always 1 and it doesn't need to be computed at runtime.

Co-authored-by: BrzVlad <[email protected]>
2020-05-07 18:22:29 +03:00
monojenkins 258c9de8d9 [mono] Fix 32bit Android build (#19724)
1) Use `HostArch` for AppBuilder tasks 
2) Remove `tkill` for Android 32bit since it's not available in recent Android NDKs anymore.
2020-05-07 15:37:43 +02:00
Katelyn Gadd 3546709b3f Disable per-collection parallelism in debugger tests (#19722)
* Disable per-collection parallelism in debugger tests because moving tests out into more classes has created more collections, causing test failures

* Fix typo
2020-05-06 21:33:52 -04:00
Ankit Jain c2795c9cb5 [master] Bump msbuild to track xplat-master (#19621)
* [master] Bump msbuild to track xplat-master

* [master] Bump msbuild to track xplat-master

* Update nuget.exe to 5.6.0-preview3

- closest version that I could find, to the one in msbuild (`5.7.0-preview.1.6592`)
2020-05-06 11:40:05 -04:00
Thad House c9e6754eea Fix incorrect arg storage for arm softfp interpreter (#19676)
In get_call_info, IS_HARD_FLOAT is checked to switch between using the FP registers and the GP registers. This was not done in arg_get_storage, which was always using the hard float registers. This would mean parameters to the native trampoline wouldn't get copied correctly, and return values would be incorrect as well.

I can't find a way to add this to tests, since no softfp test systems seem to be built. I'm working on testing the use cases I can think of on my softfp system just to be sure.

Any assignments to type RegTypeHFA were locked behind a IS_HARD_FLOAT check as well, so I think its safe to modify that case too.

Fixes #14591
2020-05-06 11:52:05 +02:00
monojenkinsandtqiu8 e96347b0ef [master] [wasm][debugger][tests] Add tests for multiple breakpoints inline (#19719)
* [wasm][debugger][tests] add tests for multiple breakpoints

* [wasm][debugger][tests] Update CheckBreakpoint with more descriptive name and use CheckLocation

* [wasm][debugger][tests] More descriptive test name

* [wasm][debugger][tests] Combine two tests in one using [Theory]

Co-authored-by: tqiu8 <[email protected]>
2020-05-05 23:18:38 -05:00
Larry Ewing f6e347ed11 [wasm][debugger] Rework setBreakpointByUrl location handling (#19688)
* [wasm][debugger] Only choose the best match when mutiple locations are found
2020-05-05 19:42:28 -05:00
Larry Ewing 5d59d699fc [wasm][debugger] Add tests for floating common types (#19712) 2020-05-05 12:16:47 -05:00
Zoltan Varga cff010202b [wasm] Delete an emscripten patch which is no longer needed. (#19705)
* [wasm] Delete an emscripten patch which is no longer needed.

* [wasm] Disable the dynamic runtime build, it has no usage.
2020-05-05 11:06:38 -04:00
Larry Ewing e911de2a8b [wasm][debugger] Fix type argument (#19704) 2020-05-05 08:53:09 -05:00
Neale Ferguson 5633d5ccef Add soft debugger support to s390x (#19697)
* Add soft debugger support to s390x
  - Implement the sdb trampoline
  - Big-endian fixes for ppdb debugger:
    - Fix utf16 strings
    - Fix decoding of ImageDebugDirectory structures
  - New implementation of OP_SEQ_POINT
  - Enable successful unwind by defining markpoints
  - Implement mono_arch_setup_resume_sighandler_ctx
  - Implement unwind ops in epilog

* Fix delegate-invoke processing
  - Handle up to 10 parameters
  - Use sigparam information to correctly "slide" parameters

* Fix floating point negative zero for s390x
  - Add a new instruction for use in OP_RxCONST -0
  - Check for -0 in constant

* Minor indentation fixes

* Fix atomic add implementation

* Check for additional hardware facilities
2020-05-05 11:04:25 +02:00
Zoltan Varga fdc474bc46 [wasm] Add docs about -fsanitize=address. (#19703) 2020-05-04 16:22:09 -04:00
monojenkinsandCoffeeFlux 6b3eb8aa82 [runtime] Mono ALC embedding: use public types, correct hook signature (#19657)
We can't have GLib types in public headers. Oops.

There is potentially still an issue with `mono_alc_get_default_gchandle` not being exported, but that can wait for later.

cc: @garuma

Co-authored-by: CoffeeFlux <[email protected]>
2020-05-04 10:56:09 -04:00
Jeffrey Stedfast ccadc65005 [Mono.Debugger.Soft] Fixed NRE in MethodMirror.FullName when param_info is not yet cached (#19675)
If param_info has not yet been fetched, then the FullName property
will throw a NRE when accessed.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1112185/
2020-05-04 10:26:09 -04:00
monojenkinsandBrzVlad 5270d49f71 [interp] Move label outside of main loop (#19696)
Wasm has control flow limitations which don't go well with gotos. Move label outside of main loop because it translates to very poor wasm code. Other gotos don't seem to negatively impact instruction dispatch code, so leave them alone, for now ..

Co-authored-by: BrzVlad <[email protected]>
2020-05-04 12:24:15 +03:00
Zoltan Varga ec646ff710 [wasm] Bump emscripten to 1.39.14. (#19690)
* [wasm] Bump emscripten to 1.39.14.

* Set PYTHONUTF8=1 to work around emscripten issue 10551.

* Set LC_ALL as well.
2020-05-04 05:02:20 -04:00
Jérôme Laban 88cae11c6b [Wasm] Additional signatures for SkiaSharp interop (#19671) 2020-05-03 13:14:05 -04:00
Zoltan Vargaandmonojenkins 9e1c7d5aa4 [wasm] Integrate the m2n-gen.cs tool into mcs/tools/wasm-tuner. (#19680)
* [wasm] Integrate the m2n-gen.cs tool into mcs/tools/wasm-tuner.

It should be ran as:
mono wasm-tuner.exe --gen-interp-to-native <output file name> <extra assemblies>.

The <extra assemblies> will be searched for pinvoke methods, and their signatures
will be added to the set of supported signatures.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-05-01 04:20:53 -04:00
Zoltan Varga 3148398386 Disable a few executables if DISABLE_EXECUTABLES is defined. (#19678) 2020-04-30 14:19:35 +02:00
Thays Grazia e934687575 Cleaning error before return Invalid_argument. (#19660) 2020-04-28 10:56:10 -03:00
Junil Um 6442041b8b Fix if already send error to http client, do not callback. (#19664)
Fixed #10435 

HttpListener already closes the Response object, closes the HTTP Client connection, and then sets the value of Response.StatusCode again. If so, an ObjectDisposedException is thrown.

```
Unhandled Exception:
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.HttpListenerResponse'.
  at System.Net.HttpListenerResponse.set_ContentLength64 (System.Int64 value) [0x00013] in <b4473693dd3c4d45883c574a53529fbe>:0
  at MonoConsoleApp1.Program.Main (System.String[] args) [0x000f8] in <a66f6d7d417445dc9f89c691941ca70b>:0
  at MonoConsoleApp1.Program.<Main> (System.String[] args) [0x0000c] in <a66f6d7d417445dc9f89c691941ca70b>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.HttpListenerResponse'.
  at System.Net.HttpListenerResponse.set_ContentLength64 (System.Int64 value) [0x00013] in <b4473693dd3c4d45883c574a53529fbe>:0
  at MonoConsoleApp1.Program.Main (System.String[] args) [0x000f8] in <a66f6d7d417445dc9f89c691941ca70b>:0
  at MonoConsoleApp1.Program.<Main> (System.String[] args) [0x0000c] in <a66f6d7d417445dc9f89c691941ca70b>:0
```

If an error is sent to the HTTP client while processing a request, it is patched so that the next process is not executed anymore.
2020-04-28 15:37:23 +02:00
Zoltan Varga 41508e493d [wasm] Update netcore build. (#19666) 2020-04-28 05:43:04 -04:00
Ankit Jain c0c18ce34b [wasm][debugger][tests] Split tests into separate files (#19647)
* [wasm][debugger][tests] Move helper methods to base class

* [wasm][debugger][tests] Extract EvaluateOnCallFrame tests to new file

* [wasm][debugger][tests] Extract Array tests to a new file

* [wasm][debugger][tests] Extract CallFunctionOn tests to a new file

* [wasm][debugger][tests] Extract Delegate tests to a new file

* [wasm][debugger] Don't inherit from another test class

.. as that ends up running all the tests of the base class too.

* [wasm][debugger][tests] Improve retries in TestHarnessStartup

Sometimes `/json/list` seems to return an empty array (`[ ]`). Treat
that also as a case for retry.

* [wasm][debugger][tests] Bump timeout from 2s to 5s

* [wasm][debugger][tests] Launch chrome in incognito mode

* debugging

* [wasm][debugger] TestHarnessStartup - don't share `ProcessStartInfo`

.. instances. Instead, create one for every use.
We seem to run into cases where the `get_Environment` is modifying it's
`_environmentVariables` field, concurrently from multiple threads,
causing exceptions like:

```
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Diagnostics.Process.CreateEnvp(ProcessStartInfo psi)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at WebAssembly.Net.Debugging.TestHarnessStartup.LaunchAndServe(ProcessStartInfo psi, HttpContext context, Func`2 extract_conn_url) in /Users/radical/dev/w1/sdks/wasm/ProxyDriver/TestHarnessStartup.cs:line 82
   at WebAssembly.Net.Debugging.TestHarnessStartup.<>c__DisplayClass10_0.<<Configure>b__1>d.MoveNext() in /Users/radical/dev/w1/sdks/wasm/ProxyDriver/TestHarnessStartup.cs:line 167
```

* [wasm][debugger] Fix regression in debug.html

Fixes https://github.com/mono/mono/issues/19653 .
2020-04-27 19:31:46 -04:00
monojenkinsandBrzVlad e76b77a6b2 [interp] Fix tracking of local refcount when adding movloc opcode (#19659)
Consider the following IL code pattern:

```
stloc.np 0
dup
stloc 1
```

When `stloc 1` instruction is processed, we know we have local 0 on the top of the stack so we can replace `dup/stloc` with `movloc 0 -> 1`. However, we need to make sure that the refcount for local 0 is increased so we don't prematurely kill instructions that store into this local (the `stloc.np` from above). This is needed because we remove a dependency from the stack but add one from the locals. If the local 0 would have been explicitly pushed with a ldloc, the refcount is updated when we handle the ldloc, and no other updates are necessary.

Co-authored-by: BrzVlad <[email protected]>
2020-04-28 00:03:58 +03:00
Filip Navara f627ac6c16 Bump corefx (#19646)
Bump CoreFX submodule to get https://github.com/mono/corefx/pull/399.
2020-04-27 11:57:58 +02:00
Neale Ferguson aff8690a9b Major fixes to s390x ABI and add TAILCALL trace support (#19611)
* Fixes to s390x ABI to support roslyn
- Add new instructions
- Fix outarg_vt processing
- Add new hardware capabilities check
- Add mono_trace_tail_method() to properly support traceing of tail events

* Further s390x enhancements:
- Support altstack
- Rework localloc
- Fix tail processing
-- Restoration of registers dependent on paramater use and RGCTX
-- Restoration of FP registers

* Reset the debug flag to FALSE

* Fix tabbing etc. to conform with standard
2020-04-25 18:43:45 +02:00
monojenkinsandvargaz fcb8bd80fe Add some missing linker exclusions. (#19652)
Co-authored-by: vargaz <[email protected]>
2020-04-25 17:08:39 +02:00
monojenkinsandfanyang-mono 429d269d0f Refactor get_time_stamp_ns (#19438)
fixes dotnet/runtime#34074

Co-authored-by: fanyang-mono <[email protected]>
2020-04-24 11:35:41 -04:00
Bernhard Urban-Forster b2ec7fbfd9 [jit] use helper to obtain method signature (#19643)
`mini_method_get ()` doesn't guarantee that `method->signature` is set.
2020-04-24 17:15:59 +02:00
Joshua Peterson d4cb7ce150 Prevent a possible assert in the debugger when a thread detaches (#19541)
When a new thread is attached to the VM, the debugger will add it to the
`thread_to_tls` hash table. When that thread detaches, it will be
removed. Once the thread is attached, if a client is debugging, the
client can use the `CMD_THREAD_GET_FRAME_INFO` command to ask for
details about that thread.

This is a possibility that the thread detaches before the
`CMD_THREAD_GET_FRAME_INFO` command is processed, so the thread may not
exist the the `thread_to_tls` hash table any more.

This is a race condition, but is a valid state, so instead of asserting,
the debugger agent should indicate to the client that the thread no
longer exists, using `ERR_UNLOADED`.
2020-04-24 12:52:09 +02:00
vidir-svanberg 7e64403cfa Prevent NullReferenceException in SurrogateProviderAdapter (#19362)
Prevents NullReferenceException in SurrogateProviderAdapter which occurred when SurrogateProviderAdapter is constructed with a null ISerializationSurrogateProvider instance, which results in NullReferenceException in GetDataContractType, GetDeserializedObject and GetObjectToSerialize.
2020-04-24 10:43:29 +02:00
Ankit Jain 0568261f20 [wasm][debugger] Fix evaluateOnCallFrame for valuetype locals (#19638)
* [wasm][debugger] Fix `evaluateOnCallFrame` for valuetype locals

.. including generic, and async methods.

* [wasm][debugger] Remove some debug spew
2020-04-24 00:22:01 -04:00
monojenkinsandCoffeeFlux 98e3eb8345 [meta] Add embedding API function to get the default ALC (#19598)
Also formatting fixes to the unstable header.

We can just return the stored GCHandle for the default ALC because it's both strong and should never change.

cc: @garuma

Co-authored-by: CoffeeFlux <[email protected]>
2020-04-23 11:49:32 -04:00
Александр Хухлаев 82c4f32882 [WinForms] Fix ListView GetItemAt high load (#19639)
When ListView.VirtualMode = true GetItemAt should not access items collection directly to
get bounds of ListViewItem. Otherwise, method OnRetrieveVirtualItem will be called.
Which costs a lot of time.
2020-04-23 16:58:46 +02:00
Mitchell Hwang 946a479729 [System.Runtime.Serialization] Work around specified cast is not valid error (#19620)
During json serialization, if the key was an enum type, then it would fail trying to cast into an XmlDictionaryString.  

The fix is to treat the enum value like a string.
2020-04-23 09:00:35 -04:00
monojenkinsandCoffeeFlux 72b3c27759 [meta] Add GCHandle embedding API v2 with pointer-sized handles (#19599)
I've adopted the v2 convention instead of attempting a more descriptive name, with the hope that it makes iteration easier in the future and simplifies opt-in obsoletion.

cc: @vargaz @garuma

Co-authored-by: CoffeeFlux <[email protected]>
2020-04-23 08:36:24 -04:00
Alexander Köplinger 1763f53a4f freebsd.sh: Fix check for python3
"env python3" would actually run the python repl which would hang in CI provisioning.
2020-04-23 11:47:30 +02:00
Maxim Lipnin 3a04bff395 [wasm] Enable System.Data tests (#19614) 2020-04-22 09:28:12 -05:00
Maxim Lipnin 2c9ccaa46f [wasm] Add throwing PNSE to where currently disabled tests throw something different (#19538)
* Add throwing PNSE to where currently disabled tests throw something different

* Add try-catch + PNSE to public SslStream ctor in case of WASM
2020-04-22 09:27:33 -05:00
Zoltan Varga 1289c36895 [wasm] Preserve MonoPInvokeCallback using a linker xml descriptor. (#19631)
Fixes https://github.com/mono/mono/issues/19594.
2020-04-22 09:46:24 -04:00
Zoltan Varga bc8f971803 Fix a warning. (#19629) 2020-04-22 12:08:56 +02:00
monojenkinsandBrzVlad 7f41366684 [amd64] Fix tailcall with arguments passed on stack (#19502)
On all platforms, OP_TAILCALL copies the arguments (that are passed on the stack) from the param area in the current frame, to the param area in the caller frame, meaning that it expects the arguments to be passed normally on the stack. However, on amd64, mono_arch_emit_call was storing these arguments directly in the param area of the caller (EMIT_NEW_ARGSTORE), instead of the param area of the current frame. OP_TAILCALL would then override the already set stack parameters with random data from the uninitialized param area of the current frame.

Fixes https://github.com/dotnet/runtime/issues/34376

Co-authored-by: BrzVlad <[email protected]>
2020-04-22 11:49:29 +03:00
Ankit Jain e5937caf97 [wasm][debugger] Fix enumerating for this on valuetypes (#19626)
* [wasm][debugger] `mono_wasm_call_function_on` - remove unncessary

.. parameter, since it is already present in the `args` object.

* [wasm] MonoProxy - convert cfo calls to MonoCommands

* [wasm][debugger] Fix enumerating `this` properties for a valuetype

* [wasm][debugger] `evaluateOnCallFrame` - fix `this` for valuetypes

* [wasm][debugger] Remove MonoCommands no longer used

* [wasm][debugger] Fix showing the name for generic methods

- We were using `method.Name` from Cecil, but that we want to the name
with the type arguments for the generic instance.
2020-04-22 01:33:34 -05:00
Larry Ewing 2c5eb1993f [wasm][debugger] Fix command response handling (#19590)
* [wasm][debugger] send failed evaluations to the browser

Make sure that we allways either send a single result or return
false from Accept Command to ensure that we don't leave
messages spinning.
2020-04-20 17:07:50 -05:00
Zoltan Vargaandmonojenkins 0a37eeadec [wasm] Reenable the managed collation code, link it out by default, add a packager option to enable it. (#19570)
* [wasm] Reenable the managed collation code, link it out by default, add a packager option to enable it.

* Disable some xunit tests which fail because the test runner can't handle the [SkipOnTargetFramework(TargetFrameworkMonikers.Mono)] attribute.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-04-20 12:53:16 -05:00
Alexander Köplinger 068b41946d [tests] Disable sgen-bridge.exe on Windows x64 FullAOT+LLVM (#19604)
See https://github.com/mono/mono/issues/19603
2020-04-20 19:44:41 +02:00
Ankit Jain 17c4a60cd2 [wasm][debugger] Support array expansion in VS via callFunctionOn (#19588)
- This was prompted by vscode/vs requiring this for array expansion.
- It runs a given js function string, on a given object id.
- We handle the case for dotnet object ids.

Implementation:

- We build a proxy object that reflects the details of the real dotnet object/array
- And run the js function on that proxy object

- Then if `returnByValue` was requested, the result is returned as-is
- Else we cache that object, with a new special object id (`dotnet:cfo_res:..`),
  and return that id in the response.

- Subsequently, we would get `Runtime.getProperties` request on this new object id
- And a `Runtime.releaseObject` request, to free that function result.

- All this is handled currently

Tests:

- A few tests were added which invoke `callFunctionOn`, on JS objects, and checks the result
  - And the same checks are performed on equivalent dotnet objects, with the same js function
  - This should help stay in sync with what we think `callFunctionOn` should work as

  - These tests have an additional parameter - `roundtrip`, which calls a simple function
	`function () { return this; }`
    .. on a dotnet object, and gets the result object id. If `roundtrip == true`, then it
    runs that same function on the result object, and gets a new result object id. And the
    tests run their checks on that.

    - this helps to check that outside of the proxy, the result object behaves same as any
      other dotnet object.

- Also, most of the tests were modified to support an additional mode (`use_cfo`).
  - With this, whenever a test wants to run `Runtime.getProperties on an object, we pass that
    object through the earlier "identity" function, and then run `getProperties` on that
    resultant object.

  - This helps to take advantage most of the existing tests, and use them for `callFunctionOn`
    testing too, by ensuring that the results of CFO returns results same as any other object.

Fixes #19229 
Fixes #19531
2020-04-20 18:55:26 +02:00
Alex Thibodeau 4821038587 Add ability to turn on just the portable executable verifier. (#19575)
Helps fix issue: https://issuetracker.unity3d.com/issues/unity-crashes-when-opening-the-project

We were hitting a crash were a corrupted dll would get loaded and then hit an assert later on when something tried to use it. The verifier suite would correctly identify the dll as corrupt but it was impossible to turn on just the PE verification. I've added a new PE only mode above "off" to allow for just the PE verifier to be used instead of the larger verification suite.
2020-04-20 16:39:05 +02:00
Alexander Köplinger 12dd904025 Fix name of MIT license in LICENSE file (#19601)
It should be MIT, not MIT X11.
Fixes https://github.com/mono/mono/issues/18133
2020-04-20 16:33:50 +02:00
imhameed b2f760ddb3 [mono] Fix multidimensional array construction when using programmer-specified lower bounds. (#19572)
* [mono] Fix multidimensional array construction when using programmer-specified lower bounds.

mono_array_full_new_checked and mono_array_full_new (which is marked as
a MONO_API function) both take two pointers to buffers containing
lengths and lower bounds. mono_array_new_n_icall can split the incoming
parameter list in two before forwarding the results to
mono_array_full_new_checked, so mono_array_full_new_checked was
receiving two buffers containing interleaved lower bounds and lengths.

ECMA-335 states that array constructors that specify both lower bounds
and lengths interleave these values. Deinterleave these
in method_to_ir.

Add some tests to iltests.il that verify that multidimensional arrays
with custom lower bounds work.

Fixes https://github.com/dotnet/runtime/issues/34377.
Fixes https://github.com/dotnet/runtime/issues/34378.
Fixes https://github.com/dotnet/runtime/issues/34381.

* Deinterleave lower bounds and array lengths in the interpreter.

Also make the condition in which the deinterleaving code path is taken
more precise. Perhaps this branch should be marked as unlikely.

* Fix a typo.
2020-04-19 16:33:10 -07:00
monojenkinsandCoffeeFlux 90560d3518 [meta] Mono ALC embedding primer (#19458)
After talking with @lambdageek , we decided we wanted the public API for ALCs to just use gchandles instead of a native struct and to handle conversion within the runtime.

This PR takes that approach, exposing the managed ALC fields to the runtime for cheap conversion and adding `mono_assembly_load_full_alc` and preload hook V3 to the unstable headers.

I considered trying to do something fancier with conversion for the preload hook but decided against it in the end; making a V3 that's intended for embedder usage is straightforward and should work fine. When invoking it, we create a new strong gchandle to keep the managed ALC pinned and then free it after the call is done.

I went with `MonoManagedAssemblyLoadContext` instead of `MonoReflectionAssemblyLoadContext` since ALCs aren't really part of reflection, but I have no strong attachment to the name and can change it if preferred to maintain consistency.

cc: @vargaz @mdh1418 @garuma

Co-authored-by: CoffeeFlux <[email protected]>
2020-04-19 04:14:05 -04:00
Larry Ewing 4eb9787f1b [wasm][debugger] Add an option to clear navigator.webdriver on attach (#19584)
* [wasm][debugger] Add an option to clear navigator.webdriver on attach

* Pass the flag when it is useful
2020-04-19 01:19:56 -05:00
Zoltan Varga e400e13324 Bump linker to the tip of mono-2020-02. (#19583) 2020-04-18 03:57:55 -04:00
Larry Ewing 5c5be77f00 [wasm][debugger] Always send the remove breakpoint command to the browser too (#19576) 2020-04-17 17:26:24 -05:00
Ankit Jain 7f0b1b1037 [wasm][debugger] Show pointer type+address (#19537)
* [wasm][debugger] Simplify the interface between debugger.c and

.. library_mono.js .

* [wasm][debugger] Show pointer type+address

* [wasm][debugger] Address review feedback

* [wasm][debugger] Small cleanups, from review feedback

* [wasm][debugger] declare mono_wasm_add_typed_value as extern
2020-04-17 22:16:23 +02:00
monojenkinsandSamMonoRT 3aa1bbb55f Minor changes - Update specific words in mono code base to be PoliCheck/TSA compliant (#19560)
Based on a recent PoliCheck tool run on the code base, there were some words in the code which weren't compliant. List of Mono bugs here :
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevdiv.visualstudio.com%2Fweb%2Fqr.aspx%3Fpguid%3D0bdbc590-a062-4c3f-b0f6-9383f67865ee%26qid%3Da5547394-def7-4a3d-ae81-fe8480583539&data=04%7C01%7Csampatel%40microsoft.com%7C0f0f005f4bf4422e7d5f08d7e1717a81%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637225749822378839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C-1&sdata=QmWt%2BP%2FzZ2PRyxwHuSJu%2FvE3dnGkahfIdjA%2FH06XoSs%3D&reserved=0

This change fixes some of those. There will be another PR for region/language/locale specific GeoPolitical issues identified by the scan too.

Co-authored-by: SamMonoRT <[email protected]>
2020-04-17 12:39:58 -04:00
Zoltan Varga 80f7953477 [aot] Fix the support for separate aot data files. Reorder the MONO_AOT_TABLE constants so they match the ordering in MonoAotFileInfo. (#19565) 2020-04-17 11:41:07 +02:00
Martin Baulig dfd748b695 Bump CoreFX to pickup https://github.com/mono/corefx/pull/398. (#19564)
Fixes https://github.com/mono/mono/issues/19451.
2020-04-17 11:07:06 +02:00
Zoltan Varga bb019312d3 [wasm] Update netcore support. (#19574) 2020-04-17 03:12:20 -04:00
Ankit Jain 354c9e68b6 [wasm][debugger] Show signature for delegate, or target, if available… (#19535)
* [wasm][debugger] Show signature for delegate, or target, if available (#19505)

* [wasm][debugger] Show signature for delegate, or target, if available

- As object properties, we return a `Target` which has the signature of
the delegate target.

Fixes https://github.com/mono/mono/issues/19382

* [wasm][debugger] Some tidying up

* [wasm][debugger] Remove unused `sig_desc`

* [wasm][debugger] Simplify code, avoid extra allocations

.. as suggested by @lewing

(cherry picked from commit 4eca12e794)

* [wasm][debugger] Fix tests broken because unused vars are not null

.. anymore, which is because we disabled optimizations in the
interpreter when debugging.

* [wasm][debugger][tests] Add tests to also check delegates in previous

.. frames.

* [wasm][debugger] Disable a test that tried to look at previous async

.. frames. This just happened to work. Currently, the trace has details
of the async machinery, instead of one reflecting what it might have
looked like for a non-async case. So, we can't reliably find the
previous method to inspect it's locals.

* [wasm][debugger] Fix test
2020-04-16 22:38:39 -05:00
monojenkinsandBrzVlad 7ae1d80f03 [jit] Add support for fconv.ovf.*.un opcodes (#19539)
These opcodes have the same effect as their non-_un_ counterpart, since floats are always signed. This means that applying a fconv.ovf.i8.un to a negative float, will result in a negative integer, despite the unsigned suffix (tested against coreclr).

Fixes https://github.com/dotnet/runtime/issues/34084

Co-authored-by: BrzVlad <[email protected]>
2020-04-17 01:38:40 +03:00
Larry Ewing a65055dbdf Change OSPlatform for wasm to "BROWSER" rather than "WEBASSEMBLY"
Following the guidance in https://github.com/dotnet/runtime/issues/33328
Change OSPlatform to BROWSER for wasm runtimeArchitecture.
2020-04-16 22:57:33 +02:00
Larry Ewing eb68dbc2b0 [wasm][debugger] Add comment about embedded pdbs and fix old regression (#19563)
With embedded pdbs we have to try to load symbols even when the
pdb is missing then handle ArgumentException if there is no embedded
pdb.  This restores the intended error path and added an explanation
while still logging unexpected errors
2020-04-16 15:49:17 -05:00
Larry Ewing 42f5d32e12 [wasm][debugger] Handle multi-doc methods and multiple results for breakpoints (#19449)
* [wasm][debugger] Rework source loading

Add logic to support embedded sources and
add all the documents from multi-doc methods

Add logic to check source hashes against
pdb hash.

* [wasm][debugger] Fix breakpoints on delegates

Only add methods to a source once by RID

When column is != 0 specified the fronted only wants results
that match that column exactly.

* Allow checking pdb hash and remove source cache

* Fixup typesByName merge
2020-04-16 11:58:55 -05:00
monojenkinsandvargaz 9545c24a15 Make the -Wsometimes-uninitialized into an error. (#19543)
Co-authored-by: vargaz <[email protected]>
2020-04-16 18:32:41 +02:00
Larry Ewing b1e4d125c7 [wasm][debugger] Fix typo and silence output (#19556) 2020-04-16 09:32:12 -05:00
Thays Grazia 039a6890f4 [debugger] Remove assert when the DebuggerTypeProxy cannot be instantiated
Remove the assert because we don't need to stop the debug if we cannot create the object when calling the constructor using invoke_method, if it's a DebuggerTypeProxy, we are not able to show the value of the DebuggerTypeProxy, but we still can show the Raw View, if it's not the case of DebuggerTypeProxy, we just cannot evaluate this expression, but we can continue the debug session.

Fix #19492
2020-04-16 10:17:09 -03:00
Zoltan Varga be24f101bd [wasm] Debugger test improvements. (#19499)
* Disable the Loaded: messages, they are not very useful.

* Add a Dotnet-test.setBreakpointByMethod protocol extensions to the proxy to set breakpoints in specific methods during testing.

* Add a new sample debugger test using the setBreakpointByMethod API.

* Remove some unrelated changes.

* Restore logging.

* Disable a log message which is printed when a pdb file is missing.
2020-04-15 16:37:50 -04:00
Larry Ewing 123bee9620 [wasm] Use browser style lang (#19544) 2020-04-15 12:12:47 -05:00
Thays Grazia aaf7a193cc [metadata] Identify when a field is overlapped by a non-object field (#19485)
Fix class loader to identify when the class has a field overlapped by a non-object field.
Fix method-to-ir to forward the TypeLoad error.
Fix transform to forward the TypeLoad error.
Fix unit test that has a class with a field that is overlapped by a non-object field.

Fixes dotnet/runtime#34388
2020-04-15 13:09:01 -03:00
Larry Ewing a73d0c028f [wasm] force set a local when running debugger tests (#19546) 2020-04-15 08:23:42 -05:00
Johan Lorensson 5a71ba15f3 Fix missing array ctor(int32,int32) test failures. (#19494)
According to ECMA, VES should add the following methods to arrays:

* A constructor that takes a sequence of int32 arguments, one for
each dimension of the array, that specify the number of elements
in each dimension beginning with the first dimension. A lower bound
of zero is assumed.

* A constructor that takes twice as many int32 arguments as there are dimensions
of the array. These arguments occur in pairs—one pair per dimension—with
the first argument of each pair specifying the lower bound for that
dimension, and the second argument specifying the total number of elements
in that dimension.

* Get.

* Set.

* Address.

In Mono's case we only added the ctor (int32, int32) for rank 1 arrays
if they were "jagged" array. Array with rank 1-4 executed an optimized
code path always using the one int32 per rank constructor.

This doesn't work in cases where the array is constructed using both
a length and a lower bound pattern, like:

newobj instance void int32[10000...10005]::.ctor(int32, int32)

this would trigger a method not found exception, since its not a jagged
array, Mono wouldn't add the ctor(int32, int32) constructor for this type.

Fix will make sure we follow ECMA's definition of this and fix code
to use right instance methods of arrays. Code also adds an assert
to trap if a jagged array is created through this code path
(shouldn't happen) since current implementation would have handled it as
a 2 ranked array.

Fix also adds several tests into iltests.il allocating different
variations of arrays using lower/upper bounds and validating length, rank,
upper/lower bounds of allocated arrays.
2020-04-15 10:44:26 +02:00
monojenkinsandvargaz e71879085d Fix an uninitialized variable error introduced by dotnet/runtime#34877. (#19540)
Co-authored-by: vargaz <[email protected]>
2020-04-14 23:18:55 +02:00
Zoltan Varga a5fd1cd86f [wasm] Fix netcore support. (#19521)
* [wasm] Remove netcore build support from builds/wasm.mk, its not used anymore.

* [wasm] Fix netcore support.

Currently, only local development with a separate dotnet/runtime repo works. The mono runtime, corelib, and the bcl will be taken from that repo.
AOT doesn't work.

* Fix running xunit tests.

* Fix some tests.

* Revert some unrelated changes.
2020-04-14 11:00:56 -04:00
Johan Lorensson a87a90463e Fix infrequent/random crashes on Windows x64 due to use of GC forwarded objects. (#19475)
* Fix infrequent/random crashes on Windows x64 due to use of GC forwarded objects.

Hard to repro and very infrequent crash. Have been analyzing a couple
of crash dumps from retail devices getting different crashes related to
vtable "corruption" on Windows x64. After some deeper analysis it turns
out the object instance has been forwarded by GC
(object vtable pointers lowest bit set to 1), but object still holds
tagged vtable. This will then cause misaligned reads, getting back random
values and pointers from vtable on next object access.

After some further analyzing it turns out that LLVM codegen and some
specific generic vt arrays lowering can cause optimized mem copies
using XMM registers. I have also identified scenarios where vt copies
gets lowered into a c-runtime memcpy that in turn uses XMM registers
as an optimization. Since Windows x64 currently don't include XMM
registers in context, any references in XMM registers will not be
visible and pinned by GC, meaning that they will point to potentially
forwarded objects after completing GC, restarting threads, leading
to these infrequent random crashes.

Fix includes xmm0-xmm15 into MonoContext on Windows x64, making sure
GC will see all references that could be held in those registers,
regardless if getting into those registers due to LLVM optimization
or other native code, like memcpy.
2020-04-14 13:46:08 +02:00
Martin Baulig 6e6f67669d Bump CoreFX to pickup https://github.com/mono/corefx/pull/397. (#19525) 2020-04-14 12:33:10 +02:00
monojenkinsandvargaz a448a33fd7 [wasm] Disable the AOT cache check on wasm as well. (#19522)
Co-authored-by: vargaz <[email protected]>
2020-04-14 00:54:56 -04:00
monojenkinsandvargaz e4eb2f0efc [runtime] Try to load assemblies from bundles on netcore before trying other methods, the ALC might not be initialized yet. (#19520)
Co-authored-by: vargaz <[email protected]>
2020-04-13 22:23:53 -04:00
Tammy Qiu bc3f47fdf5 [wasm] Add readme to package update scripts (#19507)
* add powershell and bash scripts for replacing default webassembly nuget packages from link or path

* [wasm] Add readme to package update scripts
2020-04-13 19:56:26 -05:00
Ankit Jain d5daeeab9a [wasm][debugger] Improve some valuetype visualization (#19490)
* [wasm][debugger] Fix up `value.description` for local vars too

- Local vars, when being enumerated in `mono_wasm_get_variables` don't
have an associated `name`, and so `_fixup_name_value_objects`
*incorrectly* ends up not fixing up the `value.description` field for
them.

* [wasm][debugger] For some types, use `ToString()` for their ..

.. `description`.

- `System.DateTime`
- `System.DateTimeOffset`
- `System.Decimal`
- `System.TimeSpan`
- `System.Guid`

* [wasm][debugger] Invoke all property getters, for some types

- `System.DateTime`
- `System.DateTimeOffset`
- `System.TimeSpan`

* [wasm][debugger] Show char symbol along with the code

Fixes https://github.com/mono/mono/issues/19381

* [wasm][debugger] Fix DateTime tests, don't use hardcoded strings for

.. description. Instead, use a `DateTime` object in the test to create
the expected string for the test. This should ensure that the expected
and actual were created in the same environment.

* [wasm][debugger] Special case `System.Guid` to use

.. `mono_guid_to_string`, instead of calling `ToString()` on it.

* [wasm][debugger] Don't box valuetypes for calling `ToString()`

Co-authored by @vargaz .

* [wasm][debugger] `System.Guid` is special-cased, so remove it from the

.. allowed list.

* [wasm][debugger] simple cleanup
2020-04-13 19:50:56 -05:00
Larry Ewing fdb3893c09 [wasm][debugger] explicitly pass --lang=en_US for debugger tests (#19529) 2020-04-13 17:37:40 -05:00
monojenkinsandEgorBo 585d92f375 [mono] Implement Sse41, Pclmulqdq, Aes and enable Sse41 intrinsics (#19518)
This PR implements `Sse41`, `Pclmulqdq`, `Aes`  and enables `Sse42` (was implemented but disabled). Thus, Mono-LLVM now fully supports the following intrinsics:
- [ ]  System.Runtime.Intrinsics.X86.*
  - [x] **Sse**
  - [x] **Sse2**
  - [x] **Sse3**
  - [x] **Ssse3**
  - [x] **Sse41**
  - [x] **Sse42**
  - [x] **Aes**
  - [x] **Bm1**
  - [x] **Bm2**
  - [x] **Lzcnt**
  - [x] **Pclmulqdq**
  - [x] **Popcnt**
  - [ ] **Avx**
  - [ ] **Avx2**
  - [ ] **Fma**

- [ ] System.Runtime.Intrinsics.Arm.*
  - [x] **ArmBase**
  - [x] **Crc32**
  - [ ] **AdvSimd**
  - [ ] **Aes**
  - [ ] **Sha1**
  - [ ] **Sha256**

- [ ] System.Runtime.Intrinsics.*
  - [ ] **Vector64** (needed for Arm)
  - [x] **Vector128**
  - [ ] **Vector256** (needed for Avx, Avx2 and Fma)

The changes should be LLVM-9 friendly.

Co-authored-by: EgorBo <[email protected]>
2020-04-14 00:37:11 +03:00
Kenneth Pouncey b653274418 [wasm][sdk][nuget] Bump version. (#19483) 2020-04-13 11:50:16 -05:00
Kenneth Pouncey acddc9c5c9 [wasm][browser][tests] Generate sdk version in global.json file automatically. (#19484) 2020-04-13 11:49:30 -05:00
Zoltan Varga 6e5c15cea5 [wasm] Remove .stamp-build-debugger-test-app, the app can always be built. (#19517) 2020-04-12 03:36:59 -04:00
Larry Ewing 5a45ab6630 [wasm][ci] Always build and package wasm (#19508) 2020-04-11 11:23:10 +02:00
Zoltan Varga f9faa296b8 [wasm] Disable interpreter optimizations when debugging. (#19504)
* [wasm] Disable interpreter optimizations when debugging.

* Fix tests.
2020-04-11 03:08:25 -05:00
Kenneth Pouncey 5a3e168af2 [wasm][bcl] Handle correctly backslash in file name (#19488)
* [wasm][bcl] Handle correctly backslash in file name

- Modify IO file to assign backslash to alternate directory separator by default
- Add tests for issue https://github.com/mono/mono/issues/18933

* [wasm][tests] Skip two tests for wasm

- After internal discussions decided to skip these two tests on wasm

   1. GetDirectories_Backslash
   1. GetFiles_Backslash

* [wasm][tests] PathIsRooted tests failing

- skip subtest #9 if WASM
- To fix issue https://github.com/mono/mono/issues/18933 we set the AltDirectorySeparatorChar as well.  This causes the `Assert.IsTrue (!Path.IsPathRooted ("\\"), "IsPathRooted #09");` to fail. Path.IsPathFullyQualified Method handles paths that use both the DirectorySeparatorChar and the AltDirectorySeparatorChar characters.

* [wasm][tests] PathIsRooted tests failing

- skip if WASM
2020-04-11 09:31:39 +02:00
Immo Landwerth fc7ea0aa0e Fix links to Code of Conduct (#19510) 2020-04-10 23:31:46 +02:00
Tammy Qiu e4fbccac0a add powershell and bash scripts for replacing default webassembly nuget packages from link or path (#19498) 2020-04-10 08:54:59 +02:00
Kenneth Pouncey 9bfe456516 [wasm][bcl][zoneinfo] Fix local zone info marshaling (#19493)
- Correctly marshal Intl.DateTimeFormat().resolvedOptions().timeZone information
2020-04-09 23:41:31 -04:00
Zoltan Varga f821cf4f80 [llvm] Fix a crash in JIT mode. (#19496)
* [llvm] Fix a crash in JIT mode.

* [runtime] Change some g_return_if_fails's to asserts.
2020-04-09 17:07:46 -04:00
dotnet-maestro[bot]anddotnet-maestro[bot] ced27c7de3 Update dependencies from https://github.com/dotnet/arcade build 20200401.2 (#19455)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20201.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20201.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-04-09 16:45:30 +02:00
Larry Ewing 34cccfa2fd [wasm][debugger] Wasm debug log level (#19479)
* [wasm][debugging] pass enable_debugging as debug engine log level

enable_debugging was already an int, with this change log_level
will remain at 1 by default but passing -1 will disable the log
spew

* Change the default debug level of 'Failed to lookup sequence point'
2020-04-08 23:31:52 -05:00
Jeff Smith 1167f95932 [metadata] Handle MONO_TYPE_FNPTR case in collect_type_images (#19434)
Fixes abort when PTR-FNPTR field signature is encountered.

Fixes https://github.com/mono/mono/issues/12098
Fixes https://github.com/mono/mono/issues/17113
Fixes https://github.com/mono/mono/issues/19433
2020-04-09 01:22:04 +02:00
Alexis Christoforides fda7280cea [merp] Synchronize (serialize) calls to MERP API (#19429)
Fixes https://github.com/mono/mono/issues/19406
2020-04-08 16:06:37 -04:00
Filip Navaraandmonojenkins 1fa5c1e0eb Add BeginRead/BeginWrite/EndRead/EndWrite overloads back to SslStream (#19442)
* Add BeginRead/BeginWrite/EndRead/EndWrite overloads back to SslStream and underlying implementation. Unlike the default implementation in Stream it allows parallel read along with parallel write using the standard TaskToApm pattern (https://github.com/dotnet/runtime/issues/33447). This reverts part of https://github.com/mono/mono/pull/17393 which could lead to deadlocks as reported in https://github.com/mono/mono/issues/18865.

* Move the TaskToApm pattern from MobileAuthenticatedStream to SslStream

* Bump API snapshot submodule

Co-authored-by: monojenkins <[email protected]>
2020-04-08 20:03:54 +02:00
Karl 29214fcbf1 Fix WinForms LinkLabel drawing (#19386)
MeasureCharacterRanges handles the alignment correctly by itself.

This may have been to handle poor behaviour in libgdiplus, particularly
when Pango text layout was not default.
2020-04-08 19:44:41 +02:00
Karl ed23e5e41a Three small RichTextBox / TextControl fixes (#19401)
1. Fix the height calculated by Line.RecalculatePasswordLine(). Would
   have been roughly double what it should have been - line spacing
   includes the height of the line.

2. In RichTextBox, insert a character for the Quote special character
   tags rather than just printing a message on the console.

3. In TextControl Document.Draw(), fix the criteria for skipping the
   last line. Could probably be improve further, but the previous had a
   problem because it could check with a point well off the bottom of
   the second to last line instead of its bottom. This probably arose
   because I had used line.Height for just the height of the text in the
   line, and later decided that it should be the entire height including
   all spacing, and added line.TextHeight. The earlier behaviour would
   have been wrong too, because it would have pointed to the bottom of
   the text, not the bottom of the line.
2020-04-08 19:44:03 +02:00
Александр Хухлаев a26d2f0c3e [Winforms] fix #18606 PropertyGrid didn`t edit value with CustomTypeDescriptor (#19452)
When using implemeted ICustomTypeDescriptor with overrided SetValue(object component, object value) and there value reference will be changed even if new object`s fields will be equals then in line
current_changed = Object.Equals (properties[i].GetValue (propertyOwners[i]), value);
will not detected change cause of references of object in operands are not equal.
In usual ways references will be equal. So we compare old value ref with new value ref earlier.
In .net winforms do same. And also original winforms did not check CanConvertFrom whith type of editable object. So we do not need block with
converter.CanConvertFrom ((ITypeDescriptorContext)this, valueType))

Fixes #18606
2020-04-08 19:43:10 +02:00
Александр Хухлаев e3f31b2bdc [Winforms] PropertyGrid fix disappearing of items (#19453)
When Category Name match with Item name this GridItem will disappear
2020-04-08 19:41:54 +02:00
Александр Хухлаев 198e323fc2 [WinForms] Fix ToolStripTextBox Width (#19457)
When set non-Empty Size to ToolStripTextBox it will appear with zero with, but must use width from size.
2020-04-08 19:40:21 +02:00
Александр Хухлаев 4f7cc4db7e [WinForms] PropertyGrid must show value text even if CanConvertTo return false (#19463)
If we use PropertyDescriptor with Converter like this

public class MyConverter : TypeConverter
{
    public override bool CanConvertTo(ITypeDescriptorContext context, Type destType)
    {
         return false;
    }
}

GridEntry will not show text of Value which uses MyConverter.
In .net they do not check CanConvertTo, they call ConvertToString directly.
2020-04-08 19:37:55 +02:00
Alexander Köplinger a4a19c3cb4 freebsd.h: Print commands as they're executed 2020-04-08 18:39:16 +02:00
Zoltan Varga b33943df2b [runtime] Fix Type.GetTypeCode () for generic enums. (#19470)
Fixes https://github.com/mono/mono/issues/17735.
2020-04-08 12:05:22 -04:00
Alexander Köplinger c6fa6f773a freebsd.sh: Use yes instead of 1 for ASSUME_ALWAYS_YES 2020-04-08 12:04:30 +02:00
Phil Jaenke cd024dc80f Fix for missing IGNORE_OSVERSION (#19467)
Quarterly is using 1201000, Azure is 1200086, causing forced bootstrap to fail.
2020-04-08 12:01:45 +02:00
Pranav K a0d69a4e87 Produce System.Net.Http.WebAssemblyHttpHandler (#19260)
* Produce System.Net.Http.WebAssemblyMessageHandler

* Branding changes for WebAssembly.Net.Http -> System.Net.Http.WebAssemblyMessageHandler

* Change how fetch options are configured

* Fixup tests

* More fixup

* Update submodule
2020-04-08 11:42:03 +02:00
monojenkinsandCoffeeFlux 1386af6071 [meta] Fix loader printf mistake and outdated info in Makefile (#19459)
Co-authored-by: CoffeeFlux <[email protected]>
2020-04-08 01:10:40 -04:00
Kenneth Pouncey 42c76a9728 [wasm][http] Throw HttpRequestException instead of internal WebAssembly.JSException. (#19454) 2020-04-07 17:50:07 -05:00
Phil Jaenke 8fbdc3ea44 Fix for 2020Q1 FreeBSD pkg upstream, some cleanup. (#19465)
pkg has been updated to 1.12+ which has an incompatible metadata format.
Fix this by force bootstrapping pkg when necessary. Also perform some
general cleanup and go back to the python3 metaport as 2020Q1 uses a
sufficiently current Python for the metaport.
2020-04-07 19:50:16 +02:00
monojenkinsandBrzVlad a562759d6e [sgen] Fix coverity issue (#19462)
Co-authored-by: BrzVlad <[email protected]>
2020-04-07 11:56:08 +02:00
Karl 54256b61e9 When double buffering, flush the buffered graphics (#19460)
Otherwise some stuff doesn't always get drawn.

Fixes https://github.com/mono/libgdiplus/issues/640, and probably other similar things.
2020-04-07 10:28:01 +02:00
monojenkinsandEgorBo c683499ad3 [mono] Implement Arm intrinsics: ArmBase, Crc32 (#19353)
Implement `System.Runtime.Intrinsics.Arm.ArmBase` and `*.Crc32` intrinsics for Arm64 with LLVM-backend. `ArmBase` is enabled by default, `Crc32` for LLVM-AOT can be enabled either via `mcpu=native` or `mattr=+crc` (e.g. when we crosscompile for iOS).

Asm: (`--aot=llvm,mcpu=native System.Private.CoreLib.dll` on Cortex-A57):
```asm
;; ArmBase

; int LeadingZeroCount(int value)
0000000000292c84 <System_Runtime_Intrinsics_Arm_ArmBase_LeadingZeroCount_int>:
  292c84:	clz	w0, w0
  292c88:	ret

; int LeadingZeroCount(uint value)
0000000000292c8c <System_Runtime_Intrinsics_Arm_ArmBase_LeadingZeroCount_uint>:
  292c8c:	clz	w0, w0
  292c90:	ret

; int ReverseElementBits(int value)
0000000000292c94 <System_Runtime_Intrinsics_Arm_ArmBase_ReverseElementBits_int>:
  292c94:	rbit	w0, w0
  292c98:	ret

; uint ReverseElementBits(uint value)
0000000000292c9c <System_Runtime_Intrinsics_Arm_ArmBase_ReverseElementBits_uint>:
  292c9c:	rbit	w0, w0
  292ca0:	ret

;; ArmBase.Arm64

; int LeadingSignCount(int value)
0000000000292cac <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_LeadingSignCount_int>:
  292cac:	cls	w0, w0
  292cb0:	ret

; int LeadingSignCount(long value)
0000000000292cb4 <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_LeadingSignCount_long>:
  292cb4:	cls	x0, x0
  292cb8:	ret

; int LeadingZeroCount(long value)
0000000000292cbc <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_LeadingZeroCount_long>:
  292cbc:	clz	x0, x0
  292cc0:	ret

; int LeadingZeroCount(ulong value)
0000000000292cc4 <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_LeadingZeroCount_ulong>:
  292cc4:	clz	x0, x0
  292cc8:	ret

; long ReverseElementBits(long value)
0000000000292ccc <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_ReverseElementBits_long>:
  292ccc:	rbit	x0, x0
  292cd0:	ret

; ulong ReverseElementBits(ulong value)
0000000000292cd4 <System_Runtime_Intrinsics_Arm_ArmBase_Arm64_ReverseElementBits_ulong>:
  292cd4:	rbit	x0, x0
  292cd8:	ret

;; Crc32

; uint ComputeCrc32(uint crc, byte data)
0000000000292ce4 <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32_uint_byte>:
  292ce4:	crc32b	w0, w0, w1
  292ce8:	ret

; uint ComputeCrc32(uint crc, ushort data)
0000000000292cec <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32_uint_uint16>:
  292cec:	crc32h	w0, w0, w1
  292cf0:	ret

; uint ComputeCrc32(uint crc, uint data)
0000000000292cf4 <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32_uint_uint>:
  292cf4:	crc32w	w0, w0, w1
  292cf8:	ret

; uint ComputeCrc32C(uint crc, byte data)
0000000000292cfc <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32C_uint_byte>:
  292cfc:	crc32cb	w0, w0, w1
  292d00:	ret

; uint ComputeCrc32C(uint crc, ushort data)
0000000000292d04 <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32C_uint_uint16>:
  292d04:	crc32ch	w0, w0, w1
  292d08:	ret

; uint ComputeCrc32C(uint crc, uint data)
0000000000292d0c <System_Runtime_Intrinsics_Arm_Crc32_ComputeCrc32C_uint_uint>:
  292d0c:	crc32cw	w0, w0, w1
  292d10:	ret

;; Crc32.Arm64

; uint ComputeCrc32(uint crc, ulong data)
0000000000292d1c <System_Runtime_Intrinsics_Arm_Crc32_Arm64_ComputeCrc32_uint_ulong>:
  292d1c:	crc32x	w0, w0, x1
  292d20:	ret

; uint ComputeCrc32C(uint crc, ulong data)
0000000000292d24 <System_Runtime_Intrinsics_Arm_Crc32_Arm64_ComputeCrc32C_uint_ulong>:
  292d24:	crc32cx	w0, w0, x1
  292d28:	ret
```
@tannergooding @echesakovMSFT Could you please take a quick look at asm ^ if it's ok or not.

`Vector64<T>`, `Vector128<T>`, `Vector<T>` are not enabled yet.

Co-authored-by: EgorBo <[email protected]>
2020-04-07 04:25:53 +03:00
Zoltan Varga 36c1034f4b [llvm] Define a separate symbol for the aot method info for every method. Pass this symbol to the init functions instead of the method index. (#19298)
* [llvm] Define a separate symbol for the aot method info for every method. Pass this symbol to the init functions instead of the
method index.

This removes a dependency the generated code has on the method index, hopefully allowing separate compilation etc. in the future.

* Put the method flags into a separate AOT table so they can be accessed without decoding the method info structure, which is not available to the runtime for LLVM methods.

* Make the LLVM methods set the init flag themselves. This will allow the init flag to be changed to a per-method scalar later to remove
a dependency on the method index.

Also rename init_icall_wrapper to init_func, it does much more than just calling the icall.
2020-04-06 18:14:27 -04:00
Thays Grazia c5149e31b4 [wasm][debugger] Improving evaluate expression implementation (#19444)
* - Adding tests to evaluate expression
- Adding a cache to local vars and object properties
- Fixing evaluate expressions like "this.a"
- Fixing evaluate expressions like "d + d"
2020-04-05 22:57:02 -05:00
imhameed c991c1d4c4 [llvm] Enable optimizations for LLVM 9 JIT. (#19445)
Share more code between LLVM 6 and LLVM 9.

Mangler::getNameWithPrefix takes a Twine, not a std::string, so use a
StringRef to avoid an unnecessary allocation.

InstructionCombiningPass grew a dependency on
ProfileSummaryInfoWrapperPass in
https://github.com/dotnet/llvm-project/commit/09e539fcaebb6362795d352cdcf4a818cf4d0d6a,
which depends on the enclosing FunctionPassManager having a valid
reference to a module. This commit adds a dummy module to satisfy this,
instead of recreating a new FunctionPassManager on every compilation.
(See also
http://lists.llvm.org/pipermail/llvm-dev/2019-March/130690.html.)
2020-04-05 12:08:57 -07:00
Thays Grazia 3fba2c8166 Some frames were skipped in MonoProxy but we don't skip the frame_id, doing this the call_stack has a different position than it's in the real stack on mono side. (#19447)
I changed the frame_id, so we will follow the frame_id from callstack even if we don't add all of them into the CallStack list in MonoProxy.
I also added a test.

Fixes #19379
2020-04-05 13:16:28 -05:00
Zoltan Varga 6ec38cc080 [llvm] Replace the LLVM got array with a set of scalar variables which are initialized by a callback called by the runtime. (#19267)
Long term, this will make the generated code depends less on got offsets etc., so it might be possible to
avoid recompiling it if the offsets change.
2020-04-05 10:21:10 -04:00
Thays Grazia 8ba4a2bca1 As suggested by @jbevain we should bump the protocol to implement this. (#19437) 2020-04-04 20:58:01 -03:00
Larry Ewing 2c5a02755d [wasm][debugger] Don't surface evaluation errors to the browser (#19439)
Instead of surfacing the error to the browser log it return
an empty result.
2020-04-04 14:24:08 -05:00
Kenneth Pouncey eb468076a7 [wasm] Wasm TimeZoneInfo implementation VFS (#17760)
* [wasm][bcl] TimeZoneInfo implementation for WebAssembly to read from WASM VFS `/zoneinfo`.

- Remove `TimeZoneInfo.WebAssembly.cs` file
- Remove source dependencies on `TimeZoneInfo.WebAssembly.cs`.
- Modify TimeZoneInfo for specific WASM functionality
   - Default root directory is `zoneinfo`
   -  Add icall `mono_timezone_get_local_name` for WASM in CreateLocal
- Add icall implementation
   - ves_icall_System_TimeZoneInfo_mono_timezone_get_local_name

* [wasm][build] Add `-s FORCE_FILESYSTEM=1` to mono wasm runtime build

- Required so that it includes support for loading pre preload packages

Message when generating the zoneinfo data.
   > Remember to build the main file with  -s FORCE_FILESYSTEM=1  so that it includes support for loading this file package

* [wasm] zoneinfo data and support files.

- Three files for zoneinfo VFS support.
   - `mono-webassembly-zoneinfo-fs-smd.js.metadata` - This is the separate metadata output by emscripten file-packager.  Used to parse the zoneinfo data in non browser environments.  See `runtime-tests.js`
   - `mono-webassembly-zoneinfo-fs.js` - Output by emscripten file-packager that will be referenced by browser environments to load the zoneinfo data into the VFS at `/zoneinfo`
   - `zoneinfo.data` - The binary file output by emscripten file-packager.  The file contains the actual zoneinfo data loaded into the VFS and parsed by `TimeZoneInfo`.

* [wasm][driver] Add backing support for icall `mono_timezone_get_local_name`

* Add TimeZone info test.

* [wasm][packger] Add `-s FORCE_FILESYSTEM=1`

- Required so that it includes support for loading preload packages

Message when generating the zoneinfo data.
   > Remember to build the main file with  -s FORCE_FILESYSTEM=1  so that it includes support for loading this file package

* Fix AOT tzd test

* Remove commented code lines for `TimeZoneInfo.WebAssembly.cs` module

* Regenerated zone info without --no-heap-copy as it causes problems.  This will create a warning in browser for now.

* Add zoneinfo support files to package

* Fix TimeZoneInfo tests to use the `--enable-fs` packager parameter

* Add back icall implementation after rebase
   - ves_icall_System_TimeZoneInfo_mono_timezone_get_local_name

* Add back backing support for icall `mono_timezone_get_local_name` after rebase

* Fix build error

* Add back zoneinfo tests to Makefile after rebase

* Fix packager zoneinfo sample builds.

* Address review comment about volatile not needed

* Address review comment about volatile not needed

* Address review comment about volatile not needed

* Address review comment volatile not needed
2020-04-04 13:37:21 +02:00
Ankit Jain b3210bf1e5 [wasm][debugger] Improve array visualization (#19424)
* [wasm][debugger][tests] Refactor test support methods

.. which will help in the upcoming changes.

* [wasm][debugger][tests] Add tests for arrays

- These don't include the broken cases which are fixed in the next
commits

* [wasm][debugger][tests] Fix enumerating locals in async method on a

.. struct.

* [wasm][debugger] Support ValueTypes in arrays

- ValueTypes in array elements are not expanded by default. Instead,
they get an object id like `dotnet:array:{arrayObjectId}:{index}`
- And these get expanded on request

* [wasm][debugger] Fix calls to `mono_wasm_add_null_var` from js

- when being called from `library_mono.js`, with a native string like -

    `MONO.mono_wasm_add_null_var ("string");`

.. which when passed to `Module.UTF8ToString (className)`, returns
`null`. So, if the result of that call is null, then use the `className`
argument as-is.

* [wasm][debugger][tests] Add test for local structs in an async method

* [wasm][debugger][tests] Add `TEST_ARGS` and `TEST_FILTER` arguments to

.. control `dotnet test` arguments.

Example use:

`make run-debugger-tests TEST_ARGS="--filter FullyQualifiedName~ArraysTests"`
or
`make run-debugger-tests TEST_FILTER="ArraysTests"`

* [wasm][debugger][tests] Make `label` the last argument, consistently.

* [wasm][debugger] Address review comments

- Return a `Result` response, instead of throwing an exception

* dummy commit to kick the pr
2020-04-03 20:36:44 -04:00
Thays GraziaandLarry Ewing 3122edcb7c [debugger][wasm] Implement evaluate in wasm debugger. (#19432)
* [debugger][wasm] Implement evaluate in wasm debugger.
-> evaluate expressions with constant values and variable values are working
-> evaluate expressions with methods not working yet
-> evaluate expressions with assignments not working yet

* removing extra ;

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Fix what was suggested by @radical.

* Changing what @radical suggested

* Changing what @radical suggested

Co-authored-by: Larry Ewing <[email protected]>
2020-04-03 19:14:04 -03:00
Александр Хухлаев f713aef67b [WinForms] Show ToolTips for PropertyTab`s buttons (#19430)
for custom tabs.

when the mouse is on the native tab buttons, tooltips appear, and when on the custom tab buttons, then no
2020-04-03 22:04:46 +02:00
Thays Grazia fcc6731c83 [debugger] Fix set_value and get_value after invoke a method (#19378)
* When we run an invoke in a thread and after this we do a set_value and then a get_value, when we call set_value we consider restore_state, but when we call get_value we are not considering the restore_state.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1077790
2020-04-03 16:14:41 -03:00
Mitchell HwangandMitchell Hwang c6ff50ed49 Bump corefx to get https://github.com/mono/corefx/pull/395 (#19376)
Co-authored-by: Mitchell Hwang <[email protected]>
2020-04-03 19:39:46 +02:00
Zoltan Varga 798ced842a [wasm] Avoid including config.make into wasm/Makefile, its generated by one of the builds triggered by this Makefile. (#19427) 2020-04-03 13:18:08 -04:00
monojenkinsandBrzVlad 4988c9eeca [sgen] Implement api for allocating pinned arrays (#19407)
https://github.com/dotnet/runtime/issues/33583

Co-authored-by: BrzVlad <[email protected]>
2020-04-03 15:49:23 +03:00
monojenkinsandalexischr 5fc01701ad [mono][sre] Add an expected attribute on underlying fields of enum cl… (#19349)
…asses

Follow CoreCLR's example and add the RTSpecialName type attribute to any class field with the 'value__' name when , used as the value store for enumeration class instances.

Second part of the "RTSpecialName fix" that started with https://github.com/dotnet/runtime/pull/33389 - before, our behavior was to add the RTSpecialName immediately, as soon as the EnumBuilder was constructed.
Contributes to https://github.com/dotnet/runtime/issues/2389

Co-authored-by: alexischr <[email protected]>
2020-04-03 08:48:25 -04:00
Alexander Köplinger c55dee223f Fix iOS CI device test build
MONO_ENTER_GC_UNSAFE/MONO_EXIT_GC_UNSAFE aren't in a public header yet: https://github.com/mono/mono/issues/19421
2020-04-03 14:25:32 +02:00
Steve Sanderson bd2ca880f9 Fix Mono WebAssembly regression on Firefox (#19420)
Since 3 days ago, Mono WebAssembly no longer works on Firefox. The error is `ReferenceError: SharedArrayBuffer is not defined`.

This PR adds a trivial fix.
2020-04-03 01:09:25 +02:00
monojenkinsandCoffeeFlux edf3480ade [meta] Add mono_type_get_name_full to public API (#19414)
This is needed by Xamarin.Android to be able to round-trip with names fetched via reflection, as the only public API for this (`mono_type_get_name`) uses the IL format instead.

Fixes https://github.com/mono/mono/issues/19377

Co-authored-by: CoffeeFlux <[email protected]>
2020-04-02 12:45:51 -04:00
54e6ca20b4 [loader] Add append option to loader hook installation functions (#19371)
* [loader] Add append option to loader hook installation functions

Fixes https://github.com/dotnet/runtime/issues/34273

While this specific issue only affects the preload hook, I've added the append option to all the installation functions since this seems like a generally useful capability, both for internal use and embedders.

@BrzVlad tested locally and confirmed this patch solves the problem.

* Update callers in pedump and monodis

Co-authored-by: CoffeeFlux <[email protected]>
Co-authored-by: Ryan Lucia <[email protected]>
2020-04-02 12:39:10 -04:00
Larry Ewing 82dc6e8a5e [wasm][debugger] don't try to process commands sessions without context (#19394) 2020-04-02 08:35:34 -05:00
Kenneth Pouncey 2c783ab80b [wasm][README] WASM Dynamic Runtime doc (#19416)
* update sample config

* Add dynamic runtime config

* update dynamic runtime docs
2020-04-02 08:35:11 -05:00
Ankit Jain 2709c812c0 [wasm][debugger] Add support for visualizing valuetypes (#19232)
* [wasm][debugger] Move some of the variable name/value pairs fix up to

.. library_mono.js from MonoProxy.

* [wasm][debugger] Handle invalid scope id

- this would throw NRE, and the request would fail, and so the test
passed.

* [wasm][debugger][tests] Check functionName at breakpoints

.. and fix up tests.

* [wasm][debugger][tests] Remove unnecessary `ctx` arg

* [wasm][debugger] Add support for visualizing valuetypes

- Valuetypes are expanded into json, and cached. That gets used to
answer any `getProperties` request for them.

- When a valuetype is a local var, it's whole value expansion is
returned in `members` json field. This gets cached by MonoProxy, and
used to answer any `getProperties` requests for it's members.

- When a valuetype is part of an object, it is *not* expanded. Instead,
it gets an objectId of the form `dotnet:valuetype:containingObjectId:fieldOffset`.
  - Later, when the debuggers asks for that special id, we expand *all*
  the valuetypes in that containing object, and cache it.

  - For properties, the property "number" as they are enumerated, is
  used instead of the fieldOffset.

- this cache gets cleared whenever we step/resume the debugger.

* [wasm][debugger] Add support for visualizing enums

TODO: Flags

* [wasm][debugger] Don't invoke property getters when enumerating object

.. members.

* [wasm][debugger] Override `ToString` for `Result` class
2020-04-02 09:04:42 -04:00
sharkcz 2a531e04b3 update perf jitdump for ppc64/ppc64le and s390x (#19405)
Add missing constants to handle ppc64/ppc64le and s390x platforms.

Related: https://github.com/mono/mono/commit/68a8b9290958ee5e9b21d8ff6b73a707e2504d39
Fixes: https://github.com/mono/mono/issues/19402
2020-04-01 22:07:22 +02:00
Alex Thibodeau b1d4be3bab Ensure generic parameter constraint type is included when building image (#19395)
sets.

Making associated change to type_in_image to also check the constrained type for a match. Re-adding asserts now they they no longer trigger.

fixup, accidentally used old function

adjusting coding convention to K&R
2020-04-01 16:01:01 -04:00
Zoltan Varga bd635ac8b4 [wasm] Throw an exception when trying to start a thread instead of asserting. (#19396) 2020-04-01 15:13:20 -04:00
Alexis Christoforides 5733960ac4 [ios][sdks] Fix finalizer thread init in runner (#19412)
Starting with https://github.com/mono/mono/pull/16907 , the runtime ends in GC-Safe state (mode) after mono_jit_init_version() is called. mono_gc_init_finalizer_thread() expects the GC to not already be in safe mode.
2020-04-01 21:05:55 +02:00
Alexander Köplinger 9485a66d20 Fix Windows .msi build using newer xar
The old version didn't compile anymore with OpenSSL 1.1
2020-04-01 16:31:27 +02:00
monojenkinsandEgorBo a92528508e [mono] Fix setMArch, getArchName() doesn't work as expected (#19403)
Turns out `llvm::Triple(llvm::sys::getDefaultTargetTriple()).getArchName()` doesn't work as expected on all targets so the only option is to set arch via defines.

NOTE: it accepts "x86-64", not "x86_64

Co-authored-by: EgorBo <[email protected]>
2020-04-01 14:23:31 +02:00
monojenkinsandEgorBo ad0ee22923 [mono] Fix LLVM JIT on Arm64 (#19397)
`EB.selectTarget ();` used to return `NULL` on arm64 (and hit an assert later)
Turns out it needs MArch on arm (e.g. `"aarch64"`) set. No idea why it doesn't use host arch by default on arm (but works on x86).

Co-authored-by: EgorBo <[email protected]>
2020-04-01 12:43:06 +02:00
Alexander Köplinger e10990125d Fix Windows .msi build with recent cygwin updates
Cygwin now ships libssl 1.1 which removed OpenSSL_add_all_ciphers so the xar configure fails.
Replace the configure check with another existing function to fix it.
2020-04-01 12:36:26 +02:00
monojenkinsandpi1024e 49ea50d5b4 Remove duplicated .gitignore patterns (#19370)
Co-authored-by: pi1024e <[email protected]>
2020-04-01 11:46:54 +02:00
monojenkinsandEgorBo f301481bc1 [mono] fix SIMD instructions' availability (#19387)
Fixes: https://github.com/dotnet/runtime/issues/34304

@tannergooding @davidwrighton
Now it fully matches clang/LLVM behavior, e.g. even `-mattr=+avx2` (or `-mavx2` in clang) doesn't enable `pcmul` and `aes`. However, `-mattr=+sse4.2` (`-msse4.2`) enables `popcnt` (see https://godbolt.org/z/RWAp_D). The intel manual says

![image](https://user-images.githubusercontent.com/523221/78015652-59d70e00-7352-11ea-9f88-64772e76f842.png)

So is there a CPU with AVX set but without `popcnt`? The instruction is pretty useful (it is used in BCL) users might forget to define it when they use `mattr=avx` (or `mattr=avx2`).

However in C++ world programmers use named groups e.g. `mcpu=haswell` rather than listing these sets via attributes.

Co-authored-by: EgorBo <[email protected]>
2020-04-01 11:41:50 +02:00
Thays Grazia a17ab75dea Adding static to ss_req_acquire and ss_req_count (#19390) 2020-03-31 15:37:04 -03:00
Fan Yang 68a8b92909 Initial perf jitdump implementation (#19273)
* Initial perf jitdump implementation

* Incorporate PR feedback

* Incorporate feedback round 2

* Incorporate feedback round 3

* Minor change

* Incorporate feedback round 4
2020-03-31 09:03:43 -04:00
Zoltan Varga b9f52f3681 [runtime] Fix a problem with PR #19361. (#19374) 2020-03-31 05:06:12 -04:00
Kenneth Pouncey d525e3d5a3 Document threading setup for webassembly build (#19341) 2020-03-31 06:33:30 +02:00
Kenneth Pouncey 62e6a2e0c6 Reactivate the browser threads tests. (#19365) 2020-03-31 06:32:38 +02:00
monojenkinsandimhameed 722e58cc15 [mono][llvm] Improve LLVM 9 compatibility. (#19340)
Improve LLVM 9 compatibility.

Several intrinsics were removed in LLVM 7 and above; replace them with
IR sequences that generate the desired instructions.

Fix some type mismatch problems with the generated IR (in
Sse2.StoreAlignedNonTemporal, Vector128.CreateScalarUnsafe, and
Sse2.MaskMove); these were caught by running the coreclr hardware
intrinsic JIT tests.

Support the two-argument forms of Sse.SqrtScalar, Sse.ReciprocalScalar,
and Sse.ReciprocalSqrtScalar.

llvm::linkCoreCLRGC(), in llvm/CodeGen/GCs.h, was removed and replaced with
llvm::linkAllBuiltinGCs(), in llvm/CodeGen/BuiltinGCs.h.

In mono_llvm_dump_value and mono_llvm_dump_module, flush the c library's
stdout stream before using (and flushing) LLVM's own buffered output
functions.

Co-authored-by: imhameed <[email protected]>
2020-03-30 14:01:45 -07:00
Zoltan Varga b745e20e84 [wasm] Add a try-catch around GetCallingAssembly () calls in ResourceManager, it can fail on wasm aot. (#19358)
Fixes https://github.com/mono/mono/issues/19314.
2020-03-30 13:57:26 -04:00
Jérôme Laban 5bf237a2a4 [Wasm] Fix string_decoder when running with threading enabled (#19355)
* [Wasm] Fix string_decoder when running with threading enabled

Avoids the following error message:

```
Failed to execute 'decode' on 'TextDecoder': The provided ArrayBufferView value must not be shared.
```

* [Wasm] Cache text decoder
2020-03-30 10:13:55 -05:00
Zoltan Varga 4f6b3f30f0 [wasm] Throw an exception in some blocking operations in non-threaded mode. (#19363)
* [wasm] Throw an exception from Monitor.Wait () in non-threaded mode.

* [wasm] Throw an exception when waiting for unsignalled events in non-threaded mode.
2020-03-30 08:36:22 -04:00
monojenkinsandBrzVlad 771fbddefa [interp] Optimize Span<byte>.Fill (#19351)
Co-authored-by: BrzVlad <[email protected]>
2020-03-30 11:56:02 +03:00
Zoltan Varga a30be514e2 [llvmonly] Avoid marking the rgctx/vtable variable indirect, its not needed. (#19287)
In normal llvm mode, these variables are marked as indirect and stored to a stack slot so the EH code can access them.
2020-03-30 04:54:00 -04:00
Zoltan Varga 07febe2bb0 [runtime] Allocate the memory for gshared gparams from image sets. (#19361)
Fixes https://github.com/mono/mono/issues/18127.
2020-03-30 04:43:38 -04:00
Larry Ewing cf7d5d5f45 [wasm][sdk] fix output files (#19360) 2020-03-29 16:36:48 -04:00
Zoltan Varga fbc3a9757a [wasm] Fix threads support in the packager. (#19359) 2020-03-29 16:17:07 -04:00
Zoltan Varga 6e9e5b8ef2 Bump ikvm to pick up 26a033ed9de36f462946232a9b1b7984618c5e1f. 2020-03-29 13:21:00 +02:00
Zoltan Varga f2878d45d8 [wasm] Bump emscripten to 1.39.11. (#19342)
* [wasm] Bump emscripten to 1.39.11.

* [wasm] Build the debug runtime using -g, -g4 disables dwarf.

* Add runtime debugging docs.

* Avoid copying the dotnet.wasm.map file its not generated anymore with dwarf.
2020-03-29 04:21:53 -04:00
Zoltan Varga d336d6be30 [llvm] Handle this arguments as inputs to phi nodes, i.e. for IL code doing starg.0. (#19323) 2020-03-28 03:59:37 -04:00
Vlad Brezae 450a6d6ac7 [interp] Constant fold also checked conversion opcodes (#19294)
* [interp] Constant fold also checked conversion opcodes

* [interp] Remove duplicated opcode
2020-03-27 23:06:25 +02:00
Alex Thibodeau e5be81d921 Need to convert from uint instead of cast here to maintain consistency with mono_gchandle_new_weakref_internal. (#19333)
Looks like this was missed in: https://github.com/Unity-Technologies/mono/commit/941a335ea0f20c22a02a7947945f53787a56b2d3
2020-03-27 20:58:32 +01:00
monojenkinsandBrzVlad 09f5ce3588 [interp] Fix issues around not yet supported System.Runtime.Intrinsics (#19344)
Special cases around infinite recursiveness were not handled by the interpreter.

Co-authored-by: BrzVlad <[email protected]>
2020-03-27 17:03:17 +02:00
Kenneth Pouncey 42c9cfb1fd [wasm][tests] Cleanup browser tests (#19325)
* fix clean target

* Upgrade project to netstandard2.1 and change to not append framework when publish

* Move projects to netstandard2.1

* Remove code referencing framework paths
2020-03-27 06:05:55 +01:00
Kenneth Pouncey 48871c1ff5 [wasm][sdk] Remove Required attribute from Bindings Directory. (#19326)
- Causes build error if sdk path is not being overridden.
2020-03-27 06:04:49 +01:00
Zoltan Varga 36f41ac8df [wasm] Build aot-hello using --nobinding. (#19331) 2020-03-26 16:56:15 -04:00
Thays Grazia 282320c35f [debugger] Bump protocol for this commit https://github.com/mono/mono/pull/19248 (#19318)
* Bump protocol for this commit https://github.com/mono/mono/pull/19248 as suggested by @vargaz.

* Fix coding convention.
2020-03-26 16:51:45 -03:00
monojenkinsandCoffeeFlux 1add5cbc06 [meta] Make monovm.h public (#19305)
Co-authored-by: CoffeeFlux <[email protected]>
2020-03-26 11:42:21 -04:00
monojenkinsandCoffeeFlux 4c36b361c4 [loader] Initial work to support ALC collectability (#19304)
This should enable creation of a collectible ALC from managed, though it will not actually unload properly when Unload is called. This should be enough to unblock Xamarin porting work.

If you think the `mono_alc_cleanup` work is just noise I can remove that, but I thought it helpful to check in to make future diffs clearer as I split it up with the LoaderAllocator work.

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-26 11:17:57 -04:00
Thays Grazia 78fe8c2969 [debugger] Fix suspend_policy that will be send to debugger-libs (#19317)
- Fix suspend_policy that will be sent to debugger-libs, because on debugger-libs we want to resume the VM only when it's suspended, but we were sending the wrong suspend_policy in the case of EVENT_KIND_VM_START and mono is started with suspend=y.
  With this fix we can do this PR again: https://github.com/mono/debugger-libs/pull/264/

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/999375/
2020-03-26 10:15:11 -03:00
Martin Baulig c9171a63be [wasm][debugger]: Correctly handle null sessionId. (#19327)
When the sessionId is null, don't send "null" but omit the field;
Canary does not permit "sessionId: null".
2020-03-26 14:07:14 +01:00
Zoltan Varga 151ba57d0f [llvm] Disable running the llvm verifier by default, it was enabled by mistake, and it takes a long time. (#19319) 2020-03-26 08:10:30 -04:00
Martin Baulig b0e3196b9f [wasm][debugger]: Move some of the internal helper classes into a separate file. (#19324) 2020-03-26 11:52:38 +01:00
monojenkinsandlateralusX c5a9510329 Add stack guarantee on Mono Windows to handle stack overflow. (#19296)
When hitting stackoverflow on Windows, the amount of stack available to exception handler is too small. Windows doesn't support alt stack as other platforms, but there is a way to guarantee a certain amount of available stack during stack overflow exceptions using
SetThreadStackGuarantee.

Fix add support on Windows platforms including SetThreadStackGuarantee API to reserve space needed to handle stack overflow exceptions. On Windows debug build the size needs to be larger since we might end up calling JIT that could have quite deep callstack that, on unoptimized
builds, will need more stack.

Co-authored-by: lateralusX <[email protected]>
2020-03-26 10:29:52 +01:00
Kenneth Pouncey eb9cb9b302 [wasm] Add make target to clean the framework directory (#19316) 2020-03-26 05:46:51 +01:00
d9089e914b [wasm][bindings] Remove typedarray length validation (#19315)
* [wasm][bindings] Remove typedarray length validation

fixes https://github.com/mono/mono/issues/19288

* Update sdks/wasm/framework/src/WebAssembly.Bindings/Core/TypedArray.cs

Co-Authored-By: campersau <[email protected]>

Co-authored-by: Larry Ewing <[email protected]>
Co-authored-by: campersau <[email protected]>
2020-03-26 05:46:00 +01:00
Matt Mitchell 437ca1d81c Feed cleanup (#19321)
Switch away from blob feeds
2020-03-25 19:52:08 +01:00
Alex ThibodeauandJonathan Chambers b1a1f06110 Do not access unloading domains in debugger (case 1013579) (#19299)
There was a race where a domain was being unloaded while debugger would access it. A domain was only removed from the 'appdomains_list' as a very last step. The domain was already invalid to access at this point (locks freed for example).

Worse, images would be unloaded if only referenced by that domain. All places the debugger iterates domains hold the loader lock. The loader lock is acquired by the domain unloading process, so as long as a domain is not unloading when we access it inside of the loader lock we are safe.

include domain unloading check in collect_domain_bp

Co-authored-by: Jonathan Chambers <[email protected]>
2020-03-25 14:15:08 -03:00
monojenkinsandCoffeeFlux 2881514b3d [loader] Fix some log masks in native-library (#19280)
Copy-paste error

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 16:43:41 +01:00
CzBiX 9967b065b2 Refine UnixEndPoint to support Windows Unix socket (#19274)
On windows, remote end point client will not return file name, but socket address size still isn't `0x02`.
So check empty file name after bytes readed.
2020-03-25 16:25:57 +01:00
monojenkinsandCoffeeFlux 45694f7c99 [loader] Clear filename when debugging assembly unloading (#19303)
Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 11:09:35 -04:00
monojenkinsandCoffeeFlux a89b6d3217 [metadata] Skip symbol name mangling on non-Windows for netcore (#19281)
Fixes dotnet/runtime#33246

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 15:23:00 +01:00
Kenneth Pouncey 61830e01f6 [wasm][sdk] Add functionality to override the bindings framework location (#19310)
* [wasm][sdk] Add functionality to override the bindings framework location

- Facilitate testing of intree build during the browser tests.

* white space
2020-03-25 12:02:46 +01:00
Vlad Brezae 8ec501a34f [jit] Initialize correct class for tls fields (#19300)
Initialize the klass containing the field to be accessed, rather than the class of the method from where we are accessing.
2020-03-25 10:06:27 +02:00
Zoltan Varga 8543df76a9 [llvmonly] Throw an ExecutionEngine exception with a helpful message when we try to execute a method which failed AOT compilation. (#19302) 2020-03-25 00:40:09 -04:00
dotnet-maestro[bot]anddotnet-maestro[bot] 4eb25bc4b6 Update dependencies from https://github.com/dotnet/arcade build 20200321.1 (#19289)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20171.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20171.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-25 00:40:51 +01:00
monojenkinsandCoffeeFlux 9f390a7d8a [metadata] Fix leak in custom attribute lookup (#19279)
https://github.com/dotnet/runtime/commit/9dec6ae2ea81d46884bf49d2df8b26cc1e28eb12#r37604565

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-24 19:40:34 -04:00
a9bf876bb7 [debugger] Implementing step through multithreaded code. (#19103)
* Implementing step through multithreaded code.

* Fixing concurrency in the case that more than one thread will run the same code, then when the jit is finished, the thread which jitted the code calls jit_done and the others that were waiting don't call, in this case, the other threads can run before the thread which is calling the jit_done set the breakpoint, so the other threads will not stop in the breakpoint.
As I understood we don't need to check if the try_suspend_runtime is returning false or true when we  hit a breakpoint from context, so removed the call.

Fixes #14456

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
Co-authored-by: monojenkins <[email protected]>
2020-03-24 18:24:33 -03:00
Alexander Köplinger 4cbcc9195d Reenable a bunch of disabled iOS Mono SDKs test suites (#19301)
* Reenable a bunch of disabled iOS Mono SDKs test suites

System, System.Net.Http, System.Web.Services, Mono.CSharp and Mono.Data.Sqlite test suites are running now.

Also tweaks the test UI so you can query it via Xcode's UITest in an automated way.

* Switch to APIs from public headers added in 1425b4d4d0
2020-03-24 20:41:38 +01:00
monojenkinsandlambdageek 1425b4d4d0 [mono] Make some API functions public and private (#19259)
See https://github.com/dotnet/runtime/pull/33633 and https://github.com/xamarin/xamarin-macios/blob/master/runtime/exports.t4

* `mono_install_load_aot_data_hook` was already a `MONO_API`, but it was not in a public header.
  It is added to `mono/jit/mono-private-unstable.h`
* `mono_gc_init_finalizer_thread` was [used by Xamarin.iOS](https://github.com/xamarin/xamarin-macios/blob/63ab48e679716264b33af00f86c24426da1d9f97/runtime/mono-runtime.h.t4#L278) for a long time without being in any public header.  It's one of the reasons we have to compile Mono for XI with `--disable-visibility-hidden`.  Make the function a proper `MONO_API` unconditionally - but make it do nothing if the runtime is compiled without `--with-lazy-thread-creation` (the default).  It is now a public Mono API function.
* `mono_trace_init` was a `MONO_API` function in a non-public header that was already used by embedders that need to set up logger hooks.  It is now not necessary to call this before calling the logger functions to set up logger hooks such as `mono_trace_set_log_handler`

Co-authored-by: lambdageek <[email protected]>
2020-03-24 09:29:34 -04:00
Vlad Brezae 77bcd7497d [interp] Replace ldloca + ldfld/stfld pairs with ldloc/stloc (#19286)
* [interp] Move INDIRECT flag to a separat counter variable

We will optimize away some ldlocas, so a local can stop being indirect at a future cprop iteration.

* [interp] Replace ldloca + ldfld/stfld pairs with ldloc/stloc

If loaded/stored field is the only field of the valuetype. This is meant to mainly optimize IntPtr accesses.

* [interp] Add counter for removed LDLOCA
2020-03-24 13:59:03 +02:00
Alexander Köplinger 81939f45c4 Bump nunitlite to get https://github.com/mono/NUnitLite/pull/26 2020-03-24 11:17:32 +01:00
monojenkinsandlateralusX db7c5aa72e Remove Mono runtime vcruntime140.dll release build dependency. (#19270)
CoreCLR static link vcruntime140.dll, but dynamic link rest of C-runtime using ucrt.lib in release build. Commit align Mono runtime to follow same pattern, removing dependency on vcruntime140.dll.

Co-authored-by: lateralusX <[email protected]>
2020-03-24 08:24:01 +01:00
monojenkinsandlateralusX 7d33bd36f1 Implement icall used by RuntimeHelpers.EnsureSufficientExecutionStack on netcore Mono. (#19271)
Current implementation always returned true for Windows/Linux/Android. There are however use of this API in Task library to make sure recursive tasks won't hit stackoverflow. There are several tests in System.Threading.Tasks that currently hit this, at least on Windows, causing test failures.

Fix use stack limits already setup in register_thread and used by GC when doing conservative stack scan. If the limits have not been setup or is not supported on platform, we will use old defaults, always assume there is enough stack space available. Heuristics around size of minimum execution
stack needed are picked from corresponding CoreCLR implementation.

With this fix all tests run in System.Threading.Tasks.Tests pass on Windows using Mono runtime:

```
=== TEST EXECUTION SUMMARY ===
     System.Threading.Tasks.Tests  Total: 682, Errors: 0, Failed: 0, Skipped: 3, Time: 11,419s
----- end 2020-03-20 13:43:35,72 ----- exit code 0 ----------------------------------------------------------
```

Before fix the following tests hit stackoverflow:

LongContinuationChain_ContinueWith_DoesNotStackOverflow
LongContinuationChain_Unwrap_DoesNotStackOverflow
LongContinuationChain_Await_DoesNotStackOverflow
RunStackGuardTests

Co-authored-by: lateralusX <[email protected]>
2020-03-24 08:22:32 +01:00
Johan Lorensson fa0f2ce8ab Move unbox tramp method address table to data segment, MONO_ARCH_CODE_EXEC_ONLY. (#19292)
unbox tramp method addresses are emitted as a table of calls and read
at runtime by get_call_table_entry. This is problematic on platforms
that doesn't allow data to be read from instruction stream.

Fix follow same pattern as we method address table emitted as function
pointers into table in data segment when using MONO_ARCH_CODE_EXEC_ONLY.
2020-03-24 00:28:15 -04:00
monojenkinsandlambdageek 9504df9ad2 [mono] retry mono_threads_pthread_kill if result == EAGAIN on Linux (#19291)
Try to address https://github.com/dotnet/runtime/issues/32377
(signal queue overflow) by sleeping and retrying a few times.

This is kind of a hack, but it would be good to see if it will address the issue

Co-authored-by: lambdageek <[email protected]>
2020-03-23 23:14:00 -04:00
monojenkinsandlambdageek 0102498d01 [mono] Add headers for unstable APIs (#19272)
We have functions that are de-facto part of the Mono embedding API that are
used by Xamarin.iOS or Xamarin.Android that are not in a state where we want to
commit to supporting them forever in their current form.

Nonetheless, we would like to make sure that the symbols for these functions
are visible even if we otherwise compile the runtime with
`-fvisisibility=hidden`, and we would like to discourage declaring the
functions in the embedder's own headers.

The functions that go into the
`mono/{utils,metadata,jit}/mono-private-unstable.h` headers will all be marked
with `MONO_API MONO_RT_EXTERNAL_ONLY` but we will not guarantee that the
functions will be there from one release to the next or that they will not
change their signatures or their behaviors.

Co-authored-by: lambdageek <[email protected]>
2020-03-23 19:54:25 -04:00
Zoltan Varga bf75db1199 [jit] Perform inlining earlier during call processing to avoid loading vtable/rgctx args which are not needed. (#19283)
* [jit] Perform inlining earlier during call processing to avoid loading vtable/rgctx args which are not needed.

* Avoid inlining calls marked with .tail.
2020-03-23 15:16:56 -04:00
Jay Krell 93457ec8c3 [debug_interp] Do not use uninitialized local. (#19275) 2020-03-23 12:38:37 +01:00
CoffeeFlux 47d542a1b8 [loader] Fix mistake in netcore_load_reference comment
The code here was right, the comment wrong. Also updated to mention that we now check APP_PATHS.
2020-03-23 07:11:05 +01:00
Zoltan Varga 3a08f494a9 [llvm] Unify the emission of GOT access. NFC. (#19250)
* [llvm] Use the _typed version of get_aotconst () everywhere. Lowercase aotconst variable names.

* [llvm] Unify the emission of GOT access. NFC.

* [llvm] Unify the mini_llvm_init_method icalls, use only 1 icall.
2020-03-21 21:17:58 -04:00
Vlad Brezae e0ea5d5de5 [interp] Remove MINT_TYPE_P (#19269)
MINT_TYPE_P is equivalent to the integer versions. By removing it we reduce the number of interpreter types and instructions, making it also easier to apply optimizations on instructions targeting these types.
2020-03-20 17:08:38 +02:00
campersau 12055c887b [wasm] WasmHttpMessageHandler abort connection when response or response content is disposed (#18765) 2020-03-20 16:07:52 +01:00
Aleksey Kliger (λgeek) a416864867 [coop] Make MONO_ENTER_GC_UNSAFE fatal if no_safepoints is set (#18992)
Previously it would print a warning, only, because otherwise we would get a
second assert in the same place while the crash machinery would run from the
current thread from the abort in `mono_fatal_with_history`

The solution is to turn off the "no safepoints" bit since we know we're
definitely going to assert here.

Also record the current state in the checked mode state transition history to
make it a bit easier to diagnose where the problem surfaced.

Also fixed up a pair of other debugging/tracing messages so the message strings
accurately reflect the state.
2020-03-20 10:58:41 -04:00
41d607b629 Updating the Copyright line display when running 'mono --version' (#19060)
Removes the copyright year.

```
$ mono --version
Mono JIT compiler version 6.6.0.166 (2019-08/d9001b5ae70 Wed Feb 12 19:20:16 EST 2020)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          yes(610)
	Suspend:       hybrid
	GC:            sgen (concurrent by default)
```

Co-authored-by: Alfonso, Ivan <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
2020-03-20 13:22:08 +01:00
Thays Grazia 871bfad41d [debugger] Fix NOT_IMPLEMENTED while debugging. (#19248)
- Changed the behavior on debugger-agent, if we can't parse the new behavior is to return invalid_argument and not assert and stop debugging
- Changed the mono_domain_set_fast before return from assembly_commands.
- Add error message when return INVALID_ARGUMENT

Fixes #19146
2020-03-20 09:21:26 -03:00
Jay Krell ca1e5c3214 Remove dead code from bug-10127.cs. (#18762) 2020-03-20 13:08:06 +01:00
Phil Jaenke aa688393ba [FreeBSD] Fix for libinotify support and compile (#19234)
Resolves #18528 in for both detection and compiling. This should backport cleanly to next release.

Tested on 12.1-RELEASE with libinotify; configure detects features correctly, compiles successfully, resolves some observed issues during run.

```
#define HAVE_SYS_INOTIFY_H 1
#define HAVE_LIBINOTIFY 1
#define HAVE_INOTIFY_INIT 1
#define HAVE_INOTIFY_ADD_WATCH 1
#define HAVE_INOTIFY_RM_WATCH 1
#define HAVE_INOTIFY 1
```
2020-03-20 13:00:13 +01:00
dotnet-maestro[bot]anddotnet-maestro[bot] 4a22f285e5 Update dependencies from https://github.com/dotnet/arcade build 20200312.3 (#19236)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20162.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20162.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-20 12:49:55 +01:00
Alexander Köplinger 0763fb8b13 Bump boringssl submodule 2020-03-20 11:54:54 +01:00
monojenkinsandlambdageek 2a11ec6687 [mono] Enter GC Unsafe mode in some string allocation functions (#19261)
All these functions are MONO_RT_EXTERNAL_ONLY and they all allocate managed
objects.

Fixes a coop state machine assertion if an embedder calls the function and it needs to take the GC
lock to do a collection.

Co-authored-by: lambdageek <[email protected]>
2020-03-19 19:00:26 -04:00
monojenkinsandEgorBo d1362b026e [mono] return false for AdvSimd.IsSupported and friends (#19263)
Fixes StackOverflow in https://github.com/dotnet/runtime/pull/33749#issuecomment-601271356
Intrinsify all `get_IsSupported` under `System.Runtime.Intrinsics*` to just `false` (except the sets we support, see mono_emit_simd_intrinsics).

Co-authored-by: EgorBo <[email protected]>
2020-03-20 01:30:02 +03:00
Zoltan Varga 8ca4d3e6a2 [runtime] Fix the mapping of 'int' in jit icall signatures, it should be int32 not native int. (#19249)
* [runtime] Fix the mapping of 'int' in jit icall signatures, it should be int32 not native int.

* Add a sizet type to icall signatures.
2020-03-19 06:38:19 -04:00
monojenkinsandlateralusX 729bd0a88d Ignore register unwinder callback for empty patch_info data table on Windows Mono Runtime. (#19254)
Running System.Runtime.Serialization.Xml test suite triggers calls to mono_code_manager_reserve with 0 size code block. On Windows, allocating code regions from code manager will also add callback for code region into unwinder function table, but since region could be 0 length, it will be causing a crash in RtlInstallFunctionTableCallback OS call.

Fix hardens code to not try to register callbacks for code regions of size 0, since that is considered a critical error by Windows OS API.

Co-authored-by: lateralusX <[email protected]>
2020-03-19 11:02:03 +01:00
Vlad Brezae 4359b8ff6f [interp] Optimize out intptr ctor (#19252)
* [interp] Extract nint/nfloat intrisifying code into separate method

* [interp] Optimize out intptr ctor

We avoid the newobj_vt opcode and the ctor's contents by emitting a conversion opcode directly.
2020-03-19 10:37:21 +02:00
monojenkinsandlateralusX 468ef7b362 Mono DISABLE_SIMD still emits OP codes depending on SIMD support. (#19251)
Fix makes sure runtime checks also report that SIMD has been disabled. Returning FALSE from is_simd_supported will turn of MONO_OPT_SIMD and disable all SSE/AVX cpu features will prevent code checking cpu capabilities to emit SIMD instructions.

Co-authored-by: lateralusX <[email protected]>
2020-03-18 23:31:47 -04:00
monojenkinsandlateralusX d43e869bc8 Fix warning building Mono using -Wint-conversion. (#19253)
Co-authored-by: lateralusX <[email protected]>
2020-03-18 23:29:25 -04:00
Vlad Brezae 001d4b3338 [interp] Inline also constructors of valuetypes (#19238)
* [interp] Inline also valuetype constructors

* [interp] Unify vt and non-vt inlining code

* [interp] Fix c++
2020-03-18 23:47:08 +02:00
Vlad Brezae 0d9d923e94 [interp] Do cfold for a few more unary operators (#19200)
That are accessing the stack value at level -2.
2020-03-18 23:11:30 +02:00
Johan Lorensson 56088a653f Fix additional constants embedded in instruction stream on AMD64. (#19246)
A couple of more op codes embedded constant values in instruction stream.
When using MONO_ARCH_CODE_EXEC_ONLY these needs to be moved into GOT slots.
2020-03-18 13:51:00 +01:00
Jay Krell acfa161730 Remove useless ip += from MINT_JMP, it is assigned later. (#19217) 2020-03-18 02:42:56 -04:00
Kenneth Pouncey c1b0b95549 [wasm] cleanup npm/node install. (#19244)
browser tests were failing with node command not being found if the other tests were not run before.  This makes sure that the  NPM command can be found by installing it if not there.

This also separates out the npm version that is being installed.
2020-03-17 11:11:53 +01:00
Johan Lorensson 696ec4da7e Fix usage of JIT_FLAG_CODE_EXEC_ONLY flag. (#19237)
Current implementation didn't convert the JIT flags into options on cfg
meaning that code currently checked JIT_FLAG_CODE_EXEC_ONLY never
executed. Fix handles the new JIT flag in same way as other JIT flags,
converted into bit field on cfg.
2020-03-17 00:51:10 -04:00
Alexis ChristoforidesandAleksey Kliger edf6c286a1 [merp] Add breadcrumb for StackHash (#19240)
* [merp] Add breadcrumb for StackHash

Contributes to https://github.com/mono/mono/issues/18715

* Update mono/utils/mono-state.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>

* Update mono/metadata/threads.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
2020-03-16 22:23:21 -04:00
Zoltan Varga 05880086f5 [jit] Transition the SIMD code to use the MONO_CPU_... set of flags instead of the SIMD_VERSION_... flags. (#19139)
* [jit] Transition the SIMD code to use the MONO_CPU_... set of flags instead of the SIMD_VERSION_... flags.

* Enable SIMD version checking when targeting llvm.
* AOT test assemblies using -mcpu=native.

* Remove unused mono_arch_cpu_enumerate_simd_versions (). Move the SIMD_VERSION_ constants to simd-intrinsics.c.

* Use -mcpu=native for aot testing.
2020-03-16 21:02:32 -04:00
Mitchell HwangandMitchell Hwang ab893a369a [Loader] Change mono_trace level from info to debug (#19110)
* [metadata] Change mono_trace levels from info to debug

* [mini] Change mono_trace level from info to debug

* [sgen] Change mono_trace levels from info to debug

* Amended metadata changes

* Amended mini changes

* Amend assembly.c no special case

* Amend sgen-mono.c elevate to warning level

Co-authored-by: Mitchell Hwang <[email protected]>
2020-03-16 08:27:24 -04:00
monojenkinsandEgorBo 2e73a3a25a [mono] Implement all Ssse3 and Sse42 intrinsics (#19225)
Implement all `Ssse3` and `Sse42` intrinsics for LLVM back-end.
`Sse41` is left over (that's why `Sse42.IsSupported` still returns false always outside of corelib)

Godbolt: https://godbolt.org/z/7cd7q5

Co-authored-by: EgorBo <[email protected]>
2020-03-14 23:20:24 -04:00
Zoltan Varga 4ac244261c [wasm] Add a sample for collecting coverage data. (#19223) 2020-03-14 15:16:43 -04:00
monojenkinsandEgorBo 8248c90656 [mono] Implement missing SSE2 intrinsics (#19222)
https://github.com/dotnet/runtime/pull/33465 implemented 99% of them but a few were missing. This PR implements the missing pieces and unlocks `Sse2.IsSupported`.

@vargaz I hope I didn't step on your toe (feel free to close if you already have it locally).

Co-authored-by: EgorBo <[email protected]>
2020-03-13 23:36:01 -04:00
Zoltan Varga d147c60e31 [jit] Add support for emitting r4/r8 constants outside of the text segment. (#19219) 2020-03-13 18:26:29 -04:00
Zoltan Varga 786fdf6a38 [runtime] Coverage profiler improvements. (#19213)
* [interp] Add support for the coverage profiler.

* [runtime] Add 'write-at-method' and 'send-to-method' options to the coverage profiler, they work similarly to options in the AOT profiler.

* Fix c++-isms.
2020-03-13 18:01:00 -04:00
monojenkinsandlambdageek ed2df9c816 [threads] If shutting_down, don't create new thread, kill current thread (#19082)
If the runtime is shutting down, instead of creating a new thread, or throwing
an exception on the current thread, just shut down the current thread.

This takes advantage of `mono_threads_set_shutting_down`'s behavior that if you
call it after shutdown is already started, it will suspend, detach and exit
from the current thread.

Fixes https://github.com/dotnet/runtime/issues/31842

Co-authored-by: lambdageek <[email protected]>
2020-03-13 10:45:14 -04:00
monojenkinsandjkotas 873067cb27 Fix line endings under src/mono (#19215)
Fixes dotnet/runtime#2222

Co-authored-by: jkotas <[email protected]>
2020-03-13 09:54:51 -04:00
Thays Grazia b777471fca If there is a crash and the debugger is attached, we call mini_get_dbg_callbacks ()->send_crash(). (#19191)
That call ends up in mono_debugger_agent_send_crash which does all kinds of questionable stuff like trying to take the loader lock. Eventually it ends up calling socket_transport_send which tries to do a transition to GC Safe mode.
I inserted a MONO_ENTER_GC_UNSAFE in mono_debugger_agent_send_crash function and this fixes the assert.

Fixes #18794
2020-03-13 00:46:06 -03:00
Thays Grazia 0c2f59e016 [wasm][debugger] Continue SS after leaving managed code (#19140)
* Continue stepping over/into/out of managed code to native code, without this fix the behavior was resume when a SS leaves the managed code.
2020-03-13 00:44:49 -03:00
Larry Ewing 6e7146041a [wasm][tests] be slightly more careful looking for the connection string (#19211)
be slightly more strict about connection strings from chrome
2020-03-12 21:42:31 -05:00
monojenkinsandvargaz cb39826fc6 [jit] Add more SSE2 opcodes. Enable SSE2. (#19187)
Co-authored-by: vargaz <[email protected]>
2020-03-12 21:47:42 -04:00
Larry Ewing 150ffb7af3 [wasm][bindings] rework argument marshalling in call_method (#19150)
store a converter object for each type string passed in
use that object to simplify the marshalling step. do one allocation
for arguments and exceptions.
2020-03-12 20:43:43 -05:00
Larry Ewing 35415b28a7 [wasm][bindings] clean up symbol resolution (#19149)
Instead of hard coding a switch of available types we do a one type lookup for each type by name and store that result.
2020-03-12 20:37:30 -05:00
Larry Ewing 54f64a9d08 [wasm][websocket] Buffer writes when endOfMessage == false (#19185)
* [wasm][websocket] Buffer writes when endOfMessage == false

* [wasm][websocket] Set binaryType to arrayBuffer and handle that case
2020-03-12 20:32:24 -05:00
Aleksey Kliger (λgeek) f72f47b90e [corlib] Capture the ExceptionDispatchInfo when rethrowing from TaskContinuation (#19168)
* [corlib] Capture the ExceptionDispatchInfo when rethrowing from TaskContinuation

In `task.GetAwaiter().OnCompleted(action)` if `action` throws, the
`TaskContinuation.RunOrScheduleAction method will catch the exception and then
call `RuntimeAugments.ReportUnhandledException`.  We must not simply throw the
exception again - in that case we will lose all the frames leading up to the
`catch` in `RunOrScheduleAction`.  Instead use ExceptionDispatchInfo to
caputure the original throw.

Addresses https://github.com/mono/mono/issues/19166

* [test] Add regression test

For annoying reasons we can't run this as a simple NUnit test because the
exception is unhandled on some arbitrary threadpool worker thread, and those
don't show up within an NUnit setup.

Additionally because the unhandled exception in the root domain would normally
terminate the application, we need to run this in a new appdomain.
2020-03-12 20:07:08 -04:00
Ankit Jain e0ec8de4b2 [master] Bump msbuild to track xplat-master (#19204) 2020-03-12 23:38:30 +01:00
Alexander Köplinger dcdec02b84 [metadata] Disable three icalls on netcore monodroid (#19198)
They aren't used there.
2020-03-12 19:17:36 +01:00
Johan Lorensson b58d7489be Disable Windows as default build host for WSL sdk builds. (#19196)
SDK builds done under WSL was done like Cygwin in order to be a
Cygwin replacement going forward. This has caused issues when building
sdk's under WSL when Linux is the assumed build host.

Adding an additional define where WSL build host can be overridden to
Windows, but default will now be Linux. This can in future be used
from configure scripts, setting it in sdks/Make.config when needed,
but will be an opt in feature to override Linux as default build host
for WSL.
2020-03-12 13:12:33 -04:00
Zoltan Varga 8fe10ad8e3 [wasm] Add some makefile dependencies. (#19193) 2020-03-12 11:25:08 -05:00
monojenkinsandlateralusX 70141f4959 Share managed EventSource/EventPipe implementation between CoreCLR and Mono. (#19173)
First set of changes needed in order to start sharing managed EventSource/EventPipe code between CoreCLR and Mono runtime. Shareable code has been moved into S.P.C library project and split into runtime specific source files when needed, kept within each runtime specific S.P.C project files.

Mono runtime has been extended with a set of icalls needed by EventPipeInternal, just to make sure EventPipe can be initialized, current Mono runtime implementation returns dummy values.

Current change also enables ETW provider support on Mono Windows runtime.

NOTE, this is just initial changes needed in order to share managed EventSource/EventPipe code between runtimes. Future changes will incrementally add EventPipe native code into Mono runtime.

Co-authored-by: lateralusX <[email protected]>
2020-03-12 13:36:24 +01:00
Alexis Christoforides 662f744a93 [merp] Remove unnecessary call to msync in mono_state_free_mem (#19167)
Closes https://github.com/mono/mono/issues/19136
2020-03-12 06:51:36 -04:00
Alexander Köplinger aeda6f0b26 Fix mistake in previous commit 2020-03-12 01:19:38 +01:00
Alexander Köplinger 306ebdfd30 Don't disable dllmap for mobile on netcore 2020-03-12 01:13:27 +01:00
Alexander Köplinger 0cc83ec613 Add assert when dllmap is disabled and fix support build in netcore mode
Ported from https://github.com/dotnet/runtime/commit/a54d391d9daebf7b53e7f1efc1690c1c35177570
2020-03-12 01:09:46 +01:00
Vlad Brezae b99f79fa94 [interp] Add more Math intrinsics (#19174)
* [interp] Add missing intrinsics for double Math operations

* [interp] Add math intrinsics for float operations

* [interp] Add more math intrinsics

With different type of signature

* [interp] Remove intrinsic for Round for now

Which has different semantics from the libc's round
2020-03-12 00:14:41 +02:00
Larry Ewing 691665884b [wasm][benchmarks] Add --wasm-package to bench runner to consume package builds (#19190)
This adds an option the standalone-benchmark runner to extract the package zips created with the package target and available from jenkins for commits to master.
2020-03-11 15:20:33 -05:00
Zoltan Varga 19fa221a33 Avoid passing /features:peverify-compat to csc, it generates very inefficient code in some cases. (#19144)
See https://github.com/mono/mono/issues/18572#issuecomment-595924767.

Fixes https://github.com/mono/mono/issues/18572.

Disable the IL verifier for now, it can't handle code compiled without /features:peverify-compat.
2020-03-11 19:43:19 +01:00
Vlad Brezae 94ff330b60 [interp] Optimize delegate invokes (#19065)
Add a specialized MINT_CALL_DELEGATE invoke, which is faster than executing a separate generic managed wrapper method.
2020-03-11 20:09:55 +02:00
monojenkinsandEgorBo 7d623e5e62 [mono] Optimize boxing in Pattern Matching idioms (#18812)
This PR optimizes boxing for pattern matching idioms. Basically, it copies existing optimizations from CoreCLR to mono, see [impBoxPatternMatch](https://github.com/dotnet/runtime/blob/084dd1a5aae0fbe6ed4f9164ac4ee1c5f0b2398f/src/coreclr/src/jit/importer.cpp#L5856-L6032) (one of them was recently added by me for coreclr, see https://github.com/dotnet/runtime/pull/1817)
#### 1) `box + isinst + brtrue/brfalse`  -->  `ldc.i4.0/1 + brtrue/brfalse`
```csharp
public static int Case1<T>(T t)
{
    if (t is int)
        return 42;
    return 0;
}
/*
    IL_0000: ldarg.0
    IL_0001: box !!T
    IL_0006: isinst [System.Private.CoreLib]System.Int32
    IL_000b: brfalse.s IL_0010
    IL_000d: ldc.i4.s 42
    IL_000f: ret
    IL_0010: ldc.i4.0
    IL_0011: ret
*/
```
#### 2) `box + isinst + ldnull + ceq/cgt.un`  -->  `ldc.i4.0/1`
```csharp
public static bool Case2<T>(T t)
{
    return t is int;
}
/*
    IL_0000: ldarg.0
    IL_0001: box !!T
    IL_0006: isinst [System.Private.CoreLib]System.Int32
    IL_000b: ldnull
    IL_000c: cgt.un
    IL_000e: ret
*/
```
#### 3) `box + isinst + unbox.any`  -->  `nop`
```csharp
public static int Case3_1<T>(T o)
{
    if (o is int x)
        return x;
    return 0;
}

// or

public static int Case3_2<T>(T o) => o is int n ? n : 42;

// or

public static int Case3_3<T>(T o)
{
    return o switch
    {
        int n => n,
        string str => str.Length,
        _ => 0
    };
}
```
Tests can be found here: https://github.com/dotnet/runtime/tree/master/src/coreclr/tests/src/JIT/Generics/Conversions/Boxing

[Sharplab.io link](https://sharplab.io/#v2:EYLgtghgzgLgpgJwDQxASwDYB8CwAoAAQAYACAZQAsIEAHAGQmADoAlAVwDsY0w4BufPgIBmEgQCMANjEAmEgGF8Ab3wk1Y0ROlouC6HHEAeACoA+ABTGSMAJSr1KvOuck0AMxLmYrqK652nFyCCAHYSABYZAUDgsKJo5wBfezUUjTEpEmAAe2yMPSg4GRMLK1s0xyC1UOsfPxgE9WSYkjSRDO1deX1hAH0jM0sSbIDnSqr3T2y6nW8AD1Gq5xq5xtiSeLTmts1M2YK4PuLBqxGSAF5TYZndDhIAfhI7kAiowRb2rXqDvuESoZGFTSyzC0ygAHc0DAAMYUYFVcZLIL7O6XJ5IeFIjKkWAIC5XXFMOhwDgAcxgFAxLSxal6+I2mJciTWJGaznwiSAA===) to observe IL code for the samples above.

#### Codegen diff example:
```csharp
static bool Test(int n) => Validate(n);

static bool Validate<T>(T t) => t is int;
```
Before (for `Test`):
```asm
0000000000000000	pushq	%r14
0000000000000002	pushq	%rbx
0000000000000003	pushq	%rax
0000000000000004	movl	%edi, %ebx
0000000000000006	movabsq	$0x7fdf9c5090c0, %rax
0000000000000010	movabsq	$0x7fdfad0268e8, %r14
000000000000001a	leaq	0x1be800(%r14), %rdi
0000000000000021	movl	$0x14, %esi
0000000000000026	callq	*(%rax)
0000000000000028	movl	%ebx, 0x10(%rax)
000000000000002b	movq	(%rax), %rax
000000000000002e	cmpq	%r14, (%rax)
0000000000000031	sete	%al
0000000000000034	addq	$0x8, %rsp
0000000000000038	popq	%rbx
0000000000000039	popq	%r14
000000000000003b	retq
```
After:
```asm
0000000000000000	movb	$0x1, %al
0000000000000002	retq
```

Co-authored-by: EgorBo <[email protected]>
2020-03-11 19:45:48 +03:00
imhameed b74bbe9269 [arm] Fix branch island disassembly for arm64 and thumb. (#19188)
Followup to https://github.com/mono/mono/pull/19126 and
https://github.com/mono/mono/pull/19169.

Bring arm64 `mono_arch_get_call_target` in line with other platforms by
returning NULL on failure instead of asserting. Bits 30-26 in an arm64
unconditional branch are exactly 0b00101, so 0x1f (0b11111) is used as a
mask to determine if the branch uses an immediate target--0x7 (0b111)
would incorrectly match against branches using register targets. The
extracted immediate is now sign extended in a branch-free way.

Avoid following blx entries in the method address table; ld64 can emit 4
different instruction sequences for islands involving thumb, and I'd
rather not implement support for them all now because 32-bit iOS is old
and large unlinked assemblies seem rare.
2020-03-11 13:43:08 +01:00
monojenkinsandpi1024e 30f5d34400 No need to have a "while (TRUE)" statement if we can check at the end of the loop or at the beginning. (#19111)
Whenever there must be a loop that goes indefinitely, there are either return statements or break statements in the code.

However, in this codebase, there are "infinite loops" that appear as such until the next line is checked, which is an if statement saying to break if a condition is true.

This unneeded complexity can be resolved by inverting the condition and making that the while-loop condition.

Co-authored-by: pi1024e <[email protected]>
2020-03-11 13:41:15 +01:00
Jay Krell 2746d98c1e [interp][32bit] Fix warning about precedence/parentheses. (#19184)
/s/mono2/mono/mini/interp/transform.c:3166:25: note: place parentheses around
      the '&&' expression to silence this warning
        if (mt >= MINT_TYPE_I1 && mt <= MINT_TYPE_I4
            ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
2020-03-11 10:20:40 +01:00
Jay Krell 60a192b98d [tools/sgen] Fix warnings about adding integers to string pointers. (#19180)
Kind of a silly warning.

mono/sgen/sgen-protocol-def.h:102:1: warning: adding 'unsigned long' to
      a string does not append to the string [-Wstring-plus-int]

BEGIN_PROTOCOL_ENTRY_HEAVY2 (binary_protocol_block_alloc, TYPE_POINTER, addr, TYPE_SIZE, size)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tools/sgen/sgen-grep-binprot.c:394:2: note: expanded from macro
      'BEGIN_PROTOCOL_ENTRY_HEAVY2'
        BEGIN_PROTOCOL_ENTRY2 (method,t1,f1,t2,f2)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tools/sgen/sgen-grep-binprot.c:291:26:
  note: BEGIN_PROTOCOL_ENTRY3 (binary_protocol_world_stopping,
  TYPE_INT, generation, TYPE_LONGLONG, timestamp, TYPE_POINTER, thread)exp

mono/sgen/sgen-protocol-def.h:102:1: note: use array indexing to
  silence this warning
2020-03-11 10:19:26 +01:00
Jay Krell 731b9cac54 [mini] Fix warning about unused variables. (#19182)
/s/mono2/mono/mini/mini-generic-sharing.c:1415:9: warning: unused variable
      'param_names' [-Wunused-variable]
        char **param_names;
2020-03-11 10:17:53 +01:00
Jay Krell 0ef53bae78 [wasm] Fix warnings about unused variables. (#19179)
/s/mono2/mono/metadata/sgen-mono.c:2883:16: warning: unused variable
      'last_major_gc_warned' [-Wunused-variable]
        static gint32 last_major_gc_warned = -1;
                      ^
/s/mono2/mono/metadata/sgen-mono.c:2884:16: warning: unused variable
      'num_degraded' [-Wunused-variable]
        static gint32 num_degraded = 0;
                      ^
/s/mono2/mono/metadata/sgen-mono.c:2886:9: warning: unused variable
      'major_gc_count' [-Wunused-variable]
        gint32 major_gc_count = mono_atomic_load_i32 (&mono_gc_stats.maj...
               ^
2020-03-11 10:13:36 +01:00
Jay Krell be9ced1a0f Fix warnings about misleading indentation. (#19186)
This is subtle and not actually misleading,
because it tends to be space tab or space tab, not wrong
number of tabs or significant wrong number of spaces.

/s/mono2/mono/metadata/class.c:4010:6: warning: misleading indentation;
      statement is not part of the previous 'if' [-Wmisleading-indentation]
                                 if (mono_class_implement_interface_slo...

/s/mono2/mono/metadata/class.c:4039:3: warning: misleading indentation;
      statement is not part of the previous 'if' [-Wmisleading-indentation]
        if (m_class_is_delegate (target) && mono_class_has_variant_gener...

/s/mono2/mono/mini/driver.c:1780:4: warning: misleading indentation; statement
      is not part of the previous 'if' [-Wmisleading-indentation]
                if (strncmp (argv [i], "--debugger-agent=", 17) == 0) {
2020-03-11 10:11:39 +01:00
Jay Krell fe5078961f [wasm] Fix warnings about unused variables. (#19181)
/s/mono2/mono/mini/mini-runtime.c:2284:15: warning: unused variable 'ji'
      [-Wunused-variable]
        MonoJitInfo *ji;
2020-03-11 10:08:41 +01:00
Larry Ewing 10cf3041d6 [wasm][packager] add option to pull from a different runtime directory (#19178) 2020-03-10 18:53:53 -05:00
Larry Ewing 488da04761 [wasm][standalone-benchmarks] Support mac (#19175)
This just handles systems (mac) without realpath installed and with a version of bash
that doesn't support &>>. It also trims path information from the benchmark name
2020-03-10 15:03:07 -05:00
monojenkinsandmdh1418 9ce7798c41 [mono] Include filename in Invalid image error messages (#19112)
Fixes dotnet/runtime#31649
Fixes dotnet/runtime#31650

**Testing**:
Reproduced Error via
`<Runtime Repo Root>/build.sh /p:RuntimeFlavor=mono`
Removing `[ActiveIssue("https://github.com/dotnet/runtime/issues/31650", TestRuntimes.Mono)]` in AssemblyTests.cs
Running `dotnet msbuild /t:RebuildAndTest /p:XunitMethodName=System.Reflection.Tests.AssemblyTests.LoadFile_NoSuchPath_ThrowsFileNotFoundException`

Fixing issue resulted in no errors.

Co-authored-by: mdh1418 <[email protected]>
2020-03-10 10:48:43 -04:00
monojenkinsandEgorBo 84bc62d113 [mono] Disable SIMD support for Windows target (#19172)
A temp fix for https://github.com/dotnet/runtime/issues/1933

Co-authored-by: EgorBo <[email protected]>
2020-03-10 14:45:55 +01:00
Jay Krell 5a065f628f [interp] Add EXCEPTION_CHECKPOINT to MINT_JMP consistent with other method tranform paths. (#19157) 2020-03-10 13:00:59 +01:00
Johan Lorensson fd61732c83 Convert PLT table and call site to execute only on AMD64. (#19114)
Current implementation embedded critical runtime information directly
into PLT slot. It also depends on finding call site in generic trampoline
reading call target from instruction stream in order to locate PLT slot
in use and then read GOT offset as well as PLT info offset from PLT slot
instruction stream.

This is problematic on platforms where code is execute only. Fix changes
how the metadata needed in order to correctly patch PLT is discovered.
Instead of depending on reading instruction stream, it is loaded into
RAX (free to be used when calling through PLT on AMD64) in PLT slot
before jmp takes place that moves control over to generic trampoline.

The PLT slot is the only place where we have access to both GOT index
(used in jmp), PLT info offset (currently embedded after jmp instruction)
and emitted PLT slot index. Since PLT slot index can be used to recover
GOT offset used by PLT slot as well as PLT info offset at runtime,
PLT slot index will be emitted into instruction stream and loaded into
RAX (prepared to be configurable to other reg if needed) before doing the
jump over to generic trampoline. Size of emitted imm constant is optimized
based on number of total PLT slots used in image, meaning that 1, 2, or 4
bytes could be used to store PLT slot index as an imm constant in
instruction stream. The additional jmp should have minimal to no
performance overhead since it should complete within 1 cycle, reading
imm constant from instruction stream that shouldn’t incur additional
cache misses and sine there is no data dependency between mov and jmp,
there should be options to pipeline both instructions. Since mov has
smaller latency than indirect jmp, mov should be complete once control
gets into the generic tramp (if pipelined).

In order to resolve plt info offset at runtime, needed information is
now emitted as part of got_info_offsets, increase table with 4 bytes/plt
slot, same size currently emitted into the PLT slot instruction stream.

Code size of PLT slot is currently 10 bytes (6 bytes jmp and 4 byte PLT
info offset). Since PLT info offset has been moved into got_info_offset
table, size of PLT slot will be 2, 4, 5 byte mov (depending on needed imm
size) and 6 bytes jmp instruction. As an example, mscorlib can emit all
its PLT slots using 2 byte imm constant, meaning that the PLT slot will
still be 10 bytes, but since PLT info offset of 4 bytes is moved into
got_info_offset, total image increase will be 4 bytes/PLT slot. This is
however still cheaper than alternatives that would burn 1 trampoline/PLT
slot (at least 10 additional bytes/slot) or setup lookup tables in image
or calculate more info at runtime, all-consuming more memory in total.

Note, using this approach is optional and runtime needs to be built using
MONO_ARCH_CODE_EXEC_ONLY in order to enable it since it’s only an opt in
features on platforms that can't read from instructions stream.

Current implementation is AMD64 only, but same pattern could be applied
to other architectures when needed.
2020-03-10 12:19:52 +01:00
monojenkinsandEgorBo 1ec2c9f7fd [Mono] Intrinsify Interlocked.And and Interlocked.Or using LLVM (#19135)
These APIs were introduced recently in https://github.com/dotnet/runtime/pull/33042 and already used in several places e.g. in `Task`, `InterlockedBitVector32`, `SafeHandle`, `RegexCharClass`

It should emit `%reg = atomicrmw and (or)` in LLVM IR. (and unlike `Interlocked.Add` these API return old value, see https://github.com/dotnet/runtime/pull/33102

Addresses https://github.com/dotnet/runtime/issues/32239 for Mono.

Co-authored-by: EgorBo <[email protected]>
2020-03-10 11:49:09 +03:00
monojenkinsandvargaz 5b644ae5a9 [jit] Implement support for all Sse1 intrinsics for netcore. (#19155)
* Add generic OP_XOP opcodes for opcodes which the JIT doesn't care about.
  Add a SimdOp enum to list the operations performed by these opcodes.
* Add a SimdIntrinsic struct so the mapping between the .net methods
  and the JIT opcodes can be specified declaratively.
* Add all intrinsics from the Sse class.

Co-authored-by: vargaz <[email protected]>
2020-03-09 22:48:05 -04:00
monojenkinsandpi1024e 8b447b4ccf No need to force to bool, when C does that automatically in the if statement. (#19095)
In C, the if statement checks if the value is 0 or not 0. !!, or forcing to 1 or 0, is unnecessary here.

Co-authored-by: pi1024e <[email protected]>
2020-03-09 20:28:36 +01:00
imhameed 90430277a1 [arm][ios] Follow branch islands when determining method entry addresses. (#19126)
Fixes https://github.com/mono/mono/issues/18612; `aot_code_low_addr` and
`aot_code_high_addr` were set to a range that included only first-level
branch islands, so `find_aot_module` would fail early when passed a
genuine function address.

This change only affects monotouch for now.

Why not use relative offsets stored in a read-only section? One reason
is that llvm-as cannot handle a `.long` directive containing a
subtraction expression involving externally-defined symbols. The AOT
method address table and AOT LLVM-generated code are currently emitted
in separate object files, so the method address table would include
expressions with undefined symbols.

Related: https://xamarin.github.io/bugzilla-archives/70/707/bug.html
2020-03-09 12:28:13 -07:00
Larry Ewing 9057498794 [wasm] Convert MonoString to string without utf8 intermediate (#19156)
* [wasm] Convert MonoString to string without utf8

Previously when converting strings from mono objects to js we used a utf8 intermediate buffer, this change builds the js string from the mono string without an intermediate.
2020-03-09 13:42:21 -05:00
Larry Ewing 66545a2abd [wasm] remove m_strdup (#19158) 2020-03-09 11:58:11 -04:00
Rares95andRares Sebastian Popa a25fe91245 Remove redundant mono_gc_collect during domain unload (#19130)
* Remove redundant mono_gc_collect during domain unload

Ifdef out mono_gc_collect when Boehm GC is present as it doesn't use remsets

* Fix trailing whitespace

Co-authored-by: Rares Sebastian Popa <[email protected]>
2020-03-09 16:47:45 +01:00
Aleksey Kliger (λgeek) 621a2219ef [corlib] Suppress finalization of underlying console FileStreams (#19137)
* [corlib] Suppress finalization of underlying console FileStreams

We already suppress the finalization of the CStreamReader and CStreamWriter,
but during shutdown, the underlying FileStream may be finalized which
invalidates its access field which can lead to a crash.

Addresses https://github.com/mono/mono/issues/19005
2020-03-09 10:01:16 -04:00
Marek Safar 1d575d8fe7 Bump linker (#19151) 2020-03-09 13:29:21 +01:00
dotnet-maestro[bot]anddotnet-maestro[bot] ebe382d49d Update dependencies from https://github.com/dotnet/arcade build 20200308.2 (#19163)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20158.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20158.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-09 13:27:40 +01:00
monojenkins 1713304e55 [bcl] Update BCL Linked Size 2020-03-09 07:14:17 +01:00
Marek Safar c6a3143c1e Freeze linker on mono-2020-02 branch 2020-03-08 10:35:42 +01:00
Zoltan Varga 8daffc3e4c Bump emscripten to 1.39.9. (#19147) 2020-03-07 23:16:18 -06:00
Zoltan Varga d083e2a828 [llvm] Move intrinsics definitions into a template header file llvm-intrinsics.h. (#19133)
* [llvm] Move intrinsics definitions into a template header file llvm-intrinsics.h.

* [llvm] Add a call_intrins () helper function.

* [llvm] Add globals for the SSE vector types to make code more compact.

* [llvm] Refactor the registration and lookup of intrinsics.

* Use the Intrinsics class from LLVM to register intrinsics.
* Identify intrinsics using an id instead of their name.

* Disable the wasm intrinsics on older llvm versions.

* Simplify mono_llvm_register_overloaded_intrinsic ().
2020-03-07 00:49:17 -05:00
monojenkinsandpi1024e 1690cadee2 Changed signedness of iterator to fix warnings. (#19014)
numOperands is an unsigned integer. In addition, getArgOperand has unsigned int arguments. As a result, I have no idea why i is an int, but to fix compiler warnings, I made it an unsigned int.

Co-authored-by: pi1024e <[email protected]>
2020-03-06 14:19:11 +01:00
Zoltan Varga be52633175 [jit] Use OP_EXTRACT_R4 in simd-intrinsics.c when using LLVM. (#19129)
* [jit] Use OP_EXTRACT_R4 in simd-intrinsics.c when using LLVM.

* [jit] Check for sse41 support before using the sse41.dpps intrinsic.
2020-03-06 07:32:47 -05:00
Larry Ewing 9ab10fd987 [wasm][debugger] Hash messages and sessions directly (#19128)
* [wasm][debugger] Hash messages and sessions directly
2020-03-05 16:07:19 -06:00
Larry Ewing f95d1413d3 [wasm][debugger] Check runtime state before attempting to resolve breakpoints (#19134)
Handle the case where the breakpoint request comes in very early before the runtime initialized by deferring resolution until normal runtime init.
2020-03-05 13:35:41 -06:00
Thays Grazia 93e7ef2b0f [debugger] NRE when 2 threads try to call GetThreads at same time (#19122)
As analysed by @jstedfast and me the only way to get this exception with this callstack below is 2 threads call GetThreads at same time, so while one of them is filling the items in the array, the other one is already trying to use the array with null items.

```
System.NullReferenceException: Object reference not set to an instance of an object
  at Mono.Debugging.Soft.SoftDebuggerSession.GetId (Mono.Debugger.Soft.ThreadMirror thread) [0x00000] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:3227
  at Mono.Debugging.Soft.SoftDebuggerSession.OnGetThreads (System.Int64 processId) [0x0002c] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:879
  at Mono.Debugging.Soft.SoftDebuggerSession.GetThread (Mono.Debugging.Client.ProcessInfo process, Mono.Debugger.Soft.ThreadMirror thread) [0x00008] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:902
  at Mono.Debugging.Soft.SoftDebuggerSession.HandleBreakEventSet (Mono.Debugger.Soft.Event[] es, System.Boolean dequeuing) [0x00501] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:2038
  at Mono.Debugging.Soft.SoftDebuggerSession.HandleEventSet (Mono.Debugger.Soft.EventSet es) [0x0005b] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:1687
  at Mono.Debugging.Soft.SoftDebuggerSession.EventHandler () [0x0005d] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:1625

```
 PR on mono/debugger-libs -> https://github.com/mono/debugger-libs/pull/300

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1070384/
2020-03-05 13:22:34 -03:00
Sam Patel 4c348d6567 [llvm] Don't set the LLVM JIT TLS variable in AOT. (#19127)
Should fix https://github.com/mono/mono/issues/19115. This bug was
introduced in https://github.com/mono/mono/pull/18824, which
conditionally initalized an LLVM JIT-related TLS slot while preserving
unconditonal stores to that same slot.

The value stored in this TLS slot should only be used during JIT
compilation, so set/reset it in `mono_llvm_compile_method` instead of
during IR construction in `mono_llvm_emit_method`. No non-JIT code
should depend on the existence or presence of this TLS variable, so
remove `mono_llvm_jit_set_tls_cfg`.
2020-03-05 08:29:01 -05:00
Aleksey Kliger (λgeek) a374dbb6f1 [merp] Add a test for Mono.Runtime.SendExceptionToTelemetry() (#18781)
Contributes to https://github.com/mono/mono/issues/18078
2020-03-04 21:25:57 -05:00
Larry Ewing a497dc7c5d [wasm][debugger] Rework breakpointRequest handling across sessions (#19113)
Try to follow CDP ordering constraints by not responding Debugger.enable until
we've fired scriptParsed events for any already loaded assemblies.

Send Debugger.scriptParsed asynchronously as we load assemblies instead of
waiting for them all to load first and fix some of the sequence point processing
to speed up debugger initialization time.

Rework breakpoint request handling by always passing the request to the browser
then storing both the request and the resultant id for use in when any new scripts
are loaded.

Pass any previous breakpoint requests to the new execution context on page reload.
2020-03-04 18:39:15 -06:00
Ryan LuciaandRyan Lucia a5f8905afa [loader] Clean up locking around adding assemblies to domain/ALC (#18967)
This cleans up three components of assembly loading that could potentially cause races/crashes.

First, both `add_assemblies_to_domain` and `add_assembly_to_alc` now ignore resolved references on netcore. We were seeing a crash on CI related to an assertion in here that resulted from two threads using the same MonoImage simultaneously as one is being loaded in with LoadFile, and this should solve that. Fixes dotnet/runtime#2305

Second, we take both the domain and ALC loaded assemblies locks simultaneously around the assembly being added to both lists, ensuring they stay in sync.

Finally, MonoImage now takes the image lock when writing to references instead of using the global `assemblies_mutex` in assembly.c for all images. This potentially helps reduce contention. Work here is not complete, but for this PR it's good enough. See dotnet/runtime#32889

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-04 17:09:08 -05:00
Stefan e1d3b7ba64 Minor formatting of the manuals (#19124) 2020-03-04 17:51:28 +01:00
Thays Grazia c40f1ba79f [dim] Class overriding interface method that was already override by another interface (#18962)
The problem was a parent class overriding an interface that was already override by another interface.
Refactor class-init.c
2020-03-04 12:48:25 -03:00
dotnet-maestro[bot] c1b392f02b Update dependencies from https://github.com/dotnet/arcade build 20200301.1 (#19093)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20151.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20151.1
2020-03-04 13:13:34 +01:00
Jay Krell 35cdbcb4a8 [cxx] Remove static initializers. (#19089)
In a current shipping Visual C++, 19.21.27702.2,
there is a bug involving const member data in structs, where for example:

static const struct a { // const on this line in either case
  const int b; // const on this line
} c = {1};

will have a static initializer, run usually reliably before main, but:

static const struct a { // const on this line in either case
  int b; // no const on this line
} c = {1};

will not.
The semantic difference is minuscule, so much so, that the first form
is relatively rare, and the bug shipped.

Just always use the second form.

The difference is that if you have a non-const a, b is still const, I guess.
If you have a const a, then no difference.

You can find these easily:

cd ...
for %a in (*.exe *.dll) do link /dump /disasm %a > %a.txt
findstr /c:"dynamic initializer" *.txt

Which could be automated and tested in CI.

This probably not an actual problem, but it is needlessly inefficient,
and potentially a problem. The timing/ordering of the initializers
is a bit not deterministic. You get writable data where you expect
read only.

It is in fact a compiler bug, fixed in later releases.
But it has been shipping for a while, I first noticed it over two years
ago, in code of mine that it caused to not work (a delayload implementation
which runs before main).
2020-03-04 12:54:13 +01:00
Aleksey Kliger (λgeek) 0bcc388ce8 [threads] Add a global "runtime startup is finished" flag (#18994)
This contributes to https://github.com/mono/mono/issues/18916
(to make coop locking not bother with state transitions if the runtime isn't
ready yet).
2020-03-03 18:29:04 -05:00
Steve Pfister c483eaf070 Changed Max and MinLengthAttribute validation (#19084)
It should now validate against ICollection or any other type that has a Count property for that matter.

We're still relying on the reference source implementation because corefx strips some types out.

Fixes https://github.com/mono/mono/issues/19022
2020-03-03 15:26:46 -05:00
monojenkinsandEgor Bogatov ede05b997b [Mono] Fix Sse2.ConvertScalarToVector128 and add signed overloads (#19108)
@vargaz noticed I used an incorrect opcode for ConvertScalarToVector128*
I will add a sort of fuzzy tests later (which use all kind of SRI APIs and compare results with CoreCLR) something like this: https://github.com/mono/mono/blob/master/netcore/tests/HwIntrinsics/Program.cs
Also, add signed version of ConvertScalarToVector128*.

Co-authored-by: Egor Bogatov <[email protected]>
2020-03-03 23:14:59 +03:00
Alexis Christoforides 0ec7957d7c [merp] Create a 'reason' breadcrumb for the crash dump process (#19072)
For example, the system name of a signal ('segv') or 'MANAGED_EXCEPTION' are typical arguments.
2020-03-03 11:47:19 -05:00
Johan Lorensson b5a9fe49e1 Fix Mono Tool MSVC Debug build. (#19075)
Changes done in object-offsets.h adding offsets for TARGET_AMD64 exposed
a build configuration error in the Mono Tool MSVC debug build due to
incorrectly setting both X86 and AMD64 defines causing compile errors
when including object-offset.h. Release configuration didn't have this
configuration error, meaning that CI builds wouldn't trigger build error.

Fix makes sure all MSVC build configurations for Mono Tools
(genmdesc, monodis, pedump) matches the build configuration for other
binaries and that debug build configuration is inline with release build
configuration.
2020-03-03 16:11:30 +01:00
Zoltan Varga 4622ad0635 [WIP] [aot] Add support for emitting the method address table as a table of normal pointers in the data segment. (#19086)
* [aot] Rename acfg->opts to acfg->jit_opts.

* [aot] Add support for emitting the method address table as a table of normal pointers in the data segment.
2020-03-03 07:43:11 -05:00
Zoltan Varga b91580bafa [amd64] Obtain the trampoline argument in the generic_trampoline in the AOT case without decoding the call instruction. (#19091)
* [amd64] Obtain the trampoline argument in the generic_trampoline in the AOT case without decoding the call instruction.

* [amd64] Unify the aot specific trampolines so they have the same size/layout, so its easier to compute the address before the call.
2020-03-02 20:16:53 -05:00
Ankit Jain 3aab3884f6 [master] Bump msbuild to track xplat-master (#19077) 2020-03-02 17:49:08 -05:00
monojenkinsandEgor Bogatov 0f0c284a41 [Mono] Implement missing Sse2 intrinsics (#19101)
For now we only implement intrinsics used by corlib internally to optimize code.

Fixes https://github.com/dotnet/runtime/issues/32510 crash.

`ConvertScalarToVector128UInt32/64` is expanded into LLVM's `insertelement`
`ConvertToUInt32/64` is expanded into LLVM's `extractelement`

Co-authored-by: Egor Bogatov <[email protected]>
2020-03-03 01:09:46 +03:00
monojenkinsandRyan Lucia 0e38adf329 Use GArray instead of GList when getting custom attributes from an image (#19083)
Xamarin.Forms calls this _constantly_ during startup, to the point where the malloc calls were actually showing up in the profile. This should help with that, and gives us some ms back during startup.

There are also a bunch of calls that return empty arrays, which in theory could be created in managed for a perf boost. However, it seems the Xamarin.Android team has mostly gotten rid of those calls and that optimization would complicate the code and require changes to our managed CustomAttribute implemented for both legacy and netcore, so I've opted to not check it in for now. If the empty calls continue to show up down the line, I can dig the code back up from my local branch and PR it separately.

If this is producing huge arrays and malloc continues to show up in the profile, consider https://github.com/dotnet/runtime/issues/32972. Even without that, this is a substantial improvement.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 16:46:09 -05:00
Zoltan Varga 89e8a484e1 [wasm] Allow SIMD only in netcore mode. (#19087) 2020-03-02 12:56:43 -05:00
monojenkinsandRyan Lucia f7e1a23dd8 [loader] Free some leaked strings in NativeLibrary resolution (#19085)
Oops.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 17:46:09 +01:00
monojenkinsandRyan Lucia 80ff8817d0 [threading] Switch domain assemblies lock from OS to Coop (#18918)
I don't think you can create a root domain without reaching the thread info attach call, so this should be safe to change outright.

Fixes dotnet/runtime#32177

We should complete https://github.com/mono/mono/issues/18915 and then convert over other locks as needed, but this fixes the immediate CI hang.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 17:45:57 +01:00
Zoltan Varga 89902e8d22 [wasm] Pass --exclude-feature deserialization to the linker. (#19031)
* [wasm] Pass --exclude-feature deserialization to the linker.

* Add a --linker-exclude-deserialization packager option.
2020-02-28 23:41:03 -05:00
Alexander Köplinger 1a6eb5a597 Update mono-extensions 2020-02-28 19:58:22 +01:00
Jay Krell ec9db26150 Remove non-functional spawn code.
Subject to configuration, spawn has three widely divergent behaviors:

succeed but do nothing
print and exit
do what it sounds like
Rather than mislead the callers, provide only option 3 and let the caller decide what to do otherwise.
2020-02-28 13:19:55 -05:00
Maxim Lipninandmonojenkins 68470ddfb0 [wasm] Use PNSE version of System.Net.NetworkInformation.NetworkInterface (#19074)
PNSE coverage of wasm bcl has shown that only three public members of `System.Net.NetworkInformation.NetworkInterface` class throw `PlatformNotSupportedException`.

Now all public members throw PNSE on WASM.

Relates to https://github.com/mono/mono/issues/18735


Co-authored-by: monojenkins <[email protected]>
2020-02-28 19:17:40 +01:00
Alexander Köplinger 716b73221e Update mono-extensions 2020-02-28 19:05:43 +01:00
Alexander Köplinger 00b19731c6 Bump mono-extensions 2020-02-28 15:25:03 +01:00
monojenkinsandRyan Lucia 8951940260 [loader] Remove unnecessary newline in loader mono_trace call (#19069)
Co-authored-by: Ryan Lucia <[email protected]>
2020-02-28 11:37:18 +01:00
Vlad Brezae 941a335ea0 [runtime] Use pointer size gchandles (#19039)
* [runtime] Use pointer size gchandles

This would enable in the future to use CoreCLR's GC with mono. This doesn't change the current embedding api, but, in the future, we will end up aligning with coreclr on an embedding api that uses pointer sized gchandles.

* [gc] Change format of gchandles to be netcore compatible

On netcore, the GCHandle class assumes the lowest bit to be usable, for perf improvements. Move the responsability to have this bit available down to the GC implementation, since for CoreCLR GC we shouldn't do any shifting above the gc interface.
2020-02-28 09:05:55 +02:00
Larry Ewing d11eeadb66 [wasm][debugger] use console.debug to signal runtime init (#19057)
* [wasm][debug] use console.trace instead to init runtime

Using a `debugger` breakpoint in Mono.mono_wasm_runtime_ready has
unfortunate side effects, this changes the logic to use console.trace
and checks the stackTrace when the event is arrives.

* add a console.trace stub to runtime-test.js

* Use console.debug with a magic number instead
2020-02-27 12:07:55 -06:00
Johan Lorensson 188ac1b0a7 Clean up libmono-dynamic.vcxproj dependency for netcore. (#19040) 2020-02-27 18:08:34 +01:00
Larry Ewing 76f5086687 [wasm][debugger] allow stepping out of c# (#19059) 2020-02-27 07:50:59 -06:00
Johan Lorensson d2e6bd2aff Detect build config mismatch for MSVC Windows LLVM build. (#19044)
If LLVM build is done using different build configuration, Mono build
will fail to static link LLVM libraries due to incompatibility between
release and debug C++ runtime library, generating a lot of linker errors
like this:

error LNK2038: mismatch detected for 'RuntimeLibrary':
value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug'
in mini-llvm-cpp.obj

error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL':
value '0' doesn't match value '2' in mini-llvm-cpp.obj

This is normally not a problem when doing local builds of LLVM but
when using an external provided build, like netcore Mono build,
this problem is more likely to occur.

This fix adds an additional check to make sure build configuration reported
by llvm-config.exe --build-mode matches current Mono build configuration
and if not give a more specific msbuild error:

error : LLVM build Configuration=Release doesn't match current Mono
build Configuration=Debug
2020-02-27 13:28:07 +01:00
Nikita Voronchev 4b42804803 [WinForms] Colorize PropertyGrid's Categories properly (#19045)
Assume, one have colorized `PropertyGrid propertyGrid` control:
```
propertyGrid.CategoryForeColor = Color.Red;
propertyGrid.LineColor = Color.Blue;
```

Before changes proposed by the current PR we get the following image:
![before](https://user-images.githubusercontent.com/12382656/75364102-3391fe80-58cc-11ea-9136-347bcd567731.png)
After:
![after](https://user-images.githubusercontent.com/12382656/75364115-3987df80-58cc-11ea-95d1-20691a4a8467.png)
(item labeled "_Child 2_" is selected by user)
2020-02-27 11:25:14 +01:00
imhameed 5e5b1ed425 [aot] Avoid mono_defaults when creating trampolines. (#19055)
Fixes https://github.com/mono/mono/issues/19009.

https://github.com/mono/mono/pull/18505 made PLT initialization occur at
image load time. When initializing the PLT for mscorlib,
`mono_defaults.corlib` is not yet initialized.

This change works around this by using `mscorlib_aot_module`--which is
initialized relatively early--instead of `mono_defaults.corlib`. PLT
initialization is also moved slightly later: it now happens after
`use_page_trampolines` has been initialized. The `USE_PAGE_TRAMPOLINES`
macro has also been changed to dereference `mscorlib_aot_module`.

This commit also removes an unused parameter from
`mono_aot_create_specific_trampoline`.
2020-02-27 08:56:59 +01:00
Larry Ewing 7f6a9bb783 [wasm][debugger] Always pass failed breakpoint requests to target (#19053)
* [wasm][debugger] Always pass failed breakpoint requests to target

Originally we had a simple way to detect that breakpoints were
meant for the runtime but that is no longer true.  Now if we
can't location a breakpoint we need to pass the request on to the
target so it can make an attempt.

* Fix test byUrl is a logical breakpoint and should not fail
2020-02-26 23:07:57 -06:00
Ankit Jain de54af770c [wasm][debugger] Show the typename for valuetypes (#19046)
- no details of the valuetype are accessible

Fixes #18232
2020-02-26 20:32:48 -06:00
Larry Ewing 9793a1e5be [wasm][debugger] clean up scriptid parsing (#19052) 2020-02-26 20:26:56 -06:00
Larry Ewing ac5240640c [wasm][debugger] Since System.Url normalizes the path sep also try to match on the doc.Url (#19051)
Match against the pdb document url as well as the Url we calculated in case the urls have been
processed incompatibly
2020-02-26 17:39:32 -06:00
Steve Sanderson 3b0e35d1bc Debug proxy updates (#19026)
Currently, the Mono WebAssembly debug proxy logic is treated as "shared source" with ASP.NET Core. Each time we update by copying the sources from this repo into the ASP.NET Core repo, we need to make some updates to avoid problems. The intention of this PR is to make the updates here at source, so we can simply copy the sources as-is and don't need to update them each time.

The changes are:

Making everything internal, since this is not intended as public API surface for ASP.NET Core developers. It's an internal implementation detail of the framework.
To retain convenience for the ProxyDriver and Mono test code, this PR also adds a simple public entrypoint called DebuggerProxy. This exposes nothing except the ability to connect a debugger proxy instance to a websocket. In the ASP.NET Core code, we'll simply omit this file since it's not needed, but even if we did get it in the future when the proxy feature is published as a package, it would still be fine since it doesn't expose any public API that would be expensive to support and maintain.
Replacing all the Console.WriteLine calls with use of ILogger from the standard Microsoft.Extensions.Logging package, as used by most .NET Core-based application frameworks including ASP.NET Core.
This allows external code to control what happens with log output, what severity levels are filtered in/out, etc. In ProxyDriver and Mono test code, I've configured it to behave just like before: it logs to console, and doesn't filter out anything.
This has the benefit that various bits of commented-out logging code no longer need to be commented out. Instead, when calling the logger.Log* methods, each call site can specify a severity level and then (for example) the filter level could be set to Debug so that all Trace items are filtered out by default.
2020-02-26 12:52:36 -06:00
Zoltan Varga 6c5a18ba50 [runtime] Remove MONO_LLVM_INTERNAL. (#19032) 2020-02-26 17:48:29 +01:00
Ankit Jain 546833b534 Merge pull request #19035 from radical/async-local-fixes
[wasm][debugger] Fixes to get async method locals
2020-02-26 11:18:28 -05:00
monojenkinsandJohan Lorensson 907d59798c Fix Mono netcore preload hook to work with Windows CoreRun.exe. (#19027)
Current hook compares TPA list file names using case sensitive compare. On Windows, the TPA list, when passed from CoreRun.exe makes all file paths lower case, meaning that our checks fails to find assemblies in TPA list.

Since CoreRun.exe is used for both CoreCLR and Mono, removing the tolower on file paths added to the TPA list in CoreRun.exe could be an option, but that might introduce regressions in CoreCLR, so this fix isolate the change to Mono's netcore preload hook.

Co-authored-by: Johan Lorensson <[email protected]>
2020-02-26 12:06:40 +01:00
Zoltan Varga 91e026de9d [wasm] Improve error handling in the packager. (#19037) 2020-02-25 23:56:18 -05:00
Katelyn Gadd 5c8da9e284 Add basic set of standalone benchmarks for wasm (#18977)
Adds a basic set of standalone benchmarks from multiple sources, along with simple infrastructure for running them in mono, netcore and node.js wasm. A couple of them also have original JS versions you can run from a JS shell.
2020-02-25 18:24:10 -08:00
monojenkinsandpi1024e 3cba804069 Braces around subobject of initialization warning fix (#18866)
When compiling for Mono, Apple's compiler gives a warning saying to put braces around the 0 as it is a subobject. To silence the compiler, I put braces around the 0. Simple fix.

Co-authored-by: pi1024e <[email protected]>
2020-02-25 16:42:42 -08:00
Jay Krell 79e6ce6423 Change macro CLOSE_PIPE to function mono_close_pipe. (#19030)
Extracted from https://github.com/mono/mono/pull/19025
2020-02-25 19:54:38 +01:00
Jay Krell 1bd3b7a614 Fix warnings: GPid vs. pid_t function signature mixup in glib test code. (#19028)
Extracted from https://github.com/mono/mono/pull/19025
2020-02-25 19:46:10 +01:00
Ankit Jain 5c0b1e1c83 [wasm][debugger] Fix getting async method locals, and this
- async local field names don't start with `<`, surface those
- `this` for the actual async method is in a field named like
`<>4__this`, surface it as the `this` object
2020-02-25 13:36:38 -05:00
Ankit Jain f6bef135d5 [wasm][debugger][tests] Update tests to track test api changes 2020-02-25 11:38:37 -05:00
Ankit Jain c2f8fd8bc7 [wasm][debugger][tests] Ensure to always check location when sending
.. commands.
2020-02-25 10:24:50 -05:00
Ankit Jain 5f85de2b56 [wasm][debugger][tests] extract method to set breakpoints 2020-02-25 10:23:39 -05:00
716718a4bc Bump linker. (#19006)
* Bump linker.

* Bump linker tools framework version

* [csproj] Update project files

Co-authored-by: Marek Safar <[email protected]>
Co-authored-by: monojenkins <[email protected]>
2020-02-25 12:48:18 +01:00
Matthew Baird 6b3a7986ef Ensure ListView ItemCheck event can update the checked state (#18926) (#18937)
Fixes #18926
2020-02-25 10:19:01 +01:00
Vlad Brezae 49deb4bcea [interp] Enable more interp tests (#19012) 2020-02-25 09:12:01 +02:00
Zoltan Varga 8da9361ab2 [llvm] Remove loaded llvm support, the startup perf penalty doesn't seem to exist any more. (#18868)
* [llvm] Remove loaded llvm support, the startup perf penalty doesn't seem to exist any more.

* Fix the windows build.

* Remove more mini-llvm-loaded.c references.

* Remove more --enable-loadedllvm references.

* Make --enable-loadedllvm the same as --enable-llvm.

* Fix run-jenkins.sh.

* Address review comments.
2020-02-24 20:35:00 -05:00
Larry Ewing ed012b6e89 [wasm][debugger] revert DotNetUrl changes (#19021) 2020-02-24 18:31:52 -06:00
Jay Krell 9fa3eb26e1 [interp] Reorder InterpMethod to reduce padding (#18982)
for alignment. Sizes goes from 192 to 184.
2020-02-25 01:30:26 +01:00
Jay Krell bb40a1a2d9 Whittle away at warnings: (#18874)
Return address of local (deliberate).
 Pointer cast to different size.
 Function parameter and declaration badly mismatch (pid_t vs. void*).
 Unused locals.
 Top level volatile is ignored.
 Negative shift undefined.
 Call to undeclared function (getpid).

Some are fixed, some are disabled temporarily.
2020-02-24 18:41:06 -05:00
Larry Ewing fafa41f448 Use AsLocation when creating location (#19016) 2020-02-24 14:55:48 -06:00
monojenkinsandpi1024e 655dd296bf Fix compilation warning for converting negative integer constant to unsigned. (#19013)
When compiling, Visual Studio gives two compiler warnings in the sysmath.c file about FPILOGB0/-2147483648 being converted into an unsigned integer.

The standard library math.h states that FPILOGB0 be INT_MIN, so I replaced -2147483648 with the INT_MIN macro so the behavior is much more clear as well as stifling the compiler warnings.

Changes tested manually and automatically.

Co-authored-by: pi1024e <[email protected]>
2020-02-24 13:51:14 -05:00
Ankit Jain 27c5660659 [wasm][debugger] Don't mark the runtime ready before clearing the (#19015)
.. breakpoints. Chrome tries set the breakpoints from previous sessions
when the debugger starts up. And we set that in mono, if the runtime is
ready.

But in `MonoProxy.RuntimeReady`, we mark the runtime ready *before*
clearing the breakpoints. So, we end up setting the breakpoints, as
asked by chrome debugger, but then subsequently clearing them!

This manifests as previously set breakpoints in managed code not getting
hit, but if we reset the breakpoint manually then it gets hit.
2020-02-24 13:50:10 -05:00
Ankit Jain ace2e34aae Merge pull request #19003 from radical/improve-locals-access
[wasm][debugger] Improve accessing locals
2020-02-24 13:49:45 -05:00
Alexis Christoforides 1dffbd0200 [merp] Produce hashes for unmanaged thread stacks also (#18943)
Before, this was only done for managed stacks.

Addresses https://github.com/mono/mono/issues/18941
2020-02-24 13:02:17 -05:00
Alexis Christoforides 981da38038 [merp] Increase buffer size for state dump (#18798)
Addresses https://github.com/mono/mono/issues/18796
2020-02-24 10:21:34 -05:00
Ankit Jain 881dfd770f [master] Bump msbuild to track xplat-master (#18987) 2020-02-24 15:53:08 +01:00
dotnet-maestro[bot] 9f2297bcfd Update dependencies from https://github.com/dotnet/arcade build 20200223.1 (#19011)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20123.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20123.1
2020-02-24 15:50:09 +01:00
Jay Krell 9fb0820ca5 [interp] Use alloca for InterpFrame. (#19007)
This is like https://github.com/mono/mono/pull/19002 but next_free is initialized upon allocation instead of `pop_frame` which is a slight optimization.
2020-02-24 10:30:46 +01:00
Jay Krell ae1c7c010c [interp] Decruft PowerPC. (#18981) 2020-02-24 09:12:56 +01:00
monojenkins e4cf488054 [bcl] Update BCL Linked Size 2020-02-24 08:24:19 +01:00
Jay Krell 5941141433 [interp] Remove some unused paths in newobj and cleanup a little. (#18998) 2020-02-23 15:15:14 +02:00
Jay Krell e9886db7e4 [interp] Pop leave and calli_nat frames. (#19000) 2020-02-23 15:11:14 +02:00
Ankit Jain 9049967106 [wasm][debugger] mono_wasm_get_var_info - don't repeatedly walk the
.. stack, instead find the target frame, and read in one go.
2020-02-22 18:07:06 -05:00
Ankit Jain 01e089bc46 [wasm][debugger] Fix nested class names for this 2020-02-22 17:58:37 -05:00
Ankit Jain 547e4180df [wasm][debugger] Fix issue in getting locals for previous frames
`mono_wasm_get_var_info` didn't correctly reset the `data.cur_frame`
required for `mono_walk_stack_with_ctx`, as the stack is walked
repeatedly to get the variables.
2020-02-22 17:49:56 -05:00
Zoltan Varga cc8a5b5513 [corlib] Link out more SR/SRE types (#18770)
* [corlib] Invoke TypeBuilderInstantiation.MakeGenericType through a delegate created in the TypeBuilder ctor to break a linker dependency between RuntimeType and TypeBuilderInstantiation.

* [corlib] Update linker descriptor.

* UnmanagedMarshal is not used from native code at all.
* AssemblyName is not used from native code at all.
2020-02-22 17:34:00 -05:00
monojenkins 008a10c43a [bcl] Update BCL Linked Size 2020-02-22 22:48:39 +01:00
Thays Grazia a8d34b2a03 [wasm][debugger] Enable embedded ppdb on wasm (#18966)
* Enabling PPDB on Wasm.
2020-02-22 00:35:59 -06:00
Ankit Jain d02bbe4c32 [wasm][debugger] Handle exceptions from JS
- correctly mark `Result` with the error
2020-02-21 19:43:11 -05:00
Ankit Jain cbc722ddaf [wasm][debugger] SetException on all the inspector waiters for uncaught exceptions 2020-02-21 19:43:07 -05:00
Fan Yang 041fbe0900 Revert linker change of 5a65a97 (#18991)
This is needed because  the required='false' field in
```
<type fullname="System.Reflection.Emit.TypeBuilder" required='false' preserve="fields" feature="sre">
			<method name="SetCharSet" feature="sre" />
			<!-- reflection.c mono_reflection_call_is_assignable_to () -->
			<method name="IsAssignableTo" feature="sre" />
```
Fails to preserve the IsAssignableTo method.
2020-02-22 01:27:09 +01:00
Aleksey Kliger (λgeek) 7707eda633 [metadata] Mark mono_signature_get_return_type external only (#18990)
Also mark mono_signature_get_params external only.

Runtime should use mono_signature_get_return_type_internal
and mono_signature_get_params_internal
2020-02-21 17:35:55 -05:00
Larry Ewing dce53cc9f3 [wasm][debugger] Store more per session state and force some event ordering (#18747)
* [wasm][debugger] Store more per session state and force some event ordering

Register a DebugStore and ExecutionContext per session and force some commands
to block on source loading so that scripts are loaded before attempting to use
them. This should populate the sources in vscode-js-debug properly.
2020-02-21 15:06:45 -06:00
monojenkinsandNathan Ricci 840ba017ef Added some parenthesis and a cast to control order of operations. (#18972)
This fixes https://github.com/mono/mono/issues/17140; intermittent incorrect results from GetTotalBytesAllocated for current thread. uint64_t and a pointer difference are not the same type/width on all architectures, so incorrect results can happen with out the parenthesis.

Co-authored-by: Nathan Ricci <[email protected]>
2020-02-21 14:17:39 -05:00
monojenkinsandNathan Ricci ac7a77f245 Correct values for HighMemoryLoadThresholdBytes, MemoryLoadBytes. (#18913)
Correct values fort HighMemoryLoadThresholdBytes, MemoryLoadBytes. I looked in the code for CoreCLR and spoke with Maoni Stephens and Jan Kotas about the values used in CoreCLR, so I am confident they are all correct now.

HighMemoryThresholdBytes is a system wide memory usage threshold at which CoreCLR changes its compaction strategy. There is no direct correspondance to anything in mono-sgen. But since it used by software caches to determine when memory in the system is "nearly" exhauste, mono-sgent will use .9 * total physical memory as an acceptable value.

Co-authored-by: Nathan Ricci <[email protected]>
2020-02-21 19:36:20 +01:00
Larry Ewing a3e03430ea [wasm][tests] clean up a couple of the Contains assertions 2020-02-21 10:52:41 -06:00
Aleksey Kliger (λgeek) ed017481e8 [monodis] Escape names that start with a digit (#18759)
Code like this:

```
using System;
class Program
{
    static byte[] bla = new byte[] {1,2,3,4,5};
    static void Main()
    {
    }
}
```

Compiles to a private field declaration for the array with a compiler generated
name, that name should be quoted if it starts with a digit.  For example:

```
	IL_0007:  ldtoken field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=5' '<PrivateImplementationDetails>'::'74F81FE167D99B4CB41D6D0CCDA82278CAEE9F3E2F25D5E5A3936FF3DCEC60D0'
```

Fixes https://github.com/mono/mono/issues/18750
2020-02-21 11:43:27 -05:00
Cody RussellandAlexander Köplinger 7a3c321b42 Bump bockbuild for Pango nil check fix. (#18936)
https://github.com/mono/bockbuild/pull/153

Co-authored-by: Alexander Köplinger <[email protected]>
2020-02-21 17:24:43 +01:00
monojenkinsandBernhard Urban-Forster 802d57a741 [mono/metadata] add definitions for ENCLog/ENCMap (#18976)
Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-21 14:42:33 +01:00
Jay Krell f4d5c11610 [interp] Cleanup: Replace retval-> with result. and retval with &result. (#18951) 2020-02-21 11:24:38 +01:00
Jay Krell 3e3499e89a [interp] Allow historical inlining due tO: (#18948)
1. This wasn't generally conserving the important stack.
2. Larger mechanisms are in place now.

Granted, tuning out rare paths could still be valuable.
2020-02-21 10:43:40 +02:00
Jay Krell 40187209f8 [interp] Non-recursive newobj. (#18931)
A few weird here, esp. for string, but CI passes.
Acknowledged but skipped for now, a request to also refactor the opcode -- it doesn't seem like much throwaway work to go in either order, even if the code is dead (which I don't know).

Also amend recent nonrecursive PR to simulate stack +2 instead of +1.
2020-02-20 18:19:27 -05:00
Ankit Jain 91b125af92 [wasm] Improve locals visualization (#18919)
* [wasm] Improve locals visualization in the debugger

- improves handling of generic types, null objects, delegates
- improves getting the name for nested types, arrays, generic types

top todos from this:
- Remove arity in "foo`2<int, int>" from the type name
- Convert nested type names from `foo/bar` to `foo.bar`.

* [wasm] Fix scoping for `mono_wasm_add_null_var`

- which caused runtime uncaught exceptions

* [wasm] Fix possible lost exception in GetScopeProperties

- An exception, eg., because of an invalid scope id just ends up
uncaught instead of showing up as a failed Result.

* [wasm] Address review feedback

- Replace `stringify_type` with `mono_type_full_name` which does exactly
the same thing!

* [wasm][debugger] Fixup nested class, and generic type names

```
	// Fix up generic names like Foo`2<int, string> to Foo<int, string>
	// and nested class names like Foo/Bar to Foo.Bar
```
2020-02-20 14:23:02 -05:00
sharkcz 74f90c824b Add struct offset definitions to s390x (#18958)
* [mini-s390x] standardize on using MONO_STRUCT_OFFSET

* [mono-s390x] add struct offset definitions to s390x

Add s390x specific struct offset definitions to object-offsets.h and
update the array member accesses in mini-s390x.c.

Related: https://github.com/mono/mono/commit/79135c6ea57b516addf187a08cb01826bb0197c6
Fixes: https://github.com/mono/mono/issues/18930
2020-02-20 15:05:26 +01:00
Thays Grazia dd5926cc79 [debugger] Enable reading embedded ppdb (#18895)
To fix the issue of enabling embedded pdb everywhere the problem was that when the roslyn tests are compiling sources it creates embedded pdb with source file "", and before enable reading the embedded pdb we don't read this information, and now we read and our StackTrace file written in our corlib doesn't expect a source file "", it's okay if it's NULL but not "".
I could fix in corlib or in mono, I prefer to fix on mono because I don't know if there is any other place that doesn't expect source file "".
2020-02-20 09:59:40 -03:00
Egor Bogatov e441aa4d21 Remove TestEnvVarSwitchForInnerHttpHandler test as it should be in XA/LinkerTests.cs instead (#18952)
The test will be moved to XA's `src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs` 

It only works if 
```xml
<AndroidHttpClientHandlerType>System.Net.Http.MonoWebRequestHandler, System.Net.Http</AndroidHttpClientHandlerType>
```
is set in `Xamarin.Android.Bcl-Tests.csproj` and we don't want it (other tests expect SocketsHttpHandler to be default one). As an alternative we can add it to the `Xamarin.Android.Bcl-Tests.csproj`'s Linker exceptions but `/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs` is a better place for it.
2020-02-20 11:10:58 +01:00
Jay Krell 90c66725d3 Remove LOCK_DECLARE and LOCK_INIT -- unused and incorrect comments (not idempotent). (#18945) 2020-02-20 10:51:29 +01:00
monojenkinsandBernhard Urban-Forster 1d92b01cf4 [metadata] use descriptive constants for method table columns (#18944)
Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-20 09:30:38 +01:00
Jay Krell a03dee09f9 [interp] inline newobj. (#18903)
The real goal is to make it not recursive.
This is just a step along the way.
Note that the actual not-recursive change has *not* yet been developed,
so this *might* amount to nothing useful.
2020-02-19 23:32:15 +01:00
Vlad Brezae 5d87716ea4 [sgen] Disable managed allocator when using nursery-canaries (#18932)
The managed allocator included in a full aot image does not support canaries, leading to inconsistent object layout.

Fixes https://github.com/mono/mono/issues/18925
2020-02-20 00:10:49 +02:00
Vlad Brezae 47687bdb47 [interp] Fix incorrect propagation of indirect local (#18924)
We never don't track the value of locals who have their address taken in a method, since their value can change without the pass detecting it, leading to propagation of incorrect values. We failed to check for this when doing a stloc.np optimization.

Fixes System.Reflection.Metadata.Tests.ImmutableByteArrayInteropTest.DangerousCreateFromUnderlyingArray

Enable more interp tests, which don't seem to fail on my machine.
2020-02-20 00:09:16 +02:00
monojenkinsandRyan Lucia 7d38e12a1e [loader] Add check to mono_domain_ensure_entry_assembly for AOT (#18938)
Fixes a crash with AOT because domain->setup is NULL here

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-19 22:48:50 +01:00
Zoltan Varga 5fe94fd99f [wasm] Avoid spilling refs to the stack in gsharedvt wrappers, those functions don't need safepoints. (#18820) 2020-02-19 14:30:08 -05:00
Mitchell Hwang 30c1012ed5 [bcl] Default XmlSerializer stream serialize to UTF8 Encoding (#18836)
* [mcs] Default XmlSerializer stream serialize to UTF8 Encoding

* [bcl] Add XmlSerialize Test to preserve UTF chars in stream
2020-02-19 14:11:07 -05:00
monojenkinsandRyan Lucia 45cb6fbba6 [loader] Parse in more netcore hosting properties (#18850)
This includes some other minor cleanup to the netcore hosting functions.

Setting `assemblies_path` with `APP_PATHS` despite it also being set by `MONO_PATH` doesn't scare me too much because:
1) Anyone using `MONO_PATH` is probably not going through the hosting API and is using `mono-sgen` directly
2) The default host doesn't set `APP_PATHS` (and we've worked fine without it being set at all up until now
3) This is a temporary measure; we will have to refactor our preload hook when we want to properly support the other loader properties anyway

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-19 14:10:33 -05:00
Zoltan Varga b4337dd4e8 [llvm] Remove the disable-self-init debug option, it doesn't work any more. (#18929) 2020-02-19 11:43:20 -05:00
Egor Bogatov 6ef5dd4523 Move HttpClientHandlerTests.Android.cs to nunit (#18883) 2020-02-19 16:34:37 +03:00
Alexander Köplinger 885226b1f0 [netcore] Disable ARM64 CI runs (#18922)
They were moved over to dotnet/runtime: https://github.com/dotnet/runtime/pull/32303
2020-02-19 02:36:38 +01:00
Jay Krell d1603aaca1 Remove some #if __cplusplus wrt inlining. (#18891)
The code is fine, and we should put it back at some point,
but I want things to be clearer regarding https://github.com/mono/mono/issues/18827.

i.e. That this does not cause it.
I don't know what does cause it, granted.
Maybe a preexisting race condition/deadlock, I don't know.

Repro is run a loop like:
```
type \s\1.cmd
for /l %%a in (1 1 999) do echo %%a && C:\s\mono2\msvc\build\sgen\x64\bin\Release\mono-sgen.exe \s\mono2\mono\tests\bug-10127.exe
```
2020-02-19 01:26:46 +01:00
Jay Krell 6d0895ad3b Make MONO_API_DATA clearer. (#18897)
Make MONO_API_DATA clearer, its old form -- MONO_API extern.
extern "C" { } should be added around it not within it, for clarity.
I've always found extern "C" without braces unclear, esp. for data.

i.e.

```
 extern "C" {
1   extern void function(); // redundant but clear
                            // same as void function();
2   extern int data; // not redundant, and clear
}
```

vs.
```
3   extern "C" void function(); // I guess clear.
4   extern "C" int data; // unclear -- is it extern or extern "C" or both?
```

This PR in particular turns 4 into 2.
2 is clearly a declaration, not a definition.
4 is unclear as to if it is a declaration or definition.

I was looking into https://github.com/mono/mono/issues/18827.

This does not actually change anything but makes things clearer.

This is similar/related in spirit to https://github.com/mono/mono/pull/18891.
It should not make a difference.
In this case, `#if __cplusplus` remains, but the meaning of it is clearer.
It becomes *only* about the common `extern "C" { }` and not even the less clear `extern "C"`.
2020-02-19 01:20:07 +01:00
Jay Krell 90f4013483 [interp] Make newobj_vt[st]_fast non-recursive. (#18902)
* [interp] Make newobj_vt_fast non-recursive.

* PR feedback: Attempt to provide similer primitive increase_max_stack_height
that is not so roundabout as move_stack (n) + move_stack (-n).

* [interp] Make newobj_vtst_fast not recursive.

This is builds upon https://github.com/mono/mono/pull/18875
and is easiest reviewed after it is merged.

Or, perhaps, this replaces it.

* comment-smithing
2020-02-19 01:17:14 +01:00
Alexis Christoforides 39ea4f4bbd [mono] Remove SIGTRAP handler, as it interferes with debugging (#18851)
* [mono] Remove SIGTRAP handler, as it interferes with debugging

See https://github.com/mono/mono/pull/18243#issuecomment-586013427

* Fix lack of cleanup for SIGSYS
2020-02-18 16:33:44 -05:00
Aleksey Kliger (λgeek) 972896b2f1 [merp] Capture Environment.FailFast message in crash report (#18863)
* [merp] Capture Environment.FailFast message in crash report

* [test] merp-crash-test refactor crasher data; add optional validator

Switch from a Tuple<T1,T2> to a struct for the crash items.  Add a third
optional validator field which is an action to run on the json blob that we get
back.

Not updating existing tests to do any json validation, yet.

* [test] Add new MERP crash test for Environment.FailFast

Check that the resulting json blob includes the failfast message.
2020-02-18 15:59:14 -05:00
Vlad Brezae 4b96fefa7a [interp] Unwrap RuntimeWrappedException if needed, when throwing non exception objects (#18905)
Enable the entire System.Linq.Expressions suite with interpreter
2020-02-18 19:59:18 +01:00
Marius Ungureanu 104c201053 Bump bockbuild to bring in Gtk# regression fix (#18909)
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1055130
2020-02-18 19:46:24 +01:00
Alexander Köplinger e25d05be8d [netcore] Disable a flaky test on interpreter (#18904)
See https://github.com/mono/mono/issues/18695, it's most likely fixed in dotnet/runtime already but we can't easily get updated binaries into mono/mono.
2020-02-18 15:40:01 +01:00
Jay Krell 4649b5784f Remove #if __cplusplus from MagicTypeInfo. (#18898)
This wastes about 24 static const bytes which I think is ok.
6 bytes per struct, plus 2 padding, and there are 3 of them.

I was looking into https://github.com/mono/mono/issues/18827
and wanting to rule out #if __cplusplus.
2020-02-18 10:53:17 +01:00
Zoltan Varga 7038b1a426 [wasm] Implement generation of empty assemblies in wasm-tuner. (#18880)
The linker can completely link away some assemblies, but the rest of the ninja
build still requires them. Previously they were generated by running csc, but
thats too slow, so generate them using Cecil from wasm-tuner.
2020-02-18 02:33:45 -05:00
Jay Krell 15d61562ca Make non-recursive mutexes SRWLOCK instead of CRITICAL_SECTION. (#18847) 2020-02-17 22:34:44 +01:00
Jay Krell 1288a22890 [interp] Consolidate non-recursive call sites. (#18825)
Consolidate native_stack_addr as a result (resolving a FIXME).
 Builds on:
  https://github.com/mono/mono/pull/18857
  https://github.com/mono/mono/pull/18858
  https://github.com/mono/mono/pull/18869
2020-02-17 17:02:24 +01:00
Jay Krell 38256821ed Fix THREADS_SUSPEND_DEBUG to compile (id vs. mono_thread_info_get_tid). (#18871) 2020-02-17 15:59:48 +01:00
dotnet-maestro[bot] 14ae61555b Update dependencies from https://github.com/dotnet/arcade build 20200216.1 (#18887)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20116.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20116.1
2020-02-17 15:55:48 +01:00
imhameed 78fcd4b9f4 [llvm] Disable X86 call frame size optimization. (#18645)
* [llvm] Disable X86 call frame size optimization.

Fixes https://github.com/mono/mono/issues/17285.

For calls that require parameters passed in stack memory,
X86CallFrameOptimization can sometimes replace a sequence of movs to
prologue-reserved stack memory with a sequence of pushes and a compensating sub
rsp after the call. LLVM accumulates an OpAdjustCfaOffset MCCFIInstruction for
each push and and records the ephemeral parameter space consumed by the call
using an OpGnuArgsSize MCCFIInstruction (i.e. DW_CFA_GNU_args_size).

MonoException doesn't support encoding OpGnuArgsSize MCCFIInstructions, and
attempting to encode this instruction can cause unpredictable behavior, due to
our use of llvm_unreachable with assertions disabled [1]. Moreover, even if
MonoException happens to emit valid call frame instructions despite the UB
induced by llvm_unreachable, Mono's exception handler will restore the stack
pointer to the value it contained at the time right before the protected call
instruction was made, without compensating for any ephemeral parameter space.

The stack pointer could be fixed before the jump to the EH handler by
supporting DW_CFA_GNU_args_size or by executing call frame instructions for the
EH handler's frame all the way to the end of the protected instruction address
range--which will include a sub rsp with an appropriate immediate--to determine
an appropriate delta. But it's easier to just disable this optimization. We
already disable it for AOT [2]. It's sometimes disabled by LLVM on Darwin [3].

Also, this commit adds a test (which is a nearly verbatim copy of laszk's
reproduction sample in #17285) for EH failures caused by the interaction of our
EH code and X86CallFrameOptimization.

See also [4], [5], and [6].

[1] https://github.com/dotnet/llvm-project/blob/adeaa08e7bbc9aba5d67cb16c2b348be12deb000/llvm/lib/CodeGen/AsmPrinter/MonoException.cpp#L209-L210
[2] https://github.com/mono/mono/blob/1317cf06da06682419f8f4b0c9810ad5d5d3ac3a/mono/mini/aot-compiler.c#L9872-L9875
[3] https://github.com/dotnet/llvm-project/blob/adeaa08e7bbc9aba5d67cb16c2b348be12deb000/llvm/lib/Target/X86/X86CallFrameOptimization.cpp#L146-L152
[4] https://bugs.llvm.org/show_bug.cgi?id=24792
[5] https://reviews.llvm.org/D13132
[6] http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151005/304074.html

* Address feedback: add an explanatory comment.
2020-02-17 15:41:50 +01:00
Alexander Köplinger e9981608c4 Bump linker submodule (#18884)
Contains fix for -out argument.
2020-02-17 14:54:44 +01:00
Zoltan Varga 3e1f029eb8 [wasm] Make cil-strip quiet. (#18881) 2020-02-17 14:22:39 +01:00
Bernhard Urban-Forster f6dfce1aba [cross] check against correct alignment value (#18712)
It's essentially dead code in any of our configurations.

`MONO_ALIGN_gint8` is define here:
https://github.com/mono/mono/blob/ab2321c1dc1bf0293dadd7e52436b2bd87f2b7c3/mono/metadata/abi-details.h#L26-L35

And the correct value is set here:
https://github.com/mono/mono/blob/ab2321c1dc1bf0293dadd7e52436b2bd87f2b7c3/mono/metadata/abi.c#L71-L75
2020-02-17 14:22:10 +01:00
Zoltan Varga 79ab49333e [jit] Run static cctors for inlined methods, the cctor might not have ran in AOT mode. (#18706)
* [jit] Run static cctors for inlined methods, the cctor might not have ran in AOT mode.

* Emit the class init code later since it contains a branch.

* Emit the class init in the first code bblock.
2020-02-17 08:21:49 -05:00
Bernhard Urban-Forster 79135c6ea5 [abi-details] add explaining comment for macro magic (#18713)
* [abi-details] add explaining comment for macro magic

I think the fallback was added because "let's get it back to work quickly", back in the dark times without CI: https://github.com/mono/mono/commit/b655fb35c61fd3422003de01eed8e634d46b4186

* add constant version. to be used when offset is needed for static array sizes

* missing offset for windows

* windows define fix
2020-02-17 14:21:13 +01:00
Jay Krell 9a147baef1 Remove unnecessary ifdef before undef. (#18877)
* Remove unnecessary ifdef before undef.

* PR feedback: Remove educational comment.
2020-02-17 14:01:34 +01:00
Jay Krell 9adb352348 Remove duplicated Windows header content. (#18846) 2020-02-17 13:48:53 +01:00
Jay Krell 9a6b9c8a2a Fix/quash some warnings. (#18873)
mini\mini.c(1253,59): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3100,43): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3273,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3281,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3480,51): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3490,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3626,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3664,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3722,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3944,47): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini-runtime.c(549,64): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\method-to-ir.c(11838,2): warning C4065:  switch statement contains 'default' but no 'case ' labels
metadata\sre.c(2286,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2179,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2204,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2232,11): warning C4309:  '=': truncation of constant value
mini\method-to-ir.c(11838,2): warning C4065:  switch statement contains 'default' but no 'case
mini\type-checking.c(167,3): warning C4334:  '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
2020-02-17 12:48:56 +01:00
Jay Krell e8601e7b12 Remove unnecesssary include. (#18872)
Less to build if you edit the file.
2020-02-17 12:47:20 +01:00
Jay Krell 68c9084466 [interp] Use method_entry at interp entry instead of duplicating the code. (#18869)
tracing was a mix of:
 - sometimes ifdef out, sometimes not; make it always ifdefed
 - sometimes int, sometimes gint, sometimes gboolean; make it always int

Much of this PR will end up deleted when the multiple nonrecursive calls are
consolidated, but two will remain -- entry and call: label.

https://github.com/mono/mono/pull/18825 is where this is going but it was deemed
too large and requested to be split up. This is part of it.
2020-02-17 12:46:22 +01:00
Jay Krell 2d0ddf0abd [interp] Work toward consolidating multiple nonrecursive call sites to just one. (#18858)
https://github.com/mono/mono/pull/18825 was deemed too large so this breaks
it down into smaller pieces.
To consolidate the multiple nonrecursive call sites requires they share the smae
retval variable. Go ahead and share them all.
2020-02-17 12:43:34 +01:00
Jay Krell d98b7bf9f9 [interp] Fix incorrect comment -- stackval has no size. (#18876) 2020-02-17 12:42:08 +01:00
monojenkinsandEgor Bogatov 79bb3e2031 [mono] Fix msvc mono build and add CopyMonoRuntimeFilesFromArtifactsToDestination target (#18797)
Fixes
```
fatal error C1083: Cannot open include file: '../../support/libm/complex.c': No such file or directory
```
when `mono-netcore.sln` is opened in vs2019

Also, add `CopyMonoRuntimeFilesFromArtifactsToDestination` target to mono.proj to be able to copy mono runtime bits to any folder (needed for the performance team, cc @fanyang-mono)

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-17 12:02:18 +01:00
Jay Krell fed64a7019 [interp] Step toward consolidatation of non-recursive call sites. (#18857)
https://github.com/mono/mono/pull/18825 was too large.
This starts by replacing imethod with cmethod which is a necessary
part of the consolidation of multiple non-recursive call sites to just one.
2020-02-17 09:12:52 +01:00
Zoltan Varga 56f664f2d3 [llvmonly] Allow more direct calls. (#18819)
* [runtime] Add a few m_method_is_... inline functions.

* [llvmonly] Allow direct calls to tls accessors, its not a problem in llvmonly mode.

* [llvmonly] Allow direct calls to icalls/pinvokes/generic instances in the same module.

* Fix the loaded llvm build.
2020-02-15 01:52:28 -05:00
Zoltan Varga 12c13fdf37 [llvm] Remove unused llvm.mono.load/store intrinsics. (#18853) 2020-02-15 00:57:02 -05:00
Zoltan Varga e3ad74614a [llvmonly] Avoid adding the aot init wrappers to the method table, its not needed. (#18848) 2020-02-14 20:33:01 -05:00
imhameed 013e3c07e3 [llvm] Fix LLVM JIT when used with multiple AppDomains. (#18824)
We currently reuse LLVM's command line parsing system to configure and
enable passes. When using the default code generator pass configuration,
the `ImplicitNullChecks` and `X86CallFrameOptimization` passes can only
be enabled or disabled via the command line argument parser: the
`cl::opt`s controlling these (and many other) passes are private to
their translation units, and `TargetPassConfig`/`X86PassConfig` read
from these `cl::opt`s with no other means to override these values.

Unfortunately, some LLVM command line options have restrictions on the
number of times they may be set, and running the LLVM command line
argument parser more than once can fail, because the 'number of
occurrences' counter is stored in each global `cl::opt`. This causes
several tests to fail--one such test is unhandled-exception-7.cs.

This change:
- removes the lazy LLVM JIT initialization logic and instead
  runs this initialization once during `mini_init`,

- moves some stray JIT-only code from mini-llvm.c to llvm-jit.cpp,

- wraps the declarations in domain-internals.h with a
  `G_BEGIN_DECLS`/`G_END_DECLS` pair, so that they avoid name
  mangling when included in C++ source, and

- removes `exception_cb` and `emitted_cb`: neither were actually used
  by live code.
2020-02-14 17:29:45 -08:00
Vlad Brezae c9ba14796a [interp] Fix filter clauses (#18809)
When executing a filter we create a new interp frame, that is a duplicate of the original frame that contains the filter. Once the execution of the filter is finished, we were copying the stack contents back to the base frame. The problem with this is that when the filter is run, the stack is not completely unwinded, meaning that later we can still run a finally block in a frame called by the base frame. This finally block can access the stack arguments, which would reside on the stack of base frame. This means that we must not override the stack of the base frame, only the locals.

Fixes some tests from System.Linq.Expressions suite.
2020-02-14 14:38:25 -08:00
Zoltan Varga 948d7e3b3e [llvmonly] Emit stubs for methods which failed compilation. (#18802)
Emit a stub for methods which failed llvm compilation. Currently the stub will
throw a MissingMethodException, later it can fall back to the interpreter etc.
This is required for cross-assembly direct calls, since the caller doesn't
know that the callee has failed llvm compilation leading to missing
symbols during linking.
2020-02-14 10:59:41 -08:00
dotnet-maestro[bot] 5b775170f2 Update dependencies from https://github.com/dotnet/arcade build 20200214.1 (#18856)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20114.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20114.1
2020-02-14 18:21:46 +01:00
monojenkinsandLevi Broderick 2448e5685f Fix broken MSDN blogs links (#18852)
Fixes MSDN blogs links throughout our docs and source code comments that were broken during the great MSDN migration of 2019.

Note: I am changing the contents of the _davbr Blog Archives_ folder by fixing the broken links. If we need to keep these files exactly as they were when they were copied from the blog, I can revert.

Co-authored-by: Levi Broderick <[email protected]>
2020-02-14 18:15:42 +01:00
Egor Bogatov b5df4be189 Make MonoWebRequestHandler linker friendly (#18832)
Create it using reflection so it will be always linked out. However if it's set via `XA_HTTP_CLIENT_HANDLER_TYPE` linker will preserve it in a special substep: https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/PreserveHttpAndroidClientHandler.cs#L72-L80

I had to make its ctor **public** because Linker instantiate it via `Activator.CreateInstance(string)` ([here](https://github.com/xamarin/xamarin-android/blob/60363ef45cd6994a322adac1b73d4271b0658c1e/src/Mono.Android/Android.Runtime/AndroidEnvironment.cs#L369)). But the whole type remains internal so it's not a real API change.

Fixes issue from https://github.com/mono/mono/pull/18818#issuecomment-585611647
2020-02-14 13:04:46 +01:00
Fan YangandAlexander Köplinger 549a38cd4d Bump linker submodule (#18844)
* Bump linker submodule

* Use -o instead of -out for linker output

The arguments changed in linker.

Co-authored-by: Alexander Köplinger <[email protected]>
2020-02-14 12:20:19 +01:00
Bernhard Urban-Forster 26a5d63384 [sdks] add anroid-interpreter-{arm64-v8a,armeabi-v7a,x86,x86_64} targets (#18814)
The Xamarin.Android team prefers that we ship two different `libmonosgen-2.0.so` for each target, e.g. for arm64 we will have with this PR:

* `./sdks/out/android-arm64-v8a-release/lib/libmonosgen-2.0.so`
* `./sdks/out/android-interpreter-arm64-v8a-release/lib/libmonosgen-2.0.so`

As the names indicate one has the interpreter enabled.

This is contrary to what we do for Xamarin.iOS. There, `--enable-minimal=interpreter` has the side-effect that the build will produce a `libmono-ee-interp.dylib` that can be linked next to `libmonosgen-2.0.dylib`. We _could_ do the same for Android, but that would require us to

* Add `--disable-visibility-hidden` to the Android Mono runtime builds. There are concerns however that this will slow down `dlopen (libmonosgen-2.0.so)` on the Xamarin.Android side.
* Do some changes to the interpreter infrastructure, because `--disable-minimal=interpreter` is designed to be used with a cross compiler in mind. For example, we would need a different solution for this: https://github.com/mono/mono/blob/d7e6fcbda82639fd5afe106b0e7a65a2a72a4f63/mono/mini/tramp-arm64.c#L631-L634
2020-02-14 08:26:58 +01:00
Thays Grazia fa0af37014 Removing PPDB. 2020-02-13 22:39:27 -06:00
Thays Grazia 2d0337ee31 Enabling PPDB on Wasm. 2020-02-13 22:39:27 -06:00
Zoltan Varga 2146f1f418 [runtime] Disable assertion messages for g_assertf as well if DISABLE_ASSERT_MESSAGES is used. (#18774) 2020-02-13 13:20:21 -08:00
monojenkinsandRyan Lucia b89e6208b3 [mini] Move netcore hosting functionality into separate file (#18799)
Made minor fixes to the preload hook in the process.

The runtime will still crash if you don't specify the TPA paths and include corlib in that list, but it will at least hit an assert that makes clear it can't find corlib instead of attempting to dereference a null pointer in the preload hook.

The monovm header functions are exported for testing purposes, but the file is intentionally not made public as of now.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-13 15:36:46 -05:00
Zoltan VargaandBernhard Urban-Forster 362cb3084f [wasm] Avoid compiling release configuration with -s ASSERTIONS=1 in AOT mode, it generates a lot of extra code. (#18703)
* [wasm] Avoid compiling release configuration with -s ASSERTIONS=1 in AOT mode, it generates a lot of extra code.

* Update sdks/wasm/packager.cs

Co-Authored-By: Bernhard Urban-Forster <[email protected]>

Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-13 10:25:05 -08:00
Alexander Köplinger c03b6244e0 [tests] Disable bug-10127.exe runtime test on Windows x64 (#18828)
It hangs: https://github.com/mono/mono/issues/18827
2020-02-13 13:50:00 +01:00
Vlad Brezae e8f392412b [interp] Always init locals (#18811)
Storing uninitialized locals in a managed object shouldn't crash the runtime, so they should be zeroed. This doesn't seem to regress performance on wasm since the initlocals flag is always set, currently. We should be able to remove it most of the time, once we add some multi basic block support to the cprop phase. This will enable us to further reduce call overhead.

Fixes https://github.com/mono/mono/issues/18527
2020-02-13 13:00:33 +01:00
Vlad Brezae 0db68c3f27 [interp] Add missing conversion opcodes from r4 (#18813)
Fixes tests from System.Linq.Expressions suite
2020-02-13 12:58:28 +01:00
Zoltan Varga 3129d4ca8b [runtime] Add --enable-minimal=processes,config options to disable support for processes and .config files. (#18822) 2020-02-13 12:55:02 +01:00
Vincent Povirk a645a2d3c7 [System.Drawing] Release the Graphics HDC on Dispose. (#18801)
Libgdiplus will fail when attempting to delete a Graphics object
that has an unreleased HDC. This allows Dispose to succeed in
that case.

This more closely resembles the behavior of corefx.
2020-02-13 10:40:07 +01:00
Jay Krell 19bc9a8dd3 [interp][wasm] Make newobj_fast not recursive, cleanup, fix EXCEPTION_CHECKPOINT. (#18734)
* [interp[wasm] Make newobj_fast not recursive.
This contributes to https://github.com/mono/mono/issues/18646,
and against master might even fix it.
Though much recursion remains in the interpreter/transform.

* [interp] Cleanup redundant code and child_frame vs. frame.
Only the recursive paths need child_frame separate from frame.

Callers of method_entry tended to duplicate parts of it.
Notice it is marked MONO_ALWAYS_INLINE.

And if that isn't right, it should be split into fast and slow forms,
i.e. regarding frame->imethod->transformed.

* Need to EXCEPTION_CHECKPOINT more often -- when transform occurs but does not raise an exception.

* Preserve prior faster paths.

* Making call_vararg added the following to all calls:
  storing is_void in a larger scoped local
  save is_void
  restore is_void
  check is_void, along with preexisting check of retval == null

Because retval must be set even for void call_varargs,
because the arglist is found from it.

This removes the check of retval == null, it should be redundant with is_void.

* A little cleaner -- unlikely is in method_entry.

* Comma to make comment easier to read.
2020-02-13 00:13:34 -08:00
monojenkinsandJo Shields e901d4fe1a Enable Mono ARM & ARM64 Linux builds (#18810)
Traditionally, Mono is build with native compilers (i.e. "build for this architecture" running in an ARM or ARM64 chroot on an ARM64 server), but to align with CoreCLR, let's go the cross-compile way.

This enables cross-compiler support, by passing a whole bunch of extra info to `./configure`, some of which is set in the common YAML (like the `--sysroot` value)

Co-authored-by: Jo Shields <[email protected]>
2020-02-13 01:15:38 -05:00
Zoltan Varga 0a4c3069be [wasm] Update WASM-REQUESTS.md. (#18816) 2020-02-12 18:04:49 -08:00
Zoltan Varga c76bbb23f9 Revert "[wasm] Pass direct-pinvoke to the AOT compiler. (#18775)" (#18807)
This reverts commit eb85a108a3.
2020-02-12 16:21:51 -08:00
Zoltan Varga 3776b416ad [llvm] Remove a wasm limitation which no longer exists. (#18803) 2020-02-12 16:19:21 -08:00
monojenkinsandRyan Lucia cd46f5c77c [WIP][threading] Switch to a native implementation of LowLevelLifoSemaphore (#18559)
This change is motivated on two main fronts. The first is maintainability - the current managed implementation is difficult to understand and I worry diagnosing any potential issues would be a massive time sink. The second is performance - the current implementation appears to suffer from significant lock contention when running the TechEmpower plaintext benchmark. My hope is that the simpler, cleaner native implementation here will avoid both problems, but I don't want to merge it until we have benchmarking data. However, even if the numbers are similar, I still think it's worth merging just from a maintainability perspective.

The native LifoSemaphore implementation has only ever been tested on posix-like platforms, so Windows behavior is unknown. Currently the Windows implementation of LowLevelLifoSemaphore is very different, so unless we have need for the LifoSemaphore elsewhere in the runtime this isn't a concern.

Many thanks to @filipnavara for the initial implementation.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-12 17:14:30 -05:00
Jo Shields d7e6fcbda8 Fix native mkbundle (#18739)
* Use clearer Bash conditionals, since we force Bash anyway

* Add libmono-native.so to bundle

* Add logic to add a etc/mono/config without $mono_libdir to bundle

Mostly resolves: #16991
2020-02-12 15:57:54 -05:00
Zoltan Varga 3cbc14f6ff [wasm] Remove chakra and xs from the jsvu update list. (#18804)
Chakra is no longer getting updated and we don't use XS.
2020-02-12 10:05:06 -08:00
Zoltan Varga a50f0b1197 [wasm] Switch to the lts version of node. (#18805)
* [wasm] Switch to the lts version of node.

* Fix a typo.
2020-02-12 10:02:55 -08:00
Ankit Jain 8437ecf9d1 [master] Bump msbuild to track xplat-master (#18790)
* [master] Bump msbuild to track xplat-master

* Bump roslyn to 3.5.0-beta3-20111-03 .

* Update nuget.exe to match xbuild

- the closest version available right now, to `5.5.0-rtm.6422` is
`5.5.0-preview2`, so using that for now.
2020-02-12 17:10:31 +01:00
Egor Bogatov 76e577c00f Allow users to switch to MonoWebRequestHandler on Android via UI (#18764)
* Bring back old HttpHandler for Android
2020-02-12 16:25:46 +03:00
Zoltan Varga e2c0b5ef27 [runtime] Call mono_stack_mark_record_size () only in a checked build, its slow, and the __FUNCTION__ argument creates a C string for every function name. (#18773) 2020-02-12 10:43:35 +01:00
monojenkinsandRyan Lucia 07dc4b88e7 [utils] Add unused attribute to fix warning in checked-build.c (#18800)
At least on my machine, this is the only warning we have when building netcore in the default configuration.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-11 22:57:15 -08:00
Zoltan Varga 1c7295cfdf [jit] Add an #ifdef around MONO_CPU_WASM_SIMD. (#18771) 2020-02-11 22:56:34 -08:00
monojenkinsandEgor Bogatov fc75795ed2 [mono] Implement Sse2.AddSaturate using LLVM (#18793)
Mono currently supports a limited subset of **Sse1**-**Sse42** intrinsics used only by CoreLib internally. So once a new API is used to optimize things there we also have to implement it in Mono. So it recently happened in https://github.com/dotnet/runtime/pull/32036 (`Sse2.AddSaturate` was used for the first time).

So this PR implements it for mono (with all Sse2 overloads) using named LLVM intrinsics. It turns out it's different between LLVM6 (we currently based on) and LLVM9 (we plan to migrate to soon).

for `Vector128<byte>` overload it emits
```llvm
%result = call <8 x i16> @llvm.x86.sse2.paddus.b(<16 x i8> %left, <16 x i8> %right)
```
which is then emitted as `vpaddusw`

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-12 00:55:25 +01:00
Zoltan Varga eb85a108a3 [wasm] Pass direct-pinvoke to the AOT compiler. (#18775) 2020-02-11 12:08:49 -08:00
Zoltan Varga b260b327fe [runtime] Simplify access to runtime TLS variables when DISABLE_THREADS is defined, they become simple globals. (#18772) 2020-02-11 11:49:24 -08:00
Alexander Köplinger e8527db725 [mini] Wrap check for debug info enabled in ENABLE_NETCORE in all places (#18789)
See if this fixes the weird bug-10127.exe test failure that only occurs on Windows C++ builds.
2020-02-11 16:50:14 +01:00
Alexander Köplinger 071b7259d5 Move offsets-tool into mono/tools (#18783)
This allows us to consume it in the dotnet/runtime build.
2020-02-11 12:18:45 +01:00
Bernhard Urban-Forster 742c414bde [interp] reclaim stack fragments when not used. (#18777)
* [interp] reclaim stack fragments when not used.

When a requested frame size doesn't fit in an already allocated stack fragment, we instead allocate another, larger fragment.
When doing this we missed to free the existing stack fragment, and in combination with a second problem (`current->next` not being cleared when unused) it can accumulate to a lot of unused stack fragments and in worst-case to OOM, as it happened on Linux corlib xunit tests.

Thanks to Jay for tracking it down.

* be cautious for GC

* add barrier
2020-02-11 10:59:49 +01:00
Bernhard Urban-Forster 4cd05a7be4 [cross] support arm-linux-gnueabihf cross compiling with netcore (#18714)
* [cross] support arm-linux-gnueabihf cross compiling with netcore

Some notes on usage:

```console
$ sudo apt install g{++,cc}-gnu-linux-gnueabihf python3-pip

$ ./autogen.sh --disable-boehm --target=arm-linux-gnueabihf \
    --with-cross-offsets=arm-linux-gnueabihf-cross-offsets.h \
    --with-core=only \
    CFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer" \
    CXXFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer"

$ make -C tools/offsets-tool-py/

$ python3 tools/offsets-tool-py/offsets-tool.py \
    --targetdir=/home/lewurm/work/mono \
    --abi=arm-linux-gnueabihf \
    --monodir=/home/lewurm/work/mono \
    --outfile=arm-linux-gnueabihf-cross-offsets.h \
    --libclang=/usr/lib/x86_64-linux-gnu/libclang-7.so.1 \
    --sysroot=/usr/arm-linux-gnueabihf \
    --netcore
```

* better include path guessing
2020-02-11 09:49:21 +01:00
Jay Krell b0c434280d Add volatile for cross thread communication. (#18760) 2020-02-10 21:54:41 -05:00
Alexis Christoforides 72fe53a2ee [MacSDK] Get more verbose output from fc-cache operation (#18782) 2020-02-10 21:53:00 -05:00
dotnet-maestro[bot] 900e77c868 [master] Update dependencies from dotnet/arcade (#18674)
* Update dependencies from https://github.com/dotnet/arcade build 20200201.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20101.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20101.2

* Update dependencies from https://github.com/dotnet/arcade build 20200209.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20109.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20109.1
2020-02-10 17:03:10 +01:00
Jay Krell 8082e87c4e [cxx] Fix C++ build -- int vs. enum. (#18727) 2020-02-09 11:26:52 +01:00
monojenkinsandRyan Lucia 1635c21e59 [loader] Set domain->entry_assembly earlier in startup (#18720)
When locating AppContext.BaseDirectory on netcore, if APP_CONTEXT_BASE_DIRECTORY is not set we call into the runtime for a fallback location, which checks domain->entry_assembly. On legacy this is fine because we set that property in `prepare_thread_to_exec_main`, but on netcore the new managed assembly loading algorithm will call into managed and check the property before that function is run, which will result in AppContext.BaseDirectory returning an empty string. The solution is to set entry_assembly a bit earlier in initialization. This is the appropriate location because it's what embedding API users will call to run an assembly.

In the process, I got rid of some code that would bypass the usual lookup algorithm unnecessarily.

Fixes dotnet/runtime#31800

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-07 17:18:15 +01:00
Alexis Christoforides 96e64ad4e6 [MacSDK] Keep logged post-install script output from previous installs (#18441)
This is helpful for issues such as https://devdiv.visualstudio.com/DevDiv/_workitems/edit/596552 OC, where the user is asked to both provide these logs for debugging, and also reinstall the SDK to resolve the issue
2020-02-07 11:15:48 -05:00
Alex Soto 0c200c5125 [iOS] Replace removed dsymutil -t switch with -num-threads (#18741)
Apple removed `-t` switch from dsymutil in favor of `-num-threads`
2020-02-07 11:13:31 +01:00
Alexis Christoforides d9e090e867 [merp] Add an exception type for managed exceptions (#18685)
Addresses https://github.com/mono/mono/issues/18681
2020-02-06 23:33:27 -05:00
Alexis Christoforides 081498393a [merp] Add tests for crashing via POSIX signal (#18532) 2020-02-06 23:30:11 -05:00
Alexander Köplinger a40e1ae681 Skip WebAssembly build in sdks archive build on non-master branches
WebAssembly doesn't use our usual release branches yet.
2020-02-06 22:36:38 +01:00
monojenkinsandSteve Pfister 88c44e26f6 [mono] Enable debugging for netcore configurations by default. (#18686)
What this means is:

1. --debug is no longer needed since the default is TRUE
2. --debug=ignore is a new switch to go back to the old mono default

Fixes https://github.com/dotnet/runtime/issues/31662

Co-authored-by: Steve Pfister <[email protected]>
2020-02-06 22:19:04 +01:00
Manuel de la Pena b2ad5f34e1 [NUnitLite] Bump nunitlite submodule. (#18731)
Update the nunitlite submodule to pick up some fixes in the NUnit v3
outputwriter that are needed by xamarin-macios.
2020-02-06 15:58:38 +01:00
mdh1418 7936510ec3 [corlib] Silence undocumented exceptions in ReadXdgUserDir (#18684) 2020-02-06 08:49:28 -05:00
Johan Lorensson 6fe343db8c Reduce number of mono header files copied into SDK include folder. (#18702) 2020-02-06 09:11:55 +01:00
Larry Ewing 4cb6ca5599 [wasm][debugger] Use System.Text.Json in the driver 2020-02-05 20:37:46 -06:00
Larry Ewing 8ce81f5113 [wasm][debugger] Let chrome pick an open port by default 2020-02-05 20:37:46 -06:00
Larry Ewing dd151fea35 [wasm][debugger] Silence some logging and extract the null case 2020-02-05 20:37:46 -06:00
Larry Ewing 32826eb418 [wasm][debugger] Make command formatting slightly more sane and reduce c&p code 2020-02-05 20:37:46 -06:00
campersau a2d1aec5d2 [wasm] Close WebSocket on Dispose (#18628) 2020-02-05 18:58:05 -06:00
Alexis Christoforides 31c12db374 [runtime] Improve handling crashing signals (#18243)
The default handlers for SIGTRAP, SIGSYS on macOS and Linux crash the process, so we now have Mono report this.

This patch also changes how the name of the signal is acquired for passing on to the crash reporting functions, allowing code reuse on crashing signals.
2020-02-05 19:04:57 -05:00
Jay Krell 0e49716ff8 [interp] Remove recursion from call_vararg. (#18697)
[interp] Make call_vararg non-recursive.

[interp] Remove recursion from call_vararg.

This contributes to https://github.com/mono/mono/issues/18646 but is not enough to fix it. The interpreter still recurses a lot.
2020-02-05 17:45:53 -05:00
Johan Lorensson b8369e630c Stack corruption running full AOT interpreter using ValueTuple. (#18710)
Calling a method compiled using full AOT from interpreter can lead to
stack corruption if signature include a ValueTuple without fields on
compilers where MONO_ZERO_LEN_ARRAY is not 0 (msvc, clang). This is due
to doing memset on memory allocated using MONO_SIZEOF_GENERIC_INST but
then memset using sizeof (MonoGenericInst). Fix makes sure we allocate
and memset the same size of memory since otherwise it will lead to an
overwrite of allocated memory when type_argc is 0.
2020-02-05 17:35:13 -05:00
Zoltan Varga 64eacfa900 [wasm] Pass the binding assemblies to the linker normally using -d framework_dir instead of explicitly. The latter makes the linker not link them. (#18704) 2020-02-05 11:04:49 -05:00
Zoltan Varga 2c8371c221 [wasm] Rename aot-dummy to aot-instances. (#18692) 2020-02-05 11:04:31 -05:00
monojenkins ab2321c1dc [bcl] Update BCL Linked Size 2020-02-05 08:18:03 +01:00
Zoltan Varga e9830fac8d [jit] Avoid calling mono_aot_direct_icalls_enabled_for_method () in non-aot mode, it might be a stub if aot is disabled. (#18707) 2020-02-05 00:00:27 -05:00
Alexander Köplinger 8027746aea Remove netcore/corerun/Makefile from mono-output.m4
It was removed from DIST_SUBDIRS with https://github.com/mono/mono/commit/2017e493a970a79aa32df2f05e5f1fd877cb3fdc
2020-02-04 23:36:51 +01:00
Bernhard Urban-Forster 71ad5a0771 [mini] iltests.exe depends on TestDriver.dll (#18638) 2020-02-04 23:28:37 +01:00
Matt Sylvia 2f74ab393f Update deprecated query parameter to header (#18705)
Use of `access_token` is now deprecated, switching to HTTP header

```
Please use the Authorization HTTP header instead, as using the `access_token` query parameter is deprecated.
```
2020-02-04 22:57:46 +01:00
Jay Krell b5fb8dbe6a [interp]Factor out common code. (#18672) 2020-02-04 19:53:48 +01:00
Alexander Köplinger 44fe895ab0 Treat 2020-02 as a valid release branch name too
This should give us breathing room until 2100-02 :)
2020-02-04 19:28:24 +01:00
monojenkinsandEgor Bogatov 2017e493a9 Add RunSample target to mono.proj (#18635)
1) Delete `src/mono/netcore/corerun/` folder, we can build/use coreclr's corerun (it has `--clr-path`)
2) Add `RunSample` to build & run HelloWorld with mono runtime.

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-04 19:15:56 +01:00
Zoltan Varga a592b4b871 [wasm] Add linker substitutions for fields. (#18700) 2020-02-04 12:41:31 -05:00
Alexander Köplinger 303122e94a Bump bockbuild 2020-02-04 17:41:44 +01:00
Alexander Köplinger 088692ff3a Bump xsp to get correct version number 2020-02-04 17:41:44 +01:00
Jo Shields 8d00b7cd8b Bump version to 6.13 (#18699) 2020-02-04 17:41:01 +01:00
Jay Krell cdb7944e7b Add --profiling-funcs to release wasm build. (#18663)
[wasm] Put function names in call stacks.

Add --profiling-funcs to release wasm build.
This is a weaker for of -g2 or -g3 or -g4 or --profiling.
It sounds like the weakest form that preserves names.
Documentation is around here: https://emscripten.org/docs/tools_reference/emcc.html#emcc-g1

Now when reproducing for example https://github.com/mono/mono/issues/18646, you get:
```
 e.stack

<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[generate]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke_checked]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke_array]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke_raw]@[wasm code]
<?>.wasm-function[do_icall]@[wasm code]
<?>.wasm-function[do_icall_wrapper]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_wasm_invoke_method]@[wasm code]
wasm-stub@[wasm code]
Db@[native code]
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:189579
call_method@http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:153837
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:156177
callEntryPoint@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:37229
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:33889
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:32396
i@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:31247
promiseReactionJob@[native code]
```
2020-02-04 11:36:22 -05:00
Jay Krell edb18739cb [interp] Remove goto related to code_type being unknown | interp | compiled. (#18678)
[interp] Remove goto for code_type unknown | interp | compiled.

[interp] Remove goto for code_type unknown | interp | compiled.

The goto kinda implies a loop, but it is really just handling one case
before the other, but the source misorders the checks, perhaps for perf,
which maybe, can be done equivalently with G_UNLIKELY, and also remove
some possible extra memory reads.
2020-02-04 09:15:58 -05:00
Jay Krell f85a820677 When the interpreter does recurse, use a different function (#18664)
[interp][wasm]Use per-call recursive functions in interpreter to aid debugging.

 When the interpreter does recurse, use a different function
for each call site, so you can tell from a WebAssembly stack
which case it is. i.e. with function names but no line numbers.

With this, we can see that call_varargs contributes to the problem
here: https://github.com/mono/mono/issues/18646

Also, do not inline the interpreter, it probably/hopefully never was.
2020-02-04 05:56:00 -05:00
monojenkinsandZoltan Varga 1a81bef2bd [runtime] Fix the --enable-minimal=aot build. (#18688)
Fixes https://github.com/mono/mono/issues/18675.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-04 09:51:42 +01:00
Zoltan Vargaandmonojenkins 08f4ef6d0c Bump linker. (#18691)
* Bump linker.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-02-04 09:26:09 +01:00
Jay Krell 8c0f6db26f [interp]Fix the type of clause_args. (#18671)
[interp]Fix the type of clause_args.
2020-02-04 00:25:57 -05:00
Jérôme Laban f3d2cc2176 [Wasm] Update to emscripten 1.39.7 (#18687) 2020-02-03 22:23:47 -05:00
monojenkinsandZoltan Varga 7deefe779f [aot] Avoid inflating gparams with byreflike types during generic sharing. (#18680)
Fixes https://github.com/mono/mono/issues/18676.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-03 21:35:20 -05:00
Zoltan Varga 62a57c687c [wasm] Set the 'top-runtime-invoke-unhandled' option so unhandled exceptions calls AppDomain.UnhandledException. (#18642) 2020-02-03 18:42:49 -05:00
d44a5d3a0b [mini] Add method-triggered printing for runtime stats (#18621)
* [mini] Add optional method name to --stats (#18387)

* Implement parsing for optional method name to --stats command line argument.

* Implement parsing for optional method name to --stats command line argument.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Respond to PR feedback.

* Refactored away helper functions.

* Refactored counter setup into separate function. Moved stats optional method into its own condition.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Renamed mono_setup_counters->mono_enable_counters. Restore whitespace to 0a51d89c42
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* [mini] Print header for stats. (#18442)

* Add an enum for dumping stats at points in time. Print header before dumping stats in mini_cleanup.

* Make print_stats_header private.

* Remove unused variable.

* Refactored print_stats_header to better accomodate optional methods for stats

* Change printf calls to g_printf in response to feedback.

* Undo changes in response to @CoffeeFlux 's feedback. Add g_printf calls to mini_cleanup

* Avoid always printing stats at shutdown.

* Condition on `mono_stats` instead of `mono_jit_stats`.

* [mini] Global JIT Stats Printing (#18483)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Rename mono_print_jit_stats to mono_runtime_print_stats and make it more generic.

* Rename runtime_cleanup_stats back to jit_stats_cleanup.

* Fix a line that was indented with spaces.

* [mini] parse fully qualified method name using mono_method_desc_new (#18499)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Use MonoMethodDesc struct and associated helpers to parse from command
line args. Remove unused stats_method_name and setter function.

Squashed commit of the following:

commit de7410696dab10723c76f545076522daa17e85d1
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 23:01:23 2020 -0500

    Utilize new stats printing function.

commit 1e5038adfec06c670dbaced771a8ff9344502d17
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 22:37:53 2020 -0500

    Remove unused stats_method_name and setter function. Use mono_method_desc_full_match instead of mono_method_desc_match.

commit 04f86c66c139e064bc35738f955f960232eb6cbb
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 18:11:37 2020 -0500

    Modify mono_method_desc_match to actually match namespace and class. Add global MonoMethodDesc, and parse it from command line args. Leave TODO print statement in mono_jit_compile_method_inner.

* Move mono_method_desc to mini-runtime header. Fix whitespace.

* Cherry pick @KyleMarkland 's changes, and resolve merge conflicts.

* Fix leak. Give better error message when parsing empty method name.

* Move freeing of mono_stats_method_desc.

Co-authored-by: KyleMarkland <[email protected]>

* Add a conditional to transform.c to enable stat dumping from the interpreter. (#18540)

* [mini] Use g_printerr instead of g_printf before exiting (#18574)

* Rename g_printf -> g_print

* Use g_printerr on errors instead of g_printf.

* Pass NULL to mono_counters_dump. Revert g_printf to g_print.

* Update manpage

* Feedback

Co-authored-by: rgarc <[email protected]>
Co-authored-by: KyleMarkland <[email protected]>
2020-02-03 16:05:06 -05:00
Vlad Brezae 01b2e15c06 [tests] Reenable enum equals test on interpreter (#18673)
Fixed by https://github.com/mono/mono/commit/9ea350ed3c963c77f0f4f77ec590cc3c84773424
2020-02-03 07:54:10 -08:00
Ryan Lucia 0f47ea5d80 [build] Check for Python 3 in configure.ac (#18625)
I fear this causing packaging problems, but as-is we require Python 3 for most of our Python files but don't actually check for that in configure. We should be consistent about this. Some projects (notably wasm) also use the Python version detected here, so bumping this will also bump the version used in those cases (a good thing, since there are some known bugs when using the scripts with Python 2).
2020-02-03 08:45:49 -05:00
monojenkinsandVlad Brezae 9ea350ed3c [interp] Fix enum constrained calls on netcore (#18659)
We were failing to dereference a managed pointer to the enum, due to incorrect checks, when doing a constrained call.

Fixes https://github.com/mono/mono/issues/18530

Co-authored-by: Vlad Brezae <[email protected]>
2020-02-03 03:08:15 -08:00
Phil Jaenke 20a641e4a0 Fix ppc64 build by fixing -mminimal-toc more correctly. (#18665)
The original fix back at #18554 did not work quite right, which was missed for lack of CI.
Reworked this check to instead test for `-mminimal-toc` on all powerpc hosts, and to only do the ELF version check on FreeBSD.

Fixes #18554 #18578 current CI build failures on AIX
2020-02-03 11:16:33 +01:00
Alex Thibodeau 050340ef1a Adjust mono_class_init_internal to no longer bypass cleanup when a failure is detected. (#18649)
This fixes: https://issuetracker.unity3d.com/issues/unity-physics-collisions-do-not-work-and-errors-are-thrown-when-entering-play-mode
2020-02-02 11:29:57 -05:00
imhameed 4bed68d073 [llvm] Enable jump threading with LLVM JIT. (#18655)
See https://github.com/mono/mono/issues/16243. This will allow elision
of redundant bounds checks. This pass is already used during AOT (as
part of opt -O2).
2020-02-02 01:18:31 -05:00
Fan Yang 3a93ebd6fe Abort cmov optimization, when one of the out blocks is try block (#18626) 2020-01-31 16:40:47 -05:00
mdh1418 9ea6b6a58a [wam] Rename other .stamp-build-debug-sample to debug_sample/dotnet.js (#18653) 2020-01-31 16:34:12 -05:00
Zoltan Varga 2a92593837 [wasm] Bump emscripten to 1.39.6. (#18641) 2020-01-31 13:18:11 -05:00
Zoltan Varga a475b84a3e [aot] Add support for making direct cross-assembly calls in llvm mode. (#18505)
* [aot] Handle '|' during symbol mangling.

* [aot] Clean up the construction of the linker command line.

* [aot] Pass -Wl,-install_name to the linker on osx, otherwise the shared object will be named based on the output file name i.e. <assembly>.dylib.tmp.

* [llvm] Pass the MonoAotFileInfo pointer to the init functions, not used yet.

* [jit] Add support for direct calls between methods in different AOT images in LLVM mode.

* Add a 'direct-extern-calls' AOT option.
* Initialize the PLT/GOT during method initialization,
  this normally happens in load_method () but that is
  not called when a method is directly called.
* Add self initialization support for JITted methods, they can be
  called directly since they are fallbacks if LLVM fails.

* Fix the build.

* Avoid calling BeginInvoke etc. directly, its implemented by the runtime.

* Unify some cases.

* Init PLT at image load time since directly called methods without initializers might make calls through it.

* Fix a startup problem.

* Make the mono_is_corlib_image () check for during startup.

* Check for is_inited in the init icall wrappers, since they might be called from JITted code which doesn't check it.

* Address review comments.
2020-01-31 11:41:49 -05:00
Zoltan Varga 4ef40a91ed [jit] Avoid generating wbarriers for vtype stores to the stack. (#18556)
The existing heuristic doesn't work with llvm.
2020-01-31 09:54:32 -05:00
Zoltan Varga a6707bfeb9 [wasm] Remove .stamp-build-debug-sample, use the output file instead. (#18644) 2020-01-31 01:21:21 -05:00
Zoltan Varga 745c8ff20b [wasm] Avoid overwriting emcc flags when SIMD is enabled. (#18643) 2020-01-31 01:13:17 -05:00
Johan Lorensson 1a2c433b7c Move generic_class_init out of line to assist branch prediction. (#18497)
When calling a generic class we need to make sure it has been inited.
This is done by calling generic_class_init. Since this call happens
frequently, but will only run once, it should be put out of line, assisting
branch predictor, preventing misspredictions (in all places but the first).

Currently this is not the case, since we will generate code like this:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: jne         000000000000048F
000000000000048A: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
000000000000048F: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]

Where we will always (except for first call) take the forward branch.

Instead we should mark the call to the method as out of line, at least
giving the code generator the information so it can move it out of line
and preventing a jmp, unless when it needs to init, the slow path. Doing
that will generate the following code:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: je          00000000000004C1
000000000000048A: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]
...
00000000000004C1: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
00000000000004C6: jmp         000000000000048A
2020-01-31 00:39:37 -05:00
monojenkinsandZoltan Varga 1317cf06da [aot] Fix some races if GOT entries are already set. (#18636)
decode_patch_info () would not decode patches if their corresponding GOT entry
was already set, causing half initialized MonoJumpInfo entries to be returned
to callers. Set the type of the MonoJumpInfo entry to MONO_PATCH_INFO_NONE
in these cases, and have the callers handle it.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 17:12:54 -05:00
monojenkinsandZoltan Varga 04c01efd49 [jit] Add a 'top-runtime-invoke-unhandled' option to treat exceptions which reach the top invoke frame as unhandled when embedding. (#18610)
This is not really a MONO_DEBUG option, but easier to implement it this way.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 15:59:30 -05:00
mdh1418 109a61227c [mono] Prevent direct MonoClass field access (#18634)
* [wasm] Add --enable-checked-build=private_types configure option to disable direct access of MonoClass fields.

* [mono] Indirectly access MonoClass fields
2020-01-30 15:57:55 -05:00
monojenkinsandZoltan Varga 4e90bb7e69 [wasm] Fix mono_vfree (). (#18632)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 13:28:51 -05:00
Steve Pfister e0fcb6093b [wasm][debugger] Cancel Existing Single Step Request When Creating New (#18608)
When stepping out of a c# breakpoint into the runtime (or JS... Not sure if that's accurate),
multiple single step requests can be created when the breakpoint is hit again.  In order to handle this scenario for wasm,
we just cancel the existing one.

NOTE: The debugger args addition seemed a better way to handle this than outright canceling in debugger-engine. If there
are implications to doing this, I can take it out.
2020-01-30 15:19:31 +01:00
Ryan Lucia e3624d356b [wasm] Finish renaming mono.js to dotnet.js (#18623)
This work mostly happened a while back, but this commit finishes the job.
2020-01-29 23:47:46 -05:00
Ryan Lucia cf5204021b [wasm] README updates (#18624)
Use -j with all example make invocations, mention Ninja dependency, mention raising ulimit, recommend py3 for running the server (when using py2, the port isn't properly released).
2020-01-29 23:47:28 -05:00
Larry Ewing b4cfcd5c5e Merge pull request #18602 from lewing/wasm-debug-inspect
[wasm][debugger] Initial support for proxying flat sessions
2020-01-29 17:05:43 -06:00
Alexander Köplinger 5e3e34220c [corlib] Split corlib xunit tests even more for iOS (#18619)
Follow-up to https://github.com/mono/mono/pull/18228 since two parts wasn't enough to fix the size issue on 32bits devices.
We now use three parts.

Fixes https://github.com/mono/mono/issues/18614
2020-01-29 21:28:06 +01:00
Aleksey Kliger (λgeek) 81f184c2ad [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP (#18567)
* [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP

Only prevent `gdb` or `lldb` from being invoked.

MERP is controlled by separate mechanisms --- configure flags for crash
reporting and an explicit opt-in icall.

The existing code collects crashed process data in-process, but it writes the
data out (using mono_merp_invoke) in the forked child process.  This is a bit
surprising, but in the interest of not disturbing the code too much, this PR
doesn't change that.  We should revisit `dump_native_stacktrace` again in the
future and rationalize these decisions.

Addresses https://github.com/mono/mono/issues/18565
2020-01-29 10:03:25 -05:00
Vlad Brezae 60b28b3b56 [interp] Fix virtual calls in mixed mode (#18526)
* [interp] Enable virtual calls to compiled code

If we have an aot-ed version of a method we should always go through it, because the IL for that method could be stripped.

* [interp] Disable jit call for wrappers

Some generic wrappers don't seem to work in mixed mode, probably due to the fact that the transition already reuses a gshared wrapper.
2020-01-29 00:01:58 -08:00
Larry Ewing c75a1364c5 [wasm][debugger] use PdbReaderProvider helper to try to load more pdb forms 2020-01-28 22:32:53 -06:00
Larry Ewing c6e9369de2 Merge remote-tracking branch 'upstream/master' into checkpoint 2020-01-28 22:17:07 -06:00
Larry Ewing edfe20ffc7 [wasm][debugger] start reworking logging logic 2020-01-28 18:53:36 -06:00
Larry Ewing 0fe46d8d35 [wasm] simplify EndOfMessage case 2020-01-28 18:53:36 -06:00
Larry Ewing a6ebd5cb38 Rename Ws* to something more reasonable 2020-01-28 18:53:27 -06:00
Egor Bogatov 883162e861 Use memmove in Buffer.MemoryCopy (#18547)
* Use memmove in Buffer.MemoryCopy

* add test case
2020-01-28 20:00:30 +01:00
Martin Baulig 7d2924adcb [wasm][debugger-proxy]: If we cannot load a PDB file, send a log entry. (#18601) 2020-01-28 12:31:50 -05:00
Nikita Voronchev e2b632e4a7 [WinForms] Incorrect Tree Nodes drawing at the OwnerDrawText mode (#18605)
Assume we have the following form `MyForm`:
```
using System;
using System.Drawing;
using System.Windows.Forms;

namespace test_TreeView
{
    public class MyForm : Form
    {
        TreeView treeView = new TreeView()

        public MyForm()
        {
            treeView.Nodes.Add("root node");
            treeView.Nodes[0].Nodes.Add("first child node");

            treeView.BackColor = Color.Gray;
            treeView.LineColor = Color.Yellow;

            treeView.DrawMode = TreeViewDrawMode.OwnerDrawText;
            treeView.DrawNode += treeView_DrawNode;

            Controls.Add(treeView);
        }

        private void treeView_DrawNode(object sender, DrawTreeNodeEventArgs e)
        {
            var brush = new SolidBrush(Color.FromArgb(100, 255, 0, 0));
            e.Graphics.FillRectangle(brush, e.Bounds);
        }
    }
}
```
The displayed result is
![image](https://user-images.githubusercontent.com/12382656/73263417-92872980-41e1-11ea-8f7c-7cecbb376b60.png)
One can see couple drawing issues:
* White rectangles under each `TreeNode`.
* The area of custom drawing (red rectangles) is shifted with respect to this white rectangles.

Current PR proposes the fix of this issues.
Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:38:27 +01:00
Karl 1bfd317362 [Winforms] Form fixes (#18427)
* Fix Return handling. Ensure correct ClientSize.

When pressing Return / Enter, ensure any target button is both visible
and enabled.

In CreateHandle(), ensure the ClientSize is correct if set. If not,
update it to make it right. Suspend layout while changing it so that
child controls with anchors other than Top Left are not moved.

* Select active control. PerformLayout after updating for client size.

Select the active control with SendControlFocus to avoid upsetting what
is currently selected.

* Improve Form window decoration adjustment handling

Reverts the incorrect attempted fix contained in previous commits.

Basically, when a property is changed that affects window decorations,
ensure the client size stays the same.

* Use UpdateFormStyles on FormBorderStyle too.
2020-01-28 16:32:51 +01:00
Karl 2fdafcf3a4 Don't select ListBox item if SelectionMode == None (#18492)
When the user types a letter or digit, KeySearch looks for and selects
an item starting with that character. However, it should not if
SelectionMode is None.
2020-01-28 16:31:18 +01:00
Nikita Voronchevandmonojenkins c191dba87d [WinForms] Make ContextMenuStrip behavior closer to the reference one (#18598)
* [WinForms] Make `ContextMenuStrip` behavior closer to the reference one

* Bump API snapshot submodule

Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:30:46 +01:00
Alexander Köplinger 726a9134c8 Merge pull request #18491 from PreferLinux/RTF+Textbox-work
I've implemented a heap of missing RTF and RichTextBox stuff, and also made some improvements or fixes to various aspects of it. I'm sorry this is so big...

To give an overview of the changes, here's a list:
- Removed the RTF Charcode / Charset handling that wasn't actually used, had a lot of missing functionality and was basically reimplementing character encoding handling, and replaced it with ensuring the correct character encoding is being used (changing when needed).
- Improved RTF handling of objects with object results (process the given object result). Sometimes pictures can be like this.
- Improved line wrapping and alignment
- Added or greatly improved support for:
  - paragraph spacing
  - line spacing
  - paragraph indents (left, right, hanging)
  - tabstops (left, centre, right, decimal)
  - text background colour (like highlighting)
  - superscript / subscript
  - character offset (vertical)
  - text visibility
- Significantly improved RichTextBox RTF generation
- Cache DPI, rather than doing `CreateGraphics()` every time it is needed
- A variety of other fixes and improvements, with a few I can think of relating to:
  - caret position
  - replacing text
  - invalidation and drawing
  - when tags are added or deleted
2020-01-28 16:26:36 +01:00
monojenkins b87c271b6f [csproj] Update project files 2020-01-28 15:26:03 +00:00
monojenkins e4d69eb248 Bump API snapshot submodule 2020-01-28 15:24:53 +00:00
Steve Pfister 4b4b297450 [wasm][debugger] Handle When BreakpointRequest Commands Come Out of Order
BreakpointRequest commands can come out of order and before the runtime is ready (RuntimeReady). A null check for the DebugStore was added to guard against this since these requests seem to be for JS breakpoints. These seem to be able to be safely ignored.

Fixes https://github.com/mono/mono/issues/18584
2020-01-28 09:57:40 -05:00
Cody Russell a19aaa31da Bump bockbuild for Pango patch (#18566)
Fixes bug #1048838, correct SF font not loading in VSMac
2020-01-28 13:11:04 +01:00
Larry Ewing 4b620fa84d [wasm][debugger] Make the file loading slightly more async 2020-01-27 19:14:38 -06:00
Larry Ewing e1ec438c44 [wasm][debugger] Track the sessionId with all events and commands 2020-01-27 19:14:30 -06:00
Ryan Lucia a5fe7852de [utils] Default to g_print in mono_counters_dump if the stream is NULL (#18582) 2020-01-27 13:24:53 -05:00
Ryan Lucia f41680e395 [eglib] Use the stdout handler with g_printf (#18581)
This ensures it works properly on Android.
2020-01-27 11:57:11 -05:00
Nikita Voronchev ef05316107 [WinForms] Display caret while click on ComboBox (#18523)
If one click on a `ComboBox` the caret does not appear. This is so because `document.PositionCaret` call hides the caret.
2020-01-27 17:00:33 +01:00
Steffen Kieß 1fd0748362 Fix UnixEncoding.GetBytes() for empty strings (#18520)
Current versions of the mono return a null pointer as address of the first
element for empty arrays, i.e.

    fixed (byte* ptr = new byte[0]) Console.WriteLine (ptr == null);

will print "true". This causes a problem in the UnixEncoding class where it
triggers a check for null pointers. Fix this by allowing null pointers when
the number of elements is zero.
2020-01-27 17:00:16 +01:00
dotnet-maestro[bot] 66ff74a1f3 Update dependencies from https://github.com/dotnet/arcade build 20200126.3 (#18597)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20076.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20076.3
2020-01-27 16:53:25 +01:00
felixdoerre 6358345667 Fix open port for X11 applications (#18583)
Whenever mono opens an X11 window, the application opens a tcp socket for controlling the (non-tcp)-connection to X11. This is bad practice, as a tcp socket is not needed.

This pull request cleans the logic, by replacing the tcp socket with a fifo. That way this loopback connection is not visible from outside the .NET application, and all logic surrounding it should remain unchanged.
2020-01-27 14:32:15 +01:00
Karl ba3b6a169b [Winforms] X11 Fix bounds change from within bounds-related events (#18513)
The real change here is, in `XplatUIX11.SetWindowPos()`, simply moving the `SendMessage` call to the end instead of earlier. The problem with it earlier is that any change in bounds from any of the bounds-related events (OnLocationChanged / OnMove, OnSizeChanged / OnResize, OnLayout, OnClientSizeChanged) will be overridden. When `SendMessage` is called, `UpdateBounds` is run, and that raises those events as relevant. After those events finish, `SetWindowPos` continues with setting the size of the window to the value originally specified – which is now potentially wrong. So send the message later on.

There is also no point that I can see in writing the values to `hwnd` twice, because as far as I can see the only one that can be changed is `hwnd.ClientRect` (set by `PerformNCCalc`).
2020-01-27 13:41:00 +01:00
Adrian 23b8f10cb8 Update locale tables - add Tatar CLDR files tt.xml and tt_RU.xml (Fixes #18524) (#18525)
Fixes #18524

The `tt.xml` and `tt_RU.xml` files were added in CLDR 34: http://cldr.unicode.org/index/downloads/cldr-34

Upgrading to v34 (or the latest v36) introduced a larger number of new cultures.  I went with the smallest change possible to have Tatar added.
2020-01-27 11:46:23 +01:00
Phil Jaenke 58b3ac9ad0 Fix for -mminimal-toc detection, enable BTLS on FreeBSD PowerPC (#18578)
Fixes #18554 by making autoconf actually check if `-mminimal-toc` works instead of assuming on powerpc. Tested with FreeBSD 12.x and 13.x on powerpc64 ELFv2.
While we are in here, also enable BTLS which has passed testing.
2020-01-27 11:20:54 +01:00
Vincent Povirk 21e2ebcb06 Allow passing IntPtr values with UnmanagedType.AsAny. (#18544)
This is used by the SplashScreen class of WindowsBase in .NET Core to pass a NULL pointer.
2020-01-27 11:18:55 +01:00
Pranav K c8516b2848 Remove unused variables (#18579)
These show up as build errors in Blazor when they're copied as sources. Seemed trivial enough to fix.
2020-01-27 11:17:44 +01:00
abrevet-dev ed88c509dd [WinForms] Fix #18506 ActiveTracker, do not propagate message to control when it is not enabled and visible. (#18541)
Fixes #18506
2020-01-27 11:07:58 +01:00
Ankit Jain a6a89a5ef4 [master] Bump msbuild to track xplat-master (#18575) 2020-01-27 11:00:26 +01:00
Jérôme Laban d23da5628e [Wasm] Fix threading support file in packager (#18593) 2020-01-26 22:42:52 -05:00
monojenkinsandZoltan Varga c9a97e5258 [runtime] Disable lldb backtrace display on osx, it hangs on attaching in lldb. (#18590)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-26 14:31:07 -05:00
Thays GraziaandAleksey Kliger f8c34dc56b [debugger] Access invalid memory address using PointerValue Command. (#18537)
* Validate the address that came from IDE using PointerValue. The IDE can send an invalid address and it was crashing mono.

Fixes #18191
Fixes #15612

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
2020-01-26 00:00:41 -03:00
Maher Jendoubi 4704c85c9d Fix typos (#18585)
* Contributing: fix typos

* fixed encoding and other typos

* fixed encoding and spotted other typos

* Update TransformationRules.cs
2020-01-25 19:55:22 +01:00
Zoltan Varga 9fc4cae875 [jit] Compute has_references correctly for gshared types whose constraint is a generic valuetype. (#18552)
Also emit write barriers correctly for these types.
2020-01-25 09:20:31 -05:00
Zoltan Varga 9e377af674 [wasm] Fix netcore build. (#18564)
* [wasm] Fix netcore build.

Add a DOTNET_REPO_DIR variable to Make.config which should point to the dotnet/runtime repo. Compile
the netcore runtime and corelib from this repo.

* Disable wasm+netcore on CI.
2020-01-25 06:29:58 -05:00
monojenkinsandZoltan Varga 653db97098 [runtime] Avoid loading ThreadAbortException class on netcore. (#18563)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 20:13:10 -05:00
Phil Jaenke 85b34350c6 Fix for failing compile-runtime-tests (#18569)
compile-runtime-tests is looking for python3, but $(env python3) will return error (no `/usr/local/bin/python3`); if this is the case, create a symlink from 3.6 or 3.7 to /usr/local/bin/python3 so it works. We already have this listed in `SHEBANG_FILES` for the port.
2020-01-24 18:16:44 -05:00
monojenkinsandZoltan Varga 0215d2e331 Initial CMAKE build system support. (#18561)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 17:00:49 -05:00
Radek Doulik 14570f14ac [aprofutil] Update the tool name 2020-01-24 10:04:02 +01:00
mdh1418 ae9335ac21 [mcs] Fix support for digit separators (#18539)
* [mcs] skip "_" when building number

* [mcs] Refactor digit separator feature report

* [mcs] Clean up digit separator code

* [mcs] Throw errors for invalid digit separator usage

* [mcs] Fix hex and bin usage of digit separator tests

* [mcs] Add digit separator usages that throw errors

* [mcs] Update il
2020-01-24 09:18:32 +01:00
Jay Krell 8d69965919 Remove "handles" from System.Threading.Semaphore. (#18512)
Actually, there were no handles.
This reduces MonoError.
2020-01-23 13:58:39 -05:00
monojenkinsandYoussef Victor 05da2da407 Fix typos (#18546)
Found more occurrences of the typos in dotnet/runtime#2061

The work done here is made by simple search & replace.

I'll point to replaces that needs attention in a review.

Co-authored-by: Youssef Victor <[email protected]>
2020-01-23 19:40:58 +01:00
Ankit Jain cc6e927d10 [bockbuild] preinstall: kill any running VBCSCompiler processes (#18488)
- roslyn's shared compiler server hashes it's pipename on the path to
the assembly.
- On macOS, this looks like `/Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/VBCSCompiler.exe`
- When installing a new mono, with that same `x.y.z` version, this
process will effectively be out of sync with the new updated
mono/msbuild/roslyn
- This manifests as:

```
  Using shared compilation with compiler from directory: /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn
  /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : Roslyn compiler server reports different hash version than build task. [/Users/builder/azdo/_work/3/s/xamarin-android/build-tools/xaprepare/xaprepare/xaprepare.csproj]
```

- See https://github.com/xamarin/xamarin-android/pull/4108

- So, we kill any running roslyn shared compiler servers
2020-01-23 19:34:26 +01:00
Thays Grazia 6b475d5d00 [debugger] Native thread not executing managed code considered as terminated (#18501)
* If a thread was suspended and doesn't have any managed stack, it was considered as terminated, but it wasn't really terminated because it can execute managed code again, and stop in a breakpoint so if the execution arrives in debugger_agent_breakpoint_from_context we reset the flag terminated to FALSE.

* Adding terminated = FALSE in single_step_from_context.
2020-01-23 12:41:10 -03:00
Fan Yang 4d9832a399 Add disable_omit_fp support for LLVM (#18489)
Fixes #18417
2020-01-23 16:31:14 +01:00
Alexander Köplinger 0f1ac3e5bb Disable netcore mono builds that now live in dotnet/runtime (#18549) 2020-01-23 14:45:10 +01:00
Zoltan VargaandVlad Brezae 4586c4f091 [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype. (#18496)
* [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype.

Fixes https://github.com/mono/mono/issues/18455.

* [jit] Simplify code

Fix some line damage

Co-authored-by: Vlad Brezae <[email protected]>
2020-01-23 07:57:36 -05:00
Bernhard Urban-Forster 082662cc8a [interp] context can be uninitialized for get_resume_state callback (#18533)
`jit_tls->interp_context` gets initialized lazily, that is, upon the first interpreter execution on a specific thread (e.g. via interp_runtime_invoke). However, with mixed mode the execution can purely happen in AOT code upon the first interaction with the managed debugger.

Stack trace:

```
  thread #1, name = 'tid_407', queue = 'com.apple.main-thread'
    frame #0: 0x0000000190aedc94 libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x0000000190a0f094 libsystem_pthread.dylib`_pthread_cond_wait$VARIANT$armv81 + 672
    frame #2: 0x000000010431318c reloadcontext.iOS`mono_os_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-os-mutex.h:219:8
    frame #3: 0x0000000104312a68 reloadcontext.iOS`mono_coop_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-coop-mutex.h:91:2
    frame #4: 0x0000000104312858 reloadcontext.iOS`suspend_current at debugger-agent.c:3021:4
    frame #5: 0x000000010431be18 reloadcontext.iOS`process_event(event=EVENT_KIND_BREAKPOINT, arg=0x0000000145d09ae8, il_offset=0, ctx=0x0000000149015c20, events=0x0000000000000000, suspend_policy=2) at debugger-agent.c:4058:3
    frame #6: 0x0000000104310cf4 reloadcontext.iOS`process_breakpoint_events(_evts=0x000000028351a680, method=0x0000000145d09ae8, ctx=0x0000000149015c20, il_offset=0) at debugger-agent.c:4722:3
    frame #7: 0x000000010432f1c8 reloadcontext.iOS`mono_de_process_breakpoint(void_tls=0x0000000149014e00, from_signal=0) at debugger-engine.c:1141:2
    frame #8: 0x000000010430f238 reloadcontext.iOS`debugger_agent_breakpoint_from_context(ctx=0x000000016f656790) at debugger-agent.c:4938:2
    frame #9: 0x00000001011b73a4 reloadcontext.iOS`sdb_breakpoint_trampoline + 148
    frame #10: 0x00000001008511b4 reloadcontext.iOS`reloadcontext_iOS_Application_Main_string__(args=0x000000010703a030) at Main.cs:14
    frame #11: 0x00000001010f9730 reloadcontext.iOS`wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 272
    frame #12: 0x00000001042fd8b8 reloadcontext.iOS`mono_jit_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at mini-runtime.c:3162:3
    frame #13: 0x0000000104411950 reloadcontext.iOS`do_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at object.c:3052:11
    frame #14: 0x000000010440c4dc reloadcontext.iOS`mono_runtime_invoke_checked(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, error=0x000000016f656ff8) at object.c:3220:9
    frame #15: 0x0000000104415ae0 reloadcontext.iOS`do_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5184:3
    frame #16: 0x00000001044144ac reloadcontext.iOS`mono_runtime_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5281:9
    frame #17: 0x0000000104414500 reloadcontext.iOS`mono_runtime_run_main_checked(method=0x0000000145d09ae8, argc=1, argv=0x000000016f6570d0, error=0x000000016f656ff8) at object.c:4734:9
    frame #18: 0x00000001042d3b54 reloadcontext.iOS`mono_jit_exec_internal(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1320:13
    frame #19: 0x00000001042d39a4 reloadcontext.iOS`mono_jit_exec(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1265:7
    frame #20: 0x0000000104597994 reloadcontext.iOS`::xamarin_main(argc=5, argv=0x000000016f657a80, launch_mode=XamarinLaunchModeApp) at monotouch-main.m:483:8
    frame #21: 0x00000001008510dc reloadcontext.iOS`main(argc=5, argv=0x000000016f657a80) at main.m:104:11
    frame #22: 0x0000000190af8360 libdyld.dylib`start + 4
[...]
* thread #5, name = 'Debugger agent', stop reason = signal SIGABRT
  * frame #0: 0x0000000190aedec4 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x0000000190a0d724 libsystem_pthread.dylib`pthread_kill$VARIANT$armv81 + 216
    frame #2: 0x000000019095d844 libsystem_c.dylib`abort + 100
    frame #3: 0x00000001045871b4 reloadcontext.iOS`log_callback(log_domain=0x0000000000000000, log_level="error", message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", fatal=4, user_data=0x0000000000000000) at runtime.m:1213:3
    frame #4: 0x0000000104544fc8 reloadcontext.iOS`eglib_log_adapter(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", user_data=0x0000000000000000) at mono-logger.c:405:2
    frame #5: 0x000000010456093c reloadcontext.iOS`monoeg_g_logstr(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, msg="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n") at goutput.c:134:2
    frame #6: 0x0000000104560598 reloadcontext.iOS`monoeg_g_logv_nofree(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, format="* Assertion at %s:%d, condition `%s' not met\n", args="e\x12z\x04\x01") at goutput.c:149:2
    frame #7: 0x000000010456061c reloadcontext.iOS`monoeg_assertion_message(format="* Assertion at %s:%d, condition `%s' not met\n") at goutput.c:184:22
    frame #8: 0x0000000104560674 reloadcontext.iOS`mono_assertion_message(file="../../../../../mono/mini/interp/interp.c", line=7176, condition="context") at goutput.c:203:2
    frame #9: 0x000000010459b570 reloadcontext.iOS`interp_get_resume_state(jit_tls=0x000000014900d000, has_resume_state=0x000000016fc7a9f4, interp_frame=0x000000016fc7a9e8, handler_ip=0x000000016fc7a9e0) at interp.c:7176:2
    frame #10: 0x0000000104319420 reloadcontext.iOS`compute_frame_info(thread=0x0000000104fe4130, tls=0x0000000149014e00, force_update=1) at debugger-agent.c:3422:3
    frame #11: 0x0000000104320d40 reloadcontext.iOS`thread_commands(command=1, p="", end="", buf=0x000000016fc7acf8) at debugger-agent.c:9048:3
    frame #12: 0x000000010431cca0 reloadcontext.iOS`debugger_thread(arg=0x0000000000000000) at debugger-agent.c:10132:10
    frame #13: 0x000000010447eb04 reloadcontext.iOS`start_wrapper_internal(start_info=0x0000000000000000, stack_ptr=0x000000016fc7b000) at threads.c:1232:3
    frame #14: 0x000000010447e788 reloadcontext.iOS`start_wrapper(data=0x000000028203ef40) at threads.c:1305:8
    frame #15: 0x0000000190a11d8c libsystem_pthread.dylib`_pthread_start + 15
[...]
```

Thanks to @drasticactions for helping me to reproduce.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1050615
2020-01-23 13:52:15 +01:00
dotnet-maestro[bot] 4321ba1e3b Update dependencies from https://github.com/dotnet/arcade build 20200118.1 (#18514)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20068.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20068.1
2020-01-23 10:34:41 +01:00
Alexander Köplinger a6f2903d58 configure.ac: remove AC_SEARCH_LIBS for libintl (#18531)
This causes an additional dependency of mono on libintl which we didn't have before.
Reported by a user on gitter, it's problematic on macOS since it makes the mono binary depend on the libintl we ship in the Mono .pkg.

It was introduced in Mono 6.6 by https://github.com/mono/mono/commit/18e0ebfe89be0a175d2f904b9bb1ec6816daa318
2020-01-23 00:29:56 +01:00
Alexander Köplinger b278d88a5f configure.ac: Don't look for X11 in netcore mode 2020-01-22 21:23:54 +01:00
Karl 3d168887a6 Store the current encoding code page, not encoding
The encoding itself could be null, while encoding_code_page
should always be valid.
2020-01-21 10:09:20 +13:00
Karl ff0605fe18 Use correct default encoding. 2020-01-21 10:04:57 +13:00
Karl d826bebd54 Calculate DPI before Recalculate. 2020-01-18 17:11:13 +13:00
Karl f39d652642 Fix NullReferenceException when parsing RTF.
charset_stack was never initialised.
2020-01-18 16:51:32 +13:00
Cody Russell c4aa327737 Bump bockbuild to get GTK NSPasteboardType change. (#18080) 2020-01-17 19:32:26 +01:00
Nikolay Sivov 0a96741f93 Compilation warnings fixes (#18495)
Compilation warnings fixes
2020-01-17 14:16:03 +01:00
Vlad Brezae 92bddddd7b [tests] Re-enable profiler test that used to fail (#18471)
It is not clear if it will fail again or not, but it is not reproducible.

Closes https://github.com/mono/mono/issues/8709
2020-01-17 13:10:59 +01:00
Bernhard Urban-Forster 73eeda130f [prof] remove wrong cast for format string (#18477)
[prof] remove wrong cast for format string

Fixes this warning:
```
  CC       mprof-report.o
mprof-report.c:408:62: warning: format specifies type 'char *' but the argument has type 'guint64' (aka 'unsigned long long') [-Wformat]
                        snprintf (format, sizeof (format), "%s : %%" PRIu64 "\n", (guint64)key_format);
                                                            ~~                    ^~~~~~~~~~~~~~~~~~~
                                                            %llu
/Applications/Xcode112.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:57:62: note: expanded from macro
      'snprintf'
  __builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), __VA_ARGS__)
                                                             ^~~~~~~~~~~
```

mistake in refactoring commit https://github.com/mono/mono/commit/07402062541fa9555bef99dbaac84cfb587926cd#diff-c825627ea9ddb3fa753fa3617eaec7c8R408
2020-01-17 11:05:50 +01:00
Aleksey Kliger (λgeek) 1f0e147cdd [netcore] Mark InitializeCurrentThread NoInlining (#18493)
Follow-up for https://github.com/mono/mono/pull/18480

We might want to replace calls to this with an intrinsic
2020-01-17 10:43:05 +01:00
monojenkins ed6a63bb25 [bcl] Update BCL Linked Size 2020-01-17 10:01:30 +01:00
Bernhard Urban-Forster b4c506c304 [runtime] fix warnings on linux (gcc 8.3.0) (#18479)
* [aot] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
aot-compiler.c: In function ‘emit_trampolines’:
aot-compiler.c:7675:28: warning: ‘_e’ directive writing 2 bytes into a region of size between 1 and 256 [-Wformat-overflow=]
    sprintf (end_symbol, "%s_e", symbol);
                            ^~
aot-compiler.c:7675:4: note: ‘sprintf’ output between 3 and 258 bytes into a destination of size 256
    sprintf (end_symbol, "%s_e", symbol);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At top level:
aot-compiler.c:753:1: warning: ‘link_shared_library’ defined but not used [-Wunused-function]
 link_shared_library (MonoAotCompile *acfg)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:558:1: warning: ‘mangle_symbol_alloc’ defined but not used [-Wunused-function]
 mangle_symbol_alloc (const char * symbol)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:535:1: warning: ‘mangle_symbol’ defined but not used [-Wunused-function]
 mangle_symbol (const char * symbol, char * mangled_symbol, gsize length)
 ^~~~~~~~~~~~~~~~~~~
```

* [amd64] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
tramp-amd64.c: In function ‘mono_arch_create_generic_trampoline’:
tramp-amd64.c:242:65: warning: variable ‘ex_offset’ set but not used [-Wunused-but-set-variable]
  int i, lmf_offset, offset, res_offset, arg_offset, rax_offset, ex_offset, tramp_offset, ctx_offset, saved_regs_offset;
                                                                 ^~~~~~~~~
```

* [lldb] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
lldb.c:243:1: warning: ‘buffer_add_buffer’ defined but not used [-Wunused-function]
 buffer_add_buffer (Buffer *buf, Buffer *data)
 ^~~~~~~~~~~~~~~~~
lldb.c:215:1: warning: ‘buffer_add_id’ defined but not used [-Wunused-function]
 buffer_add_id (Buffer *buf, int id)
 ^~~~~~~~~~~~~
lldb.c:208:1: warning: ‘buffer_add_long’ defined but not used [-Wunused-function]
 buffer_add_long (Buffer *buf, guint64 l)
 ^~~~~~~~~~~~~~~
lldb.c:188:1: warning: ‘buffer_add_short’ defined but not used [-Wunused-function]
 buffer_add_short (Buffer *buf, guint32 val)
 ^~~~~~~~~~~~~~~~

```

* [interp] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
interp/interp.c:3425:1: warning: ‘g_error_xsx’ defined but not used [-Wunused-function]
 g_error_xsx (const char *format, int x1, const char *s, int x2)
 ^~~~~~~~~~~
  CC       interp/libmono_ee_interp_la-mintops.lo
  CC       interp/libmono_ee_interp_la-transform.lo
  CCLD     libmono-ee-interp.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       libmono_dbg_la-debugger-agent.lo
  CC       libmono_dbg_la-debugger-state-machine.lo
  CC       libmono_dbg_la-debugger-engine.lo
  CCLD     libmono-dbg.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CCLD     mono-sgen
  AR       libmain.a
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       interp/libmaintest_a-whitebox.o
In file included from interp/whitebox.c:3:
interp/transform.h:163:12: warning: ‘stack_type’ defined but not used [-Wunused-variable]
 static int stack_type [] = {
            ^~~~~~~~~~
interp/transform.h:155:20: warning: ‘stack_type_string’ defined but not used [-Wunused-variable]
 static const char *stack_type_string [] = { "I4", "I8", "R4", "R8", "O ", "VT", "MP", "F " };
                    ^~~~~~~~~~~~~~~~~

```

* [thread state] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
  CC       libmonoutils_la-mono-threads-state-machine.lo
mono-threads-state-machine.c:33:1: warning: ‘get_thread_no_safepoints’ defined but not used [-Wunused-function]
 get_thread_no_safepoints (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
mono-threads-state-machine.c:26:1: warning: ‘get_thread_suspend_count’ defined but not used [-Wunused-function]
 get_thread_suspend_count (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
```

* [domain] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
appdomain.c:2415:1: warning: ‘get_app_context_base_directory’ defined but not used [-Wunused-function]
 get_app_context_base_directory (MonoError *error)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC       libmonoruntimesgen_la-assembly-load-context.lo
  CC       libmonoruntimesgen_la-domain.lo
domain.c:135:1: warning: ‘mono_domain_alcs_lock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_lock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~
domain.c:138:1: warning: ‘mono_domain_alcs_unlock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_unlock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~~~
domain.c:2129:1: warning: ‘mono_alc_free’ defined but not used [-Wunused-function]
 mono_alc_free (MonoAssemblyLoadContext *alc)
 ^~~~~~~~~~~~~
domain.c:2108:1: warning: ‘mono_domain_create_default_alc’ defined but not used [-Wunused-function]
 mono_domain_create_default_alc (MonoDomain *domain)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
domain.c:2084:1: warning: ‘create_alc’ defined but not used [-Wunused-function]
 create_alc (MonoDomain *domain, gboolean is_default)
 ^~~~~~~~~~

```

* [native library] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libmonoruntimesgen_la-native-library.lo
native-library.c:295:1: warning: ‘mono_loader_register_module_locking’ defined but not used [-Wunused-function]
 mono_loader_register_module_locking (const char *name, MonoDl *module)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* [sample] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libtestlib_la-test-mono-embed.lo
In file included from test-mono-embed.c:7:
../../samples/embed/teste.c:20:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 gimme () {
 ^~~~~

```

* fix for checked builds
2020-01-17 09:36:32 +01:00
Aleksey Kliger (λgeek) c1498b8d4c [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread (#18480)
* [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread

This function (and its call to mono_handle_new) was showing up in the
flamegraph for some async-heavy benchmarks
2020-01-16 21:57:54 -05:00
Aleksey Kliger (λgeek) 2ba25542e3 [interp] Add missing files to tarball (#18482) 2020-01-16 19:27:23 -05:00
Thays Grazia e2d8f5ee91 Revert "On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)" (#18490)
This reverts commit 61894d4a07.
2020-01-16 17:23:58 -05:00
Alexander Köplinger e4552f54fb Bump xsp to fix issues with removed Mono.Security APIs (#18478)
See https://github.com/mono/xsp/pull/88
2020-01-16 21:49:21 +01:00
Thays Grazia 61894d4a07 On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)
Fixes #18106
2020-01-16 17:34:22 -03:00
Alexander Köplinger bafdd7865f Bump corefx to disable two tests. (#18464)
HashTableTests.Ctor_Int_Int_GenerateNewPrime and BitConverterTests.ToString_ByteArrayTooLong_Throws allocate a lot of memory and this causes issues on Windows i386 and other platforms.
2020-01-15 23:52:44 +01:00
Bernhard Urban-Forster bdde94b8e2 [arm] account for signed bit when calculating offset to PLT entry (#18454)
[arm] account for signed bit when calculating offset to PLT entry

See `bl` documentation (make sure you look at the thumb2 one):
<img width="1003" alt="Screenshot 2020-01-14 at 22 43 33" src="https://user-images.githubusercontent.com/75403/72385601-2bb53b00-3720-11ea-9858-d73bc2d02a7e.png">


Fixes https://github.com/mono/mono/issues/18247
2020-01-15 18:35:53 +01:00
Aleksey Kliger (λgeek) e7e81a47c9 [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects (#18457)
* [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Match .NET Framework behavior.

Addresses https://github.com/mono/mono/issues/18388

* [tests] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Regression test for https://github.com/mono/mono/issues/18388
2020-01-15 10:47:27 +01:00
Maher Jendoubi f4567e83dd Contributing: fixing some typos (#18453) 2020-01-14 22:30:06 +01:00
Jay Krell 6eea4a18f0 Reduce some stack use and cleanup. (#17809) 2020-01-14 22:17:00 +01:00
Vlad Brezae fef06df537 [utils] Make sure we include WRAP_ATOMIC_INTRINSIC if needed
Currently, this doesn't impact any targets, but do it for consistency.

Fixes https://github.com/mono/mono/issues/10651
2020-01-14 22:03:48 +01:00
Vlad Brezae 7fb011aa5a [utils] Include missing define of MONO_GNUC_VERSION
We were adding the WRAP_ATOMIC_INTRINSIC all the time on arm64, due to missing MONO_GNUC_VERSION.
2020-01-14 22:03:48 +01:00
Jay Krell 0740206254 Fix many printf warnings (#16484)
* Do not `printf` `long long` or `unsigned long long`.

*Do* printf `gint64` or `guint64`.

Do not printf %ll.
Do not printf %z.

 gint64 ==  int64_t, which could be long, or long long, or maybe other
guint64 == uint64_t, ditto

For printing gint64/guint64, use PRId64, PRIx64, etc.
For printing gsize/gssize, use `G_GSIZE_FORMAT` (seems like excess `G`).

%z definitely would warn for every single use, with Mingw.

And %ll might also, since on 64bit platforms, gint64 tends to be
long, not long long, and would merit %l not %ll.
(No matter that %ll is merited on 32bit, and the behavior
would then be correct on both.).

If there becomes a system w/o PRId64 etc., and it is a 64bit system,
then we could use %l under a portable name.
Or %I64. This appears unlikely (unless reach is extended to older Windows runtime).

Note that PRIxPTR for gsize/intptr_t may be a good idea too.
2020-01-14 22:03:22 +01:00
Ryan Lucia 3b19228a66 [netcore] Implement Thread.GetCurrentProcessorId (#18450)
CoreCLR has a managed cache surrounding this, but for the moment the naive implementation will suffice. We can later move their managed cache to shared on top of this PR with minimal trouble.

We can't guarantee the id will be [0..CpuCount) for a variety of reasons, and a followup PR to the dotnet docs will clarify this. I've chosen not to intentionally add an offset, but no one should be relying on that behavior, and if someone thinks adding an offset would drive the point home the perf cost is irrelevant.
2020-01-14 22:01:13 +01:00
Bernhard Urban-Forster 05f2651a12 [interp] initial infrastructure for whitebox testing of optimization passes (#18268) 2020-01-14 21:59:48 +01:00
Vlad Brezae d9db79b4b2 [sgen] Fixes for max-heap-size option (#18446)
* [sgen] Don't subtract twice the nursery size from max_heap_size

We were subtracting the nursery size when initializing max_heap_size as well as adding it to the allocated_heap when allocating the nursery (alloc_nursery).

* [sgen] Trigger collection if failing to allocate los section

If the memory governor says we are exceeding the max_heap_size by allocating a los section (1MB) for a new los object, attempt to clear some space by triggering a major collection, before completely bailing out.
2020-01-14 11:42:14 -08:00
Jay Krell 5237696f65 [cxx] Make the externC of icalls configuration-dependent instead of always. (#17573)
There are pluses and minuses here:

 + Minimize externC.

 + Maximimze typesafe linkage.

 - Minimize configuration-sensitivity.

 - Maximize commonality across all configurations, so that
   success in one configuration maximally but not 100% implies
   success in others (which is why to avoid `#if` in general,
   to contribute toward compilation success).

 + Use higher level macros to explain reason.
   Even if ICALL_EXTERN_C was always externC (presently not),
   using it instead of plain G_EXTERN_C offers a builtin comment.
2020-01-14 19:28:54 +01:00
Egor Bogatov 08bb5660f6 Optimize typeof(T).IsValueType (#18307)
* Optimize Type.IsValueType

* Ignore gsharedvt types

* use mini_is_gsharedvt_variable_klass

* test

* fix build
2020-01-14 09:37:16 -06:00
Alexander Köplinger 55393c2c9b Add normalized .bat files after the .gitattributes EOL change 2020-01-14 15:52:06 +01:00
Jay Krell c32414966c Remove comment about unexplained C++/LLVM problem. (#18447)
The explanations/fixes were:
 Compile consistently as C++:
   https://github.com/mono/mono/commit/893486638cae8efc46323d6c2209fc99febcd05d

 Make inconsistent compilation safe since it was fragile:
   https://github.com/mono/mono/commit/b2817e6404d573e837e1d8c7e047135974a3b311
2020-01-14 14:20:15 +01:00
Jonathan Chambers 480bef7217 Add mono_gc_thread_detach callback. (#18408)
Add mono_gc_thread_detach callback for Boehm

Use under Boehm to detach without threads lock as Boehm will take it's own lock internally.

Note in `on_gc_notification` we take threads lock after Boehm already has it's own lock. For consistency and to prevent deadlocks always take lock ordering of Boehm then threads lock.
2020-01-14 12:56:09 +01:00
Jay Krell dc866c2aad [cxx] Fixes for all lanes to build as C++ but w/o configure.ac change. (#17452)
* [cxx] dlmalloc remove extern "C" and remove NO_MALLINFO.

* [cxx] Move CallInst bitfields to the end to minimize padding for alignment
and fix an unexplained LLVM problem.
2020-01-14 12:32:08 +01:00
Egor Bogatov f5e1bd3cdf Merge pull request #18293 from EgorBo/vt-gettype-boxing
Optimize boxing for var.GetType()
2020-01-14 14:04:21 +03:00
Zoltan Varga 71a1a84c19 [wasm] Pass --used-attrs-only true to the linker. (#18439) 2020-01-14 09:42:10 +01:00
Alexander Köplinger 53ade46065 [netcore] Fix build warning 2020-01-13 21:27:23 +01:00
Aleksey Kliger (λgeek) 6abab691ec [netcore] Require use of MonoClass getters in Debug builds (#18267)
1. Update `build.sh` to use `--enable-checked-build=private_types` which forces the use of getter methods to access `MonoClass` fields.
2. Fix up some places that were accessing MonoClass directly.
3. Get rid of a spurious comma operator usage from an earlier refactoring:
   ```c
      z = a, b, c;
   ```
   is the same as
   ```c
      z = a; b; c;
   ```
   which is the same as just `z = a;` if `b` and `c` have no side-effects.
2020-01-13 21:15:09 +01:00
Alexander Köplinger a5cc4cd9ec Write version.h for netcore and allow preview VS for msvc builds 2020-01-13 20:47:35 +01:00
Karl 8bd438f318 [Winforms] Calculate GroupBox preferred size the same way as Panel (#18429)
GroupBox's GetPreferredSize() is not great. I've simply copied the code from Panel, and used that. It uses LayoutEngine.GetPreferredSize(), which does things a lot better than the code this used.
2020-01-13 17:15:09 +01:00
Karl 4ee4d368c6 [Winforms] Always draw at least one line of a Label. (#18422)
Previously, if the Label wasn't quite high enough nothing would be drawn as the line didn't quite fit. It should draw the line anyway, clipped if necessary.
2020-01-13 17:12:36 +01:00
Karl 42956f554b [Winforms] Improve X11 keyboard handling. (#18428)
- Ignore key events with vkey == 0. They won't match a real key code so don't try. Besides, trying can cause segfaults on ChromeOS Linux mode.
- Check special-case keys (numpad) before looping to get them. No point leaving them for afterwards...
2020-01-13 17:12:10 +01:00
Karl 7a8242ed36 [Winforms] Don't have non-active controls focused (#18426)
- Make the state of Control.CanSelect depend on whether the control is visible and enabled.
- When setting Control.Visible to false, if this control is focused, move the focus to the next control.
2020-01-13 17:11:38 +01:00
Karl d7e84ac6d1 [Winforms] Implement correct Control.Region / ClipRegion support on X11 (#18421)
Fixes https://xamarin.github.io/bugzilla-archives/20/20233/bug.html

Previously, on Linux, setting Control.Region just set the clipping for painting, but didn't influence the visibility of other controls behind the one it was set on. This implements this correctly via X11 Shape extension.
2020-01-13 17:10:57 +01:00
Maxim Lipnin 87cfaf15ae [netcore] Make System.Diagnostics.TraceSourceTests.DefaultTraceListenerClassTests.EntryAssemblyName_Null_NotIncludedInTrace Pass (#18403)
* [netcore] Extract Assembly.GetEntryAssembly method to shared part
2020-01-13 17:06:55 +01:00
Johan Lorensson 407898621e Make sure .bat and .cmd files have crlf in wd. (#18437)
Looks like https://github.com/mono/mono/commit/c9dc20f59ebe7fbed2a439d69c2d03a2d4500577 broke Windows build for some scenarios using .bat files. Fix makes sure we convert .bat and .cmd files to use crlf in wd.
2020-01-13 15:01:41 +01:00
dotnet-maestro[bot] 49c5101200 Update dependencies from https://github.com/dotnet/arcade build 20200112.1 (#18438)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20062.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20062.1
2020-01-13 15:00:54 +01:00
Egor Bogatov 423192ed85 [netcore][llvm] Implement Sse1-4.2 subsets used by corlib (#18103)
* Implement SSE41 subset used by corlib

* Implement Sse2.MoveMask

* only works with llvm

* Implement Sse3 and Ssse3 subsets used by corlib

* Implement a few SSE1 methods

* Address feedback, also, implement Sse.Add/Subtract

* Fix build

* Implement Sse.Multiply, Sse.Store

* Implement Sse.CompareNotEquals

* Implement Sse.MoveScalar

* Finish SSE1 corlib subset

* Implement Sse.LoadVector128, Sse.Shuffle

* Sse.Shuffle cleanup

* Implement Sse2 APIs

* More of SSE2: LoadAlignedVector128, Compare*

* Implement Sse2.Unpack* and Sse2.StoreScalar

* Implement Sse2.PackUnsignedSaturate

* Implement Sse2.ShiftRightLogical

* Implement Sse2.Shuffle

* Implement Vector128<T>.Zero

* Fix CreateScalarUnsafe

* Implement Vector128.As*, Fix Vector128.CreateScalarUnsafe

* Fix failures

* Fix failures

* Fix Sse.MoveMask

* remove redundant null checks

* Fix AOT failures

* fix compilation warrning

* rename create_vector_mask_*

* Fix failures found via tests

* Index in Sse41.Insert has to be a constant

* add local tests for mono

* Update tests (cleanup)

* Code cleanup

* test

* fix typo

* Clean up

* Clean up

* Implement And, AndNot, Or, Xor, Divide for Sse1

* Cleanup

* Fix build

* limit emit_vector128 with LLVM

* enable IsSupported for corlib

* Fix build on wasm (if-defs issue)

* Don't intrinsify Vector256

* Address feedback
2020-01-13 13:51:51 +01:00
Karl e92fe19fa6 [Winforms] X11Keyboard LookupString fixes (#18423)
Two small fixes to X11 keyboard handling here:
- For Xutf8LookupString, use the actual buffer size rather than the starting buffer size. Using the previous fixed size made the buffer overflow handing code immediately after the change pointless.
- For XLookupString, handle similar to Xutf8LookupString, getting a byte array. The old was sometimes causing problems where input on the numeric keypad didn't enter text. I believe using a StringBuilder similar to how it was _should_ work, but for some reason I could not get it to work reliably. So I changed it to this, which seems to work well for me. The only question I have is whether it should be Encoding.ASCII like I used, Encoding.Default, or another specific encoding.
2020-01-13 12:19:36 +01:00
Zoltan Varga 4b718ca1f2 [jit] Fix an assertion in the Thread.VolatileRead () intrinsic. (#18430)
Fixes https://github.com/mono/mono/issues/18418.
2020-01-13 12:09:09 +01:00
Vlad Brezae a13f5ccbb6 [interp] Fix struct copying during pinvoke transitions (#18402)
If we are in pinvoke wrapper, we need to use the marshalled native size, not the managed size.
2020-01-13 12:05:25 +01:00
Zoltan Varga 09a7317e5a [wasm] Disable a test which OOMs. 2020-01-13 10:00:49 +01:00
Karl ff494e6065 Only calculate scrollbars once handle created 2020-01-13 17:54:00 +13:00
Karl b8f747e452 Significantly improve DPI caching.
Particularly, don't call CreateGraphics() in TextBoxBase's constructor.
2020-01-13 17:54:00 +13:00
Karl e9abb44242 Correct code style problems. 2020-01-13 17:54:00 +13:00
Karl c41ee8050e Improve font name and charset handling.
- Use alternative names as a fallback if present.

- Use the untagged name (\fname, name that doesn't
identify charset ) if present.

- Use font codepage or charset correctly to get the correct encoding.

Rip out the old nonfunctional RTF charset handling.
This was not used, and furthermore was fundamentally flawed.
I have replaced it by simply using the correct text Encoding, mentioned
above.
2020-01-13 17:54:00 +13:00
Karl 9b87a23ee6 Replace Document Height / Width Changed events with SizeChanged 2020-01-13 17:54:00 +13:00
Karl 150d5e209d Round font ascent / descent up in LineTag. 2020-01-13 17:54:00 +13:00
Karl 24c2e505a3 Fix updating to cover lines modified while suspended. 2020-01-13 17:54:00 +13:00
Karl e586084ae8 Rearrage slightly, fix kerning. 2020-01-13 17:54:00 +13:00
Karl 3d9e4fa7f4 Fix CharCount, CharIndexToLineTag, recalculate wrapped lines.
CharCount / length was inconsistent about counting line endings. Fixed
to include them all, as that was much easier than the other way.

When shortening a wrapped line, it is necessary to start the recalculate
at the previous line, in case the wrap point has changed.

In CharIndexToLineTag, if there is a line boundary at the specified
index, return the line starting at the index.

Fixed a few other things too.
2020-01-13 17:54:00 +13:00
Karl 11a89a53ab Recalculate after changing canvas. Fix scrolling. 2020-01-13 17:54:00 +13:00
Karl 0b6dd983dd Later RTF para overrides earlier. Emit \nosupersub.
RTF paragraph controls override earlier instructions, rather than
keeping the maximum.
Emit \nosupersub between superscript and subscript text. Missing it out
technically gives a superscript (or subscript) of the subscript (or
superscript), which is not what we want.
rather than keeping the maximum.
2020-01-13 17:54:00 +13:00
Karl df3b5a26b4 Improve indent and kerning handling. 2020-01-13 17:53:59 +13:00
Karl ca7352ad24 Add tabstop support. Use floats for spacings.
Floats are now used to preserve paragraph spacing and indents. The RTF
format uses twips (1/20") for these, which is usually less than 1px.
2020-01-13 17:53:33 +13:00
Karl b35c560df3 Lots of TextControl stuff, plus others.
* RTF.cs: Add support for JPEG and EMF pictures.
  Allow for attribute etc. groups inside the picture group.

* Line.cs: In InsertString, ensure that the tag passed in is currently
  in the line – not necessarily valid when the passed tag is the
  cursor, for instance.

* LineTag.cs: Only break tags for formatting when needed.

* TextControl.cs: Fix invalidating text that isn't left-aligned.
Fix caret stuff:
  Positioning after the line ending no longer happens
  Up and down when not left-aligned
  Tag used when moving left to start of line
  Page up / down
  Remove empty tags when navigating away from them
Change tag background draw location – I think it is a correction!
When inserting at a position without being given the tag, choose an
  empty tag at that position if one exists.
Change ReplaceSelection order back (corrected some formatting and
  similar stuff), and format the new text to match the tag.
Change IncrementLines to transverse the tree (using a new function)
  rather than repeatedly call GetLine.

* RichTextBox.cs: Don't streamline the line in case it removes tags we
  want to keep – we'll skip them ourselves instead.
Emit pictures.

* TextBoxBase.cs: Remove redundant check – a previous check means
  value != acutal_border_style, therefore at least one of them !=
  BorderStyle.Fixed3D, therefore the if condition was always true.
2020-01-13 16:42:44 +13:00
Karl 681b514e88 Use a float for the RTF font size to avoid rounding error when dividing by two to get the point size. 2020-01-13 16:42:44 +13:00
Karl dbf98ffc93 Add line spacing support. Move dpi to TextControl.
Line spacing seems to be working perfectly.
2020-01-13 16:42:44 +13:00
Karl f25ca77bfe Fix ReplaceSelection() again. Line spacing support. Fix drawing non-multiline selection.
It was inserting at the start, and then removing what had been inserted +/- whatever length as needed.
2020-01-13 16:42:44 +13:00
Karl e01e43025d Fix TextControl.ReplaceSelection to insert before removing so that setting SelectedText doesn't lose formatting. 2020-01-13 16:42:44 +13:00
Karl 1821cc8adf RichTextBox: Cache dpi
The dpi isn't likely to change during the lifetime of the object, so is relatively safe to cache.
Creating a Graphics object each time the dpi is wanted when reading RTF is also rather silly performance-wise.
Even worse, when generating RTF it can cause a recalc, which can mean that some lines <= end_line don't exist.
2020-01-13 16:42:44 +13:00
Karl be446f04f3 Three fixes in RichTextBox, one GenerateRTF, two ContentResized.
In GenerateRTF, check tag != null before trying to generate anything based on it. Not sure
why it hasn't been a problem before, but either way I was getting NullReferenceExceptions
because of it today.

Run ContentSizeChanged() when either width or height changes, irrespective of Multiline status.
This fixes various problems with stuff jumping around vertically.

Correct ContentSizeChanged to consider the left and right margins instead of the left margin twice.
Was probably a typo at an earlier stage, otherwise would have been a * 2.
2020-01-13 16:42:43 +13:00
Karl 42cecb6ed2 Fix line calculation for vertical positioning of tags.
* Line.cs: Fix handling of differing tag Ascents and CharOffsets in recalculateLine.

* TextControl.cs: Remove unneeded assignments.
2020-01-13 16:42:43 +13:00
Karl fee6509476 * Line.cs: Finished off indent support, also added tabstops.
* LineTag.cs: Added support for visibility. Corrected handling of tabs
  and added tabstop support.

* RichTextBox.cs: Added support for text visibility, indents,
  tabstops.

* TextControl.cs: Make invisible text work correctly. Fixed a bug with
  tag backcolor. Added support for indents.
2020-01-13 16:42:35 +13:00
Karl f98c1e0087 Fixed so that it sets measuredText to an empty rectange if not
measuring text.
2020-01-13 16:09:02 +13:00
Karl e4a5b4a720 * RichTextBox.cs: Align Justified as Left instead of as Center.
* TextControl.cs: Provide an output on PictureTag.Draw(... out
  Rectangle measuredText ...) if requested.
2020-01-13 16:08:29 +13:00
Karl cc91b61cea * RTF.cs: More work on ReadObjGroup
* RichTextBox.cs: Fixed up the insertion of pictures.
2020-01-13 16:06:41 +13:00
Karl aab3dca9fc All sorts of work, mostly on RichTextBox. 2020-01-13 16:03:11 +13:00
Zoltan Varga 73468eda91 Bump linker. (#18424) 2020-01-12 10:57:28 +01:00
Ankit Jain fc7b254953 [master] Bump msbuild to track xplat-master (#18432) 2020-01-12 10:56:21 +01:00
monojenkins ee3da3821b [bcl] Update BCL Linked Size 2020-01-11 08:58:48 +01:00
Martin Baulig a702e4ff02 [Sockets]: Always reset internal SAEA completion when reattempting connection in MultipleConnectAsync. (#18389)
This should fix #18030.
2020-01-10 19:01:36 +01:00
Zoltan Varga 3563bd90dc Revert "[wasm] Pass --used-attrs-only true to the linker. (#18401)" (#18415)
This reverts commit 6f4fe05209.
2020-01-10 16:49:43 +01:00
Ankit Jain 9666bcfae7 [master] Bump msbuild to track xplat-master (#18412) 2020-01-10 12:45:01 +01:00
Vlad Brezae d33a100961 [interp] Add gc unsafe transition during pinvokes (#18399)
Native methods can be run in gc safe mode, meaning we can be in gc safe mode while setting/getting the arguments for a native transition. Call context computation requires us to be in unsafe mode.

Fixes https://github.com/mono/mono/issues/16969
2020-01-10 09:32:04 +02:00
Zoltan Varga 5a65a97c58 [corlib] Mark the SRE types with required=false so they can be linked out. (#18379)
Currently SRE types are still kept by the reference in RuntimeType::MakeTypeBuilderInstantiation ().
2020-01-10 04:22:45 +01:00
Larry Ewing 439091eddf [wasm] use updated node to run jsvu (#18386)
* [wasm] use updated node to run jsvu

* [wasm] make the browser tests use the node we've installed

* [wasm] Fix browser tests to reference dotnet.js
2020-01-09 22:42:03 +01:00
Zoltan Varga 6f4fe05209 [wasm] Pass --used-attrs-only true to the linker. (#18401) 2020-01-09 19:12:16 +01:00
Alexander Köplinger c9dc20f59e Fix line endings in .bat files
They need to be CRLF otherwise we'll get conflicts when moving into dotnet/runtime which has a .gitattributes setting forcing .bat to CRLF.
2020-01-09 18:06:53 +01:00
Ankit Jain 46f834fda7 [master] Bump msbuild to track xplat-master (#18391) 2020-01-09 17:45:05 +01:00
Maxim Lipnin f5aca4235e [netcore][xunit tests] Re-enable System.Diagnostics.Tests.ProcessTests.ProcessStart_UseShellExecute_ExecuteOrder (#18397)
#14903
2020-01-09 17:01:57 +01:00
monojenkins e55a1a4f62 [bcl] Update BCL Linked Size 2020-01-09 14:21:06 +01:00
Vlad Brezae 345550fb99 [interp] Throw overflow exception when converting NaN (#18384)
Fixes https://github.com/mono/mono/issues/18061
2020-01-09 12:08:15 +02:00
Bernhard Urban-Forster 207cf805e5 [memory barrier] use pagesize determined by helper (#18383)
[memory barrier] use pagesize determined by helper

`limits.h` doesn't always define `PAGESIZE` properly (e.g. iOS), so it would fall back to `0x1000` (4096) but it's really `0x4000` (16384) on iOS. We already have a helper in the runtime to determine the pagesize, so let's use that instead.


Fixes https://github.com/mono/mono/issues/18273
2020-01-09 10:46:03 +01:00
Marek Safarandmonojenkins 9cc751d618 [corlib] Switch to CharUnicodeInfo from corefx (#18305)
* [corlib] Switch to CharUnicodeInfo from corefx

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-01-09 09:36:55 +01:00
Zoltan Varga 147c7972a8 [wasm] Reenable linker substitutions. (#18381) 2020-01-09 05:18:58 +01:00
Zoltan Varga 46fe5ad3f6 Bump linker. (#18380) 2020-01-09 05:18:17 +01:00
Martin Baulig ba7c00e4a9 Add workaround for #17710. (#18275) 2020-01-08 17:50:55 -05:00
Ryan Lucia 9fd95db929 [reflection] Fix check for Type.FullName to match CoreCLR (#18385)
Updated test to follow in dotnet/runtime
2020-01-08 17:02:25 -05:00
Ryan Lucia 1178ab3a1a [metadata] Propagate error in mono_runtime_try_invoke_array (#18382)
This was prompted by https://github.com/mono/mono/issues/18339, which seems to be failing to locate a native library and then hitting an assertion. We should propagate the error instead, so that if others run into something similar they can see what's actually going wrong instead of just getting a runtime crash.
2020-01-08 15:57:16 -05:00
Egor Bogatov ee5850a2b5 Update FieldInfoTest.cs 2020-01-08 23:21:41 +03:00
dotnet-maestro[bot] 5dbc923cb3 [master] Update dependencies from dotnet/arcade (#18206)
* Update dependencies from https://github.com/dotnet/arcade build 20191215.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19615.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19615.1

* Update dependencies from https://github.com/dotnet/arcade build 20191222.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19622.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19622.1

* Update dependencies from https://github.com/dotnet/arcade build 20191229.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19629.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19629.1

* Update dependencies from https://github.com/dotnet/arcade build 20200105.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20055.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20055.1
2020-01-08 20:47:44 +01:00
Jay Krell e8d38961ee [Coop] Remove handles from Mono.Interop.ComInteropProxy. (#16993) 2020-01-08 13:34:11 -05:00
Alexander Köplinger ba505510f2 FreeBSD: fix provisioning script to force reinstall perl5 at the end
It didn't work otherwise.
2020-01-08 16:46:58 +01:00
Zoltan Varga 34dee0ea4e [runtime] Initialize field info before accessing it in ves_icall_System_RuntimeFieldHandle_SetValueDirect (). (#18378)
Fixes https://github.com/mono/mono/issues/18364.
2020-01-08 16:39:39 +01:00
Ankit Jain d31f6ac4da [master] Bump msbuild to track xplat-master (#18376) 2020-01-08 15:01:03 +01:00
Zoltan Varga e999846088 [jit] Implement fast TLS access in gshared methods. (#18377)
Fixes https://github.com/mono/mono/issues/18370.
2020-01-08 13:32:52 +01:00
Zoltan Varga d0a3fb1b1b [wasm] Rename binaries mono -> dotnet. (#18373) 2020-01-08 03:04:55 +01:00
Ankit Jain aa87d3db6d [master] Bump msbuild to track xplat-master (#18372) 2020-01-07 20:56:41 +01:00
Bernhard Urban-Forster 6481e8f7a3 [interp] run regression with all combinations of available optimizations (#17644)
[interp] run regression with all combinations of available optimizations

Like we do for the JIT.
2020-01-07 19:35:54 +01:00
Zoltan Varga 3d4cd57a52 [wasm] Pass --explicit-reflection to the linker only in aot mode. (#18367) 2020-01-07 18:50:22 +01:00
Phil Jaenke f3f11d1081 FreeBSD: Fix for perl5.26 / perl5-5.30 conflict (#18368)
The base image appears to have perl5.26 which conflicts with perl5-5.30, resulting in the /usr/local/bin/perl binary not being placed by perl5-5.30, which results in failed builds. The `perl5.26` appears to be part of the base image and has no packages actually dependent on it.
2020-01-07 18:14:55 +01:00
Calvin Buckley 524a8298d0 Force Python 3.x from env in shebang lines (#18285)
* Fix byte/string incompatiblity in Python 3
2020-01-07 17:28:06 +01:00
Zoltan Varga ec7f4c68f9 [netcore] Remap String ctors to the String:Ctor () methods. (#18357) 2020-01-07 17:12:34 +01:00
Ankit Jain 71070d34c8 [master] Bump msbuild to track xplat-master (#18297)
This enables parallel build support with msbuild!
2020-01-07 16:21:51 +01:00
Calvin Buckley b3ce5bfa88 More disambiguation of Python in makefiles (#18284) 2020-01-07 15:44:03 +01:00
Vlad Brezae 376c46bcf3 [sgen] Fix xref computation with tarjan bridge (#18239)
* [sgen] Include also derived classes as bridges

When using `MONO_GC_DEBUG=bridge=` debug option

* [sgen] Fix xref computation with tarjan bridge

Between C# and java (on android) there are objects that live on both worlds. This means that there exists a C# object with a corresponding java object. The relationship between them is strong, meaning if C# object is alive then java object must stay alive, and vice-versa. We keep java bridge objects always alive through a GCHandle (on the java gc). When doing a C# collection we select all bridge objects that appear to be dead on the C# side. These objects are candidates for collection, assuming the java side has nothing against it. Before triggering a collection on the java side (following the mono gc) we switch all the strong gchandles to the java objects to be weak (for these objects that are candidate to be collected) and we recreate the reference graph from the C# side to the java side (if C# Bridge1 can reference C# Bridge2 then, on the java side, we add a reference from Java Bridge1 to Java Bridge2; this is done by adding to an array of references inside Java Bridge1). In order to minimize the amount of work that needs to be done on the java side, we compute the minimal amount of references that need to be added, by computing the strongly connected components of the object graph.

An optimized way to construct the SCCs and the xrefs is by using the tarjan algorithm (https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm). Our algorithm is non recursive (scan_stack emulates the recursive order of traversal in a dfs algorithm, while loop_stack is the stack used by the algorithm). A color represents an SCC. We have some optimizations in place where we might merge colors if they don't contain bridges, since the client only cares about SCCs containing bridge objects and the links between them. color_merge_array is used to keep track of all neighbors of a node until we are creating the scc for that node. It is populated when scanning all the refs inside an object (compute_low). All the colors in the color_merge_array will be cross references with that scc.

Before this commit we were only clearing the color_merge_array when creating an SCC. This is problematic because we could end up with xrefs inside of an SCC that belong to another SCC. Consider the simple graph of nodes 0,1,2,3 where 0 <=> 2, 0->1, 2->3. Assume we start scanning with node 0. When creating the SCC for node 3 the followed path is 0 -> 2 -> 3, while the loop stack will contain (0,1,2,3). After creating SCC for node 3, we will finish scanning node 2 which would detect the xref to Bridge3, which would have been added to the color_merge_array. Because node 2 is not the root of the SCC it belongs to (its lowlink points towards node 0 which has a lower index), we are not creating an SCC with it, and the link to node 3 remains in color_merge_array. Because the next node from the scan_stack is node 1, which is also the root of the SCC that it belongs to, we will create an SCC for it and wrongly add the node 3 reference from color_merge_array to it. In order to fix this issue, we will always clear the color_merge_array once we finished scanning the xrefs for a node. If the node in question is not the root of an scc, then we will remember them as xrefs pointing out from this object. When we finally reach node 0 (which will be the root of the SCC containing nodes 0 and 2), we will then know that all xrefs for this color are the union of the xrefs of all objects belonging to this color (which represents the objects that we are popping from the loop_stack until we encounter the root node).

Even though this change adds required bookeeping for xrefs, I didn't notice any change in performance on the bridge tests that we have in mono/tests.

* [sgen] Some logging improvements in tarjan bridge

* [sgen] Disable optimization when comparing bridge outputs

When this optimization is enabled, the tarjan bridge will create more SCCs in order to reduce amount of xrefs in the graph. This would render the `bridge-compare-to` debug flag unusable with tarjan bridge.
2020-01-07 16:31:49 +02:00
Jay Krell 2129ac6c28 Set thread pool thread name just based on string pointer check. (#16637)
* Set thread pool thread name just based on string pointer check.

When the generation-based solution went in, there were not
yet constant thread names, so a pointer based approach could fail
due to reuse. We can do slightly simpler now.

* Remove the generation field which is no longer needed.
Remove the duplicated knowledge of how to set a constant and put
it behind a funny sounding but perhaps reasonable flag.
Revise corlib version.
2020-01-07 09:01:22 -05:00
EgorBo 322ba565b6 disable a test 2020-01-07 17:00:23 +03:00
Steve Pfisterandmonojenkins 0ccf503f0a Add SignedCms to the TypeForwards in the System.Security.Cryptography.Pkcs facade (#18325)
* Add SignedCms to the typeforwards in the System.Security.Cryptography.Pkcs facade

* Added additional missing types from System.Security.Cryptography.Pkcs.

* It would be nice if the files I added had namespaces ;-)

* Bump API snapshot submodule

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>

Fixes https://github.com/mono/mono/issues/18323
2020-01-07 14:03:15 +01:00
EgorBo e6c0cdae0c Address Zoltan's feedback (enable for AOT) 2020-01-07 15:18:51 +03:00
EgorBo 1c5a295082 Merge branch 'master' of github.com:mono/mono into vt-gettype-boxing 2020-01-07 14:50:16 +03:00
Marek SafarandBernhard Urban-Forster adcd62248d [System.Private.CoreLib] Cleanup intrinsic tracking (#17884)
* [System.Private.CoreLib] Cleanup intrinsic tracking

1. Removed icalls where intrinsic version for the method exists

2. Add [Intrinsic] attribute to methods which have runtime intrinsic
   to make them easier to track and to tell linker not to analyze them

3. Use recursive syntax for any intrinsic which can be called via
   reflection to correctly apply the intrinsic when invoked.

```c#
public static bool IsSupported { get => IsSupported; }
```

When one calls X86.IsSupported, the call gets replaced by an
intrinsics. When one calls it with an Invoke, the body of IsSupported
gets replaced by intrinsics. It's just a hack to get Invoke support for free.

* [interp] implement System.Type::op_Equality

* Undo changes which are not yet supported by interpreter

* [interp] Implement System.Runtime.CompilerServices.RuntimeHelpers::OffsetToStringData

Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-01-07 11:17:06 +01:00
Zoltan Varga 487dacce79 [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods. (#18334)
* [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods.

Fixes https://github.com/mono/mono/issues/18276.

* Fix compilation with mcs.
2020-01-07 02:57:10 +01:00
Jay Krell 6a8ca2252b Do not depend on gboolean to have multiple bits. (#17825) 2020-01-06 16:11:21 -05:00
Zoltan Varga 384db9e890 [netcore] Optimize RuntimeType::IsSZArray. (#18347) 2020-01-06 16:36:59 +01:00
Zoltan Varga 7236dadd32 [netcore] Mark SRE types with required=false so they are only retained when they are actually used. 2020-01-06 15:44:50 +01:00
Jo Shields 4c9e28b80a Don't run make install in boehm submodule (#17907)
Don't run `make install` in boehm submodule
2020-01-06 09:20:46 -05:00
Zoltan Varga 090e25ec2d [wasm] Rename the interp-sample targets to interp-hello. (#18342) 2020-01-06 07:40:40 +01:00
Zoltan Varga 47424e34ae Bump emscripten to 1.39.5. 2020-01-05 21:45:10 -06:00
monojenkins 59a1eade7c [csproj] Update project files 2020-01-05 13:43:39 +01:00
monojenkins 36e89d52d1 Bump API snapshot submodule 2020-01-05 13:43:39 +01:00
Zoltan Varga 3acab49ce7 Bump linker+cecil. 2020-01-05 13:43:39 +01:00
Aleksey Kliger (λgeek) 07fee8a91d [metadata] Size 0 Blob heap is ok when resolving assembly refs (#18313)
* [metadata] Size 0 Blob heap is ok when resolving assembly refs

Sometimes ILasm can produce images with a Blob heap of size 0.  In cases where
we're loading assembly references from such an image, the hash (which is
optional) can be at index = 0 and the Blob heap size is also 0.

Also: add the hash value to the output of `monodis --assemblyref`

* [metadata] Add a separate assertion for the index == size == 0 case

And a comment explaining how it is likely to be triggered.  If the assembly is
reasonable the caller of mono_metadata_blob_heap should be updated to use
mono_metadata_blob_heap_null_ok instead

* [bcl] Allow Mono's ILASM to produce a size 0 Blob heap

ECMA 335 II.24.2.4 says that the user string and blob heaps should have an
entry at index 0 consisting of the single byte 0.  However .NET Framework (and
.NET Core) ilasm will entirely omit the Blob heap (that is, create a blob heap
entry of size 0) if it is not needed.  This PR changes Mono's ILASM to emit the
initial byte on demand only if one of the MetaDataStream.Add() methods is
called. Otherwise we will also emit a stream of size 0.

This is needed to compile some test cases.

* [tests] Add regression test for loading assemblies with size 0 Blob heap

Depends on ILASM that can emit a size 0 Blob heap
2020-01-03 10:42:44 -05:00
Zoltan Varga 22afdb80e1 [netcore] Add some linker exclusions. 2020-01-03 13:11:39 +01:00
monojenkins 95101b19fc [bcl] Update BCL Linked Size 2020-01-03 09:22:27 +01:00
Larry Ewing 14a3add47f [wasm][debugger] put the testharness base url in one place 2020-01-02 19:28:18 -06:00
Larry Ewing 11bf192d1a [wasm][deubgger] clean up task usage and remove delay 2020-01-02 19:28:18 -06:00
Marek Safar c9036e7555 Cleanup a few dependencies to reduce wasm bcl size 2020-01-02 14:24:44 -06:00
Ryan Lucia 33ca3d33e1 [loader] Add assembly name matching on netcore to fix DefaultContext test (#18272)
* [loader] Rename assembly name check strictness flag

* [loader] Add predicate for assembly name matching on netcore

* [loader] Enable LoadInDefaultContext test

* Make mono_assembly_names_equal external, change flags in domain search

* Move ifdef inside mono_loader_get_strict_assembly_name_check

* Fix pedump

* No reason to make this external

* Feedback
2020-01-02 15:21:45 -05:00
Egor Bogatov 458d80002d Add !align 8 to all managed allocators (calls) (#18283) 2020-01-02 20:45:40 +01:00
Jérôme Laban 52674e238a [Wasm] Use Path.Combine in Packager (#18332)
Improves Windows compatibility
2020-01-02 20:38:41 +01:00
Ryan Lucia ff6294d231 [reflection] Convert internal GetCustomAttributes calls to Attribute[] (#18176)
* [reflection] Avoid creating object[] in GetCustomAttributesBase

* [reflection] Migrate Attribute type checking to CustomAttribute.cs
2020-01-02 11:58:15 -05:00
Ryan Lucia 675f1eb321 [eglib] Fix whitespace in memory.c (#18320) 2020-01-02 11:56:50 -05:00
Thays Grazia e20956305e The behaviour expected of filter exceptions is: (#18309)
-> throw in a async task with a try catch in the caller -> don't stop
-> throw in a new thread with a try catch in the caller -> stop
-> throw in the main thread without any try catch -> stop

Including a new test to test the fix of #17601.

Fixing #16588, create a while 1 to stops the program execution when an exception is thrown and no try catch is found and MONO_DEBUG=suspend-on-unhandled.

Fixes #17601
Fixes #16588
2020-01-02 11:26:14 -03:00
Jonathan Nagy 5470386775 Set inited flag _after_ TermInfoDriver initialisation completes (#18318)
Fixes #18317
2020-01-01 19:02:48 -05:00
Steve Pfister 6f24832646 Pull in DirectoryInfo Invalidate After Create Fix (#18310)
* Bump to mono/corefx@bff1d77
2019-12-30 21:05:14 -05:00
Martin Baulig 400fb9c2cb Socket.BeginMConnect() should not attempt connections on unsupported address families. (#18302)
Fixes #16513.
2019-12-30 13:08:03 -05:00
Aleksey Kliger (λgeek) 61eb2be2e1 [eglib] Don't define _FORTIFY_SOURCE on Cygwin builds (#18308)
See https://github.com/msys2/MINGW-packages/issues/5803

Without -lssp (or -fstack-protector), -D_FORTIFY_SOURCE=2 leads to linker
errors.
2019-12-30 09:54:57 -05:00
imhameed 9719635e58 Use byte-aligned LLVM loads and stores for unaligned mini IR loads and stores. (#18296)
Should fix https://github.com/mono/mono/issues/18221.
2019-12-26 14:34:02 -05:00
EgorBo 28cae98ef0 Optimize box(T)+GetType 2019-12-25 02:32:46 +03:00
Aleksey Kliger (λgeek) ffdffa54f2 [domain] Don't add NULL runtime to runtimes list (#18260)
Also when iterating over the runtimes, actually look at the current element,
not the first element.

Also mention the unsupported runtime version in the warning message when it is
passed to mono_init_internal.

Fixes https://github.com/mono/mono/issues/17916
2019-12-23 10:44:32 -05:00
Fan Yang a1a4bcc49d Fix llvm jit assertion (#18280)
* Fix assertion when Alignment is greater than 32
2019-12-23 08:38:36 -05:00
Fan Yang 7ffcaaf59a Add cast to ((align) - 1) for Alignment macros (#18255)
* Add cast to ((align) - 1) for Alignment macros

* Add test for Alignment macro change
2019-12-23 08:34:37 -05:00
Alexander Köplinger 720767a9ff Fix invalid assetManifestPlatform in pipeline-netcore-runtime.yml
Spaces are note valid there.
2019-12-23 12:24:53 +01:00
Larry Ewing 1587c49f5f [wasm][debugger] Clean up startup and improve comments
Properly await more of the async logic and add retries when we get
an empty response from the "json/list" endpoint.
2019-12-20 21:41:05 -06:00
Larry Ewing 14b736e942 [wasm] resolve the full path of pdbs so that ../ doesn't break inclusion 2019-12-20 21:41:05 -06:00
Larry Ewing 04b9e5cb8f [wasm][debugger] rename the debugger target to run-debugger-tests 2019-12-20 21:41:05 -06:00
Larry Ewing 1669812689 [wasm][debugger] get the debug harness working again 2019-12-20 21:41:05 -06:00
Maxim Lipnin b55d8f9700 [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes (#18223)
* [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes

* [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate

* [wasm][xunit tests] Exclude XLinqTests.TreeManipulationTests.IEnumerableOfXNodeRemoveWithEvents test method
2019-12-20 19:18:51 -06:00
Ryan Lucia 0f8aac3c62 [loader] Check AppContext.BaseDirectory on netcore in preload hook 2019-12-20 19:17:01 -06:00
Ryan Lucia 57b6c8775b [loader] Remove unnecessary newlines in tracing 2019-12-20 19:16:11 -06:00
Ryan Lucia 5c87445310 [netcore] Re-enable some BinaryFormatter tests (#18277) 2019-12-20 19:52:53 -05:00
Alexander Köplinger cfba7c550f [netcore] Update dotnet sdk version in global.json (#18264)
Should unbreak the publishing build of netcore-mono.
2019-12-20 17:26:09 +01:00
Larry Ewing 47bca852c1 [wasm] fix the wrapped function signatures to match current documentation 2019-12-20 09:38:18 -06:00
Larry Ewing 5db64e27e9 [wasm] fix the mono_clear_bps condition 2019-12-20 09:38:18 -06:00
Aleksey Kliger (λgeek) d6f3de3c4c [tests] Add test files to tarball (#18261)
Some test sources weren't included.

Fixes https://github.com/mono/mono/issues/18202
2019-12-20 11:11:26 +01:00
Alexander Köplinger 7fd24a40bb Bump corefx to get test fix 2019-12-20 00:37:55 +01:00
Larry Ewing 83a898d25b [wasm] Make sure we're comparing the same date (#18251) 2019-12-19 15:35:05 +01:00
Alexander Köplinger c8e19bec68 [System] Add additional hardening against race in MonoTlsStream (#18250)
See https://github.com/mono/mono/issues/15805#issuecomment-567131297
2019-12-19 02:25:06 +01:00
Bernhard Urban-Forster e7b98d3587 [interp] add guard that a local hasn't been modified for stlocfld fusion (#18129)
[interp] add guard that a local hasn't been modified for stlocfld fusion

Fixes https://github.com/mono/mono/issues/18120
2019-12-19 02:05:57 +01:00
Nikita Voronchev 0beb9611df [WinForms][UIA] Add new internal event to track form's owner change (#18117)
This PR is needed for https://github.com/mono/uia2atk/pull/8
2019-12-18 22:24:06 +01:00
Alexander Köplinger a74416d950 [corlib] Move a few more test cases to part2, turns out part1 was still too large
Follow up to 3891fe97.
2019-12-18 22:18:33 +01:00
Alexander Köplinger 3891fe97d2 [bcl] Split corlib and System.Core xunit test assemblies for monotouch profiles (#18228)
They cannot deal with large test assemblies so we need to split them.

We're doing this by introducing a new `#skip <PROFILE>` directive in the .sources format which defines where to split the assembly.
The logic is very simple: files up until the first `#skip` are common files which get included in each part, subsequent files are only in the respective part assembly.

Fixes https://github.com/mono/mono/issues/18213
2019-12-18 21:11:17 +01:00
Larry Ewing 2ff93e4c76 [wasm] Fix test case spelling (#18242) 2019-12-18 13:51:21 -06:00
Maxim Lipnin 56f397bef3 [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate (#18234) 2019-12-18 06:49:45 -06:00
Larry Ewing 60f081c077 [wasm] handle completed Tasks and exceptions when resolving Task prom… (#18215)
* [wasm] handle completed Tasks and exceptions when resolving Task promises

* Preserve the old void invoke behaviour
2019-12-17 22:47:23 -06:00
Larry Ewing f46d5fda1d [wasm][debugger] fix exception when we can't locate debug assembly (#18230) 2019-12-17 21:37:22 -06:00
Nathan Ricci ed7432d988 Change ICONST to I8CONST when propagating constants through sign extension. (#18222)
Change ICONST to I8CONST when propagating constants through sign extension.
2019-12-17 21:35:56 -05:00
Evgenii Stepanov 06673e723e Relax assert in mono_arch_build_imt_trampoline. (#18195)
This assert fails when the trampoline code size worst case is realized.

Observed on a Android device with HWASan [1] enabled, where heap
addresses have a non-zero tag in the most significant byte, and thus
require 4 instructions to materialize in emit_imm64.

[1] https://source.android.com/devices/tech/debug/hwasan
2019-12-17 01:18:53 +01:00
Maxim Lipnin df3837eb7d [wasm][xunit tests] Re-enable System.Buffers.Text.Tests.ParserTests (#18220) 2019-12-17 01:18:34 +01:00
Zoltan Varga aa9227f7c8 [llvm] Generate correct LLVM IR for OP_XEQUAL. (#18217) 2019-12-17 00:52:02 +01:00
Zoltan Varga 970d695718 [wasm] Disable linker substitutions for now, it generates invalid IL when running make build-aot-hello-netcore. (#18218) 2019-12-17 00:10:04 +01:00
Zoltan Varga 6bf2c17f34 [wasm] Pass --deterministic to the linker in msbuild. (#18214) 2019-12-17 00:09:46 +01:00
Zoltan Varga 9f5e38229e [netcore] Add AssemblyLoadContext:OnAssemblyLoad() to linker descriptor. (#18219) 2019-12-17 00:09:24 +01:00
Jay Krell de359d4930 Remove handles/MonoError from some marshal icalls. (#18036)
Remove handles/MonoError from some marshal icall functions.

This does some small other things.
  In particular, if a function is not static for some platforms, make it not static for all.
  static historically was valuable, because of default linker semantics around visibility.
  I believe it is less important now, since visibility is default constrained.
  Also being consistent across platforms seems good.

Maybe split off a separate PR for that.
2019-12-16 23:06:11 +01:00
Ryan Lucia e19914fe69 [metadata] Check for GTDs in setup_generic_array_ifaces (#18101)
Judging by https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/metadata.c#L3515 the cached_class is sometimes intentionally a GTD, which can get passed all the way down to setup_generic_array_ifaces and cause a crash.
2019-12-16 14:17:44 -05:00
Zoltan Varga ac1a12f997 [wasm] Regen wasm_m2n_invoke.g.h. (#18212) 2019-12-16 18:54:54 +01:00
Marek Safar 47199fede8 Revert "[wasm][ci] Deactivate linker on browser tests (#18152)"
This reverts commit 079acc2fb8.
2019-12-16 17:12:55 +01:00
Marek Safar f8266639ea Bump linker 2019-12-16 17:12:55 +01:00
Maxim Lipnin ba69304049 [wasm][xunit tests] Re-enable System.Tests.MathFTests (#18208) 2019-12-16 09:53:02 -06:00
Jérôme Laban 5562065e73 [Wasm] Enable dynamic runtime tests (#18098)
* [Wasm] Enable dynamic runtime tests

- Add dynamic runtime browser tests in CI
- Change `release-dynamic` to `dynamic-release` to align with other runtimes
- Add `-dynamic-runtime` flag for packager.exe
- Add debug configuration for dynamic runtime
- Fix invalid MAIN_MODULE=2 flag, which forces the emscripten linker to pass even on exported functions.
2019-12-16 07:28:26 -06:00
Calvin Buckley 84221f04b5 Fix locating Python interpreter when it isn't "python" (#18197)
Fix locating Python interpreter when it isn't "python"

The "python" interpreter is specified by PEP-394. In it, an OS
can decide not to ship a "python" executable, and instead only
ship python2/3, providing some ambiguity for scripts that are
version-neutral, like the one Mono uses.

This uses an automake macro to locate a suitable Python, and
changes the makefile to use the located Python, whatever it may be.
2019-12-16 11:06:00 +01:00
Zoltan Varga 61a103f371 [wasm] Set the main thread, some EH code depends on it. (#18203) 2019-12-16 08:41:43 +01:00
Marek Safar afcafb6c5e [monolinker] Fix path to wasm references 2019-12-15 17:05:37 +01:00
Maxim Lipnin c454cb92eb [wasm][xunit] Re-enable System.MemoryTests and System.SpanTests tests (#18179) 2019-12-14 14:59:48 +01:00
Jay Krell e76fb5dbf9 Add full compiler barriers to any fence that lacked a compiler barrier. (#18177)
A compiler barrier generates no code.
It "only" inhibits certain compiler optimizations, which I do not consider
particularly worrisome, as most code is "fully" optimized, and
compiler barriers are relatively rare.

It could be a concern in code paths that greatly benefit
from compiler optimizations and for which barriers greatly
inhibits the compiler.

I would further suggest, though it is not here, that any compiler
half barrier (read, write) be a compiler full barrier instead,
as they sound pretty cheap, so simplify.

Similar cannot be said about CPU barriers.
They do not always generate code, but when they do, it can cost.

This is follow-up from https://github.com/mono/mono/pull/17849#issuecomment-556273801
https://github.com/mono/mono/pull/17849#issuecomment-557661604.
2019-12-13 22:49:15 +01:00
Marek Safar 6dc5295ed0 [wasm] Improve virtualenv installation script 2019-12-13 22:37:08 +01:00
monojenkins b0e7a1e4f9 [bcl] Update BCL Linked Size 2019-12-13 22:24:52 +01:00
Marek Safar 15a8c2a7fa [wasm] Run linker tests with wasm substitution settings 2019-12-13 22:15:59 +01:00
Aleksey Kliger (λgeek) d0f27ce4fe [cominterop] Attach to runtime from IUnknown and IDispatch methods (#18175)
* [cominterop][test] Calling QueryInterface from an unattached thread asserts

If we pass a managed COM object to native code and the native code calls one of
the IUnknown methods, we assert because the thread is not attached but we want
to enter GC Unsafe mode.

* [cominterop] Attach to runtime from IUnkown and IDispatch methods

If the COM objects for a managed object is passed to a thread that hasn't
interacted with Mono before, we need to attach before doing GC thread state
transitions.

Addresses https://github.com/mono/mono/issues/18137

* [test] Call managed method through COM wrapper from unattached thread

This works, but I don't think we had a test previously.  Call a managed method
via a COM interface from a thread that is not attached to the runtime, to
ensure that it is attached.
2019-12-13 15:32:06 -05:00
Ankit Jain ea50ca48ba Bump msbuild to track xplat-master (#18183)
- bump roslyn to 3.5.0-beta1-19606-04, to match msbuild
- bump nuget.exe to 5.5.0.6319 (5.5.0-preview1), to match msbuild
2019-12-13 20:06:16 +01:00
Jay Krell 1310043953 Fix on-demand initialization race conditions [threads.c]. (#18166) 2019-12-13 19:57:33 +01:00
Jay Krell 2eca6a1e37 Fix on-demand initialization race conditions [marshal.c]. (#18169)
* Fix on-demand initialization race conditions [marshal.c].

* PR feedback: fix incorrect merge
2019-12-13 19:57:17 +01:00
Ryan Lucia 7bc25cef6d [netcore] Implement AppDomain.AssemblyLoad (#18143) 2019-12-13 11:36:08 -05:00
Jay Krell d2539babf4 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:30:08 +01:00
Jay Krell b637c1db67 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:29:30 +01:00
Alexander Köplinger 7e65a73d4d [sdks] Use Xcode 11.3 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-12-13 17:27:24 +01:00
Kenneth Pouncey 96fa4e8c6f [wasm][debug-proxy] Fix crash and send back failed message. (#18146)
* [wasm][debug-proxy] Fix crash and send back failed message.

* Remove debugging error message

* Remove debug Console.WriteLine
2019-12-13 10:22:55 -06:00
Jay Krell 57c6b81b96 Fix on-demand initialize race conditions [cominterop.c] (#18158) 2019-12-13 12:28:34 +01:00
Jay Krell 64ad646066 Fix on-demand initialize race conditions [custom-attrs.c] (#18159) 2019-12-13 12:27:39 +01:00
Steve Pfister 97bf0a1bcc Relax faultcode parsing for Soap11 message errors. (#18154)
* Relax faultcode parsing for Soap11 message errors.

The mono implementation was reading the faultcode content as an XmlQualifiedName, which applied the rule where you could not
start a name with a number. This caused interoperability problems with the .net framework because their parsing just looks
for <namespace>:<name>, where name can be any string.

Example:

<faultcode>s:1</faultcode>

On mono, a soap response that contains a fault code name of 1 will throw an error.  If you run the same code on the .net framework, it will not.

Fixes https://github.com/mono/mono/issues/12995

* [csproj] Update project files
2019-12-13 12:26:53 +01:00
Jay Krell 0dc19600d1 Fix on-demand initialization race conditions [object.c]. (#18163)
Fix on-demand initialization race conditions [object.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 09:25:56 +01:00
Zoltan Varga e391861c01 [wasm] Add linker substitutions. (#18149)
* [wasm] Add linker substitutions in netcore mode.

* Enable non-netcore mode as well.

* Disable conditionals which are not true in threaded mode for now.
2019-12-13 06:46:33 +01:00
Jay Krell 9cefdf59ad Fix on-demand initialization race conditions [sgen]. (#18170)
Fix on-demand initialization race conditions [sgen].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 04:55:58 +01:00
Jay Krell a93e4712ec Fix on-demand initialization race conditions [remoting.c]. (#18168)
Fix on-demand initialization race conditions [remoting.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:46:07 +01:00
Jay Krell 943a2760ec Fix on-demand initialization race conditions [native-library.c]. (#18162)
Fix on-demand initialization race conditions [native-library.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:26:07 +01:00
Jay Krell 1fe3d7eda6 Fix on-demand initialization race conditions [reflection.c]. (#18167)
Fix on-demand initialization race conditions [reflection.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:45:36 +01:00
Jay Krell 51bc81919f Fix on-demand initialization race conditions [mini] (#18161)
Fix on-demand initialization race conditions [mini].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:36:19 +01:00
Jay Krell 620d7cf84d Fix on-demand initialization race conditions [sre.c]. (#18164)
Fix on-demand initialization race conditions [sre.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:55:33 +01:00
Jay Krell f96ba075a1 Fix on-demand initialization race conditions [threadpool.c]. (#18165)
Fix on-demand initialization race conditions [threadpool.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:46:06 +01:00
Ryan Lucia 88dde8b763 [netcore] Implement AppDomain.TypeResolve (#18141) 2019-12-12 15:00:28 -05:00
Jay Krell a1e16d3273 Remove handles/MonoError from 12 easy socket icalls. (#18035)
Also put _icall back on ends so the search coincides between managed and native.
2019-12-12 13:32:14 -05:00
Maxim Lipnin 6d9c1359ec [wasm] Disable particular failing System.Net.Security.Tests tests rather than the whole namespace (#18151) 2019-12-12 13:07:44 -05:00
Ryan Lucia 7d88d4d502 [netcore] Enable AppDomainTests.AssemblyResolveInvalidAssemblyName (#18157)
I don't remember why this wasn't enabled originally, but it seems to be passing now. I probably accidentally fixed it back when I made changes to our assembly name parsing.
2019-12-12 12:27:51 -05:00
Zoltan Varga fb67916c0f [wasm] Change mono_wasm_runtime_invoke () to return the exception itself in the out argument instead of a boolean. (#18135) 2019-12-12 17:23:42 +01:00
Kenneth Pouncey 079acc2fb8 [wasm][ci] Deactivate linker on browser tests (#18152)
There are problems on the CI when using the linker building the Browser Tests.

https://github.com/mono/mono/commit/3142396c3c6a62f389420113129783bb156de399

PR https://github.com/mono/mono/pull/18128
2019-12-12 12:53:05 +01:00
Ryan Lucia 3ddcbda8b6 [netcore] Enable AppDomainTests.MonitoringIsEnabled (#18142) 2019-12-11 21:19:45 -05:00
Ryan Lucia acc8c0c696 [netcore] Remove MonoDomain.DoAssemblyResolve (#18139)
This is no longer called in netcore builds as of a while ago, so get rid of it and the corresponding linker exclusion.
2019-12-12 00:16:50 +01:00
Jay Krell 256eda8c7f Make Socket.Socket_icall static, as most icalls are, removing the first, unused parameter. (#18130)
Also partial undo recent and append _icall to end.
This is verbose, but maybe worthwhile to aid search.

This is extracted from https://github.com/mono/mono/pull/17978.
2019-12-11 14:40:27 -05:00
Ryan Lucia 0e05bda235 [utils] Correctly convert MONO_DL_LOCAL to RTLD_LOCAL (#18121)
On Linux, neither flag being specified will result in a default of RTLD_LOCAL. Unfortunately, on MacOS it's the opposite - the default is RTLD_GLOBAL. This means currently there is no way to set RTLD_LOCAL on Macs, so despite the potential for this to break someone it directly contradicts our documentation and needs to be changed.
2019-12-11 10:42:10 -05:00
Marek Safar 3142396c3c Bump linker (#18128)
* Bump linker

* [csproj] Update project files
2019-12-11 16:38:32 +01:00
Alexander Köplinger 8ae9708e87 [corlib] Rename internal LogLevel enum to avoid conflict with Android LogLevel (#18124)
After https://github.com/mono/mono/pull/18040 we now have InternalsVisibleTo for the Mono.Android assembly.
This exposed the internal LogLevel enum in BCLDebug which causes an issue when consumed in XA:

```
Java.Lang/Object.cs(119,18): error CS0104: 'LogLevel' is an ambiguous reference between 'Android.Runtime.LogLevel' and 'System.LogLevel'
```

To fix this we rename the internal enum in corlib since it's not really used.
2019-12-11 13:39:38 +01:00
Jay Krell 97dd6cf04c Sockets: Foo_internal to Foo_icall and ves_icall_Foo_internal to ves_icall_Foo. (#18017)
The first is to avoid requiring the signature matching in icall resolution.
The second is just to shorten names.
2019-12-11 10:40:00 +01:00
Maxim Lipnin 93c18d6fd1 [wasm] Re-enabling a test that was fixed by the corefx bump (#18108)
* [wasm] Clean up rsp-file

* Bump corefx
2019-12-11 10:39:35 +01:00
Kenneth Pouncey d6d74e819f [wasm][debugger] Prevent crash on disconnect of client (#18107) 2019-12-11 06:37:19 +01:00
Bernhard Urban-Forster a65b7af6d5 [interp] relop result is always int (#18114)
This can lead to problems on big endian.

```
result.i = (int) 1;
--> 0x0000_0001_XXXX_XXXX;

result.l = (long) 1;
--> 0x0000_0000_0000_0001;
```

On little endian it's always `0x0000_0000_0000_0001`, so it doesn't matter.
2019-12-10 22:06:48 +01:00
Ryan Lucia a778c46362 [utils] Make mono_dl_open default to RTLD_LOCAL on netcore (#17912)
* [utils] Make mono_dl_open default to RTLD_LOCAL

The exception is for fallback handlers added via the embedding API, in which case we convert the flags over to preserve legacy behavior

* Convert flags properly for embedding API use

* Changes

* Turn off -fvisibility=hidden temporarily

TESTING

* Undo some previous crimes, maybe fix failure

* Leave legacy mono unchanged, defaulting to RTLD_GLOBAL
2019-12-10 14:58:58 -05:00
Bernhard Urban-Forster ef422d9cb7 [interp] fix build when DEBUG_INTERP is enabled (#18112)
broke with c4136ebd94
2019-12-10 18:55:09 +01:00
Maxim Lipnin cd2e0e8f7a [wasm] Clean up rsp-file (#18113) 2019-12-10 18:08:52 +01:00
Bernhard Urban-Forster 80b1e103cb [debugger] skip suspend for unattached threads (#18105)
When the debugger tries to suspend all the VM threads, it can happen with cooperative-suspend that it tries to suspend a thread that has previously been attached, but then did a "light" detach (that only unsets the domain). With the domain set to `NULL`, looking up a JitInfo for a given `ip` will result into a crash, but it isn't even necessarily needed for the purpose of suspending a thread.

More details: Consider the following:
```
(lldb) c
error: Process is running.  Use 'process interrupt' to pause execution.
Process 12832 stopped
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
    frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
   2713         MonoDomain *domain = (MonoDomain *) mono_thread_info_get_suspend_state (info)->unwind_data [MONO_UNWIND_DATA_DOMAIN];
   2714         if (!domain) {
   2715                 /* not attached */
-> 2716                 ji = NULL;
   2717         } else {
   2718                 ji = mono_jit_info_table_find_internal ( domain, MONO_CONTEXT_GET_IP (&mono_thread_info_get_suspend_state (info)->ctx), TRUE, TRUE);
   2719         }
Target 0: (WatchWCSessionAppWatchOSExtension) stopped.
(lldb) bt
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  * frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
    frame #1: 0x004e177b WatchWCSessionAppWatchOSExtension`mono_thread_info_safe_suspend_and_run(id=0xb0767000, interrupt_kernel=0, callback=(WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical at debugger-agent.c:2708), user_data=0xb04dc718) at mono-threads.c:1358:19
    frame #2: 0x00222799 WatchWCSessionAppWatchOSExtension`notify_thread(key=0x03994508, value=0x81378c00, user_data=0x00000000) at debugger-agent.c:2747:2
    frame #3: 0x00355bd8 WatchWCSessionAppWatchOSExtension`mono_g_hash_table_foreach(hash=0x8007b4e0, func=(WatchWCSessionAppWatchOSExtension`notify_thread at debugger-agent.c:2733), user_data=0x00000000) at mono-hash.c:310:4
    frame #4: 0x0021e955 WatchWCSessionAppWatchOSExtension`suspend_vm at debugger-agent.c:2844:3
    frame #5: 0x00225ff0 WatchWCSessionAppWatchOSExtension`process_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0, il_offset=0, ctx=0x00000000, events=0x805b28e0, suspend_policy=2) at debugger-agent.c:4012:3
    frame #6: 0x00227c7c WatchWCSessionAppWatchOSExtension`process_profiler_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0) at debugger-agent.c:4072:2
    frame #7: 0x0021b174 WatchWCSessionAppWatchOSExtension`thread_startup(prof=0x00000000, tid=2957889536) at debugger-agent.c:4149:2
    frame #8: 0x0037912d WatchWCSessionAppWatchOSExtension`mono_profiler_raise_thread_started(tid=2957889536) at profiler-events.h:103:1
    frame #9: 0x003d53da WatchWCSessionAppWatchOSExtension`fire_attach_profiler_events(tid=0xb04dd000) at threads.c:1120:2
    frame #10: 0x003d4d83 WatchWCSessionAppWatchOSExtension`mono_thread_attach(domain=0x801750a0) at threads.c:1547:2
    frame #11: 0x003df1a1 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop_internal(domain=0x801750a0, cookie=0xb04dcc0c, stackdata=0xb04dcba8) at threads.c:6008:3
    frame #12: 0x003df287 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop(domain=0x00000000, dummy=0xb04dcc0c) at threads.c:6045:9
    frame #13: 0x005034b8 WatchWCSessionAppWatchOSExtension`::xamarin_switch_gchandle(self=0x80762c20, to_weak=false) at runtime.m:1805:2
    frame #14: 0x005065c1 WatchWCSessionAppWatchOSExtension`::xamarin_retain_trampoline(self=0x80762c20, sel="retain") at trampolines.m:693:2
    frame #15: 0x657ea520 libobjc.A.dylib`objc_retain + 64
    frame #16: 0x4b4d9caa WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke + 279
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb26e Foundation`____addOperations_block_invoke_4 + 20
    frame #24: 0x65de007b libdispatch.dylib`_dispatch_call_block_and_release + 15
    frame #25: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #26: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #27: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #28: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #29: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #30: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #31: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```

Going further, `info` is about this thread:
```
(lldb) p/x *(int *)(((char *) info->node.key) + 0xa0)
(int) $2 = 0x01243f93
(lldb) thread list
Process 12832 stopped
  thread #1: tid = 0x1243ee1, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_303', queue = 'com.apple.main-thread'
  thread #2: tid = 0x1243ee6, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10
  thread #3: tid = 0x1243ee7, 0x65f81aea libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'SGen worker'
  thread #4: tid = 0x1243ee9, 0x65f7e3d2 libsystem_kernel.dylib`semaphore_wait_trap + 10, name = 'Finalizer'
  thread #5: tid = 0x1243eea, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10, name = 'Debugger agent'
  thread #6: tid = 0x1243f1d, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.uikit.eventfetch-thread'
  thread #8: tid = 0x1243f93, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
* thread #9: tid = 0x12443a9, 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  thread #10: tid = 0x1244581, 0x65f7fd32 libsystem_kernel.dylib`__workq_kernreturn + 10
(lldb) thread select 8
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
    frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
libsystem_kernel.dylib`mach_msg_trap:
->  0x65f7e396 <+10>: retl
    0x65f7e397 <+11>: nop

libsystem_kernel.dylib`mach_msg_overwrite_trap:
    0x65f7e398 <+0>:  movl   $0xffffffe0, %eax         ; imm = 0xFFFFFFE0
    0x65f7e39d <+5>:  calll  0x65f85f44                ; _sysenter_trap
(lldb) bt
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
  * frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
    frame #1: 0x65f7e8ff libsystem_kernel.dylib`mach_msg + 47
    frame #2: 0x66079679 libxpc.dylib`_xpc_send_serializer + 104
    frame #3: 0x660794da libxpc.dylib`_xpc_pipe_simpleroutine + 80
    frame #4: 0x66079852 libxpc.dylib`xpc_pipe_simpleroutine + 43
    frame #5: 0x66043a8f libsystem_trace.dylib`___os_activity_stream_reflect_block_invoke_2 + 30
    frame #6: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #7: 0x65de3d71 libdispatch.dylib`_dispatch_block_invoke_direct + 257
    frame #8: 0x65de3c62 libdispatch.dylib`dispatch_block_perform + 112
    frame #9: 0x6604349a libsystem_trace.dylib`_os_activity_stream_reflect + 725
    frame #10: 0x6604ef19 libsystem_trace.dylib`_os_log_impl_stream + 468
    frame #11: 0x6604e44d libsystem_trace.dylib`_os_log_impl_flatten_and_send + 6410
    frame #12: 0x6604cb3b libsystem_trace.dylib`_os_log + 137
    frame #13: 0x6604f4aa libsystem_trace.dylib`_os_log_impl + 31
    frame #14: 0x4b4eb4e9 WatchConnectivity`WCSerializePayloadDictionary + 393
    frame #15: 0x4b4d7c4d WatchConnectivity`-[WCSession onqueue_sendResponseDictionary:identifier:] + 195
    frame #16: 0x4b4da435 WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke.411 + 35
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb067 Foundation`____addOperations_block_invoke_2 + 20
    frame #24: 0x65dedf49 libdispatch.dylib`_dispatch_block_async_invoke2 + 77
    frame #25: 0x65de4461 libdispatch.dylib`_dispatch_block_async_invoke_and_release + 17
    frame #26: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #27: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #28: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #29: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #30: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #31: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #32: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```
which is a thread in a "light" detach state (aka. coop detach), where we only unset the domain:
https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/threads.c#L6084-L6111

Fixes https://github.com/mono/mono/issues/17926
2019-12-10 12:16:05 +01:00
Aleksey Kliger (λgeek) 7704bd1b92 [mini] Fix delegate trampoline virtual call via delgate Invoke (#18073)
* [mini] Fix delegate trampoline virtual call via delgate Invoke

If we need to jit the Invoke method of a delegate, we get tramp_info with a
NULL method.

Background: normally when we create a delegate around a virtual method,
handle_delegate_ctor will just create a virtual invoke trampoline with
mono_arch_get_delegate_virtual_invoke_impl which doesn't get here.  But if
we're asked to compile the delegate's Invoke method, then compile_special ()
will create a tramp_info with a null method, and return a delegate trampoline.

That's the case here - we had
  var del = SomeDelegate(obj.VirtualMethod);
  var invoke_method = del.GetType().GetMethod ("Invoke");
  invoke_method.Invoke (del, args);
or
  var del = SomeDelegate(obj.VirtualMethod);
  var another_del = OtherDelegate (del.Invoke);
  another_del (args);

in both cases, we end up in mono_delegate_trampoline with tramp_info->method ==
NULL.

in the second case the IL is like this:
   newobj instance void Derived::'.ctor'
   ldvirtftn instance void class Base::VirtualMethod()
   newobj instance void class SomeDelegate::'.ctor'(object, native int)

So delegate->target is a derived instance but delegate->method is some base
class method.

Addresses https://github.com/mono/mono/issues/17718

* [tests] Add tests for compiling delegate's Invoke method

Where the delegate calls a virtual method
2019-12-10 12:12:50 +01:00
Bernhard Urban-Forster fd44435a84 [debugger] remove unused thread suspend count (#18102) 2019-12-10 08:25:31 +01:00
Nikolay Sivov d1fc2ffa84 [support] Remove some dead code (#16674)
Both 'ptv' and 'tv' are unused since select() was replaced with poll() in 57b521e.
2019-12-10 00:21:00 +01:00
Steve Pfister 4cefdcb7ce Bump to mono/corefx@f3815a9 (#18090)
Picks up https://github.com/mono/corefx/pull/376
2019-12-09 21:29:40 +01:00
Jay Krell 5cbb1f4be8 [cxx] Fix mono-boehm to compile as C++. (#18086)
https://jenkins.mono-project.com/job/test-mono-pull-request-i386/32263/consoleText

```
      ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
boehm-gc.c: In function 'void mono_gc_base_init()':
boehm-gc.c:202:42: error: invalid conversion from
'void (*)(size_t) {aka void (*)(unsigned int)}' to
GC_on_heap_resize_proc {aka void (*)(long unsigned int)}' [-fpermissive]
  GC_set_on_heap_resize (on_gc_heap_resize);
                                          ^
In file included from ../../mono/utils/gc_wrapper.h:44:0,
                 from ../../mono/metadata/gc-internals.h:16,
                 from boehm-gc.c:16:
../../external/bdwgc/include/gc.h:132:21: note:
 initializing argument 1 of 'void GC_set_on_heap_resize(GC_on_heap_resize_proc)'
 GC_API void GC_CALL GC_set_on_heap_resize(GC_on_heap_resize_proc);
```
2019-12-09 18:59:10 +01:00
abrevet-dev 3d83c7429e [WinForms] No event should be triggered when calling ComboBox.Items.RemoteAt() (#18069)
* [WinForms] Fix wrong behavior when calling ComboBox.Items.RemoteAt() with index equals to SelectedIndex

* [WinForms] Fix #10643 No event should be triggered when calling ComboBox.Items.RemoteAt()

Fixes #10643
2019-12-09 15:51:28 +01:00
dotnet-maestro[bot] 612f2a8e88 Update dependencies from https://github.com/dotnet/arcade build 20191208.1 (#18089)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19608.1
2019-12-09 15:26:16 +01:00
Maxim Lipnin 17bba9e7dd [wasm] Clean up rsp-file (#18087) 2019-12-09 07:57:00 -06:00
Egor Bogatov 2befdcfa52 [netcore] Enable more tests for interpreter and llvm on CI (#17957)
* Enable more tests

* fix build?

* forward XUNIT_MONO_ENV_OPTIONS

* increase timeout

* ignore some tests for interpreter

* ignore more tests

* move interp specific issues to a separate rsp

* fix CI

* fix CI

* undo build.sh chmod

* revert chmod for xunit-summary-py

* improve xunit summary

* Ignore the whole System.Net.Tests namespace

* Ignore System.Text.Json

* disable the whole BinaryFormatterTests

* Disable System.Net.Http.Functional.Tests
2019-12-09 14:55:44 +01:00
Kenneth Pouncey 1462208991 [wasm][bcl][websockets] Add WebSocket support. (#18062)
Add the ability to initialize a `WebAssembly.Net.WebSockets::ClientWebSocket` from `System.Net.WebSockets.ClientWebSocket`.

`System.Net.WebSockets.ClientWebSocket` will delegate all calls to the WebAssembly ClientWebSocket.  

- Add support for marshaling `Uri` class correctly.  This was a cause of failures.
- Add bindings tests for `Uri` marhaling.
2019-12-09 13:32:17 +01:00
Jay Krell ed6b8024b0 fix warning: threadpool-worker-default.c: In function 'worker_try_create': 'now' may be used uninitialized (#18056)
https://jenkins.mono-project.com/job/build-source-tarball-mono-pullrequest/854/console
2019-12-09 08:17:54 +01:00
Jay Krell a6c4eafee8 Add some missing barriers when dealing with global caches (#18032)
* Add some missing barriers when dealing with global caches
accessed from multiple threads without locks.

This is based on earlier PR https://github.com/mono/mono/pull/17849.

Read-side barriers are removed.
  More analysis is needed to determine if they are needed (which pointer
  read vs. written).
Write-side barriers are compiler barrier + store-release instead of full barrier.

Note all candidates are converted to new form yet.

Cases that do motivate read-wide barriers are skipped.
e.g.
`mono_arch_start_dyn_call`
`get_agent_domain_info`
`init_jit_info_dbg_attrs`

* PR: Remove 'Try one more time..'
2019-12-08 15:30:48 +01:00
Sam Patel 74567a0a34 Update glib-remap.h to include g_module_address (#18076) 2019-12-07 07:11:59 +01:00
abrevet-dev 4be2a9a207 [WinForms] Handle menu for wheel button click (close the contextmenu) (#18049)
Fixes #18019
2019-12-06 18:41:47 +01:00
Ankit Jain d1db937e78 Bump msbuild to track xplat-master (#18050)
.. to get changes from https://github.com/mono/msbuild/pull/162 .

And bump roslyn to 3.4.0-beta4-19569-03 to match msbuild.
2019-12-06 16:55:44 +01:00
Aleksey Kliger (λgeek) 3dabeddfc6 [threads] Add back mono_threads_attach_tools_thread as a public API (#18048)
* [utils] Add back mono_threads_attach_tools_thread

In https://github.com/mono/mono/commit/a5da7b21f4b6dbc5eaa09c2addee91b84dc1dbd5
we got rid of "tools" threads internally to the runtime.

However since the API was previously marked with MONO_API it was an internal
API that some embedders depended on.

This PR adds back an (external-only) limited form of tools thread.

The runtime is aware of the Tools thread in that FOREACH_THREAD_* macros will
iterate over them, and the thread has a coop thread state machine. (That is,
mono_thread_info_current() and GC Safe and GC Unsafe transitions all work.)

However the thread is:
1. Not stopped by the GC
2. Is not interrupted by profiler sampling.
3. Does not have a "current domain"
4. (As a consequence of the above) cannot call managed methods or touch managed
objects.

Such threads are useful for low-level interaction with the runtime such as
querying metadata, the JIT state and other coordination.

mono_threads_attach_tools_thread should be called no more than once.  It should
not be called on a thread that is already attached with mono_thread_atach, and
vice versa.

Addresses https://github.com/mono/mono/issues/18011

* [threads] Make mono_threads_attach_tools_thread into a public API
2019-12-06 10:50:34 -05:00
Bernhard Urban-Forster 121928d29a [ppc32] delete leftover code from pre-monoctx area (#18072)
Also update assert which doesn't make sense since we have moved over to MonoContext

Some context: e7011c780f

Fixes: https://github.com/mono/mono/issues/18064
2019-12-06 11:35:11 +01:00
Jay Krell 12bc4717fd [cxx] Remove use of register keyword. (#18057)
[cxx] Remove use of register keyword.

https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html

```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
4 warnings generated.
```
2019-12-06 03:55:56 +01:00
Jay Krell a131e9bbb0 [cxx][netcore] Fix goto around init. (#18058)
https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html
```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
```
2019-12-05 15:41:09 -06:00
Egor Bogatov 63f155ac46 Let user overwrite opt -Ox level (#18012) 2019-12-05 13:44:05 -06:00
Jay Krell b99ea70e43 Remove handles/MonoError from two socket icalls. (#17977)
Remove handles/MonoError from two socket icalls.

Ongoing removal of handles() icalls.
This is small, to pause and remove `mono_class_get_field_from_name_full` from w32socket.c.
2019-12-05 19:36:11 +01:00
Egor Bogatov 84839ab297 Merge pull request #18053 from EgorBo/fix-fneg-llvm
LLVM: use fneg instead of fsub for `-x`
2019-12-05 21:13:11 +03:00
Jay Krell ae9192f96f [netcore] Remove approx. 10 icalls. (#18018)
* [netcore] Remove ves_icall_System_Runtime_InteropServices_Marshal_PtrToStringUni and _len icalls.
They are implemented in C#.
The C# is equivalent to the C, and could be used for classic runtime, except for the treatment of large/negative lengths.

* Remove a few more icalls.
Instead of ifdef and leaving them uncompiled when compiling netcore,
declare them and put asserts in them.
2019-12-05 13:10:29 -05:00
Alexander Köplinger 269d509e42 Remove DNS lookups of the local hostname in tests (#18059)
This allows us to workaround issues we currently have in the macOS bot network where DNS lookup of the hostname fails.
2019-12-05 18:08:26 +01:00
Zoltan Varga e67673fac8 [llvm] Avoid decomposing OP_STOREV_MEMBASE early for types which contains no references. (#18041) 2019-12-05 16:02:43 +01:00
Martin Baulig cc720b83aa [System.Net.Http]: Add workaround for mono/mono#17897. (#17992)
Use `(UriKind)300` (which is the same as setting `MONO_URI_DOTNETRELATIVEORABSOLUTE=true`)
when parsing Uri headers.
2019-12-05 14:36:07 +01:00
Jay Krell 5a5e092a53 Fix ifndef include guard in icall-decl.h. (#18055)
There was no clash so ok either way, but this is more like intended.
2019-12-05 10:19:24 +01:00
Egor Bogatov a3fbb64493 Convert to 0/1 in OP_XEQUAL (#18038) 2019-12-05 02:38:34 +01:00
abrevet-dev 186f9cd8c4 [WinForms] Fix wrong behavior of RadioButton auto-check (#18044)
Fixes #18020
2019-12-05 00:41:49 +01:00
Alexander Köplinger 4590bf6cac Bump bockbuild to get https://github.com/mono/bockbuild/pull/136 (#18043) 2019-12-05 00:33:08 +01:00
Alexander Köplinger f965e5d3e8 Fix formatting of README.netcore.md 2019-12-05 00:11:14 +01:00
EgorBo 55a3c10e72 remove some items from rsp 2019-12-05 02:09:11 +03:00
EgorBo 92047c453f use fneg instead of fsub 2019-12-05 02:03:24 +03:00
Bernhard Urban-Forster 9d58bc1d12 [amd64] align application stack pointer in signal handler (#18021)
* [amd64] align application stack pointer in signal handler

16byte stack pointer alignment is only required in certain places by the ABI, so the assumption that it holds all the time is wrong (though, it passed CI 😅). We have to setup the frame properly, i.e. we have to ensure the stack pointer is properly aligned. That's what we have done previously too.

Regression of https://github.com/mono/mono/pull/17922
Fixes https://github.com/mono/mono/issues/18006

* add ALIGN_DOWN_TO macro

* remove unnecessary ifndefs
2019-12-04 15:23:19 -06:00
Alexander Köplinger 59554712cc Update submodules before running external/bdwgc/autogen.sh (#18042)
Otherwise we'll run into issues because in configure.ac we update submodules _after_ we try to call external/bdwgc/configure (which won't be there if autogen.sh didn't run).
This would manifest itself as `*** No rule to make target 'all'.` in external/bdwgc when you build as there's no Makefile.
2019-12-04 22:16:21 +01:00
Jonathan Peppers 8511ae20f0 [corlib] add InternalsVisibleTo for Mono.Android (#18040)
* [corlib] add InternalsVisibleTo for Mono.Android

Context: https://github.com/xamarin/xamarin-android/pull/3991

In Xamarin.Android, we have a bit of System.Reflection usage on
startup:

https://github.com/xamarin/xamarin-android/blob/bbbe2ed36e6eeb7cce0905dcbfdaeaa30754070f/src/Mono.Android/Android.Runtime/UncaughtExceptionHandler.cs#L47-L67

These methods are in `mscorlib.dll` and `internal`. If we had
`[assembly: InternalsVisibleTo ("Mono.Android")]` we could avoid the
System.Reflection.

This is already done for `Xamarin.iOS`, `Xamarin.Mac`, etc. so
hopefully it's OK to add one more?

We could then, just call these methods directly:

* `System.Diagnostics.Debugger.Mono_UnhandledException`
* `AppDomain.CurrentDomain.DoUnhandledException`

The results are possibly a 100-250ms improvement to startup:

    Before:
    12-03 14:27:51.450  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s727ms
    12-03 14:27:57.310  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s723ms
    12-03 14:28:03.216  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s703ms
    12-03 14:28:09.054  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s679ms
    12-03 14:28:14.953  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s711ms
    After:
    12-04 09:13:18.248  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s608ms
    12-04 09:13:24.087  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s570ms
    12-04 09:13:29.997  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s566ms
    12-04 09:13:35.805  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s498ms
    12-04 09:13:41.670  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s475ms

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Bump API snapshot submodule
2019-12-04 21:01:20 +01:00
Kenneth Pouncey 83cac0306c [wasm][build] Modify to use -s USE_ZLIB=1 instead of zlib support headers (#18031)
* [wasm] Modify to use -s USE_ZLIB=1 instead of zlib support headers

- This will remove the dependency on the mono zlib header files.
- Moves the zlibtest.cs to tests/src directory to help clean up the main directory.

* [wasm][packager] Remove the zlib headers from the zlib build

- use `-s USE_ZLIB=1` instead when building.
2019-12-04 20:11:43 +01:00
Alexander Köplinger 4e8ead7caf Remove mk/ folder (#18039)
The plan to put additional helpers into this directory didn't materialize and we can just inline the one thing we have in common.mk where needed.
2019-12-04 19:55:22 +01:00
Kenneth Pouncey 42c5ccc298 [wasm][reference] Update binary references to include wasm websocket assemblies. (#18029) 2019-12-04 11:52:19 +01:00
Jonathan Peppers b816189b94 [android] don't probe directories for .config files (#18024)
* [android] don't probe directories for .config files

In Xamarin.Android, on startup we see logging like this for every .NET
assembly:

    12-03 14:27:42.104 26842 26842 D Mono    : Prepared to set up assembly 'FormsViewGroup' (/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll)
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly FormsViewGroup[0xebcee9c0] added to domain RootDomain, ref_count=1
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly Loader loaded assembly from location: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll.config'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config'.

In most cases, no `.dll.config` file exists and so it probes this
macOS-looking path:

    /Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config

It looks like this is happening because nothing in Xamarin.Android
calls `mono_set_dirs` or `mono_set_config_dir` and so it falls back to
using `MONO_CFG_DIR`. `MONO_CFG_DIR` is set to a path on the build
machine.

Even if Xamarin.Android *did* call `mono_set_config_dir`, the code
appends `mono/assemblies`:

    cfg = g_build_filename (cfg_dir, "mono", "assemblies", aname, cfg_name, (const char*)NULL);

So it doesn't seem useful to use this fallback at all, there would
never be a file found.

I wrapped the appropriate code in `#ifndef TARGET_ANDROID` blocks.

I was able to see a minor *maybe* ~20ms performance improvement on
startup:

    Before:
    12-03 14:27:48.494 26903 26903 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:601::443900
    12-03 14:27:54.356 26963 26963 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::495400
    12-03 14:28:00.263 27021 27021 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:586::147100
    12-03 14:28:06.135 27074 27074 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:593::639100
    12-03 14:28:11.988 27129 27129 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:571::194900
    After:
    12-03 14:46:50.590 27737 27737 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:503::918700
    12-03 14:46:56.517 27794 27794 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:570::533600
    12-03 14:47:02.368 27850 27850 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:537::435000
    12-03 14:47:08.289 27904 27904 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::830900
    12-03 14:47:14.130 27965 27965 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:555::593400

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Add --enable-minimal=cfgdir_config

* This also defines `DISABLE_CFGDIR_CONFIG`
* Used this setting for android
2019-12-04 11:50:59 +01:00
abrevet-dev a79c4d11a4 [WinForms] Editing the DialogResult of the form in the button click event was ignored. (#18025)
Fixes #12022
2019-12-04 11:49:43 +01:00
Jérôme Laban 91d3d63858 Update to emscripten 1.39.4 (#18016)
Fixes for i64 returning values with dynamic runtime configuration.

See https://github.com/emscripten-core/emscripten/issues/9850
2019-12-04 09:11:22 +01:00
monojenkins 37ba834ee1 [bcl] Update BCL Linked Size 2019-12-04 08:29:56 +01:00
Aleksey Kliger (λgeek) 467d46f8ab [configure.ac] Build on FreeBSD/PowerPC64 (#18015)
Community contribution from https://github.com/mono/mono/issues/18009
from @igorzigorz
2019-12-03 19:56:30 -05:00
Egor Bogatov 85826e5c27 Merge pull request #18013 from EgorBo/intrin-string-memcpy
[netcore] Improve jitdiff tool
2019-12-04 02:19:32 +03:00
Maxim Lipnin e7ef2f3d5f [Wasm] Enable system nunit tests on ci (#17716)
* [wasm][nunit tests] Enable nunit tests for System on CI

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Mail.SmtpClientTest tests

* Fix test suit name

* [wasm][nunit tests] Exclude MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest nunit tests

* [wasm][nunit tests] Exclude MonoTests.System.Net.NetworkInformation.NetworkInterfaceTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.EventDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.AsyncOperationManagerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.ServicePointTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebClientTestAsync nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Timers.TimerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest.PropertiesWhenClosedSet3 test

* [wasm][nunit tests] Increase timeout for System nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest.Find_FindByIssuerName test

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Mail.SmtpClientTest tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerTest nunit tests

* Increase CI timeout for System.Core nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Update NetworkInterface class to throw PNSE

* [wasm][nunit tests] Ignore failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Adjust the use of Ignore attribute

* [wasm][nunit tests] Categorize failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Remove the category for passing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing part of MonoTests.System.Net.ServicePointTest.DnsRefreshTimeout test

* [wasm][nunit tests] Remove PNSE expectation in failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests.

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests due to libc issue

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.WebRequestTest.DefaultWebProxy test

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests due to compare info issue
2019-12-03 22:40:09 +01:00
abrevet-dev 174f2f6a70 [WinForms] SetRowSpan and SetColumnSpan does not trigger the good exception. (#18002)
According to .Net documentation:

Exceptions
ArgumentOutOfRangeException

value is less than 1.
2019-12-03 21:38:38 +01:00
EgorBo 5f58bb0959 Improve TryParseFunctionName in jitdiff 2019-12-03 18:13:48 +03:00
EgorBo ea0824cb22 Remove dead docs 2019-12-03 18:11:42 +03:00
Egor Bogatov 8150a38dae Don't throw NRE when len == 0 in Buffer.Memmove (#18004) 2019-12-03 15:21:41 +01:00
Steve Pfister a3c25e7b4f Added Obsolete to AppDomain.CreateDomain for monodroid. (#18000)
* Added Obsolete to AppDomain.CreateDomain for monodroid.  The intent is to be helpful for when .NET 5 comes along

* Changed AssemblyLoadContext link to a better doc

* Bump API snapshot submodule
2019-12-03 14:23:49 +01:00
Jo Shields a2f9fb77a3 Bump version to 6.11 (#17990) 2019-12-02 15:58:38 -05:00
Di Wu 0370372ce3 Fix raw security descriptor parsing (#17482)
Fix security identifier parsing error with the following SDDL:

```
O:S-1-5-21-1356517589-3987021482-2501375073-1001G:S-1-5-21-1356517589-3987021482-2501375073-513D:(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;S-1-5-21-1356517589-3987021482-2501375073-1001)
```
2019-12-02 20:03:49 +01:00
Jay Krell 72aa81e901 Remove handles/MonoError from Mono.RuntimeClassHandle. (#17805) 2019-12-02 19:18:03 +01:00
Bernhard Urban-Forster a9a0a1e5b9 [profiler] disable inlining when coverage profiler is used and fix allocation sizes (#17986)
* [profiler] let compiler decide about the size of an entry

* [profiler] disable inlining when coverage profiler is used
2019-12-02 19:04:13 +01:00
Vlad Brezae 640ffa7a62 [sgen] Don't trigger collections during allocation of thread objects (#17970)
Allocation of thread objects usese the mono_object_new_mature API. These objects are allocated before the thread is finished attaching to the runtime. This means that a collection can happen on an unattached thread as well as all its callbacks, which is a counter-intuitive behavior.

On android this is problematic because at the end of the SGen collection we need to call the java collection, which needs to have the thread attached. This could probably be fixed instead from the Xamarin.Android side, but this approach seems simpler and saner.

Fixes https://github.com/mono/mono/issues/17878
2019-12-02 12:54:21 -05:00
Jay Krell d7f5a5fdfe Remove handles/MonoError from Mono.Security.Cryptography.KeyPairPersistence icalls. (#17967) 2019-12-02 12:51:16 -05:00
dotnet-maestro[bot] 9bd1b2e065 Update dependencies from https://github.com/dotnet/arcade build 20191201.1 (#17985)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19601.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19601.1
2019-12-02 18:03:11 +01:00
abrevet-dev 1454b010ae [WinForms] the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true (#17981)
* [WinForms] Fix #17980 Prevent from crashing when DataGridView  is null.

Reproducer:

DataGridViewLinkColumn links = new DataGridViewLinkColumn();
links.UseColumnTextForLinkValue = true;

* [WinForms] Fix #17980 The field useColumnTextForLinkValue of DataGridViewLinkCell was not cloned.

* [WinForms] Fix #17980 the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true

Fixes #17980
2019-12-02 17:39:43 +01:00
abrevet-dev 865e955840 [WinForms] Fix #9726 imageLayout of DataGridViewImageCell was not cloned. (#17974)
Fixes #9726
2019-12-02 17:38:08 +01:00
Thays Grazia 3834be3d22 [DIM] Fix behavior when there is an override of a method in a generic interface. (#17963)
* Fix behavior when there is an override of a method in a generic interface.
Fixes #17869
2019-12-02 13:20:00 -03:00
Jay Krell 7356aedc77 Refactor Monitor to eliminate near identical duplicated code. (#17964)
The only difference in the copies, was that one had an infinite timeout
so implied TRUE result unless interrupted.

Where the other copy had a timeout and could return FALSE as a result.

They both used the same underlying, with timeout, so it wasn't really an optimization,
just duplicated code.

Also there is subtelty around having a return value or an out parameter.
That is carefully preserved.

Also, coop is fixed, by creating a frame only in a slow path.
Presumably creating a thread is quick and stateless enough such that this
is not too late.
2019-12-02 11:06:11 -05:00
Calvin Buckley 369806e3e6 Disable ConnectFailAsync test on AIX (#17975)
Hangs runtime; prevents CI from completing run. (Another test can
also hang the runtime, but is hard to reproduce and inconsistent
about that.)

This should be fixed eventually, but...
2019-12-02 10:34:07 +01:00
Zoltan Varga f13e6ba91a [wasm] Add beginnings of SIMD support in AOT mode. (#17971)
* [jit] Implement beginnings of SIMD support for WASM.

* [wasm] Add a DISABLE_WASM_NO_NETCORE Make.local conditional to disable non-netcore builds during development.

* [wasm] Add a --simd argument to the packager.

* Remove some duplicate defines.

* Fix an windows assertion.
2019-12-02 07:21:17 +01:00
campersau b0f0b14023 [wasm][bcl][http] Fix descriptive message for the PlatformNotSupportedException (#17919)
* Fix descriptive message for the `PlatformNotSupportedException`

* Add more descriptive message to the `Properties` `PlatformNotSupportedException`
2019-12-02 05:20:15 +01:00
EgorBo d33ce1c3ec enable only for netcore 2019-12-01 17:38:33 -06:00
EgorBo 376d3acb29 fix failing tests 2019-12-01 17:38:33 -06:00
EgorBo b77f757679 fix build 2019-12-01 17:38:33 -06:00
EgorBo 1b7c7e6075 Address feedback and fix C++ warnings 2019-12-01 17:38:33 -06:00
EgorBo e2241969f5 Optimize box(X) == null to false 2019-12-01 17:38:33 -06:00
Jay Krell fffeb23239 sockets: Remove piecemeal ifndef ENABLE_NETCORE since the file has it broadly. 2019-12-01 17:33:24 -06:00
Zoltan Varga ae4b678b94 [wasm] Avoid rebuilding corelib in build-wasm-bcl-netcore. (#17969) 2019-11-30 03:18:42 +01:00
Bernhard Urban-Forster 4d0feffa42 [interp] track valuetype stack for CEE_RET too (#17968)
[interp] track valuetype stack for CEE_RET too

Usually the stack depth is determined via basic block predecessors, but  that only works for forward branches.

Fixes https://github.com/mono/mono/issues/17924
2019-11-30 00:06:05 +01:00
Zoltan Varga 6c0bfdc3f3 [wasm] Add more netcore test suites. (#17965)
* [wasm] Handle more exceptions in the xunit runner.

* [wasm] Add more corefx test suites.

* Add more icall signatures.

* Add support for running the System.Compression test suites.
2019-11-29 18:17:40 +01:00
Zoltan Varga d1735a8356 Interpreter fixes. (#17966)
Interpreter fixes.
2019-11-29 16:26:05 +01:00
Ankit Jain c92a96ccb5 Bump msbuild to track xplat-master (#17950)
* Bump msbuild to track xplat-master

* Bump nuget.exe to 5.4.0
2019-11-29 12:01:06 +01:00
Egor Bogatov 9a18b6fd6f [netcore] Improve Jitdiff tool (#17960)
use `objdump` arguments to reduce verbosity (no raw bytes, etc) and make the dasm more diff-friendly.
On Linux all functions have a random prefix (address) - replace it with `0xD1FFAB1E` constant). On macOS it's fine by default.

Also, allow to generate diffs for a single assembly via `dump-asm-lib` rule.
2019-11-29 11:54:01 +01:00
Kenneth Pouncey 608c93643e [wasm][http] Fix race condition as a result of using TaskCompletionSource: RangeError: Maximum call stack size on Safari (#17946)
There is a race condition on Safari as a result of using TaskCompletionSource that causes a stack exceeded error being thrown during Blazor startup.

Safari throws the following error:

```
[Error] Unhandled Promise Rejection: RangeError: Maximum call stack size exceeded.
    step (blazor.webassembly.js:2530)
    fulfilled (blazor.webassembly.js:2500)
    promiseReactionJob

```

- Modify `WebAssemblyHttpMessageHandler.cs` to create the TaskCompletionSource to run continuations asynchronously.
   - `TaskCompletionSource<HttpResponseMessage>(TaskCreationOptions.RunContinuationsAsynchronously);`

- There is a second part to this on AspNetCore in `Renderer.cs` to process it's pending startup tasks to run continuations asynchronously to compliment the TaskCompetionSource of the Http response message.

The following link discusses the potential stack/race problem that can occur when using TaskCompletionSource:  https://devblogs.microsoft.com/premier-developer/the-danger-of-taskcompletionsourcet-class/
2019-11-29 07:14:19 +01:00
Zoltan Varga 59a9af6d7c Revert "Use range metadata (#17956)" (#17958)
This reverts commit fde058518d.
2019-11-28 21:52:01 +01:00
Egor Bogatov fde058518d Use range metadata (#17956) 2019-11-28 14:26:35 +01:00
Kenneth Pouncey d6dea097ca [wasm] Marshal null string correctly. (#17955) 2019-11-28 07:31:59 +01:00
Zoltan Varga 79db9b2a87 [interp] Make calli non recursive as well. (#17953) 2019-11-28 06:26:36 +01:00
EgorBo 6360ccbfe3 Add quick diff 2019-11-27 19:15:56 -06:00
EgorBo e333f13710 Add jit-diff tools 2019-11-27 19:15:56 -06:00
Ryan Lucia 9c1cae02ca Change signature, add cast 2019-11-27 19:13:30 -06:00
Ryan Lucia 0c3036abc7 [mini] Move bundled native library loading into the runtime 2019-11-27 19:13:30 -06:00
Jay Krell b2817e6404 [cxx] MonoJitICallId same between C and C++.
This is optional.
The real problem was -enable-cxx not actually compiling some files as C++.
This is still reasonable, to ward off the danger of that occuring again.
2019-11-27 19:12:32 -06:00
Jay Krell c3da0ad518 Update netcore. 2019-11-27 19:11:35 -06:00
Jay Krell 4a15e73429 Remove handles/MonoError from Mono.RuntimeMarshal and Mono.SafeStringMarshal. 2019-11-27 19:11:35 -06:00
Nikita Voronchev 7db2389846 [WinForms][UIA] Add to the PropertyGrid new internal event to track grid items expanded state update (#17731)
This PR is needed for the https://github.com/mono/uia2atk/pull/6 one.
2019-11-27 23:59:04 +01:00
abrevet-dev f135312b44 [WinForms] DrawToBitmap() did not draw children controls (#17676)
Fixes #13777
2019-11-27 23:38:04 +01:00
Sergey Podobry abc75476d7 [Mono.Security] Do not decode data beyond detected length in ASN1 parser (#17751)
If `byte[] data` has additional data at the end `Decode` will try to parse it and will generate incorrect tags or throw an exception. This fix makes `Decode` to use only `nLength` bytes from `byte[] data`.
2019-11-27 23:37:20 +01:00
Aleksey Kliger (λgeek) b6a351c330 [threadpool] Decrement max_working when worker times out (#17927)
[threadpool] Decrement max_working when worker times out

The corresponding code in CoreCLR:
https://github.com/dotnet/runtime/blob/bf88f146412d1e2d41779422337184a802f186c9/src/coreclr/src/vm/win32threadpool.cpp#L2156-L2164

The issue is that counter._.max_working is only ever changed by:
1. the monitor_thread when it detects starvation (increment by 1).
2. the hill climbing algorithm (increment or decrement).

Creating a periodic load of creating many workers at once followed by a minute
or more of quiescence increases the variable worker.counters._.max_working in
monitor_thread(). Once that counter matches worker.limit_worker_max,
monitor_thread() keeps looping without unparking or creating a thread, even
though the actual number of threads/workers is small. If the existing threads
are all waiting on work that needs a new thread, then a deadlock occurs.

With this change, when a parked worker times out, it will lower max_working to
the number of active (working + parked + starting) threads minus itself (but no
less than limit_worker_min). As a result, monitor_thread will only increment
max_working as long as none of the already running workers are timing out.

Attempt to address https://github.com/mono/mono/issues/17833

---

Also allow monitor_thread to start workers if there aren't enough active, even if the max worker limit has been reached.

Work around a pathological condition where the work_item_count is non-zero, and
the max_working limit has been reached, but the number of active threads is
still below the max.  In that case, unpark some workers and start some new
threads.

May address http://work.azdo.io/827206
2019-11-27 22:06:17 +01:00
Jay Krell 3e369ba8c9 Cleanup initialize_object_slots, remove unused GetHashCode (ghc) handling. (#17834)
Fix possible race conditions involving multiple writes to globals, by
removing one of the two globals involved in finalize.
2019-11-27 14:25:14 -05:00
abrevet-dev 6145a397d6 [WinForms] Refactor selected indexes in TabControl.Remove (#17608)
Fixes #12337
2019-11-27 19:55:00 +01:00
abrevet-dev 5944e72d72 [WinForms] Returns real installed input languages on Windows (#17653)
The method InstalledInputLanguages() always returned InvariantCulture. Now, installed input languages are detected properly on Windows.

Reproducer:

`Console.WriteLine(InputLanguage.CurrentInputLanguage.Culture.TwoLetterISOLanguageName.ToUpper());`

Related issue: #13552
2019-11-27 19:46:04 +01:00
abrevet-dev 41ba5846b3 [WinForms] DefaultCellStyle was not cloned deeply in DataGridViewRow.Clone() (#17755)
Fixes #16557
2019-11-27 19:41:15 +01:00
HebaruSan d62eed676e [WinForms] Fix X11 theming (#17888)
## Background

In Mono 5 and earlier, the active GTK or KDE theme would be sourced for system colors, allowing dark themes to be obeyed:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

This allowed WinForms apps to blend in and look more like native apps.

## Problem

In Mono 6, this capability was lost (as noted in passing in #14325). WinForms apps now only look like this, regardless of the system color settings:

![image](https://user-images.githubusercontent.com/1559108/69486114-dccce280-0e0d-11ea-8107-fe1653b2d816.png)

This looks somewhat jarring and worse than previous Mono releases, for users with a dark system theme.

## Cause

During load of the system settings (in the static constructor of `X11DesktopColors`), `Theme.SetSystemColors` passes system color data to `System.Drawing.KnownColors.Update`, which populates them into `System.Drawing.KnownColors.ArgbValues`. In Mono 5 and earlier, this flowed through to the visible UI.

It seems that as of Mono 6, this table no longer affects the visible WinForms UI (although it is still being updated!). Instead, a same-but-different table in `System.Drawing.KnownColorTable` drives the system colors now. I was not able to identify the commit in which this change was made.

## Changes

Now `Theme.SetSystemColors` accesses `System.Drawing.KnownColorTable.s_colorTable` instead, and WinForms apps once again obey GTK and KDE themes:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

Fixes KSP-CKAN/CKAN#2881.
2019-11-27 19:13:11 +01:00
RizvanKuliev a73e7b8af2 [WinForms] Fix Recalculate in ScrollableControl (#17163) 2019-11-27 19:09:19 +01:00
Alexander Köplinger 9954579c2a [sdks] Call mono_gc_init_finalizer_thread() on iOS device runtimes
These runtimes are configured with `--with-lazy-gc-thread-creation=yes` so they don't start a GC finalizer threads by themselves.
XI does this because they want to delay creating that thread for some time after startup.

We can just do it immediately, the important part is that it _does_ exist.
2019-11-27 18:36:02 +01:00
Alexander Köplinger c66c7814d0 [sdks] Add option to enable debug server for iOS test app
Include .pdb files in iOS test app if the DEBUG variable is set and wait for managed debugger to attach.
2019-11-27 18:36:02 +01:00
Alexander Köplinger 18ddc2e45d [sdks] Fix loading of assemblies in assembly_preload_hook()
We need to check whether the assembly name passed to us already contains .exe/.dll and append it otherwise.
Fixes loading of assemblies via the hook, we were falling back to the usual path resolving logic (which doesn't work in netcore mode).
2019-11-27 18:36:02 +01:00
Alexander Köplinger a0d79a4886 [sdks] Add netcore option for iOS SDK 2019-11-27 18:36:02 +01:00
Alexis Christoforides 167913524f [merp] Use timeline (and generate timeline breadcrumbs) for SendExceptionToTelemetry() (#17913)
Fixes https://github.com/mono/mono/issues/17790
2019-11-27 11:56:12 -05:00
Bernhard Urban-Forster 4c595069e7 [arm] if mtriple is provided, do not set -march (#17937)
[arm] if mtriple is provided, do not set -march

E.g. when passing `-march=arm -mtriple=armv7s-ios`, `-march` will win and thus ignore `armv7s`.

Fixes https://github.com/mono/mono/issues/17931

* Caused by 5e318afd2e
* Related with 4cd5068230
* Almost fixed by e431093f73

Hopefully we are done with it.
2019-11-27 17:36:03 +01:00
monojenkins 9dfeb545d5 [bcl] Update BCL Linked Size (#17322) 2019-11-27 13:43:31 +01:00
Bernhard Urban-Forster 690af29ad9 [amd64] do not stack allocate on the application stack for the transtion from alstack handling (#17922)
### Backstory
This test is failing in the interpreter (ONLY on Debian9/amd64. Not Ubuntu/amd64. Not macOS/amd64):
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mcs/class/Mono.Debugger.Soft/Test/dtest.cs#L2484-L2499
Where the debuggee is just some unsafe code that tries to read from unmapped memory, thus causing a segfault.

So the test expects the runtime to send a CRASH event via the managed debugger interface. This happens very late in the crash handling machinery:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L1105

However, the runtime would never reach that, because it silently crashes when calling `backtrace`:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L939

At the point where the SIGSEGV happens, the native stack trace looks like this:
```
Thread 1 "mono-sgen" received signal SIGSEGV, Segmentation fault.
interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
4463                            sp[-1].data.i = *(gint32*)sp[-1].data.p;
(gdb) bt
%0  interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
%1  0x000055555569237a in interp_runtime_invoke (method=<optimized out>, obj=<optimized out>, params=<optimized out>, exc=0x0, error=0x7fffffffd8b0) at interp/interp.c:1900
%2  0x0000555555590745 in mono_jit_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=0x7fffffffd858, exc=0x0, error=0x7fffffffd8b0) at mini-runtime.c:3001
%3  0x0000555555778c4c in do_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=<optimized out>, exc=<optimized out>, error=0x7fffffffd8b0) at object.c:3052
%4  0x000055555577b7d0 in do_exec_main_checked (method=0x555555bfdc38, args=<optimized out>, error=0x7fffffffd8b0) at object.c:5184
%5  0x000055555559a2aa in mono_jit_exec_internal (argv=0x7fffffffdcb8, argc=1, assembly=0x0, domain=0x555555bf9d40) at driver.c:1320
%6  mono_jit_exec (domain=domain@entry=0x555555bf9d40, assembly=assembly@entry=0x555555c66ae0, argc=argc@entry=1, argv=argv@entry=0x7fffffffdcb8) at driver.c:1265
%7  0x000055555559b8e2 in main_thread_handler (user_data=<synthetic pointer>) at driver.c:1402
%8  mono_main (argc=<optimized out>, argv=<optimized out>) at driver.c:2622
%9  0x000055555558b757 in mono_main_with_options (argv=<optimized out>, argc=<optimized out>) at main.c:52
%10 main (argc=<optimized out>, argv=<optimized out>) at main.c:434
```
Stepping into the crash handling machinery, the picture looks a bit different:
```
Thread 1 "mono-sgen" hit Breakpoint 1, altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
871     {
(gdb) bt
%0  altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
%1  0x00005555556957de in interp_exec_method_full (frame=0x555555c53f08, context=0x555555c38350, clause_args=0x0, error=0x555555c53f18) at interp/interp.c:4461
%2  0x0000000000000000 in ?? ()
```
So it looks like we messed up in the transition from altstack back to the program stack.

 ### The fix

The problem is that `backtrace` consults the unwind information emitted at the point where SIGSEGV happens in `interp_exec_method_full ()`. However at that given program code, there is no function call that allocates space to pass arguments on the stack. Still, we massage the stack to do that in the signal handler, which does not add up with the emitted unwind information by the C compiler.

Note that without the fix, crashing in `backtrace` is the worst case. In other cases we might (1) get still a correct stack trace, or (2) we get a wrong stack trace, and `backtrace` will not crash. So this PR might improve the crash experience overall.

This also reverts commit cff400f3fae9775ac734f8a691339e59a272872c.
2019-11-27 12:20:47 +01:00
Zoltan Varga c4136ebd94 [interp] Non-recursive interpreter (#16461)
* Aling alloca_size to 8.

* Prevent some false pinning in the monitor-resurrection.cs test when using the non-recursive interpreter.

* [runtime] Add a mono_compiler_barrier () function.

* Add a MONO_GET_SP opcode to pass to enter/exit_gc_safe_region, ldloca might not return a native stack address with the interpreter.

* Add an interpreter/gc callback to mark the interpreter stack.

* Copy only an integer in ENDFILTER.

* Add a native_stack_addr field to InterpFrame so the EH code has something to compare against native stack addresses.

* Add a FrameStack structure to allow allocation of InterpFrame structures and stack data outside the native stack.

* Allocate InterpFrames/stack data from a frame stack instead of the native stack.

* Make some calls non-recursive.

* Remove an assert.

* Use non-recursive calls in CALLVIRT_FAST as well.

* Fix stack size allocation for vararg calls.

* Add some micro optimizations to the calling code.

* Add more micro optimizations.

* [dtest] disable Crash test on interpreter

* Add some implicit conversions to STSFLD to fix a corefx test failure.

* Disable the new test.
2019-11-26 04:31:32 +01:00
Zoltan Varga 5b4cd5ce50 WASM + Netcore. (#17915)
* [runtime] Throw an exception from Thread::StartInternal () if threads are disabled.

* Add more test suites and a corefx exclusions file.
2019-11-26 02:59:31 +01:00
Larry Ewing e917c95053 Add the reference assembly path for the Http implementation (#17908) 2019-11-25 19:21:38 -06:00
dotnet-maestro[bot] cd190481bd Update dependencies from https://github.com/dotnet/arcade build 20191124.1 (#17899)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19574.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19574.1
2019-11-25 21:22:41 +01:00
Alexander Köplinger e437330808 Remove mkinstalldirs invocation from two places (#17910)
These can be easily replaced so we don't need to mirror over the mkinstalldirs file to the new repo
2019-11-25 21:10:26 +01:00
Alexander Köplinger 8aa1cdfe17 [configure.ac] Don't run compiler server check in netcore mode (#17911)
We don't need it there.
2019-11-25 21:10:08 +01:00
Calvin Buckley 836f453906 Fix BinaryReader on BE (#17893)
Fix BinaryReader on BE

The ints passed into the int array ctor needed to be endian swapped.
Interestingly, the BinaryWriter implementation currently being used
(not refsrc AFAIK) does this on write just fine.

Fixes ReadDecimal test on AIX, and adds another test to ensure that
Write(decimal) can be read back by ReadDecimal in case.
2019-11-25 19:15:55 +01:00
Alexander Köplinger bdf4790fa0 Fix boehm on Windows i386 builds (#17901)
After https://github.com/mono/mono/pull/16832 was merged the run-msbuild.bat
script has an additional required parameter for specifying the GC.

However the script only passed the MONO_TARGET_GC property to msbuild
if the script had no additional parameters which doesn't make sense.

Always pass the parameter, this should fix the boehm build on Windows i386.
2019-11-25 18:58:42 +01:00
Larry Ewing b42845c567 Fix issue in debugger variable inspection (#17900) 2019-11-25 11:55:21 -06:00
Marek Habersack 10795da1c0 [WIP] Make debugger agent startup faster (#17898)
Mono SDB's debugger agent takes a `host:port` parameter to specify the host and
port on which to listen for debugger connection. `host` is passed, via
`mono_get_address_info`, to the `getaddrinfo(3)` standard library call which
does the job of translating the name to the IP address. However, there's no
special treatment for situations when `host` is already an IP address. In this
case `getaddrinfo` takes a lot of time to perform at least two DNS lookups (if
there's just a single DNS server configured) which will time out as the IP
address in `host` will not be resolved by any of the servers.

`getaddrinfo` takes a "hints" parameter where caller can specify some
information about the `host:port` pair to make the lookup faster/easier.
However, Mono doesn't specify the address family in the `hints` parameter, thus
forcing the slow path of DNS lookups on `getaddrinfo`.

This commit adds a new hint named `MONO_HINT_NUMERIC_HOST` (which maps to
`AI_NUMERICHOST` for `getaddrinfo`) which will allow, if set, `getaddrinfo` to
take a faster route in address discovery. Debugger agent will, in most cases,
get an IP address to instead of host name so we call `mono_get_address_info` up
to 3 times (IPv4, IPv6 and unspecified) in order to make the lookup faster.
2019-11-25 17:52:58 +01:00
Calvin Buckley f47b7bbf4f Fix flawed BufferTest on big-endian (#17894)
Fix flawed BufferTest on big-endian

Test ignored byte order when considering integer comparison. This
tests different cases for each endian, and explains why the byte
order is different.
2019-11-25 16:25:55 +01:00
Kenneth Pouncey 5d981c2679 [wasm] [bindings] Method invoke signature argument validation. (#17853)
* Implement validation for signature and argument count consistency.

- throw error `Parameter count mismatch.` when arguments and signature counts are not consistent.

* Fix tests that have Parameter count mismatch.

* Add browser binding tests for `Parameter count mismatch
2019-11-25 08:24:33 -06:00
Kenneth Pouncey e8d70c333f Add tests for Promise library.
- Right now only Bluebird promise library is tested right now.
- If more are needed later then we can add them.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 2f1e2f8132 Add a more reliable way of identifying a Promise to be marshaled.
- This is left as generic as possible for now.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 64e2864fef Add a more descriptive error in case of fetch response not returning back what it should.
- An example of this is when a Promise library is being used and the Promise can not be identified correctly.  Thus the internal error.
2019-11-25 08:23:45 -06:00
Vlad Brezae 1ef7a29836 [interp] Resolve tokens also from wrappers during optimization (#17880)
[interp] Resolve tokens also from wrappers during optimization

Remove some duplicated code.
2019-11-25 14:45:56 +01:00
Calvin Buckley 004fd06b96 [eglib] In AIX/generic POSIX, handle null addresses better (#17892)
Fixes crash in tests on AIX.

Matches b0d966bd7a
2019-11-25 02:52:54 -05:00
Calvin Buckley a3f4def8b6 Allow runtime to be built with C++ on AIX (#17672)
* Allow runtime to be built with C++ on AIX

Many AIX/PPC/BE specific codepaths didn't do casting properly,
since C++ is much stricter than C about pointer typing.

Also specify the C99 format macros early as possible, since
inttypes may get included before eglib gets a chance to set the
macros.

Also explicitly use -pthread, since not using it is the cause of
many libstdc++ crashes on AIX.

* Oops, should have been that define instead

* Suggested changes from Jay on style

* Remove unneeded cast
2019-11-24 17:08:19 -05:00
Alexander Köplinger 280e9d2423 Fix warnings in runtime build (#17887)
This makes at least the netcore-mono build in dotnet/runtime warning-free.
2019-11-24 00:16:20 +01:00
Alexander Köplinger 165e28ae97 [netcore] Disable libmonoruntime-support.la build on netcore (#17885)
It's used for the zlib compression helpers used by System.IO.Compression in the old Mono BCL.
On corefx this is handled differently so we don't need it there.

To avoid automake complaining about missing files we include the .c files directly instead of in the Makefile.
2019-11-23 10:06:40 +01:00
Alexander Köplinger 2ccd6122bd [netcore] Disable a test which fails on the interpreter
The test is wrong, will be fixed upstream.
2019-11-22 23:00:28 +01:00
Alexander Köplinger 81e305421d Fix run-msbuild.bat invocation in pipeline-netcore-runtime.yml 2019-11-22 22:48:53 +01:00
Alexander Köplinger 0d80372173 Make configure.ac and Makefile.am work without certain directories (#17871)
Those folders won't be present in the dotnet/runtime repository.

We need to factor out AC_OUTPUT into a separate file so we can continue
to share configure.ac but not the list of files in AC_OUTPUT since
autotools doesn't support variables there.
2019-11-22 22:48:11 +01:00
Zoltan Varga 93e46c687b [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case. (#17832)
* [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case.

* [amd64] Fix the return value mapping in gsharedvt calls, the return address might be in different registers in the caller and callee.

* [runtime] Always initialize del->method_ptr even if interp_method is set, JITted code makes calls through it in mixed mode.

* Add a test for open delegates and gsharedvt.

* Increase nrgctx trampoline count, its needed by dynamic-method-churn.exe in full-aot-interp mode.
2019-11-22 20:34:14 +01:00
Jonathan Chambers cc137237d8 Replace embedded libgc with Unity fork of recent Boehm (bdwgc) (#16832)
* Initial commit to migrate libgc to bdwgc.

* Fix submodule url

* Don't force Boehm as runtime wrapper. Was just for testing locally.

* Build bdwgc as single object file. Helps with performance.

* Bump bdwgc submodule.

* Attempt to fix windows MSVC based build.

* Remove any references to GC_INSIDE_DLL as we manually managed threads on Windows.

* Allow building boehm on Windows amd64.

* Allow building boehm with msvc.

* Bump bdwgc to fix line endings

* Use LF for sh, am, m4, and ac files in bdwgc

* Bump bdwgc & libatomic_ops with fix for cygwin

* Bump bdwgc to fix mono-boehm being generated as libtool wrapper script.

* Link the static boehm library.

* Revert "Link the static boehm library."

This reverts commit 1f90d81d08308a61c890743cd79d8f6bb791bc99.

* Use --export-all-symbols on cygwin/mingw to fix issues with boehm causing mono symbols not to be exported.

* Fix
2019-11-22 18:03:23 +01:00
Filip Navara 0fed03ed63 Fix InternalGetHashCode for boehm/null GC (#17859)
It was unintentionally shifting the address to left instead of right. It should be shift right to get rid of zeroes from the alignment.
2019-11-22 07:51:43 -05:00
Alexander Köplinger 5fedc42ccf Fix automake warning 2019-11-22 13:18:10 +01:00
Kenneth Pouncey 9e1d59b8cc Remove unused GetHttpMessageHandler from WebAssembly.Net.Http.
- Remove linker.xml from project
2019-11-22 13:12:17 +01:00
Zoltan Varga ecd67a91c4 [wasm] Add aot+netcore support. (#17798)
* [wasm] Compile corelib as 32 bit.

* [wasm] Add a netcore cross compiler.

* [wasm] Add netcore+aot support.

* [aot] Load the dedup module lazily instead of at startup.

The previous version didn't work on netcore because it couldn't resolve references to corlib etc.
since they were not yet registered in the root domain at the time of loading.

* Add more icall trampoline signatures.

* Add more netcore pinvokes.

* Fix the build.

* Add more icall trampoline signatures.

* Add an --embed-file option to the packager.

* Add more netcore pinvokes.

* [interp] Return FALSE for IsDynamicCodeCompiled.

* Add a netcore project and targets for running BenchmarkDotNet.

* Fix pinvoke tables.

* Add missing files.

* Fix make clean.

* Add a comment.

* Revert "Fix pinvoke tables."

This reverts commit 998065e77614b7e621d2873510aabc3930669ca9.

* Make xunit-runner into a dotnet project. Add a netcore version.
2019-11-22 12:42:55 +01:00
Egor Bogatov 6a5c44b15e [netcore] Run tests with Interpreter (#17862)
* Run tests with interp and llvm

* fix build

* Run tests with llvm

* Add a few tests to rsp for LLVM
2019-11-22 09:14:23 +01:00
Kenneth Pouncey 202cb6a648 Add more descriptive message to the PlatformNotSupportedException (#17864) 2019-11-22 08:40:30 +01:00
Jay Krell dadc3d3117 Remove handles/MonoError from Mono.RuntimeGPtrArrayHandle. (#17806) 2019-11-21 14:15:29 -05:00
Jo Shields 9e00330af3 Stop referencing files from Mono BCL in Netcore System.Private.CoreLib (#17866) 2019-11-21 19:48:00 +01:00
Bernhard Urban-Forster 058d157648 [interp] fix array_element_size intrinsic (#17857)
[interp] fix array_element_size intrinsic

`mono_class_array_element_size` gives us the size of the provided MonoClass if it would be an array element. But here we want the element size of a given array's MonoClass. That's what `mono_array_element_size` is returning.

This leads to all kind of weird crashes otherwise, specifically here:
https://github.com/mono/mono/blob/2c20649539ac16e069a65f2a750c793eb341e50f/netcore/System.Private.CoreLib/src/System/Array.Mono.cs#L64

Which is then used later to compute the size to memset in order to clear the content of an array. If it's larger than it should be, then this will cause memory corruption. For example this would crash eventually:

```csharp
using System;
using System.Collections.Generic;

namespace HelloWorld {
    class Program {
        static void Main(string[] args) {
            for (int j = 0; j < 0x1000; j++) {
                var d = new Dictionary<string, string> ();
                for (int i = 0; i < 197; i++)
                    d.Add (i + "", i + "foo");
                d.Clear ();
            }
        }
    }
}
```

Thanks to @EgorBo for reporting.
2019-11-21 11:56:03 +01:00
Alexander Köplinger b6e4ddd73c [metadata] Remove native threadpool from netcore build (#17856)
We use the managed threadpool there.
2019-11-21 10:01:39 +01:00
Egor Bogatov 2c20649539 [llvm] Recognize GEP (#17844)
* initial impl

* improve jit

* fix copy-paste

* disable for non-netcore

* clean up
2019-11-20 21:25:24 +01:00
Zoltan Varga 4961d411e7 [interp] Move the freeing of interp method info into a callback. (#17852)
[interp] Move the freeing of interp method info into a callback.
2019-11-20 19:56:09 +01:00
Egor Bogatov 27e6e812f1 enable gss on android (#17838) 2019-11-20 10:53:14 -05:00
Alexander Köplinger 0396037d31 [debugger-agent] Fix CMD_VM_ALL_THREADS returning wrong value due to unitialized variable (#17847)
`remove_gc_finalizing` wasn't initialized so its value is undefined.
The `count_thread_check_gc_finalizer()` function only sets it in some cases so we could end up with the variable having an undefined (and probably not 0) value.
This resulted in the returned thread count from the debuggee being wrong because we'd decrement `count` even though we shouldn't.

Regression from https://github.com/mono/mono/pull/15618
2019-11-20 08:28:58 +01:00
dotnet-maestro[bot] a6db160596 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17690)
* Update dependencies from https://github.com/dotnet/arcade build 20191102.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19552.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19552.1

* Update dependencies from https://github.com/dotnet/core-setup build 20191104.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19554.1

* Update dependencies from https://github.com/dotnet/corefx build 20191103.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19553.8

* Update dependencies from https://github.com/dotnet/arcade build 20191108.11

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19558.11
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19558.11

* Update dependencies from https://github.com/dotnet/core-setup build 20191109.5

- Microsoft.NETCore.App - 5.0.0-alpha.1.19559.5

* Update dependencies from https://github.com/dotnet/corefx build 20191109.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19559.6

* Update dependencies from https://github.com/dotnet/arcade build 20191117.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19567.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19567.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191114.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19564.1

* Update dependencies from https://github.com/dotnet/corefx build 20191113.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19563.6

* Use NETCoreApp version 5.0.0-alpha.1.19563.3

Later versions have an issue and the new dotnet/runtime assets aren't ready yet.

* Disable failing tests

* Fix merge error
2019-11-19 16:47:57 +01:00
Steve Pfister 0c3ffb9cc9 Fixes https://github.com/mono/mono/issues/15805 at least in terms of what may be making it crash. (#17270)
Adds a null check around the sslStream before trying to dispose.
2019-11-19 14:23:51 +01:00
Zoltan Varga b8fe43da3e [interp] Add some missing netcore intrinsics. (#17782) 2019-11-18 13:55:15 +01:00
Steve Pfister 0269f7b479 Bump corefx to pick up https://github.com/mono/corefx/pull/370 (#17758)
Backport of https://github.com/dotnet/corefx/pull/34560

Added HAVE_LCHFLAGS & HAVE_STAT_FLAGS to configure.
2019-11-18 12:13:16 +01:00
Filip Navara e2ccdac3f4 [llvm] Add support for LLVM JIT on ARM64 (#17827)
* [llvm] Add support for LLVM JIT on ARM64

* Fix tabs
2019-11-18 00:24:11 +01:00
Jérôme Laban 965efade84 Update emscripten to 1.39.3 (#17816)
Includes updates for https://github.com/emscripten-core/emscripten/issues/9562 and side modules.
2019-11-16 13:59:28 +01:00
Dotnet-GitSync-Bot 6785b27567 Mirror changes from mono/runtime (#17803)
It looks like the mirror didn't fully quiesce prior to snapping coreclr/corefx for the runtime consolidation.  As a result the libraries build is breaking.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-16 11:27:32 +01:00
Nikolay Sivov 18dd48cad8 [mini] Fix NULL pointer crash in mono_decompose_vtype_opts(). (#17801)
* [mini] Fix NULL pointer crash in mono_decompose_vtype_opts().

All other call sites for mono_type_size() are already passing non-null pointer there.

* [mini] Rename variable dummy -> align.
2019-11-15 17:56:55 +01:00
Marcos Cobeña Morián 92dd4d3883 [Wasm] Forced filesystem creation (#17789)
* Forced filesystem creation

* [Wasm] Added new hello-io sample
- Goal is testing file preloading through System.IO
- Filesystem is now always forced to support later file preloading
- build-io-sample target showcases such

* Added --enable-fs flag to packager.exe

* Update packager.md
2019-11-15 15:24:50 +01:00
Jan Kotas 3ceecb8db0 Fix build breaks
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Carlos Sanchez Lopez d3fbf4a1d3 Fix NetStandard issue in System.IO.FileSystem.AccessControl in 5.0 (#42546)
Fix NetStandard issue in System.IO.FileSystem.AccessControl

We built the fix incorrectly so that it would only apply for netcoreapp3.0 but customer needs it on netstandard2.0 (desktop + core).  Customer impact is that they’ll see a PlatformNotSupported exception rather than the new API.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 00b0170bdc Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#42531)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Tanner Gooding c93801d458 Adding tests for the new apis to convert between S.Numerics and S.R.Intrinsic vectors (#27481)
* Updating Vector128.AsVector3 to have a functioning implementation

* Adding test template metadata covering the new methods for converting between S.Numerics and S.R.Intrinsics

* Regenerating the existing tests

* Generating the new Vector conversion tests

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Emmanuel AndréandStephen Toub 5645032fc6 Add span-based CopyTo and CopyToAsync methods (dotnet/coreclr#27639)
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <[email protected]>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
John adcaf88db0 Fix Vector128 and Vector256 WithUpper documentation (dotnet/coreclr#27274)
* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector256.cs

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 1f2f9b929b Add tracing for handling of dependencies for Assembly.LoadFrom (#27833)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 3242b5f354 Add tracing for Resolving/AssemblyResolve event handler invocation (#27788)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
ts2do 35d94dc8ff Address bugs in BigInteger (dotnet/coreclr#27280)
* Method Add(ref BigInteger lhs, uint value, ref BigInteger result) would store most of the result blocks into lhs instead of result.
* Method ShiftLeft(ulong input, uint shift, ref BigInteger output) with a shift argument exceeding 32 would generally compute the higher blocks incorrectly.
* Multiply(ref BigInteger lhs, uint value, ref BigInteger result) would not set result._length in some cases.
* IsZero() would incorrectly return false for non-canonical zeros with _length > 0.

Fix:
* Inline Add(ref BigInteger, uint, ref BigInteger) into Add(uint).
* Inline ShiftLeft(ulong, uint, ref BigInteger) into Pow2.
* Inline ExtendBlock and ExtendBlocks into Pow2.
* Properly handle 0 in SetUInt32 and SetUInt64.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 2a108e958f Use Stream's BeginReadInternal from FileStream (dotnet/coreclr#27737)
FileStream has two modes, decided upon at construction time.  When it's created in non-async mode, the Read/WriteAsync methods end up queueing work items to invoke the synchronous Read/Write methods.  To do this, the base methods on Stream delegate to Begin/EndRead/Write (since they were around first) and then the resulting IAsyncResult is wrapped.  However, Stream has an optimization that checks to see whether the derived stream actually overrides Begin/EndXx, and if it doesn't, then it skips using those and goes straight to queueing a work item to Read/Write.  However, FileStream does override those, but when it delegates to the base implementation because it's in non-async mode (rather than because it's a type derived from FileStream), going through Begin/EndXx is just unnecessary overhead.  So, in the right circumstances, we can call to Stream's special helper instead.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Zoltan Varga 25b7e6bd42 [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files. (#17795)
* [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files.

* Fix a typo.

* Revert unrelated changes.
2019-11-15 09:04:35 +01:00
Vsevolod Kukol 805c7f5436 [MacSDK] Bump xamarin-gtk-theme.py to latest revision from private bockbuild (#17785) 2019-11-14 09:27:33 +01:00
Jay Krell 19a44cd32b Remove handles from System.Diagnostics.Debugger. (#17713)
* Remove handles from System.Diagnostics.Debugger.
memset (&x, 0, sizeof (type)) to memset (&x, 0, sizeof (x)).

* Bump API snapshot submodule
2019-11-14 08:18:02 +01:00
Zoltan Varga 1d81534d27 [interp] Add some inline checks from the JIT. (#17784) 2019-11-13 22:34:01 +01:00
Zoltan Varga a0fcc29d5d [runtime] Fix some handle usage in native-library.c (#17781) 2019-11-13 18:12:09 +01:00
Jay Krell 73431403c7 Remove handles from ves_icall_System_Array_CanChangePrimitive. (#17708)
Remove handles from ves_icall_System_Array_CanChangePrimitive.
2019-11-13 17:55:52 +01:00
Thays Grazia dd18ec40dd [debugger] Assert when async debug a generic method (#17727)
* When we try to call a method to get the async_id to do an async debug and we are trying to do this in a generic method like this:

async Task<T> ExecuteAsync_Broken<T>()
 {
            await Task.Delay(2);
            return default;
  }

We need to inflate the generic type before call the method or we will get the error:  Could not execute the method because the containing type 'System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[T_REF]’, is not fully instantiated.

Fixes #17549
Fixes #17569
2019-11-13 16:29:33 +01:00
imhameed fe0f824db5 [llvm] Use explicit null checks with LLVM. (#16954)
* [llvm] Use explicit null checks with LLVM.

When the LLVM backend is used for a method, explicit null checks will
unconditionally be used at the mini IR level. We had previously used
implicit null checks at the mini IR level combined with llvm-volatile
loads and stores to avoid mis-optimization of the generated code.

During AOT compilation, LLVM's `ImplicitNullChecks` pass is enabled.
This conservatively elides these explicit null checks. This pass only
modifies branches tagged with `make.implicit` metadata.

Null check branches are now tagged with `make.implicit` metadata, with
three exceptions:

1. For platforms with backends that do not support zero page fault
handling, null check branches are not tagged with `make.implicit`.

2. This commit also adds a `MONO_DEBUG` option:
`llvm-disable-implicit-null-checks`. When it is enabled, null check
branches are not tagged with `make.implicit`.

3. This commit alters the behavior of the `explicit-null-checks`
`MONO_DEBUG` option. Enabling `explicit-null-checks` also implies
`llvm-disable-implicit-null-checks`, because this option is documented
(e.g. in https://www.mono-project.com/docs/debug+profile/debug/) to
completely disable signal/seh-based null checks.

* Add llvm-disable-implicit-null-checks to the MONO_DEBUG help string.

* Add llvm-disable-implicit-null-checks to mono.1.

* Don't tag null checks in EH regions with `make.implicit`.

This relocates the logic formerly used to guard individual LLVM-level
loads and stores with explicit null checks inside EH regions.

* Use atomic/volatile loads and stores for the terimation flag in thread-suspend-suspended.

LLVM was hoisting the load of and branch against `finished` out of
thread t1's loop, resulting in an infinite spin.

* Use ImplicitNullChecks with LLVM JIT.

* Suppress emission of the .llvm_faultmaps section.

Bump external/llvm to release_60
(7a8dc89adbe7e123220e070a527e096ee91e66d5).

* [ci] use LLVM built from repository
2019-11-13 01:35:44 +01:00
Zoltan Varga eaf895f2d0 [wasm] Change netcore support to use a prebuilt corefx runtime. (#17773)
* [wasm] Change netcore support to use a prebuilt corefx runtime.

* [wasm] Build and test netcore support on CI.

* [wasm] Disable netcore tests on CI for now.

* [wasm] Add an error check.

* [wasm] Add a linker exclusion.

* [wasm] Add netcore runtimes to the nupkg.
2019-11-13 01:30:09 +01:00
Filip Navara bd278dd00d Mirror changes from mono/corefx,coreclr (#17777)
* Rewrite the rest of the array non-generic fast paths in C# (#27703)

* Rewrite the rest of the array non-generic fast paths in C#

* Add missing resource string

* Use CorElementType and handle native ints to match the unmanaged implementation

* Avoid try pattern for Sort and Reverse

* Match the native implementation more closely

Signed-off-by: dotnet-bot <[email protected]>

* Rewrite Enum.CompareTo in C# (dotnet/coreclr#27792)

* Rewrite Enum.CompareTo in C#

* Handle floating point too for match the existing native implementation

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused argument from GetRegionDisplayName (dotnet/coreclr#27806)

* Remove unused argument from GetRegionDisplayName

* Address race conditions in CultureData

While most of the race conditions in this file are benign, in these cases, a thread could end up returning an intermediate value due to multiple potential writes to the same field.

Signed-off-by: dotnet-bot <[email protected]>

* Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#27716)

Will enable the rule in corefx once these changes have propagated.

Signed-off-by: dotnet-bot <[email protected]>

* Remove non-shared Enum.CompareTo/InternalCompareTo implementation, add new Array icalls used by shared code
2019-11-12 18:46:00 +01:00
Bernhard Urban-Forster 4728cec01f [llvm] use multiple cores to build llvm if ninja is not available (#17778) 2019-11-12 18:45:45 +01:00
Jo Shields 8bceb40878 Move zlib to its own directory, and re-enable subdir-objects
This has the following benefits:

* Clearer structure for mirroring
* Slight build time improvement on bundled zlib use (as it only builds once)
* Removes pages of subdir-objects spew from autogen
2019-11-12 09:31:08 +01:00
abrevet-dev 8683fa6db7 [WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask setted (#17660)
[WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask se…





<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-11-12 01:55:53 +01:00
abrevet-dev 2b4801d9f1 [WinForms]: Fix #17546 Fix TabPage position when enabling MultiLine (#17670)
[WinForms] Fix TabPage position when enabling MultiLine

Fixes #17546
2019-11-12 00:15:51 +01:00
abrevet-dev 61811075ca [WinForms]: Fix #16632 special values (-1 and -2) of ListView Column header Width was not handled when using List.View. (#17669) 2019-11-11 23:00:45 +01:00
abrevet-dev 9eb0bcae57 [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs (#17662)
* [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs of ScrollBar.cs

* [WinForms] Fix 12249 scroll orientation was not defined in ScrollEventArgs of DataGridView.cs
2019-11-11 22:59:53 +01:00
Jo Shields c256fc86b6 Delete some LLVM test cases from make dist (#17772)
Many of the tests from the llvm-project monorepo cause problems to `make dist`
2019-11-11 17:42:51 +01:00
Alexis Christoforides 722c6f7dd5 [netcore] Complete Monitor.LockContentionCount implementation (#17719)
* [netcore] Complete Monitor.LockContentionCount implementation

Fixes https://github.com/mono/mono/pull/16538
2019-11-11 13:03:03 +01:00
Jan Kotas 594bbbef25 Relax visibility
Signed-off-by: dotnet-bot <[email protected]>
2019-11-10 09:29:18 +01:00
Marek Safar d90030d273 Fixes build 2019-11-09 20:45:42 +01:00
Ryan Lucia 9664df7eb3 Move AssemblyLoadContext.GetResolvedUnmanagedDll to shared (#27775)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
rob313663 e46f29313d Fixed typos in comments. (dotnet/coreclr#27789)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
Joseph Musser 5bf661d7f9 StackTrace.GetFrames should return an array of non-nullable frames (#27787)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 09:37:25 +01:00
Levi Broderick ae02b6677b Improve Encoding.UTF8.GetString / GetChars performance for small inputs (#27268)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 08:13:11 +01:00
Alexander Köplinger f9b3b5276d Fix llvm path in CODEOWNERS file 2019-11-09 00:44:57 +01:00
Alexander Köplinger ed2d5f69e0 [netcore] Fix RuntimePropertyInfo.GetValue() in FullAOT scenarios (#17757)
It tried to use dynamic codegen which is not available in FullAOT.
We can use the `RuntimeFeature.IsDynamicCodeSupported` to check for this, added a JIT intrinsic which returns false in FullAOT.

Used a linker friendly way to check for TypeBuilder by adding an `internal virtual IsTypeBuilder` method in the `Type` class which returns false, and overwrote it in `TypeBuilder`.
2019-11-09 00:33:00 +01:00
Zoltan Varga 73b50890cc [wasm] Bump emscripten. (#17748)
* [wasm] Bump emscipten to 1.39.2.

* Disable some code in libtest.c which causes emscripten to fail.

* [wasm] Source emscripten env before make install as well.
2019-11-08 14:43:32 -05:00
Nathan Ricci 5b0a822afb Implement GC.GetGCMemoryInfo (#17607)
Implement the GC.GetGCMemoryInfo API.
2019-11-08 13:58:37 -05:00
Ryan Lucia d5b6cf33b3 [loader] Unmanaged library refactoring and NativeLibrary implementation (#17369) 2019-11-08 13:42:56 -05:00
Ryan Lucia 68d142912d [netcore] Run individual CoreCLR test suites (#17753)
This is implemented in a lazy way for convenience when manually testing until the tests are wired up to xunit
2019-11-08 13:14:06 -05:00
Alexis Christoforides 23f2baa71e [merp] Remove extraneous waitpid invocation (#17723)
The extraneous call basically guarantees that the call right after will fail. Fixes merp stating that its process has failed when it hasn't.
2019-11-08 11:40:04 -05:00
Jo Shields 3948a065af Merge pull request #17749 from directhex/dotnet-llvm-monorepo
[LLVM] Change llvm submodule to dotnet-org fork of official LLVM git repo
2019-11-08 10:42:20 -05:00
lateralusX fad91a1c1e Fix Windows LLVM build using new repository structure. 2019-11-08 09:21:30 -05:00
Dotnet-GitSync-Bot 759a8c1847 Mirror changes from mono/corefx,coreclr,corert (#17761)
* Add EventWaitHandle creation extension method that takes an ACL (#42213)

* Add EventWaitHandle creation extension method that takes an ACL

* Call OpenExisting, add basic unit tests, move MAX_PATH in csproj for all Windows platforms, ensure same exceptions are thrown in both netcore and netfx.

* Address suggestions: using for the created handle, simplify try finally, merge methods, remove an unnecessary Debug.Assert

* simplify using, reorganize PInvokes a bit, let VS format resx as it expects it, remove BOM from csproj

* Add more unit tests

* Save basic access rights in a constant, generate name randomly

* Use FullControl (EVENT_ALL_ACCESS); update unit tests to use EventWaitHandleRights

* Apply suggestion of creating an EWH, then replacing its SWH.

* Remove using causing chaos, add more unit tests, remove null string unit test.

* spacing

* Address comments

* Remove unnecessary checks/exceptions, update unit tests

* Fix netfx x86 ut failure - different exception for mode validation.

* Small documentation fix

* Remove documentation for exception thrown with null security

* more documentation fixes

* New file for unit tests

* Dispose and remove windows only attributes in tests

* suggestion to not use var

* Remove duplicate resx after merge

* Readd cs files lost during merge

* Remove resx modification since it was untouched for this PR

* Address test comments

Signed-off-by: dotnet-bot <[email protected]>

* Move NativeLibrary.Free null check to managed (dotnet/coreclr#27744)

Signed-off-by: dotnet-bot <[email protected]>

* Arm64: Add S.P.CoreLib instrinsics UnzipEven and UnzipOdd (#27731)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 07:11:13 -05:00
Tamar Christina 771b5fb18d Arm64: Update S.P.CoreLib for ZipLow and ZipHigh (dotnet/coreclr#27730)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 11:01:17 +01:00
Carlos Sanchez Lopez 229d3cc097 Add Semaphore creation extension methods that take an ACL (#42377)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Semaphore with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Semaphore, then set its ACLs. This presents a few problems:
- Potential security hole as semaphores can be accessed between creation and modification.
- Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Semaphore and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 01:59:07 +01:00
Zoltan Varga c0a9d6bf76 [aot] Improve the aot mangler a bit, handle bool/char as a primitive type and avoid emitting a System prefix. (#17739) 2019-11-07 18:18:12 -05:00
Tanner Gooding 8637912a99 Updating NumberToFloatingPointBitsSlow to handle the remaining parsed fractional digits being zero. (#27688)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 23:39:21 +01:00
Jo Shields 4d579d00ff Bump LLVM submodule 2019-11-07 15:52:56 -05:00
Tamar Christina b924ffa5e0 Arm64: Add S.P.CoreLib for AddV. (dotnet/coreclr#27663)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 21:47:22 +01:00
Ryan Lucia 2c22d0de2e [netcore] Propagate ALCs through reflection functions (#16949)
* [reflection] Pass assembly load context through mono_reflection_get_type_with_rootimage

The entry points are mono_reflection_type_from_name_checked and
mono_reflection_get_type_checked which for now pass in an ambient ALC, but
should be updated to take an ALC argument.

Internally mono_reflection_get_type_with_rootimage is recursive through a half
dozen helper functions which all get an ALC argument too.

The principle place where the ALC is used is in
_mono_reflection_get_type_from_info which passes it to
mono_assembly_loaded_internal and to replace the call to mono_assembly_load by
a direct call to mono_assembly_request_byname with an explicit ALC set in the
request.

* Mark mono_assembly_load as external only

* Pass an alc through non-static reflection functions

* Fix usage in wasm debugger

* Relax assertions due to wasm
2019-11-07 15:02:23 -05:00
Marcos Cobeña Morián 8cc0402967 Enabled --preload-files without AOT (#17732) 2019-11-07 15:00:36 -05:00
Jo Shields fe69f5b699 Fix .gitmodules typo 2019-11-07 14:03:06 -05:00
Jo Shields c1c483cef8 LLVM changes for SDKs 2019-11-07 12:16:29 -05:00
Jo Shields d0d779b8d9 Update AzDo pipeine for new LLVM location 2019-11-07 12:09:58 -05:00
Alexander Köplinger 71b78536ad [sdks] Add xunit to iOS test runner and add results reporting (#17744)
- Allows the on-device runner to work with the xunit framework in preparation for netcore.

- Added a simple UI to show the test run progress

- Report the test result XML back to the host so we can process it on Jenkins/AzDO
2019-11-07 18:09:51 +01:00
Jo Shields b61263e994 [llvm] redirect from mono llvm-mirror fork to dotnet llvm-project fork 2019-11-07 12:03:29 -05:00
Jo Shields 9e6b57ce5a delete old llvm 2019-11-07 11:47:32 -05:00
Aleksey Kliger (λgeek) 2c56ac54c0 [profiler] Fix log profiling of native to managed wrappers (#17740)
* [profiler] Add test case for pinvokes and reverse pinvokes

* [profiler] Fix crash in native to managed wrappers

When we call back to managed from a pinvoke, we're in GC Safe mode and the
profiler instrumentation is inserted in the wrapper before it does the
transition to GC Unsafe.  So we need an extra transition before the profiler
calls its locking functions

Fixes https://github.com/mono/mono/issues/17687

* [coop] Mark internal GC Unsafe functions with MONO_PROFILER_API

Allow the profiler to transition to GC Unsafe mode

* [profiler] ptestrunner.pl fix profiler modules directory

we used to specify mono/mini/.libs but that's not where the profiler plugins live, they're in mono/profiler/.libs.   We probably picked up the profiler plugins from the system Mono install.
2019-11-07 11:42:52 -05:00
Thays Grazia 4bdacea237 [embed] Assert when call mono_runtime_object_init (#17589)
* Fix assert when calling externally call mono_runtime_object_init, mono_object_get_domain, mono_string_to_utf8
* Adding samples/embed as unit-test
2019-11-07 11:22:25 -03:00
Carlos Sanchez Lopez 3c84633c7c Add Mutex creation extension methods that take an ACL (#42281)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Mutex with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Mutex, then set its ACLs. This presents a few problems:

Potential security hole as mutexes can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Mutex and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 12:27:50 +01:00
Elinor Fung 0886ddb3a0 Populate properties of AssemblyLoadStart/Stop events (dotnet/coreclr#27558)
* Populate properties of AssemblyLoadStart/Stop events
* Add tests for basic start/stop binder tracing events
* Add property for ALC of requesting assembly

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 09:54:23 +01:00
Jay Krell 7b4b5e72d9 Remove handles from ves_icall_System_Array_InternalCreate. (#17681)
Remove handles from ves_icall_System_Array_InternalCreate.

Also combine two allocas.
2019-11-07 07:36:07 +01:00
Zoltan Varga 0680073b87 [runtime] Fix locking in mono_get_seq_points (). (#17738)
Fixes https://github.com/mono/mono/issues/17737.
2019-11-06 19:50:59 -05:00
Johan Lorensson bda239f44c Switch away from Start-Process, Wait-Process in build.ps1. (#17730)
Running Start-Process, Wait-Process on msbuild.exe have showed a couple
of hangs in the build script. This is most likely due to msbuild.exe
spawned child processes (like compile servers) running longer than
spawned msbuild.exe process. Since there is no specific requirements
when spawning the process from build.ps1, switch to regular process
exec and LastExitCode.
2019-11-06 16:51:31 +01:00
Johan Lorensson 1a6c092019 Explicit update/init only LLVM BTLS repro on external MSVC build. (#17691)
Reduce the scope of git submodule update --init to only LLVM and/or BTLS
repositories when building external MSVC dependencies.
2019-11-06 16:51:14 +01:00
Marek Safar 965beadc1e [System.Private.CoreLib] Remove unused Array methods 2019-11-06 12:46:30 +01:00
Stephen Toub e506008ebb Add Span.Sort, and make Array.Sort span-based (dotnet/coreclr#27700)
* Add Span<T>.Sort

Shares existing Array-based implementation by changing that implementation to be span based.

* Additional work to enable span-based sorts

- Cleans up changes from the previous commit, e.g. corrects nullable annotations, removes TODOs, using GetRawSzArrayData instead of GetRawArrayData, etc.
- Passes spans around rather than spans+lo+hi.
- Deletes the native TrySZSort, preferring to use the managed implementation in all cases.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-06 12:46:30 +01:00
Bernhard Urban-Forster 29ba75c154 Bump Corefx (#17694)
* Bump Corefx

New commits in mono/corefx:

* mono/corefx@c44efe7297 [mono] limit DegreeOfParallelism to 16 (#369)

Diff: https://github.com/mono/corefx/compare/6e65509a17da898933705899677c22eae437d68a..c44efe7297f8e11cab157c7e96b998dc936d53b7

* another bump
2019-11-06 07:36:17 +01:00
Jay Krell 1e8e357831 Handles reduction -- 4 MERP functions. (#17711)
Handles reduction -- 4 MERP functions.
2019-11-06 03:56:10 +01:00
Jay Krell aa506ca1bb error_init reduction. (#17712) 2019-11-05 16:38:40 -05:00
JesperTreetopandStephen Toub 104abd7fe7 Prevent DivideByZeroException in EventCounter.ToString() (#27581)
* Prevent DivideByZeroException in EventCounter.ToString()

Maybe it should print something other than 0. Maybe "Mean" should be excluded. But at least with this it doesn't divide by zero. Ternary operator in interpolated string is not super readable; didn't know if making the code more descriptive would have performance implications.

* Update src/System.Private.CoreLib/shared/System/Diagnostics/Tracing/EventCounter.cs

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 21:18:22 +01:00
Ryan Lucia f477bce548 [loader] Skip the full pinvoke resolution process for __Internal (#17701) 2019-11-05 14:35:00 -05:00
Tanner Gooding 3d4f44d85c Fixing the return type of the ArmBase.ReverseElementBits methods in the PlatformNotSupported file (#27672)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 20:20:17 +01:00
Bernhard Urban-Forster 93b97d5582 [mini] Fix Coverity CID 1455161 & 1455162 (#17615)
* [mini] Fix Coverity CID 1455161 & 1455162

> Null-checking "ji" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.

* do not rely on assert to guard ji-nullness.  refactor a bit
2019-11-05 19:09:32 +01:00
Johan Lorensson 6f73c2dab8 Fix MSVC intellisense for LLVM sources. (#17706) 2019-11-05 13:16:32 +01:00
Johan Lorensson 44e6226c31 Fix correct accumulated time for parallelized jobs in SGEN. (#17518)
* Fix correct accumulated time for parallelized jobs in SGEN.

Current implementation incorrectly accumulate time for scan jobs for
the follow metrics when running parallel minor GC:

time_minor_scan_major_blocks,
time_minor_scan_los,
time_major_scan_mod_union_blocks,
time_major_scan_mod_union_los

Commit fix so updates are atomic making sure values are correct in cases
where parallel minor GC is used.

* Implement 64-bit atommic add fallback in SGEN.
2019-11-05 01:57:06 -05:00
Nikolay Sivov 95743afa80 [mono] Fix ProcessExit handler argument. (#17680)
Problem was spotted with mono tracing hitting ProcessExit handler,
and crashing on first argument.
2019-11-05 01:56:43 -05:00
Egor Bogatov fdc6495a54 [netcore] Intrinsify Buffer.IsPrimitiveTypeArray (#17438)
* Improve IsPrimitive intrinsic

* Implement it for interpreter

* clean up

* clean up
2019-11-05 01:50:37 -05:00
Zoltan Varga ebb4598096 [wasm] Build the tests with --no-native-strip. (#17698) 2019-11-04 17:46:52 -05:00
Vlad Brezae b3b9960ed1 [interp] Use GetType instrinsic also on net4x (#17654)
We didn't do this because it had some issues when GetType was invoked through reflection. This issue would also be present with jit but, because jit doesn't use direct wrappers, the GetType call can't be intrinsified in that scenario. See https://github.com/mono/mono/pull/5350 for the context of the issue.
2019-11-04 12:58:38 -05:00
Dotnet-GitSync-Bot db49be3cbb Update only S.P.CoreLib for ReverseElementBits (dotnet/coreclr#27582) (#17695)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 12:55:57 -05:00
Egor Bogatov cf8495e2f7 [netcore] Copy Array.Clear impl from CoreCLR (#17411)
* Address feedback

* fix build error

* Fix has_reference usage in object.c

* Fix fallback impl

* Rename to ArrayHasReferences

* Rename to ObjectHasReferences

* Rollback to ArrayHasReferences

* Rename to ObjectHasReference

* remove redundant is_array variable

* ObjectHasReferences can accept any object

* undo changes in Program.cs

* undo changes in Program.cs
2019-11-04 12:48:04 -05:00
Filip Navara 07fa0bd3b4 Intrinsify Activator.CreateInstance<T> for value types with no ctor (#17688)
* Intrinsify Activator.CreateInstance<T> for value types with no ctor

* Fix failed test, address feedback
2019-11-04 12:40:13 -05:00
Zoltan Varga 929a8a7fd8 [jit] Allow Unsafe.As<TFrom, TTo> on gsharedvt types. (#17692) 2019-11-04 18:14:59 +01:00
Bernhard Urban-Forster 237f210ea2 [interp] use mask instead of bool expression (#17683)
Discovered by Coverity:
```
>>>     CID 1455205:    (CONSTANT_EXPRESSION_RESULT)
>>>     The expression "flag && 2" is suspicious because it performs a
        Boolean operation on a constant other than 0 or 1.
```
2019-11-04 15:56:46 +01:00
Alexander Köplinger 49d5c4d14d [sdks] Use Xcode 11.2 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-11-04 15:27:42 +01:00
Tarek Mahmoud Sayed 6630bcc67f Fix Buggy code in TimeZoneInfo (dotnet/coreclr#27591)
Although couldn’t reach the corner case cause executing the buggy code, the fix here is done according to the similar code in TimeZoneInfo which show the intention of the change

https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1638
https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1568

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 09:59:54 +01:00
Jan Kotas f3e30a0ad0 Delete two more FCalls (dotnet/coreclr#27640)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas deb3115b15 Use simple Array.Copy overload where possible (dotnet/coreclr#27641)
The simple Array.Copy overload does not come with a performance penalty anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas 6ecf9e87ba Consolidate *.Fast.cs (dotnet/coreclr#27644)
System.Memory OOB package that this split was originally done for does not exist anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Johan Lorensson c0e237b734 Mono NetCore Windows only build/test. (#17646)
Support for Windows Mono NetCore build/test wihtout any need for
cygwin/wsl/automake just MSBuild and MSVC. Windows build support mirror
technologies used by coreclr/aracade builds msbuild + powershell.

Windows only Mono NetCore build, makes a port of existing build.sh and
Makefile, implementing a build.ps1 and build.targets. So far full build
support + execution of corefx test suite is implemented. Other make file
rules can be added when needed.

MSVC build runtime seems to hits a couple of additional test failures
in corefx test suite that needs to be investigated further.

Building NetCore Mono on Windows can now be done without cygwin or wsl.
Build depends on regular VS install (or build tools), if xunit summary
is requested, python needs to be installed.
2019-11-04 08:55:36 +01:00
Egor Bogatov 137ffe8b7c [netcore] Improve Array.CreateInstance (#17673)
* Rewrite Array.CreateInstance

* Rewrite Array.CreateInstance

* Address feedback

* Convert the icall to NOHANDLES

* fix build errors, remove NOHANDLES

* Fix GC.KeepAlive usage

* Remove icall definition for GC.KeepAlive (it's no an icall)

* Fix failing tests (expect details in the AOORE)
2019-11-03 22:39:19 -05:00
Filip Navara 8131518bb1 [netcore] Improve default constructor lookup, (#17666)
* [netcore] Improve default constructor lookup, cache when no default constructor is found (common for struct)

* Address PR feedback
2019-11-03 18:28:32 +01:00
Levi Broderick 8700c04b02 Fix typo in Tuple.cs (dotnet/coreclr#27637)
Reported by https://github.com/dotnet/corefx/issues/42321

Signed-off-by: dotnet-bot <[email protected]>
2019-11-03 18:17:23 +01:00
Alexander Köplinger 4888c49c26 [netcore] Disable shared compilation for System.Private.CoreLib
There's a dotnet process hanging around which seems to lock the build when called from the SDKs Makefiles.
2019-11-03 15:43:40 +01:00
Alexander Köplinger 40c1cd3576 [netcore] Fix strong name key for System.Private.CoreLib
It's using the silverlight key: https://github.com/dotnet/coreclr/blob/34fe045a27e150bde7ee54e0d5f0df635922519d/src/System.Private.CoreLib/System.Private.CoreLib.csproj#L112
2019-11-03 15:43:40 +01:00
Alexander Köplinger d89b96a298 [netcore] Fix out of tree build for netcore/ folder 2019-11-03 15:43:40 +01:00
Jan Kotas ad385f98b3 Normalize trailing whitespaces (dotnet/coreclr#27626)
Normalize trailing whitespaces in frequently changing files (docs and sources)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-02 16:51:03 +01:00
Bernhard Urban-Forster 62ee01b36b [interp] s/MONO_API_ERROR_INIT/error_init_reuse/g (#17650)
[interp] s/MONO_API_ERROR_INIT/error_init_reuse/g
2019-11-02 16:45:36 +01:00
Zoltan Varga 803c657b24 [jit] Call mono_class_setup_fields () before accessing field->offset. (#17667)
Fixes https://github.com/mono/mono/issues/17665.
2019-11-02 03:57:25 -04:00
Manuel de la Pena 36539ed67f [iOS] Match changes done in xamarin-macios in the SDK runtime. (#17664)
* [iOS] Match changes done in xamarin-maciosn in the SDK runtime.

* Address reviews.
2019-11-02 01:40:40 -04:00
Jay Krell 4b9dc08ac1 Move MonoError from managed wrappers to native wrappers. (#15894)
Move MonoError from managed wrappers to native wrappers.

Move MonoError from managed wrappers (ilgen) to native wrappers (C preprocessor).

Alternative to https://github.com/mono/mono/pull/15869.
Which says: Previously, we would emit a call to memset
since the struct is large (> 100 bytes). Only the error code
field needs to be initialized.
Which that PR and this PR both fix, in different ways.

In the past:
- This did not work due to a possible dependency on precise GC,
  interacting with what gets zeroed or not here.
  That is why I put it on the managed side.
  We'll see if that reproduces.
  (https://github.com/mono/mono/pull/11294/files#diff-efe0070415f3f0c0f24ed0221aa1962aR339
  external/coreclr/tests/src/CoreMangLib/cti/system/weakreference/weakreferenceisaliveb.exe.)

- Some indication that embedding API should expose MonoError
  and not set_pending_exception. This does not seem relevant
  presently, and the code seems better structured and
  more efficient this way. It can revisited if/when new public API
  is the dominant concern, or do it this way for these functions,
  and another way for actual public functions.
  (https://github.com/mono/mono/pull/11294#issuecomment-432443485)
2019-11-01 22:36:06 +01:00
Zoltan Varga e9d68e9485 [netcore] Cleanups. (#17661)
* [netcore] Remove the type_info field from RuntimeType, extend the TypeCache structure instead.

* [netcore] Remove some linker exclusions which are not needed.
2019-11-01 12:52:45 -04:00
Steve Pfister b9f185b74a Bump mono/corefx@6e65509 (#17631)
Pulls in PR https://github.com/mono/corefx/pull/368
2019-11-01 11:04:27 -04:00
Jo Shields 16d1a2e6f1 [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. (#17628) 2019-11-01 10:26:08 -04:00
monojenkins a4790c892d [csproj] Update project files 2019-11-01 14:25:26 +00:00
monojenkins cc28706e58 Bump API snapshot submodule 2019-11-01 14:24:11 +00:00
Vlad Brezae 166fdf60b0 [interp] Handle remoting field access same as jit (#17641)
Doing this we avoid using slow ldrmfld in the common case of loading fields from instance methods, when we execute on the real object. We could do this also for strmfld but we need to track on the stack if the instance object is this, which is not worth the trouble.
2019-11-01 14:55:58 +02:00
Vlad Brezae 170d33c35b [interp] Optimize call path (#17623)
* [interp] Optimize void calls

By avoiding setting of retval

* [interp] Avoid computing param count for every call

* [interp] Remove wrong profiler check

From the times when interp had its own EH mechanism. We raise this event from mono_handle_exception_internal for interp.

* [interp] Remove profiling check code from method exit

We emit MINT_PROF_EXIT opcode when under profiler. Unify profiling enter/exit with tracing enter/exit since they are very similar.
2019-11-01 14:55:27 +02:00
Filip Navara 8aa3654de5 Lower min/max SSE operations on x64 (#17536)
Follow up on #17511.
2019-11-01 13:30:41 +01:00
Zoltan Varga dc8d55d446 [llvm] Fix a case where we treated the dreg of a store_membase instruction as a dreg, its actually the base reg. (#17648) 2019-11-01 03:43:42 -04:00
Steve Pfister 9566027142 Modify run-jenkins.sh to adapt wasm build for the OSX CI lane (#16785)
* Modify run-jenkins.sh to adapt wasm build for the OSX CI lane

* Round 2... Go

* Make ENABLE_WASM_DYNAMIC_RUNTIME conditional on non-OSX

* Add runtime-threads target

* Ignore 'multiple rules generate' on osx

* Fix typo

* Remove blank line

* Exclude CoreXml.Test.XLinq.FunctionalTests.EventsTests.EventsReplaceNodes.XElementReplaceNodes

* Revert "Ignore 'multiple rules generate' on osx"

This reverts commit 5010721bd710d6955aaee47aa7f49ad7845763fc.

* Check if OSX lane can follow general build flow
2019-11-01 01:44:50 -04:00
Stephen Toub 8b86349c2c Disable another static analysis rule (dotnet/corefx#42265)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 22:23:35 +01:00
Bernhard Urban-Forster 0be756d609 [threadpool] cache processor count (#17642)
On top of https://github.com/mono/corefx/pull/369 this improves the execution time of `System.Core-xunit` on Linux/ARM64 by 2x, so from:

```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 131.143s
```
to
```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 74.636s
```

This is only relevant for non-netcore. The CoreCLR folks just recently fixed something similar (thanks to Marek sharing this link): https://github.com/dotnet/coreclr/pull/27543
2019-10-31 22:08:14 +01:00
Vlad Brezae b484979aa9 [interp] Avoid emitting MINT_SAFEPOINT for every single call (#17625) 2019-10-31 16:49:16 +01:00
Jonathan Chambers 3648cb087b Avoid caching of System.dll image and types as they may be unloaded with appdomain. (#17602) 2019-10-31 10:13:01 -04:00
Zoltan Varga dff84a2464 [wasm] Fix xunit test ninja errors. (#17639)
* [wasm] Error out on duplicate assemblies in the packager.

* [wasm] Switch search path order to avoid duplicate references to assemblies.

* Copy all xunit binaries to test_lib_dir.

* Revert "Copy all xunit binaries to test_lib_dir."

This reverts commit b15b03c16b508c04dc6121196fddca0e6d949e2f.
2019-10-31 10:10:51 -04:00
Alexander Köplinger ca72deed17 Fix build 2019-10-31 14:36:53 +01:00
Ben Adams c9b7f767a8 Don't needlessly refresh proc count (dotnet/coreclr#27543)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 14:36:53 +01:00
Vlad Brezae 1d043a31e7 [interp] Fix interp logging (#17636)
* [interp] Don't print NOPs during cprop

They can become numerous and distracting.

* [interp] Fix mono_interp_print_code

Regressed after code_size was changed to byte count instead of short count.
2019-10-31 15:32:50 +02:00
abrevet-dev 6b3aa3a05b [WinForms] Change HotkeyPrefix default value in TabControlPainter.cs (#17579)
The default behavior on .NET Framework is _HotkeyPrefix.None_ when creating a _TabControl_ element (ampersands not escaped). 
If an user wants to override this behavior, he has to declare a _DrawItemEventHandler_ (with _DrawMode = OwnerDrawFixed_).

Fixes #16206
2019-10-31 13:48:12 +01:00
Stephen Toub a27aa7afd0 Update analyzer versions (dotnet/corefx#42249)
* Update analyzer packages to latest NuGet versions

* Fix new warnings

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Stephen Toub 6ae3360342 Clean up Uri.UnescapeDataString (dotnet/corefx#42225)
- Use string.IndexOf rather than an open-coded, unsafe loop.
- Avoid an unnecessary SequenceEquals at the end: we're only at this point if a `%` was found highlighting that something escaped was found.
- Use stack memory for smaller inputs if possible, to avoid unnecessary ArrayPool interaction
- Remove an unnecessary argument to a helper function.
- Fix ValueStringBuilder.Grow to only copy the contained data.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Egor Chesakov a94dda5b5a Add BitwiseSelect (dotnet/coreclr#27430)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 08:51:46 +01:00
Zoltan Varga dc76a95aa9 [runtime] Add a --enable-minimal=threads configure option to disable threading support. Use it on wasm. (#17611) 2019-10-31 01:33:50 -04:00
Zoltan Varga 5811404e55 [eglib] Assert in g_strlcpy () if src/dest is NULL, instead of printing a warning. (#17507)
Related to https://github.com/mono/mono/issues/17283.
2019-10-30 16:07:44 -04:00
Martin Baulig 578e0df42f [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. 2019-10-30 15:37:25 -04:00
Jo Shields 27921fbbd0 Rearrange System.Private.CoreLib for easier monorepo integration (#17517)
* [netcore] Move corlib around to match CoreCLR directory layout better

This is a prerequisite to making a sane migration of this whole
directory from mono.git to coreclr.git
2019-10-30 15:09:06 -04:00
Thays Grazia fffb1201bd [dim][regression] Explicit interface override (#17583)
* Precedence order:
-> Override interface method in class
-> Method in class
-> DIM
2019-10-30 14:47:15 -03:00
Egor Chesakov feca5e042d Add ComputeCrc32, ComputeCrc32C (dotnet/coreclr#27421)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Egor Chesakov 744d77ea7f Remove unsigned overloads of LeadingSignCount (dotnet/coreclr#27533)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Martin Baulig c6f71167a1 Bump CoreFX to pickup corefx PR #367 to fix #17133. (#17590) 2019-10-30 12:32:21 -04:00
Bernhard Urban-Forster 95cad24526 [interp] fix signature mismatch between jit<>interp for string constructor (#17595)
[interp] fix signature mismatch between jit<>interp for string constructor
2019-10-30 16:36:04 +01:00
Marek Safar e4c0b50dec [System.Private.CoreLib] Remove NativeRuntimeEventSource 2019-10-30 16:07:43 +01:00
Jan Kotas c9b88b45fe Move runtime specific methods out of the runtime partition
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 16:07:43 +01:00
Jay Krell 10ac2750e2 Inline TLS access. (#17119)
Inline TLS access.

This attempt differs from the previous in that the TLS accessor functions are presented to far less code, i.e. not to main.c, so their data references do not need to be resolved so much.

See https://github.com/mono/mono/commit/bffd4c6b99213a0b154d7efac9b0581df67775e8 etc.
2019-10-30 16:06:24 +01:00
Zoltan Varga 79f5e6c288 Remove some unused icalls. (#17612) 2019-10-30 10:08:50 -04:00
Filip Navara 3d397805d4 [netcore] Report errors on Windows CI (#17616)
* [netcore] Report errors on Windows CI

* Fix Windows build
2019-10-30 14:39:47 +01:00
Zoltan Varga 2823ac5d95 [wasm] Print a useful error message instead of a signature mismatch error on missing icalls. (#17600) 2019-10-30 00:04:15 -04:00
Dotnet-GitSync-Bot 3695d8704e Mirror changes from mono/coreclr,corefx (#17609)
* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp() (#27473)

* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp()

* Insert GC_POLL before statement with unmanaged call.

* JIT test for insertion of GCPoll

Signed-off-by: dotnet-bot <[email protected]>

* Add file creation method that takes an ACL (dotnet/corefx#42099)

Approved API Proposal: #41614
Related change for directory creation method that takes an ACL: #41834 -merged and ported to 3.1 Prev2

Description
We have extension methods in System.IO.FileSystem.AclExtensions that let the user get and set ACLs for existing files, but we do not have methods that create files with predefined ACLs.
.NET ACL (Access Control List) support is Windows specific. This change will reside inside the System.IO.FileSystem.AccessControl assembly.

Customer impact
Before this change, customers had to create a file or filestream, then set its ACLs. This presents a few problems:

Potential security hole as files can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
Stability issues with background processes (file filters) can prevent modifying ACLs right after creation (typically surfaces as a security exception).
This change addresses those problems by adding a new extension method that allows creating a file and ensuring the provided ACLs are set during creation.
This change is expected to be backported to 3.1.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 00:03:47 -04:00
Zoltan Varga 9adce0b632 [wasm] Fix build problems. (#17596)
* [runtime] Add a stub for MacOsIPInterfaceProperties::ParseRouteInfo icall on non-osx platforms to fix the wasm aot build.

* [wasm] Avoid generating invalid ninja build files if an assembly is specified on the command line and its also an implicit reference.

* Put back and ifdef.

* Fix the windows build.
2019-10-30 00:03:08 -04:00
Filip Navara d49d3efd5d Temporary build fix 2019-10-29 23:34:11 +01:00
Noah Falk 67ff64a381 Support large EventSource filter args (dotnet/coreclr#27500)
1. Fix NullReferenceException. When the filter args exceeded the
hard-coded size limit GetDataFromController would return data = null.
The code previously asserted that data was not null and triggered NRE
when it was. The fix correctly null checks the value instead of
asserting and uses an empty args dictionary in this case, the same as if
filter args had been empty to begin with.

2. ETW has always limited filter args to 1024 bytes but EventPipe has no
such restriction. When using DiagnosticSourceEventSource it can be
useful to specify a larger filter arg blob. I can't do anything about
ETW's restriction but there is no need for the runtime to force
EventPipe to be equally limited. The larger size also reduces the chance
that we need to hit the fallback path above causing filter args to be
ignored.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 8a03f60897 Cleanup (dotnet/coreclr#27502)
* Delete empty FireCustomerDebugProbe FCall

* Delete duplicate C++ compiler command line options

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Elinor Fung 17cef4c4f9 Basic infrastructure for binder tracing (dotnet/coreclr#27383)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub 249522036d Fix StreamReader to pass cancellation token into ReadBufferAsync (#27464)
ReadAsyncInternal is correctly passing its token into the two call sites where it delegates directly to the underlying stream, but in one place it calls through ReadBufferAsync, which is currently not defined to take a token.  Fix that, and pass the token through.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 447cb03399 Delete unnecessary abstraction from stubbed out SecurityElement (#27474)
The stubbed out SecurityElement implementation does not support extensibility that ISecurityElementFactory was designed for.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Dan Moseley 9aeef958c6 Add path to not-absolute-path exception (dotnet/coreclr#27470)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Filip Navara db02ebc88b Fix building portable thread pool on Windows (dotnet/corert#7849)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub f26e73b49b Move PreserveDependencyAttribute to shared (dotnet/corefx#42174)
We need to use it in corelib, too.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Bernhard Urban-Forster d274cfbacd [mini] Initial tiered compilation work (#17551)
[mini] Initial tiered compilation work

Enable it with `./autogen.sh --enable-experiment=tiered`.

Let's consider `Simple.cs`:
```csharp
using System.Runtime.CompilerServices;
using System;

public class Simple {
    public static void Main (string []args) {
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
    }

    static int cnt = 0;

    [MethodImplAttribute (MethodImplOptions.NoInlining)]
    public static void HotMethod () {
        for (int i = 0; i <= 1000; i++)
            cnt += i;
    }
}
```

```console
$ csc Simple.cs
$ MONO_LOG_LEVEL=debug MONO_LOG_MASK=tiered ./mono/mini/mono-sgen --trace=M:Simple:HotMethod --interp=-all Simple.exe
[0x10ec275c0: 0.00000 0] ENTER:i Simple:HotMethod ()()
Mono: tiered: queued Simple:HotMethod ()
[0x10ec275c0: 0.00010 0] LEAVE:i Simple:HotMethod ()(
Mono: tiered: patching 0x7fe855803224 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x34 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
Mono: tiered: patching 0x7fe8558031f2 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x2 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
cnt: 500500
[0x10ec275c0: 0.04093 0] ENTER:c Simple:HotMethod ()()
[0x10ec275c0: 0.04095 0] LEAVE:c Simple:HotMethod ()(
cnt: 1001000
```
Note the suffix after `ENTER:`
* `i` indicates it's executed by the interpreter
* `c` indicates it's a compiled method (JIT)

Another example:
```console
$ make -C mono/mini gshared.exe
$ ./mono/mini/mono-sgen --interp=-all --stats ./mono/mini/gshared.exe
Regression tests: 84 ran, 0 failed in Tests
[...]
Tiered statistics
Methods promoted                    : 68
```

It's a basic proof-of-concept for now. An incomplete list of future work items:

* Right now it only works for direct calls, need to expand it to virtual/interface calls.
* Calls of the JIT leading into the interpreter again can't be patched yet.
* Kind of related, no concept of versioning compiled methods does exist yet. The interpreter maintains its own table of "transformed" methods, however, when doing the transition from JIT->interpreter, the wrapper+interpmethod will end up in the JIT table. We need a way to have multiple JIT compilation results for the same MonoMethod exist.
* Investigate actual performance. We might have to optimize the interp<>JIT transition.
* Patching is racy. Need to make that atomic.
* All the FIXMEs in this PR



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-29 22:36:01 +01:00
Alexander Köplinger 1340095581 [acceptance-tests] Bump mono/roslyn to get build fix
See https://github.com/mono/roslyn/commit/6bfcd4e770609a38b8523e83ce1ec943162f30bb
2019-10-29 20:27:09 +01:00
Zoltan Varga b0d966bd7a [eglib] Handle dli.dli_sname being NULL in g_module_address (). (#17592)
Should fix the warnings/assertions in g_strlcpy ().
2019-10-29 09:21:23 -04:00
Aniruddha Shastri 7f91972ded Support platform-independent newlines in line wrapping in Mono… (#17431)
* Add newLine parameter to StringCoda methods

* Update Options.cs to handle both kinds of newlines

* Add test to OptionSetTest

* Directly handle \r\n instead of using the TextWriter.NewLine
2019-10-29 13:53:41 +01:00
dotnet-maestro[bot] cddf90dda4 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17577)
* Update dependencies from https://github.com/dotnet/arcade build 20191027.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19527.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19527.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191027.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19527.2

* Update dependencies from https://github.com/dotnet/corefx build 20191027.5

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19527.5

* Update CoreFX.issues.rsp

GetGCMemoryInfo was moved to GCExtendedTests in https://github.com/dotnet/corefx/commit/d6dac9e6a7875862b91139334c8ec620f599cd4d.

GetTotalAllocatedBytes and GetAllocatedBytesForCurrentThread seem to pass now.
2019-10-29 12:51:33 +01:00
Zoltan Varga e4cc70f4b5 [runtime] Implement support for SuppressGCTransitionAttribute. (#17534)
Fixes https://github.com/mono/mono/issues/17367.
2019-10-28 10:59:52 -04:00
Johan Lorensson c5edcd1ed4 Additional and adjusted logging for parallel minor GC. (#17522)
* Additional and adjusted logging for parallel minor GC.

When running parallel minor GC we currently get incorrect timing
on several loggings and counters due to work being put on thread pool
and accumulated. This commit fix so that the logging is accurate
when running with and without parallelization. In parallel mode a new
logging is also added to show the accumulated split between major and los
remset scan jobs.

The counters time_minor_scan_remsets and time_minor_scan_roots are only
updated when not running parallel GC since they are not correct when
running parallel GC.

* Fix WASM build error.
2019-10-28 15:34:42 +01:00
Alexander Köplinger 450d474684 [netcore] Fix offsets generation when cross compiling
The offsets-tool.py uses some magic to map `typename` to `<underscore>typename` for looking up runtime types.
This caused issues on netcore because we aliased it to a different name.

To fix this let's keep the `_MonoInternalThread` name instead and just fix the one location where `_MonoThread` is used.
2019-10-28 13:57:17 +01:00
Alexander Köplinger fb7b075e49 [netcore] Don't load types in aot-compiler.c that don't exist in corefx
Fixes https://github.com/mono/mono/issues/16026
2019-10-28 13:57:17 +01:00
dotnet-maestro[bot] 6f04054541 Update dependencies from dotnet/corefx (dotnet/coreclr#27431)
* Update dependencies from https://github.com/dotnet/corefx build 20191024.13

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.13
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.13

* Update dependencies from https://github.com/dotnet/corefx build 20191024.14

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.14
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.14

* Update dependencies from https://github.com/dotnet/corefx build 20191025.6

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19525.6
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19525.6

* Workaround https://github.com/dotnet/coreclr/issues/27465

Signed-off-by: dotnet-bot <[email protected]>
2019-10-27 17:54:12 +01:00
Alexis Christoforides db568c6fe0 [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping (#17537)
* dump_native_stacktrace should always register the sigterm handler.

Cherry picked from the https://github.com/mono/mono/pull/17310

* [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping

The primary reason for this is gaining the ability to interpret SIGTERM properly as a crashing signal when outside of 'dump mode', and as a utility signal during the 'thread summarizer' dumping process.
2019-10-26 18:48:40 -04:00
Aleksey Kliger (λgeek) 0e3caf00df [runtime] Unbalanced GC Unsafe transitions before shutdown (#17566)
When embedders call mono_runtime_quit or mono_jit_cleanup, we need to do unbalanced transitions to GC Unsafe, because after mini_cleanup runs we don't have GC thread states anymore and MONO_EXIT_GC_UNSAFE will assert.

* [runtime] Do an unbalanced GC Unsafe transition in mono_jit_cleanup

After mini_cleanup, we don't have GC thread states anymore because all that
stuff was cleaned up, so MONO_EXIT_GC_UNSAFE would assert

* [runtime] Mark mono_runtime_quit external only.

Runtime should use mono_runtime_quit_internal.

After we call the quit_function (aka mini_cleanup) we don't have any GC thread
states anymore because all that stuff got cleaned up.  So MONO_EXIT_GC_UNSAFE
can't work.
2019-10-26 18:45:35 -04:00
Ben Adams 2da4e54a7f Avoid mod operator when fast alternative available (dotnet/coreclr#27299)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 19:07:15 +02:00
Jay Krell 5f3446bd12 [cxx][x86] int/ptr casts (#17570)
[cxx][x86] int/ptr casts
2019-10-26 18:26:15 +02:00
Dotnet-GitSync-Bot c5e3ca90de Mirror changes from mono/coreclr (#17571)
* Use ValueStringBuilder instead of StringBuidler{Cache} in multiple places

Signed-off-by: dotnet-bot <[email protected]>

* Avoid unnecessary allocations in Exception.ToString

Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 07:48:46 -04:00
Dotnet-GitSync-Bot fec907397d Mirror changes from mono/coreclr,corert,corefx (#17565)
* Update Corelib to include Stopwatch (dotnet/coreclr#27417)

* Update Corelib to include Stopwatch

* Delete HighPerformanceCounter

Signed-off-by: dotnet-bot <[email protected]>

* Share completed Task between Task and AsyncTaskMethodBuilder (#27437)

Signed-off-by: dotnet-bot <[email protected]>
2019-10-25 21:23:46 -04:00
Caleb Cornett 1801bdad6a Add --preload-file packager option 2019-10-25 14:53:15 -05:00
Leonard 4ec9bd03e5 Fixes #17190: SerializationException on ListViewItemCount (#17195)
* Fix issue #17190: SerializationException on ListViewItemCount

ListViewGroup doesn't (always) have a serialized member "ListViewItemCount".

This patch accounts for that and also adds support for the "ItemsCount"
property that upstream [1] seems to use.

Also items should be initialized regardless of the count, otherwise we get a NPE.

[1] https://github.com/dotnet/winforms/blob/master/src/System.Windows.Forms/src/System/Windows/Forms/ListViewGroup.cs#L144
2019-10-25 19:30:08 +02:00
Kenneth Pouncey 1bd76c6069 Spacing 2019-10-25 12:19:54 -05:00
Kenneth Pouncey f1a3f3c58d Add CompileFunction to Runtime that returns a WebAssembly.Core.Function
- The code passed to `CompileFunction` is not a function definition. Instead it must create and return a function instance.
- Solves the problem of accessing `Node.js` functionality as defining a JavaScript function executes in the global space and does not get `Node.js` access.
- Returning a function instance allows the function to initialize and maintain encapsulated state.
2019-10-25 12:19:54 -05:00
Jay Krell 893486638c [cxx] Compile mini-llvm.c as C++ if configure -enable-cxx. (#17558)
[cxx] Compile mini-llvm.c as C++ if configure -enable-cxx.

This is presently catastrophic, because of `MonoJitICallId` varying between C and C++, and therefore the layout of `MonoCallInst`.

Related to:
https://github.com/mono/mono/pull/17557

https://github.com/mono/mono/pull/17445#issuecomment-546175352
2019-10-25 17:46:01 +02:00
Jay Krell 84d64d314b [jit][x86][cxx] int/ptr casts (#17382) 2019-10-25 11:34:50 -04:00
Aleksey Kliger (λgeek) d6ee245c94 [runtime] Make mono_thread_manage external only (#17554)
* [runtime] Make mono_thread_manage external only

runtime should use mono_thread_manage_internal
2019-10-25 11:28:18 -04:00
Vlad Brezae 768fefbad1 [interp] Optimize stelem (#17519)
* [interp] Optimize stelem instructions

* [intepr] Optimize stelem_ref and slowpath casts

mono_object_isinst_checked uses coop handles and is very slow. Attempt to use the straightforward mono_class_is_assignable_from_checked.

Makes stelem_ref 5x faster.
2019-10-25 16:25:08 +02:00
Egor Bogatov dc4a7f5ed7 [llvm] Use -place-safepoints in JIT mode too (#17116)
* always use -place-safepoints pass in JIT mode

* ignore loaded llvm

* remove gc.safepoint_poll after optimizations

* fix typo

* cleanup

* cleanup

* cleanup

* cleanup

* Do not emit safepoints on entry thus simple functions (without calls and loops) won't have them at all

* Undo  -spp-no-entry=true

* fix failing tests

* do not emit sp in Allocators
2019-10-25 16:10:57 +02:00
Kenneth Pouncey 6ac749ed20 [wasm][bcl] Fix System.Diagnostics.Trace.TraceError crash. (#17509)
* Fix `System.Diagnostics.Trace.TraceError` crash.

- WASM does not have access to the command line arguments through the browser so the length is 0 causing index out of range error.
- The zero index of the command line arguments is what decides the `AppName` so under WASM the AppName will be blanks.

* Address review comments

* Remove #ifdef for WASM only.

- Addresses review note.
- As per channel discussion some embedding scenarios might be the same.
2019-10-25 11:17:00 +02:00
Zoltan Varga 9d781eb78c [wasm] Propagate exit code from Main in the test runner. (#17559)
* [wasm] Propagate exit code from Main in the test runner.

* Disable some System tests as they currently fail.
2019-10-25 04:59:30 -04:00
Dotnet-GitSync-Bot 8800bfdf9c Fix build break (#17553)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 18:30:09 -04:00
Jay Krell 4ef7c566af [cxx] externC for some icalls, for wasm and ios. (#17444)
Maybe all JIT icalls?
2019-10-24 17:59:11 -04:00
Jay Krell fc390a5f24 Include signal.h unconditionally. (#17430)
Some of the includes already were unconditional, including both on Win32 and Unix.
It is part of ANSI C 89, albeit in a reduced form compared to typical Unix (no sigaction).

Remove configuration of MSG_NOSIGNAL which has no other uses.

Preserve configuration of signal.h temporarily due to other uses of ifdef HAVE_SIGNAL_H.
 i.e. LLVM?
2019-10-24 17:44:57 -04:00
Jay Krell ec81a868a1 [cxx][x86] int/ptr casts. (#17441)
* [cxx][x86] Another int/ptr cast.

* More casts. Notice these are under
`#if defined (MONO_ARCH_USE_SIGACTION) && !defined (MONO_CROSS_COMPILE)`
which would explain them not showing up.
2019-10-24 17:37:12 -04:00
Dotnet-GitSync-Bot aea7214eb4 Optimize GC.AllocateUninitializedArray and use it in StringBuilder (#27364) (#17550)
* use GC.AllocateUninitializedArray for allocating internal char buffers

* force inlining of AllocateUninitializedArray to have no perf hit on the small buffers hot path

* insrease the threshold from 256 to 2048 bytes

* use Unsafe.As instead of a cast

* remove the size precondition from AllocateNewArray method, the called AllocateSzArray is responsible for handling negative size

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:02:11 -04:00
dotnet-maestro[bot] a7287bdae5 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17458)
* Update dependencies from https://github.com/dotnet/arcade build 20191018.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19518.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19518.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191021.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19521.2

* Update dependencies from https://github.com/dotnet/corefx build 20191020.7

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19520.7

* Update dependencies from https://github.com/dotnet/arcade build 20191023.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19523.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19523.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191023.10

- Microsoft.NETCore.App - 5.0.0-alpha1.19523.10

* Update dependencies from https://github.com/dotnet/corefx build 20191023.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19523.8

* Disable a file system watcher test on mac

See https://github.com/mono/mono/issues/17547

* Reenable tests that were fixed
2019-10-24 21:06:46 +02:00
Bernhard Urban-Forster 25b53a642b [interp] fix code length for JitInfo (#17538)
* [interp] fix code length for JitInfo

* fix c++ build
2019-10-24 20:13:27 +02:00
Tanner Gooding 0595c0e6d7 Move down S.N.Vector APIs and expose new conversion support (#27401)
* Updating S.P.Corelib to start building Vector2/3/4 and supporting types

* Removing HAS_INTRINSICS usages from Matrix4x4

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 14:10:07 -04:00
Marek Safar 81a4a19f0a [netcore] Fix corerun build deps 2019-10-24 14:09:14 -04:00
Martin Baulig 1553889bc5 Cleaning up SslStream, MobileAuthenticatedStream and TaskToApm. (#17393)
### Overview

After the removal of the Legacy TLS Provider, we can make some cleanups and improvements to the `SslStream` class.  Since both the remaining providers use the internal `MobileAuthenticatedStream` and support Task-async code, we can get rid of some of the Begin/End async logic and the the underlying
`Stream` class handle it instead.

This also fixes a nasty issue with some of those task objects leaking unobserved exceptions.  The problem was that `SslStream.Dispose()` needs to clear out all resources, but in doing so also nulls out the `Impl` handle to the `MobileAuthenticatedStream` - so a subsequent `End*()` call will not reach it's corresponding `TaskToApm.End()` method.

There are also some cleanups in the internal APIs that were only ever used internally and not intended for public consumption.

### Mono.Security.dll

* `Mono.Security.Interface`: removed internal `ICertificateValidator2` interface.

* `Mono.Security.Interface.IMonoSslStream`: this internal interface is only used by the web-tests, which only use it as a way to get the `SslStream` instance, so all those `Begin/End` async methods are now gone.

* `Mono.Security.Interface.IMonoSslStream2`: removed another internal interface.

* `Mono.Security.Interface.MonoTlsProvider`: removed some `internal abstract` methods; these have been moved into a new `Mono.Net.Security.MobileTlsProvider` inside `System.dll`.

### System.dll

* `Mono.Net.Security.MobileTlsProvider`:
  New abstract internal base class, which received the aforementioned abstract internal methods.  The naming is such to make it consistent with the already existing `MobileTlsContext` and `MobileAuthenticatedStream`.

Since both `AppleTlsProvider` and `BtlsProvider` now have an abstract internal base class inside `System.dll`, they can use the `ChainValidationHelper` class directly and we can also avoid some `Mono.Security` dances like for instance that `MonoSslPolicyErrors` enum.

This will also help make the code easier to read and maintainer.
  
* `Mono.AppleTls.AppleTlsProvider` and `Mono.Btls.MonoBtlsProvider`: change base class into `MobileTlsProvider` and reflect above mentioned overload changes.

* `Mono.AppleTls.AppleCertificateHelper`: use `ChainValidationHelper` as well.

* `Mono.Net.Security.ChainValidationHelper`: only implement `ICertificateValidator`.
  The `ICertificateValidator` interface is still required because it is in use by the web-tests, but the previous `ICertificateValidator2` was an internal dance.

* `Mono.Net.Security.MobileAuthenticatedStream`: largely cleaned up this class.
  - we now implement the current slimmer version of `IMonoSslStream`.
  - moved all those overloads into `SslStream` that only fill in default values for some of their arguments and then call another overloaded version of their own.
  - removed lots of `Begin/End` methods that are not used anymore.

* `Mono.Net.Security.MonoTlsProviderFactory`: use `MobileTlsStream` everywhere.

* `Mono.Net.Security.MonoTlsStream`: cleanup `SslStream` creation logic; instead of going throw `provider.CreateSslStream()`, we can now use an internal constructor directly, thus avoiding the `IMonoSslStream` interface while also making the code a lot easier to read and understand.

* `System.Net.Mail.SmtpClient`: same here.

* `System.Net.HttpWebRequest`: use `MobileTlsProvider`.

* `System.Net.Security.SslStream`: largely cleaned up this class.
  - we now use `MobileAuthenticatedStream` directly instead of the `IMonoSslStream` interface and therefore could also get rid of `IMonoSslStream2`.
  - cleanup constructor logic to allow the internal constructor being used with a `MobileTlsProvider` instead of going through `provider.CreateSslStream()`.
  - all those overloads that are only filling in default values for some of their arguments before calling another overloaded version of their own now live here instead of in the `MobileAuthenticatedStream` class.  The reason is to simply that `MobileAuthenticatedStream` is already quite a complex and difficult class, and removing some of that complexity out of it will make it easier to understand.
  - handle the `TaskToApm.Begin()` / `TaskToApm.End()` here to ensure that we are not leaking any task objects with unobserved exceptions.
  - removed `BeginRead()` and `BeginWrite()` overloads to let the underlying `Stream` class handle these.


* Bump API snapshot submodule

* [csproj] Update project files
2019-10-24 19:42:54 +02:00
Calvin Buckley 638e2c6a86 Fix eglib on AIX/PASE (#17476)
* fix eglib on AIX/PASE

	* Work around access(2) syscall bug on PASE
	  On PASE, X_OK will always returns true. This is contrary to
	  how AIX behaves (though corresponds to documentation!). This
	  is probably because it's an upcall to the ILE version of
	  access, and that seemingly always returns true.

	* Always export the dummy function in the test suite
	  The AIX linker didn't export this properly, without using the
	  giant sledgehammer of -bexpall. Write an export file and use
	  it. This required some build system changes to expose an
	  AIX-only automake conditional.

	* Use a file that exists on PASE for tests
	  PASE doesn't use Unix-style DNS resolution, so /etc/hosts is
	  missing. Use /etc/magic, since that seems to be present.

	* Don't use /bin on AIX/PASE for tests
	  This is a symlink to /usr/bin, and it seems chdir/getcwd
	  follows links. Use /usr instead, that is unlikely to be a
	  symlink.

	* Make a note what directory we're in if we fail test_cwd

* Change manual executable access check change per Jay

* Change test on POSIXy platforms to always use root directory
2019-10-24 18:04:20 +02:00
Vlad Brezae b7ef736338 [interp] Add super instructions for field storing (#17525)
* [interp] Add also floating versions for stloc.np

* [interp] Add super instructions for stfld

When storing into a field of a local or an argument (ldloc/ldarg -> random instructions that end up pushing one value -> stfld). We add this instruction to the cprop pass because the instructions that are optimized together are not consecutive and we need the stack information to be able to access the instruction that loaded the argument / local and to make sure that the local wasn't dirtied. Alternatively we could make the super instruction pass stack aware, which we should probably do if more super instruction candidates turn out to require stack state information.

* [interp] Enable cprop for methods without locals

We do some optimizations here that don't apply to locals.

* [interp] Fix result stack type of CEE_CONV_OVF_I_UN
2019-10-24 18:05:04 +03:00
Vladimir Sadov 6fd07e4745 Moving StopWatch to src\Common\src\Corelib (dotnet/corefx#42072)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:54:28 +02:00
Egor Bogatov 7952e73780 Bump to mono/corefx@fb41040
Includes upstream corefx changes that makes NTLM auth work for more scenarios.
2019-10-24 10:04:24 -04:00
Bernhard Urban-Forster 861812252a [lldb] another fix for handling interp frames (#17535)
[lldb] another fix for handling interp frames

And improved error message in case it breaks again.
2019-10-24 14:07:22 +02:00
Dotnet-GitSync-Bot 2c8e0f68c1 Add AdvSimd (LeadingZeroCount, LeadingSignCount, PopCount) (#27382) (#17532)
* AdvSimd.LeadingSignCount

* AdvSimd.LeadingZeroCount

* AdvSimd.PopCount

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 07:16:49 -04:00
Zoltan Varga 951769b279 [netcore] Add a few more missing SSE intrinsics. (#17511)
* [netcore] Add a few more missing SSE intrinsics.

* Disable some intrinsics for non-llvm.
2019-10-24 06:57:13 -04:00
Johan Lorensson fe7654f745 Adjust restart world logging level to match start major/nursery logging. (#17520)
With this change we will get the following logging on level 1:

2019-10-22 15:46:59 Start nursery collection 38898 00000205A0000000-00000205A1000000, size: 16777216
2019-10-22 15:46:59 restarted (pause time: 2728 usec, max: 689233 usec)

showing the pause time and when runtime was restared, before this fix
we will only see the start log message on level 1.
2019-10-24 04:57:55 -04:00
Filip Navara 7a15b71873 [netcore] Managed ThreadPool implementation (#17387)
* Wire up the managed side of the portable thread pool

* Remove old native thread pool code

* Fix non-netcore build

* Fix Thread.IsThreadPoolThread

* Fix MSVC builds

* Code style fix

* Make the locking primitives uninterruptible

* Revert icall changes for non-netcore

* Implement Thread.ResetThreadPoolThread

* Compile out more thread pool code for netcore

* Remove ThreadPoolBoundHandle, it is shared now

* Address PR feedback on code style
2019-10-24 04:56:25 -04:00
Dotnet-GitSync-Bot 2dfd682ac9 Enable pooling for async ValueTask/ValueTask<T> methods (#26310) (#17531)
Today `async ValueTask/ValueTask<T>` methods use builders that special-case the synchronously completing case (to just return a `default(ValueTask)` or `new ValueTask<T>(result))` but defer to the equivalent of `async Task/Task<T>` for when the operation completes asynchronously. This, however, doesn't take advantage of the fact that value tasks can wrap arbitrary `IValueTaskSource/IValueTaskSource<T>` implementations.

This commit gives `AsyncValueTaskMethodBuilder` and `AsyncValueTaskMethodBuilder<T>` the ability to use pooled `IValueTaskSource/IValueTaskSource<T>` instances, such that calls to an `async ValueTask/ValueTask<T>` method incur 0 allocations (ammortized) as long as there's a cached object available.

Currently the pooling is behind a feature flag, requiring opt-in via the DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKS environment variable (setting it to "true" or "1").  This is done for a few reasons:
- There's a breaking change here, in that while we say/document that `ValueTask/ValueTask<T>`s are more limited in what they can be used for, nothing in the implementation actually stops a `ValueTask` that was wrapping a `Task` from being used as permissively as `Task`, e.g. if you were to `await` such a `ValueTask` multiple times, it would happen to work, even though we say "never do that". This change means that anyone who was relying on such undocumented behaviors will now be broken. I think this is a reasonable thing to do in a major release, but I also want feedback and a lot of runway on it.
- Policy around pooling. Pooling is always a difficult thing to tune. Right now I've chosen a policy that limits the number of pooled objects per state machine type to an arbitrary multiple of the processor count, and that tries to strike a balance between contention and garbage by using a spin lock and if there's any contention while trying to get or return a pooled object, the cache is ignored. We will need to think hard about what policy to use here. It's also possible it could be tuned per state machine, e.g. by having an attribute that's looked up via reflection when creating the cache for a state machine, but that'll add a lot of first-access overhead to any `async ValueTask/ValueTask<T>` method.  For now, it's tunable via the `DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKSLIMIT` environment variable, which may be set to the maximum number of pooled objects desired per state machine.
- Performance validation.  Preliminary numbers show that this accomplishes its goal, having on-par throughput with the current implementation but with significantly less allocation.  That needs to be validated at scale and across a variety of workloads.
- Diagnostics.  There are several diagnostics-related abilities available for `async Task/Task<T>` methods that are not included here when using `async ValueTask/ValueTask<T>` when pooling.  We need to evaluate these (e.g. tracing) and determine which pieces need to be enabled and which we're fine omitting.

Before shipping .NET 5, we could choose to flip the polarity of the switch (making it opt-out rather than opt-in), remove the fallback altogether and just have it be always on, or revert this change, all based on experimentation and data we receive between now and then.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 03:09:50 -04:00
Kenneth Pouncey 1b5658d590 [wasm][sdk] Bump SDK version to 0.2.2 (#17516)
* [wasm][sdk] Bump SDK version to 2.2

* Update SDK version for browser tests
2019-10-24 07:37:28 +02:00
Dotnet-GitSync-Bot e5d88dbd04 Mirror changes from mono/coreclr,corefx,corert (#17526)
* Apply SuppressGCAttribute to some SPCL functions. (dotnet/coreclr#27369)

Apply SuppressGCAttribute to some SPCL functions.

Only insert GC_POLL in first morph call.

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused methods (dotnet/coreclr#27370)

Signed-off-by: dotnet-bot <[email protected]>

* Move Matrix3x2/4x4, Plan, and Quaternion to the shared CoreLib partition (#42021)

Signed-off-by: dotnet-bot <[email protected]>

* Remove Socket.InnerSafeCloseSocket (dotnet/corefx#41888)

* Remove Socket.InnerSafeCloseSocket

* Move IsInvalid checks in SetHandle function

* PR feedback

* Remove unnecessary 'SocketPal.'

* Remove SafeSocketHandle.SetHandle

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 23:01:00 -04:00
Maxim Lipnin 43a938fefb [wasm][xunit tests] Exclude some System.Data tests (#17527) 2019-10-23 22:55:56 -04:00
Thays Grazia 8fda1738f7 [debugger] Changing how debugger handles exception when hybrid suspend is enabled (#17478)
* Removing all fixes to fix #15203 and #17083, and trying a new approach to fix this #17084, together.

The thing is when we are executing on hybrid suspend the caller is executed inside try catch in the runtime_invoke_ method that is generated. And this try catch should be ignored for the debugger exception, otherwise it's the last try catch in main thread, but should always be considered to generate the stack trace.

Fixes #17084

* This test check the case that we were missing. On preemptive this test always runs perfectly, on Hybrid, never run well until this PR.

* Changing how call unit test.
Changing variable name.
2019-10-23 17:56:58 -03:00
Jay Krell 4d7adf4d2a [cxx] Put extern "C" in MONO_LLVM_INTERNAL. (#17443) 2019-10-23 15:36:35 -04:00
Johan Lorensson df30c7228c Optimize LOS for better parallelization. (#17173)
Scanning LOS list as part of minor GC didn't parallelize well since
all scan jobs needed to walk the complete LOS list. This led to touching
a lot of memory walking the list, increasing minor GC pause times
when the number of items in the LOS list increased. As an example,
stressing LOS data structure using ~600 MB's of random sized byte arrays
caused minor GC pause times between ~20ms without parallelization and ~19ms
with parallelization. NOTE, the more work each job gets (more memory to scan)
we will get a better parallelization, but still a lot of room for improvement.

Changing from a normal linked list to a SgenArrayList makes it more effective
to parallelize the scan jobs and since the list is using sequential
memory, this will also reduce cache misses for each thread iterating LOS
objects. Tagging objects in SgenArrayList with reference information reduce
cache misses dramatically in cases where objects not including references
exists on the list, since scanning jobs can skip these items without any
need to touch object memory causing cache misses.

Same scenario running with fix gives us minor GC pause times, without
parallelization in ~7ms (down from 20~ms) and with parallelization,
< 5ms (down from 19ms). NOTE, the more work each job gets (more memory to scan)
we will get a better result using parallelization, so the delta will increase
between using and not using parallelization, this scenario primarily stress
the extreme case where most LOS objects doesn't have references.

In total, this optimization reduce minor GC pause time 4x, from ~19ms when
using parallelization down to < 5ms.
2019-10-23 14:58:04 -04:00
Zoltan Varga 31b475e25a [wasm] Avoid adding an extra arg to native-to-managed wrappers. (#17508)
[wasm] Avoid adding an extra arg to native-to-managed wrappers.

Fixes https://github.com/mono/mono/issues/17389.




<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-23 17:06:18 +02:00
Jay Krell 8b8b8b5010 [coop] Fix big endian problem in MonoThradStateMachine and cleanup. (#17502) 2019-10-23 10:21:54 -04:00
Filip Navara e29f8e32b7 [netcore] Add Unsafe.SkipInit intrinsic 2019-10-23 09:26:09 -04:00
Tanner Gooding e73aa17e23 Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe (#27377)
* Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe

* Updating the runtime to handle Unsafe.SkipInit

* Updating Decimal.DecCalc to use Unsafe.SkipInit

* Removing 'workaround for CS0165' comments

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 09:26:09 -04:00
Marius Ungureanu bed1b39c6e Bump Bockbuild to pick-up gtk# binding change (#17486)
Fixes https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1005448
Which causes quite a few broken clipboard bugs and possible memory corruption

(cherry picked from commit 989333d6ae)
2019-10-23 14:36:46 +02:00
Kenneth Pouncey bc769e7376 [wasm] move jsvu repo back after fixes for v8 (#17514) 2019-10-23 13:34:36 +02:00
Kenneth Pouncey cdf8b839ae [wasm][binding] Add bindings for calling the entry point of an '.exe' (#17510)
- Add `mono_bind_assembly_entry_point`
- Add `mono_call_assembly_entry_point`
   - Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);

Example:

```
   Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);
```

```
   var main = Module.mono_bind_assembly_entry_point("WasmHttpStreamSync", []);
   main();
```
2019-10-23 12:43:16 +02:00
Vlad Brezae 479d2dff36 [interp] Add a new super instructions pass (#17489)
Together with a few very simple super instructions. We replace common instruction patterns to avoid indirection via interp stack, instruction dispatch overhead and to allow the compiler to generate better code for the instruction.
2019-10-23 11:57:07 +03:00
Vlad Brezae 4419dc919f [interp] Retry cprop pass if killing stloc (#17491)
When replacing a stloc with a pop, we should retry the pass, since it is very likely that we will be able to kill the instruction that pushed that value on the stack.
2019-10-23 10:39:27 +03:00
Jay Krell 92c08fab4d SetLastError slightly later, in case last error not preserved, and it is just as good. (#17483)
SetLastError slightly later, in case last error not preserved, and it is just as good.
2019-10-23 09:36:15 +02:00
Vlad Brezae 6159407846 [interp] Optimize ldelem instructions (#17494)
Remove the switch and have each instruction have its own code. Seems to improve speed on a benchmark that heavily uses ldelem by about 10-20%.
2019-10-23 10:23:49 +03:00
Zoltan Varga a2cd8b2495 [wasm] Enable threads again on CI, it seems to work again. (#17503) 2019-10-23 02:41:46 -04:00
Dotnet-GitSync-Bot b99a3e9514 Move Vector2/3/4 to the shared CoreLib partition (dotnet/corefx#42005) (#17505)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 00:36:20 -04:00
Filip Navara 1e60703a19 [netcore] Disable debugging code in lookup_intrins (#17464) 2019-10-23 00:01:51 -04:00
Aaron Robinson 11dca2a3b8 Add documentation for SuppressGCTransition (dotnet/coreclr#27368)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 21:30:52 -04:00
Zoltan Varga 745e5ba3c3 [wasm] Bump emscripten to 1.39.0. (#17484)
* [wasm] Bump emscripten to 1.39.0.

* [wasm] Fix threads build.
2019-10-22 20:09:07 -04:00
Larry Ewing d08713198f [wasm] fix package target 2019-10-22 18:19:19 -05:00
Larry Ewing 433d7c5004 [wasm] Bump DebugProxy to netstandard 2.1 / netcore 3.0.0 2019-10-22 18:19:19 -05:00
Larry Ewing 15d61d3f47 [wasm] work around jsvu / v8 changes 2019-10-22 15:50:18 -05:00
Tanner Gooding 69407517bc Adding back more arm intrinsics that existing before refactoring (#27153)
* Adding AdvSimd.And

* Adding AdvSimd.AndNot

* Adding AdvSimd.Or

* Adding AdvSimd.OrNot

* Adding AdvSimd.Xor

* Adding AdvSimd.Not

* Removing a trailing whitespace from AdvSimd.PlatformNotSupported.cs

* Adding AdvSimd.Subtract

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Filip Navara fdca1630dc Fix nullability error in portable thread pool (dotnet/coreclr#27366)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Jan Kotas b98911d4f8 Delete Uap quirks from CoreLib (dotnet/coreclr#27356)
This matches cleanup done in CoreFX.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Adam Sitnik 0663433389 remove double bound check from StringBuilder.Append(char) (#27340)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Aleksey Kliger (λgeek) fb5b2c949d [reflection] mono_method_get_base_method on a GTD should use canonical ginst (#17475)
* [test] GetCustomAttributes on generic type definition

* [reflection] mono_method_get_base_method on a GTD should use canonical ginst

The issue is in how we pass along the generic instantiation.  So there are two
parts here: we start with a method on a GTD which has to decompose into the GTD
and it's instantiation with its own generic parameters, and then we go up to
its parent and grandparent inflating and decomposing the resulting
instantiation. The instantiations keep passing the last parameter as the number
of parameters shrinks so that the gparam index from the original type is now
out of bounds for the parent and grandparent.  If we mess up, we'll get a
BadImageFormatException from the runtime

Fixes https://github.com/mono/mono/issues/17278
2019-10-22 14:07:44 -04:00
Maxim Lipnin bc9b64b807 Get resource string 2019-10-22 12:13:38 -05:00
Aleksey Kliger (λgeek) ebb5846e0d [coop] Use bitfields for coop state machine state representation (#17298)
* [checked] Mask the thread state in GC thread state assertions

Because we have extra bits (like suspend count and the no_checkpoints bit) on
the thread state, we have to mask it to get the actual thread state enum value

* [coop] Use bitfields to represent coop thread state

Instead of using a 32-bit int, use bitfields and type-punning through a union
type.  We need to type pun because we need to access the entire state as a 32-bit int so that we can do atomic CAS.

* use atomic load of thread state in unwrap_thread_state.

Saw one stack trace on CI on the full coop lane where check_thread_state saw STATE_RUNNING
but with a positive suspend count.  Another thread was starting a GC a
the time.  Seems like we saw the state as runnning, but the suspend
count as if we were already in async_suspend_requested.

Thread 22 (Thread 0x7f43c03ff700 (LWP 39860)):
2019-10-22 12:14:05 -04:00
Marek Safar f34fd20fbf [System.Private.CoreLib] Remove unused methods 2019-10-22 11:28:22 -04:00
Filip Navara cf1a9e0033 Update Mono for changes in shared CoreLib partition 2019-10-22 09:46:29 -04:00
Jan Kotas 64c3405594 Cleanup memory copy helpers (dotnet/coreclr#27307)
- Merge RuntimeImports that has just a few FCalls into Buffer.
- Delete assembly implementation of memcpy for ARM. The implementation was outdated and it was used only on Windows ARM and only for fraction of memcpy callsites.
- Delete unnecessary PInvoke into SysStringLen. We have a managed equivalent on System.Runtime.InteropServices.Marshal.
- Delete large amount of redundant code between SecureString.Windows.cs and SecureString.Unix.cs, and switch SecureString to use Spans
- Fold null check into IsWin32Atom

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas d4a2f01064 Rewrite Buffer.BlockCopy in C# (dotnet/coreclr#27216)
* Rewrite Buffer.BlockCopy in C#

Fixes #27106

* Workaround to enable type check optimizations for BlockCopy only

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Tanner Gooding 8ea0426488 Improve codegen for IsNan (dotnet/coreclr#27272)
* Updating double.IsNaN and float.IsNaN to just `return x != x`

* Updating x86 codegen to optimize `x != x` for floating-point

* Changing the isnan optimization to only check the operand assigned registers

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas 113d855adb Delete ENABLE_WINRT define (dotnet/coreclr#27240)
.NET Native specific code not needed anymore

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Ben Adams d5b182c6d8 Improve Dictionary TryGetValue size/perfomance (dotnet/coreclr#27195)
* Dictionary avoid second bounds check in Get methods

* Add NullRef methods to Unsafe

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 274db87e60 Fix DecoderNLS.Convert to out the correct value for 'completed' (#27210)
This is a complementary fix to https://github.com/dotnet/coreclr/issues/23020

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Del Myers 0e02fa55af Enable ETW/EventSource logging of task IDs for boxed state machines (#27115)
* Wrap MoveNext Action for TPL event tracing

Wraps the MoveNext action of the AsyncStateMachineBox in a
continuation wrapper when async causality tracing is on so that the
TPL event source can find the task that is associated with a
continuation. Does not wrap otherwise.

* Clarifying comment

* removing trailing whitespace

* code review feedback.

Makes AsyncMethodBuilderCore.TryFindContinuationTask check to
whether the target of a continuation is itself a task, as a fall-back
to checking to see if the continuation is a ContinuationWrapper

* Got rid of unnecessary null checks

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 5fdea0b84b Update Utf8String to match latest prototype (dotnet/coreclr#27046)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Stephen Toub d34ba480f9 Use stackalloc expressions with builder ctors (dotnet/coreclr#27194)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Aaron Robinson 296b8d40cf Conditionally remove the GC transition from a P/Invoke (#26458)
* Provide mechanism to remove the GC transition from a P/Invoke.
  New Attribute: 'System.Runtime.InteropServices.SuppressGCTransitionAttribute'

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas cd2388fd6f Replace primitive Array fcalls with managed implementations (#27123)
Prep-work for #27106

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Filip Navara 858fc11900 Move portable thread pool to shared partition (dotnet/corert#7828)
* Move portable thread pool to shared partition

* Rename ClrThreadPool to PortableThreadPool

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Dan Moseley 565b81e26f Annotate System.Resources.ResourceWriter for nullable ref types (#41880)
* Annotate System.Resources.ResourceWriter for nullable ref types

* ref

* Mark common as nullable enable explicitly

* _resourceList

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Eriawan Kusumawardhono f731db32dd Replace tfm based constants with SDK ones (dotnet/corefx#41601)
* replace DefineConstants with SDK ones

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Zoltan Varga b96260e603 [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now. (#17466)
* [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now.

* Fix checks so stack overflows work again.
2019-10-22 06:31:23 -04:00
Larry Ewing 20752df324 [debugger][wasm] Check the length of the vars array. Fixes #17473 2019-10-21 18:45:57 -05:00
Vlad Brezae c1461e4a2d [interp] Replace loading of static readonly fields with LDC (#17420)
LDC is faster and also allows for constant propagation. Cleanup code a little bit.
2019-10-21 23:42:24 +03:00
Jay Krell fd82c6036e No type_traits on Android.
In file included from /Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall.c:42:
/Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall-internals.h:89:10: fatal error: 'type_traits' file not found
2019-10-21 11:15:16 -05:00
Kenneth Pouncey 7e522ef179 Modify the check for streaming to also check that the browser Response object exists. 2019-10-21 10:27:05 -05:00
Jay Krell 21d7d41bfe [cxx][debug-interp] Cast once from void* to non-void*. 2019-10-21 10:22:43 -05:00
Kenneth Pouncey 2cea0a89d1 Fix host object existence check.
- The checking for the existence of the object is incorrect: Missing parens around undefined.
- Modify error description returned back to managed code.
- Fix capitalization in global object check error.
2019-10-21 10:21:16 -05:00
Thays Grazia 0d357c1347 [tests] Replace __WATCHOS__ with MONOTOUCH_WATCH (#17459)
Fixes the conditional from mono/mono#17423, we don't define __WATCHOS__ in the Mono BCL builds. We use MONOTOUCH_WATCH instead.
2019-10-21 16:20:12 +02:00
Kenneth Pouncey f07e93da2f [wasm] Make sure message pump is working when executing via NODEJS (#17453)
- Tests if we are running in ENVIRONMENT_IS_NODE
   - run message pump if we are.

- This also fixes GC not running.

- Clean up some messages.
2019-10-21 14:29:18 +02:00
Kenneth Pouncey 3ff90ac0c2 Export mono_load_runtime_and_bcl so that is available in nodejs. (#17456) 2019-10-21 13:50:57 +02:00
Jay Krell 9f50bdb2d9 [Coop] Unconvert Microsoft.Win32.NativeMethods. (#16992)
* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_CloseProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcessId.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetExitCodeProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessTimes.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_TerminateProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_WaitForInputIdle.
2019-10-21 10:29:39 +02:00
Zoltan Varga 4b9226619c [interp] Small fixes. (#17442)
* [interp] Really abort in ves_real_abort ().

* [interp] Fix stack usage in TRACE_EXIT.

* [interp] Add a callback to free ThreadContext.

* Fix warnings.

* Remove an empty statement.

* Fix a double free.
2019-10-20 23:28:25 -04:00
Filip Navara 32ab6bcf00 [netcore] Implement Array.GetElementSize as intrinsic (#17437)
* [netcore] Implement Array.GetElementSize as intrinsic

* Remove unnecessary icall

* alloc_preg -> alloc_ireg

* Implement Array.GetElementSize as intrinsic in interpreter

* Address PR feedback
2019-10-20 18:39:39 -04:00
Filip Navara 67c8d0d812 [netcore] Remove unused code for Array.Unsafe[Mov/Load/Store] intrinsics 2019-10-20 12:44:52 +02:00
monojenkins 5fcd00669c [csproj] Update project files 2019-10-20 12:43:59 +02:00
monojenkins b19d9aa658 Bump API snapshot submodule 2019-10-20 12:43:59 +02:00
Zoltan Varga 1d44139d01 Bump cecil. 2019-10-20 12:43:59 +02:00
Filip Navara bc678a2c10 Use correct variable in mini_get_cpu_features (#17436)
* Use correct variable in mini_get_cpu_features

* Address PR feedback
2019-10-20 01:26:44 -04:00
Filip Navara 91f10aeee4 Remove dead code left out in #17341 (#17433) 2019-10-20 01:23:05 -04:00
Jay Krell 91413f78f3 [jit][x86][cxx] int/ptr casts in mini-x86.c (#17380) 2019-10-19 19:20:56 +02:00
Jay Krell 84f4e1dcbf Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same, and it is only used once. (#17429)
Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same and it is only used once.
2019-10-19 13:25:54 +02:00
Thays Grazia 105ead8707 Changing behavior of some tests in WatchOs
Needs this ifdefs because of this https://github.com/mono/mono/pull/17260. (#17423)
2019-10-18 21:23:44 -04:00
Aleksey Kliger (λgeek) d8ae32a9ee [bcl] Add null reference checks to Interlocked.Exchange<T> (#17400)
* [bcl] Add null reference checks to Interlocked.Exchange<T>

and Interlocked.CompareExchange<T>

This is to mimic the old behavior in unmanaged when the intrinsic cannot be
used.

The issue was detected via the coreclr acceptance test:
https://github.com/mono/coreclr/blob/mono/tests/src/baseservices/threading/interlocked/exchange/exchangetneg.il

* and netcore
2019-10-18 19:11:22 -04:00
Jay Krell a2992d634b [cxx] [wasm] int/ptr casts esp. m2n-gen. (#17379) 2019-10-19 00:03:00 +02:00
Martin Baulig fffad7e3a8 Remove the Legacy TLS Provider. (#17391)
* `MonoTlsProviderFactory` now treats "legacy" as an alias for "default".

* `Mono.Net.Security/LegacySslStream`: Removed.  This class also wasn't using the `MobileAuthenticatedStream` base class, thus preventing cleanups simplifications in that area.

* `Mono.Net.Security/LegacyTlsProvider`: Removed.

* `Mono.Security/Mono.Security.Protocol.Tls*`: Removed all the old legacy code.

* `Mono.Security.Interface.MonoTlsProviderFactory`: Bump internal version to 4.
2019-10-19 00:02:48 +02:00
imhameed 8836231459 Merge pull request #17279 from imhameed/bug-gh-16935-netcore-equalsall-codegen
Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers.
2019-10-18 12:47:41 -07:00
Alex Rønne Petersen d44e0e80d7 Some updates to make RISC-V buildable again. (#17408)
Some updates to make RISC-V buildable again.

Also some updates to the toolchain setup documentation.
2019-10-18 20:16:14 +02:00
Kenneth Pouncey a0c7d73d6f [sdks] add .gitignore python __pycache__ (#17414)
This will ignore the __pycache__ files generated after the tools offset build.
2019-10-18 18:08:57 +02:00
Vlad Brezae 37f628750a [profiler] Fix coverage profiler on macos (#17396)
If we have coverage profiling enabled for a method (mono_profiler_coverage_instrumentation_enabled, with coverage_filter callback), then we should be able to allocate a coverage info structure (mono_profiler_coverage_alloc). After https://github.com/mono/mono/commit/52429673dc617fd437518757c04c7448211075e9, coverage_filter was reporting wrapper methods as ok to profile, while mono_profiler_coverage_alloc was refusing to allocate the necessary structure, leading to crashes.
2019-10-18 16:31:47 +02:00
Jay Krell 958b9b2658 [interp][wasm] Remove more varargs to conserve stack. (#17388) 2019-10-18 10:39:05 +03:00
Egor Bogatov 4908d1f19a Enable GSS on Linux and Android (#17223) 2019-10-17 19:01:44 -04:00
Imran Hameed 7c7d406fc0 Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers. 2019-10-10 16:05:34 -07:00
2041 changed files with 114204 additions and 126856 deletions

No files matched your search

+4 -1
View File
@@ -10,6 +10,7 @@ mkinstalldirs crlf=input
*.sources crlf=input
.gitattributes crlf=input
*akefile* crlf=input
m4/mono-output.m4 crlf=input
# ensure native line endings on checkout
*.c crlf
@@ -19,8 +20,10 @@ mkinstalldirs crlf=input
*.sln crlf
*.*proj* crlf
*.bat text eol=crlf
*.cmd text eol=crlf
# don't do anything to line-endings. Let CRLFs/LFs go into the repo, and CRLF/LFs on checkout
*.bat -crlf
*.xml -crlf
# CRLF Handling
+25 -24
View File
@@ -18,11 +18,11 @@
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/data @directhex @akoeplinger
/data/gdb @lewurm @vargaz
/data/lldb @lewurm @vargaz
/data/gdb @vargaz
/data/lldb @vargaz
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp @lewurm
/docs/riscv.md @alexrp
/docs/sources/mono-api-profiler.html @BrzVlad
/external @akoeplinger @marek-safar
@@ -34,7 +34,7 @@
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/linker @marek-safar
/external/llvm @SamMonoRT @vargaz @imhameed
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
@@ -65,24 +65,24 @@
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/System.Data* @egorbo @steveisok
/mcs/errors @marek-safar @lewurm
/mcs/ilasm @marek-safar @lewurm
/mcs/jay @marek-safar @lewurm
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar @lewurm
/mcs/errors @marek-safar
/mcs/ilasm @marek-safar
/mcs/jay @marek-safar
/mcs/mcs @marek-safar
/mcs/tests @marek-safar
/mcs/tools @marek-safar @akoeplinger @steveisok
/mcs/tools/gensources @kg @akoeplinger
/mcs/tools/resx2sr @baulig
/mcs/tools/upload-to-sentry @directhex @akoeplinger
/mono/arch @vargaz @lewurm
/mono/arch @vargaz
/mono/btls @baulig @marek-safar
/mono/dis @lambdageek @vargaz
/mono/eglib @vargaz @lambdageek
/mono/benchmark @SamMonoRT @lewurm
/mono/benchmark @SamMonoRT
/mono/metadata @vargaz @lambdageek @thaystg
/mono/metadata/*profiler* @lambdageek
@@ -99,14 +99,14 @@
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lewurm @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz @BrzVlad
/mono/mini @vargaz @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @vargaz
/mono/mini/*exceptions* @vargaz @BrzVlad
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/*type-check* @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/interp/* @BrzVlad
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
@@ -116,10 +116,12 @@
/mono/sgen @BrzVlad @lambdageek
/mono/utils @vargaz @lewurm @lambdageek
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
/mono/utils/mono-mem* @lewurm @vargaz
/mono/tools/offsets-tool @rolfbjarne
/mono/utils @vargaz @lambdageek
/mono/utils/atomic* @vargaz
/mono/utils/mono-hwcap* @vargaz
/mono/utils/mono-mem* @vargaz
/mono/utils/mono-merp* @alexischr @lambdageek
/mono/utils/mono-state* @alexischr @lambdageek
/mono/utils/mono-threads* @lambdageek @vargaz
@@ -135,14 +137,14 @@
/runtime @akoeplinger @marek-safar
/samples @lambdageek @lewurm
/samples @lambdageek
/samples/profiler @BrzVlad @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
/scripts/ci @akoeplinger @directhex
/sdks @marek-safar @lewurm @vargaz @lambdageek
/sdks @marek-safar @vargaz @lambdageek
/sdks/builds/android* @alexischr @jonpryor
/sdks/builds/mxe* @alexischr @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
@@ -156,6 +158,5 @@
/tools @lambdageek @vargaz
/tools/locale-builder @marek-safar @egorbo
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/pedump @lambdageek
/tools/sgen @brzvlad @vargaz @naricc
+1 -1
View File
@@ -43,5 +43,5 @@ Please paste the stack trace here if available.
```
<!--
You can join us at https://gitter.im/mono/mono to discuss your reported issue
You can join us on Discord (https://aka.ms/dotnet-discord) in the #monovm channel to discuss your reported issue
-->
+1 -2
View File
@@ -18,6 +18,7 @@
*.vssscc
*_i.c
*_p.c
*.mdb
*.ncb
*.suo
*.tlb
@@ -139,8 +140,6 @@ llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
tools/offsets-tool-py/offtool/
##############################################################################
# Arcade output directories
+7 -4
View File
@@ -58,13 +58,16 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/llvm"]
path = external/llvm
url = git://github.com/mono/llvm.git
branch = release_60
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+1 -1
View File
@@ -76,7 +76,7 @@ The Licenses
These are the licenses used in Mono, the files are located:
### MIT X11 License
### MIT License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
+13 -50
View File
@@ -2,33 +2,11 @@ ACLOCAL_AMFLAGS = -I m4
AM_CFLAGS = $(WERROR_CFLAGS)
MONOTOUCH_SUBDIRS = $(libgc_dir) mono
SUBDIRS = @MONO_SUBDIRS@
noinst_SUBDIRS = @MONO_NOINST_SUBDIRS@
DIST_SUBDIRS = $(SUBDIRS) m4 netcore
# Some tools might not build when cross-compiling
if CROSS_COMPILING
tools_dir =
else
tools_dir = tools
endif
if ENABLE_MSVC
build_with_msvc = msvc
build_without_msvc =
else
build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
SUBDIRS = mk llvm mono runtime netcore netcore/corerun
else
update_submodules = update_submodules
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
endif
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore netcore/corerun
if !ENABLE_NETCORE
all: $(update_submodules)
@@ -67,6 +45,9 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
@@ -113,30 +94,10 @@ mcs-do-compiler-tests:
bootstrap-world: compiler-tests
$(MAKE) install
if INSTALL_MONOTOUCH
monotouch-do-build: config.h
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-build";; \
* ) target="all";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-build)
monotouch-do-clean:
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-clean";; \
* ) target="clean";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-clean)
endif
install:
for mydir in $(filter-out $(noinst_SUBDIRS),$(SUBDIRS)); do \
(cd $${mydir} && ${MAKE} install) \
done
update-csproj:
-rm msvc/scripts/order
@@ -180,3 +141,5 @@ update-solution-files:
update-solution-files-with-tests:
$(MAKE) "GENPROJ_ARGS=2012 true true" update-solution-files
endif
+1 -1
View File
@@ -2,7 +2,7 @@
<configuration>
<packageSources>
<clear />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
+7 -6
View File
@@ -5,7 +5,13 @@ standards for C# and the Common Language Runtime.
The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.org/)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/mono/mono?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Join us on [Discord](https://aka.ms/dotnet-discord) in the `#monovm` channel:
<a href="https://aka.ms/dotnet-discord">
<img src="https://img.shields.io/discord/732297728826277939?style=flat-square&label=Discord&logo=discord&logoColor=white&color=7289DA">
</a>
### Contents
1. [Compilation and Installation](#compilation-and-installation)
2. [Using Mono](#using-mono)
@@ -434,7 +440,6 @@ Disables compilation for the SSA optimization
framework, and the various SSA-based optimizations.
* `--enable-llvm`
* `--enable-loadedllvm`
* This enables the use of LLVM as a code generation engine
for Mono. The LLVM code generator and optimizer will be
@@ -447,10 +452,6 @@ full details and up-to-date information on this feature.
* You will need to have an LLVM built that Mono can link
against.
* The `--enable-loadedllvm` variant will make the LLVM backend
into a runtime-loadable module instead of linking it directly
into the main mono binary.
* `--enable-big-arrays` - Enable use of arrays with indexes larger
than Int32.MaxValue.
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "13a40eedacc799a78710aba5c8e4b2dcc80a7e61",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+17 -8
View File
@@ -104,12 +104,10 @@ if grep "^AM_PROG_LIBTOOL" configure.ac >/dev/null; then
fi
fi
#
# Plug in the extension module
#
# Parse parameters
has_ext_mod=false
ext_mod_args=''
has_disable_boehm=false
for PARAM; do
if [[ $PARAM =~ "--enable-extension-module" ]] ; then
has_ext_mod=true
@@ -117,8 +115,14 @@ for PARAM; do
ext_mod_args=`echo $PARAM | cut -d= -f2`
fi
fi
if [[ $PARAM =~ "--disable-boehm" ]] ; then
has_disable_boehm=true
fi
done
#
# Plug in the extension module
#
if test x$has_ext_mod = xtrue; then
pushd $top_srcdir../mono-extensions/scripts
sh ./prepare-repo.sh $ext_mod_args || exit 1
@@ -150,10 +154,15 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
# Update all submodules recursively to ensure everything is checked out
if test -e $srcdir/scripts/update_submodules.sh; then
(cd $srcdir && scripts/update_submodules.sh)
fi
if test x$has_disable_boehm = xfalse -a -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
+4 -48
View File
@@ -1,50 +1,6 @@
# Contributor Code of Conduct
# Code of Conduct
As contributors and maintainers of this project, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating
documentation, submitting pull requests or patches, and other activities.
This project has adopted the code of conduct defined by the Contributor Covenant
to clarify expected behavior in our community.
We are committed to making participation in this project a harassment-free
experience for everyone, regardless of level of experience, gender, gender
identity and expression, sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, or nationality.
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing other's private information, such as physical or electronic
addresses, without explicit permission
* Other unethical or unprofessional conduct
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
By adopting this Code of Conduct, project maintainers commit themselves to
fairly and consistently applying these principles to every aspect of managing
this project. Project maintainers who do not follow or enforce the Code of
Conduct may be permanently removed from the project team.
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting a project maintainer at miguel@xamarin.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. Maintainers are
obligated to maintain confidentiality with regard to the reporter of an
incident.
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 1.3.0, available at
[http://contributor-covenant.org/version/1/3/0/][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/3/0/
For more information, see the [.NET Foundation Code of Conduct](https://dotnetfoundation.org/code-of-conduct).
+569 -395
View File
File diff suppressed because it is too large. Load diff
+4 -3
View File
@@ -1,4 +1,5 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
@@ -19,11 +20,11 @@ def print_frames(thread, num_frames, current_thread):
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
except Exception as e:
print("DBG: execfail:" + str(e))
traceback.print_exc()
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -242,7 +242,7 @@ namespace HtmlAgilityPack
}
/// <summary>
/// Checks for existance of attribute with given name
/// Checks for existence of attribute with given name
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
+1 -1
View File
@@ -22,7 +22,7 @@ Layout
as well as the upcoming Monodoc tools and index creation
tools).
The files can contain any ammount of extra information.
The files can contain any amount of extra information.
The documentation from the source code is then merged with
these files and the header/footer information to produce
+61 -39
View File
@@ -2,36 +2,58 @@
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documents
## Useful RISC-V documentation
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications/)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa/)
* [RISC-V Debug Specification Standard](https://github.com/riscv/riscv-debug-spec/blob/master/riscv-debug-spec.pdf)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual/blob/master/riscv-asm.md)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md)
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V Linux](https://github.com/riscv/riscv-linux)
* [RISC-V LLD](https://github.com/riscv/riscv-lld)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V QEMU](https://github.com/riscv/riscv-qemu)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
* [lowRISC Organization](https://github.com/lowrisc)
* [RISC-V LLVM](https://github.com/lowrisc/riscv-llvm)
## Setting up a cross environment
Setting up a cross environment with a Linux toolchain and QEMU is quite easy.
### Debian/Ubuntu packages
First, add these to your `.bashrc` (or some other script that you run):
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
@@ -39,47 +61,42 @@ export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Install some dependencies needed to build the toolchain:
Next, install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev libusb-1.0-0-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev device-tree-compiler pkg-config libexpat-dev libglib2.0-dev libpixman-1-dev
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
```
Now you can build the toolchain:
Finally, build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-tools.git
$ cd riscv-tools
$ ./build.sh
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-multilib
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ make linux
$ cd ../..
$ git clone --recursive [email protected]:riscv/riscv-qemu.git
$ cd riscv-qemu
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --disable-werror
$ make
$ make install
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
@@ -89,19 +106,24 @@ Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 $RISCV/sysroot/bin/mono hello.exe
$ qemu-riscv64 ./mono/mini/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono-sgen --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono-sgen
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
```
## Things to be done
In no particular order:
Things that need to be done beyond the basic 64-bit port, in no particular
order:
* Complete the soft float port.
* Complete the 32-bit port.
+8 -8
View File
@@ -1,13 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha1.19515.14">
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>b50bd0a1d3f0326fc717706cb5c5950ab41cf437</Sha>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha1.19515.17">
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>0ceceab994c8e965bf50b11fc230dc2ea67aa1ec</Sha>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
@@ -15,13 +15,13 @@
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.19511.1">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.20201.2">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>64a027b8d8b3a8e8e214bb00da02148335e3bd92</Sha>
<Sha>bce0a98620c1c5a110b2bba9912f3d5929069c6b</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.19511.1">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.20201.2">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>64a027b8d8b3a8e8e214bb00da02148335e3bd92</Sha>
<Sha>bce0a98620c1c5a110b2bba9912f3d5929069c6b</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
+2 -2
View File
@@ -5,9 +5,9 @@
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha1.19515.14</MicrosoftPrivateCoreFxNETCoreAppVersion>
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha1.19515.17</MicrosoftNETCoreAppVersion>
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
-158
View File
@@ -1,158 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SymbolToolPath # Full path to directory where dotnet symbol-tool was installed
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $TargetServerParam, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $SymbolsPath
)
$FileName = [System.IO.Path]::GetFileName($FullPath)
$Extension = [System.IO.Path]::GetExtension($FullPath)
# Those below are potential symbol files that the `dotnet symbol` might
# return. Which one will be returned depend on the type of file we are
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
}
elseif (Test-Path $SymbolPath) {
return "Module"
}
else {
return $null
}
}
function CountMissingSymbols {
param(
[string] $PackagePath # Path to a NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
# How many files are missing symbol information
$MissingSymbols = 0
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
# Makes easier to reference `symbol tool`
Push-Location $SymbolToolPath
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
}
else {
Write-Host "No symbols found on SymWeb!"
}
}
}
}
Pop-Location
return $MissingSymbols
}
function CheckSymbolsAvailable {
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-Error "Missing symbols for $Status modules in the package $FileName"
}
Write-Host
}
}
CheckSymbolsAvailable
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PushToStaticFeed task that will read the build asset
manifest and publish the assets described in the manifest to
informed target feeds.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(MSBuildThisFileDirectory)DefaultVersions.props" Condition="Exists('$(MSBuildThisFileDirectory)DefaultVersions.props')" />
<!--
This won't be necessary once we solve this issue:
https://github.com/dotnet/arcade/issues/2266
-->
<Import Project="$(MSBuildThisFileDirectory)ArtifactsCategory.props" Condition="Exists('$(MSBuildThisFileDirectory)ArtifactsCategory.props')" />
<Import Project="$(NuGetPackageRoot)microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<Target Name="PublishToFeed">
<Error Condition="'$(ArtifactsCategory)' == ''" Text="ArtifactsCategory: The artifacts' category produced by the build wasn't provided." />
<Error Condition="'$(AccountKeyToStaticFeed)' == ''" Text="AccountKeyToStaticFeed: Account key for target feed wasn't provided." />
<Error Condition="'$(ManifestsBasePath)' == ''" Text="Full path to asset manifests directory wasn't provided." />
<Error Condition="'$(BlobBasePath)' == '' AND '$(PackageBasePath)' == ''" Text="A valid full path to BlobBasePath of PackageBasePath is required." />
<ItemGroup>
<!-- Include all manifests found in the manifest folder. -->
<ManifestFiles Include="$(ManifestsBasePath)*.xml" />
</ItemGroup>
<Error Condition="'@(ManifestFiles)' == ''" Text="No manifest file was found in the provided path: $(ManifestsBasePath)" />
<!--
For now the type of packages being published will be informed for the whole build.
Eventually this will be specified on a per package basis:
TODO: https://github.com/dotnet/arcade/issues/2266
-->
<PropertyGroup>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCORE'">https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCOREVALIDATION'">https://dotnetfeed.blob.core.windows.net/arcade-validation/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORETOOLING'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ENTITYFRAMEWORKCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETEXTENSIONS'">https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORECLR'">https://dotnetfeed.blob.core.windows.net/dotnet-coreclr/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORESDK'">https://dotnetfeed.blob.core.windows.net/dotnet-sdk/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSINTERNAL'">https://dotnetfeed.blob.core.windows.net/dotnet-tools-internal/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSET'">https://dotnetfeed.blob.core.windows.net/dotnet-toolset/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'WINDOWSDESKTOP'">https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'NUGETCLIENT'">https://dotnetfeed.blob.core.windows.net/nuget-nugetclient/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETENTITYFRAMEWORK6'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframework6/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETBLAZOR'">https://dotnetfeed.blob.core.windows.net/aspnet-blazor/index.json</TargetStaticFeed>
</PropertyGroup>
<Error
Condition="'$(TargetStaticFeed)' == ''"
Text="'$(ArtifactsCategory)' wasn't recognized as a valid artifact category. Valid categories are: '.NetCore' and '.NetCoreValidation'" />
<!-- Iterate publishing assets from each manifest file. -->
<PushArtifactsInManifestToFeed
ExpectedFeedUrl="$(TargetStaticFeed)"
AccountKey="$(AccountKeyToStaticFeed)"
BARBuildId="$(BARBuildId)"
MaestroApiEndpoint="$(MaestroApiEndpoint)"
BuildAssetRegistryToken="$(BuildAssetRegistryToken)"
Overwrite="$(OverrideAssetsWithSameName)"
PassIfExistingItemIdentical="$(PassIfExistingItemIdentical)"
MaxClients="$(MaxParallelUploads)"
UploadTimeoutInMinutes="$(MaxUploadTimeoutInMinutes)"
AssetManifestPath="%(ManifestFiles.Identity)"
BlobAssetsBasePath="$(BlobBasePath)"
PackageAssetsBasePath="$(PackageBasePath)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
</ItemGroup>
</Project>
-82
View File
@@ -1,82 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PublishSymbols tasks to publish symbols to MSDL and SymWeb.
Parameters:
- PDBArtifactsDirectory : Full path to directory containing PDB files to be published.
- BlobBasePath : Full path containing *.symbols.nupkg packages to be published.
- DotNetSymbolServerTokenMsdl : PAT to access MSDL.
- DotNetSymbolServerTokenSymWeb : PAT to access SymWeb.
- DotNetSymbolExpirationInDays : Expiration days for published packages. Default is 3650.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(NuGetPackageRoot)microsoft.symboluploader.build.task\$(SymbolUploaderVersion)\build\PublishSymbols.targets" />
<Target Name="PublishSymbols">
<ItemGroup>
<FilesToPublishToSymbolServer Include="$(PDBArtifactsDirectory)\*.pdb"/>
<PackagesToPublishToSymbolServer Include="$(BlobBasePath)\*.symbols.nupkg"/>
<!--
These packages from Arcade-Services include some native libraries that
our current symbol uploader can't handle. Below is a workaround until
we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
-->
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Darc.*" />
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Maestro.Tasks.*" />
</ItemGroup>
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
<Message
Importance="High"
Text="No symbol package(s) were found to publish."
Condition="$(PublishToSymbolServer) == false" />
<!-- Symbol Uploader: MSDL -->
<Message Importance="High" Text="Publishing symbol packages to MSDL ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenMsdl)"
SymbolServerPath="https://microsoftpublicsymbols.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
<!--
Symbol Uploader: SymWeb
Watson, VS insertion testings and the typical internal dev usage require SymWeb.
Currently we need to call the task twice (https://github.com/dotnet/core-eng/issues/3489).
-->
<Message Importance="High" Text="Publishing symbol packages to SymWeb ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenSymWeb)"
SymbolServerPath="https://microsoft.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.SymbolUploader.Build.Task" Version="$(SymbolUploaderVersion)" />
</ItemGroup>
</Project>
+141
View File
@@ -0,0 +1,141 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile $(Build.SourcesDirectory)/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
if (!$Password) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Please supply a valid PAT'
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
$dotnet3Source = $sources.SelectSingleNode("add[@key='dotnet3']")
if ($dotnet3Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$dotnet31Source = $sources.SelectSingleNode("add[@key='dotnet3.1']")
if ($dotnet31Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$doc.Save($filename)
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $(Build.SourcesDirectory)/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
Write-PipelineTelemetryError -Category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [ -z "$CredToken" ]; then
Write-PipelineTelemetryError -category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Please supply a valid PAT"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
PackageSources=()
# Ensure dotnet3-internal and dotnet3-internal-transport are in the packageSources if the public dotnet3 feeds are present
grep -i "<add key=\"dotnet3\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal')
grep -i "<add key=\"dotnet3-internal-transport\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal-transport')
fi
# Ensure dotnet3.1-internal and dotnet3.1-internal-transport are in the packageSources if the public dotnet3.1 feeds are present
grep -i "<add key=\"dotnet3.1\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3.1-internal\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal')
grep -i "<add key=\"dotnet3.1-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal-transport')
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources+="$IFS"
PackageSources+=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" $ConfigFile
fi
done
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
- PackageBasePath : Directory containing all files that need to be validated.
- SignCheckVersion : Version of SignCheck package to be used.
- SignValidationExclusionList : ItemGroup containing exclusion list to be forwarded to SignCheck.
- EnableJarSigningCheck : Whether .jar files should be validated.
- EnableStrongNameCheck : Whether strong name check should be performed.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<!--
From 'Signing.props' we import $(SignValidationExclusionList)
-->
<Import Project="$(MSBuildThisFileDirectory)Signing.props" Condition="Exists('$(MSBuildThisFileDirectory)Signing.props')" />
<Target Name="ValidateSigning">
<PropertyGroup>
<SignCheckToolPath>$(NuGetPackageRoot)Microsoft.DotNet.SignCheck\$(SignCheckVersion)\tools\Microsoft.DotNet.SignCheck.exe</SignCheckToolPath>
<SignCheckInputDir>$(PackageBasePath)</SignCheckInputDir>
<SignCheckLog>signcheck.log</SignCheckLog>
<SignCheckErrorLog>signcheck.errors.log</SignCheckErrorLog>
<SignCheckExclusionsFile>signcheck.exclusions.txt</SignCheckExclusionsFile>
</PropertyGroup>
<ItemGroup>
<!--
Documentation for these arguments is available here:
https://github.com/dotnet/arcade/tree/master/src/SignCheck
-->
<SignCheckArgs Include="--recursive" />
<SignCheckArgs Include="--traverse-subfolders" />
<SignCheckArgs Include="--file-status AllFiles" />
<SignCheckArgs Include="--log-file $(SignCheckLog)" />
<SignCheckArgs Include="--error-log-file $(SignCheckErrorLog)" />
<SignCheckArgs Include="--input-files $(SignCheckInputDir)" />
<SignCheckArgs Include="--exclusions-file $(SignCheckExclusionsFile)" Condition="'@(SignValidationExclusionList)' != ''" />
<SignCheckArgs Include="--verify-jar" Condition="'$(EnableJarSigningCheck)' == 'true'" />
<SignCheckArgs Include="--verify-strongname" Condition="'$(EnableStrongNameCheck)' == 'true'" />
</ItemGroup>
<WriteLinesToFile
File="$(SignCheckExclusionsFile)"
Lines="@(SignValidationExclusionList)"
Condition="'@(SignValidationExclusionList)' != ''"
Overwrite="true"
Encoding="Unicode"/>
<!--
IgnoreExitCode='true' because the tool doesn't return '0' on success.
-->
<Exec
Command="&quot;$(SignCheckToolPath)&quot; @(SignCheckArgs, ' ')"
IgnoreExitCode='true'
ConsoleToMsBuild="false"
StandardErrorImportance="high" />
<Error
Text="Signing validation failed. Check $(SignCheckErrorLog) for more information."
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
<Message
Text="##vso[artifact.upload containerfolder=LogFiles;artifactname=LogFiles]{SignCheckErrorLog}"
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.SignCheck" Version="$(SignCheckVersion)" />
</ItemGroup>
</Project>
-184
View File
@@ -1,184 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SourceLinkToolPath, # Full path to directory where dotnet SourceLink CLI was installed
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
$RepoTreeURL = -Join("https://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
Measure-Command { ValidateSourceLinkLinks }
+56 -41
View File
@@ -18,6 +18,7 @@ Param(
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
@@ -25,49 +26,55 @@ Param(
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
# Unset 'Platform' environment variable to avoid unwanted collision in InstallDotNetCore.targets file
# some computer has this env var defined (e.g. Some HP)
if($env:Platform) {
$env:Platform=""
}
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
. $PSScriptRoot\tools.ps1
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot "restore-toolset.ps1"
$script = Join-Path $EngRoot 'restore-toolset.ps1'
if (Test-Path $script) {
. $script
@@ -78,8 +85,8 @@ function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'Build.binlog') } else { '' }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { '' }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
@@ -113,7 +120,15 @@ function Build {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($clean) {
if (Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host 'Artifacts directory deleted.'
}
exit 0
}
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
@@ -123,7 +138,7 @@ try {
$nodeReuse = $false
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
if ($restore) {
InitializeNativeTools
}
@@ -131,7 +146,7 @@ try {
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+14 -1
View File
@@ -26,6 +26,7 @@ usage()
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
@@ -62,6 +63,7 @@ publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
@@ -82,6 +84,9 @@ while [[ $# > 0 ]]; do
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
@@ -196,7 +201,15 @@ function Build {
ExitWithExitCode 0
}
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
fi
if [[ "$restore" == true ]]; then
InitializeNativeTools
fi
@@ -1,41 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)
## Specify the toolchain
set(TOOLCHAIN "arm-linux-androideabi")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
add_compile_options(-mfloat-abi=soft)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/arm-linux-androideabi/)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
@@ -1,42 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
## Specify the toolchain
set(TOOLCHAIN "aarch64-linux-android")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
## Needed for Android or bionic specific conditionals
add_compile_options(-D__ANDROID__)
add_compile_options(-D__BIONIC__)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+45 -51
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
set -e
__NDK_Version=r14
__NDK_Version=r21
usage()
{
@@ -16,11 +16,11 @@ usage()
echo.
echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"
echo "you can set the NDK_DIR environment variable to have this script use that installation of the NDK."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.21-arm64. This file is to replace '/etc/os-release', which is not available for Android."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.28-arm64. This file is to replace '/etc/os-release', which is not available for Android."
exit 1
}
__ApiLevel=21 # The minimum platform for arm64 is API level 21
__ApiLevel=28 # The minimum platform for arm64 is API level 21 but the minimum version that support glob(3) is 28. See $ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/glob.h
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
@@ -53,13 +53,20 @@ for i in "$@"
done
# Obtain the location of the bash script to figure out where the root of the repo is.
__CrossDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__ScriptBaseDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__Android_Cross_Dir="$__CrossDir/android-rootfs"
__NDK_Dir="$__Android_Cross_Dir/android-ndk-$__NDK_Version"
__libunwind_Dir="$__Android_Cross_Dir/libunwind"
__lldb_Dir="$__Android_Cross_Dir/lldb"
__ToolchainDir="$__Android_Cross_Dir/toolchain/$__BuildArch"
__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"
if [[ ! -f "$__CrossDir" ]]; then
mkdir -p "$__CrossDir"
fi
# Resolve absolute path to avoid `../` in build logs
__CrossDir="$( cd "$__CrossDir" && pwd )"
__NDK_Dir="$__CrossDir/android-ndk-$__NDK_Version"
__lldb_Dir="$__CrossDir/lldb"
__ToolchainDir="$__CrossDir/android-ndk-$__NDK_Version"
if [[ -n "$TOOLCHAIN_DIR" ]]; then
__ToolchainDir=$TOOLCHAIN_DIR
@@ -78,60 +85,47 @@ echo "Target Toolchain location: $__ToolchainDir"
if [ ! -d $__NDK_Dir ]; then
echo Downloading the NDK into $__NDK_Dir
mkdir -p $__NDK_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__Android_Cross_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__CrossDir
fi
if [ ! -d $__lldb_Dir ]; then
mkdir -p $__lldb_Dir
echo Downloading LLDB into $__lldb_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
fi
# Create the RootFS for both arm64 as well as aarch
rm -rf $__Android_Cross_Dir/toolchain
echo "Download dependencies..."
__TmpDir=$__CrossDir/tmp/$__BuildArch/
mkdir -p "$__TmpDir"
echo Generating the $__BuildArch toolchain
$__NDK_Dir/build/tools/make_standalone_toolchain.py --arch $__BuildArch --api $__ApiLevel --install-dir $__ToolchainDir
# combined dependencies for coreclr, installer and libraries
__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"
# Install the required packages into the toolchain
# TODO: Add logic to get latest pkg version instead of specific version number
rm -rf $__Android_Cross_Dir/deb/
rm -rf $__Android_Cross_Dir/tmp
for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\
grep -A15 "Package: \(${__AndroidPackages// /\\|}\)" | grep -v "static\|tool" | grep Filename); do
mkdir -p $__Android_Cross_Dir/deb/
mkdir -p $__Android_Cross_Dir/tmp/$arch/
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu-dev_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb
if [[ "$path" != "Filename:" ]]; then
echo "Working on: $path"
wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"
fi
done
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob-dev_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support-dev_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma-dev_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind-dev_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb
echo Unpacking Termux packages
dpkg -x $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
cp -R $__Android_Cross_Dir/tmp/$__AndroidArch/data/data/com.termux/files/usr/* $__ToolchainDir/sysroot/usr/
cp -R "$__TmpDir/data/data/com.termux/files/usr/"* "$__ToolchainDir/sysroot/usr/"
# Generate platform file for build.sh script to assign to __DistroRid
echo "Generating platform file..."
echo "RID=android.${__ApiLevel}-${__BuildArch}" > $__ToolchainDir/sysroot/android_platform
echo "RID=android.21-arm64" > $__ToolchainDir/sysroot/android_platform
echo Now run:
echo CONFIG_DIR=\`realpath cross/android/$__BuildArch\` ROOTFS_DIR=\`realpath $__ToolchainDir/sysroot\` ./build.sh cross $__BuildArch skipgenerateversion skipnuget cmakeargs -DENABLE_LLDBPLUGIN=0
echo "Now to build coreclr, libraries and installers; run:"
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory coreclr
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory libraries
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory installer
+72 -28
View File
@@ -2,15 +2,16 @@
usage()
{
echo "Usage: $0 [BuildArch] [LinuxCodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "Usage: $0 [BuildArch] [CodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "BuildArch can be: arm(default), armel, arm64, x86"
echo "LinuxCodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine"
echo "CodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo " for FreeBSD can be: freebsd11 or freebsd12."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine and FReeBSD"
echo "--skipunmount - optional, will skip the unmount of rootfs folder."
exit 1
}
__LinuxCodeName=xenial
__CodeName=xenial
__CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__InitialDir=$PWD
__BuildArch=arm
@@ -25,8 +26,9 @@ __UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
__AlpinePackagesEdgeTesting=" lldb-dev"
__AlpinePackagesEdgeMain=" llvm9-libs"
__AlpinePackagesEdgeMain+=" python3"
# symlinks fixer
__UbuntuPackages+=" symlinks"
@@ -52,6 +54,15 @@ __AlpinePackages+=" krb5-dev"
__AlpinePackages+=" openssl-dev"
__AlpinePackages+=" zlib-dev"
__FreeBSDBase="12.1-RELEASE"
__FreeBSDPkg="1.10.5"
__FreeBSDPackages="libunwind"
__FreeBSDPackages+=" icu"
__FreeBSDPackages+=" libinotify"
__FreeBSDPackages+=" lttng-ust"
__FreeBSDPackages+=" llvm-90"
__FreeBSDPackages+=" krb5"
__UnprocessedBuildArgs=
while :; do
if [ $# -le 0 ]; then
@@ -80,7 +91,7 @@ while :; do
__BuildArch=armel
__UbuntuArch=armel
__UbuntuRepo="http://ftp.debian.org/debian/"
__LinuxCodeName=jessie
__CodeName=jessie
;;
x86)
__BuildArch=x86
@@ -109,36 +120,36 @@ while :; do
unset __LLDB_Package
;;
trusty) # Ubuntu 14.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=trusty
if [ "$__CodeName" != "jessie" ]; then
__CodeName=trusty
fi
;;
xenial) # Ubuntu 16.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=xenial
if [ "$__CodeName" != "jessie" ]; then
__CodeName=xenial
fi
;;
zesty) # Ubuntu 17.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=zesty
if [ "$__CodeName" != "jessie" ]; then
__CodeName=zesty
fi
;;
bionic) # Ubuntu 18.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=bionic
if [ "$__CodeName" != "jessie" ]; then
__CodeName=bionic
fi
;;
jessie) # Debian 8
__LinuxCodeName=jessie
__CodeName=jessie
__UbuntuRepo="http://ftp.debian.org/debian/"
;;
stretch) # Debian 9
__LinuxCodeName=stretch
__CodeName=stretch
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
buster) # Debian 10
__LinuxCodeName=buster
__CodeName=buster
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
@@ -148,14 +159,22 @@ while :; do
usage;
exit 1;
fi
__LinuxCodeName=
__CodeName=
__UbuntuRepo=
__Tizen=tizen
;;
alpine)
__LinuxCodeName=alpine
__CodeName=alpine
__UbuntuRepo=
;;
freebsd11)
__FreeBSDBase="11.3-RELEASE"
;&
freebsd12)
__CodeName=freebsd
__BuildArch=x64
__SkipUnmount=1
;;
--skipunmount)
__SkipUnmount=1
;;
@@ -191,25 +210,50 @@ if [ -d "$__RootfsDir" ]; then
rm -rf $__RootfsDir
fi
if [[ "$__LinuxCodeName" == "alpine" ]]; then
if [[ "$__CodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.7
__AlpineVersion=3.9
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeMain
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeTesting
rm -r $__ApkToolsDir
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__LinuxCodeName $__RootfsDir/etc/apt/sources.list
elif [[ "$__CodeName" == "freebsd" ]]; then
mkdir -p $__RootfsDir/usr/local/etc
wget -O - https://download.freebsd.org/ftp/releases/amd64/${__FreeBSDBase}/base.txz | tar -C $__RootfsDir -Jxf - ./lib ./usr/lib ./usr/libdata ./usr/include ./usr/share/keys ./etc ./bin/freebsd-version
# For now, ask for 11 ABI even on 12. This can be revisited later.
echo "ABI = \"FreeBSD:11:amd64\"; FINGERPRINTS = \"${__RootfsDir}/usr/share/keys\"; REPOS_DIR = [\"${__RootfsDir}/etc/pkg\"]; REPO_AUTOUPDATE = NO; RUN_SCRIPTS = NO;" > ${__RootfsDir}/usr/local/etc/pkg.conf
echo "FreeBSD: { url: "pkg+http://pkg.FreeBSD.org/\${ABI}/quarterly", mirror_type: \"srv\", signature_type: \"fingerprints\", fingerprints: \"${__RootfsDir}/usr/share/keys/pkg\", enabled: yes }" > ${__RootfsDir}/etc/pkg/FreeBSD.conf
mkdir -p $__RootfsDir/tmp
# get and build package manager
wget -O - https://github.com/freebsd/pkg/archive/${__FreeBSDPkg}.tar.gz | tar -C $__RootfsDir/tmp -zxf -
cd $__RootfsDir/tmp/pkg-${__FreeBSDPkg}
./autogen.sh && ./configure --prefix=$__RootfsDir/host && make install
rm -rf $__RootfsDir/tmp/pkg-${__FreeBSDPkg}
# install packages we need.
$__RootfsDir/host/sbin/pkg -r $__RootfsDir -C $__RootfsDir/usr/local/etc/pkg.conf update
$__RootfsDir/host/sbin/pkg -r $__RootfsDir -C $__RootfsDir/usr/local/etc/pkg.conf install --yes $__FreeBSDPackages
elif [[ -n $__CodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__CodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__CodeName $__RootfsDir/etc/apt/sources.list
chroot $__RootfsDir apt-get update
chroot $__RootfsDir apt-get -f -y install
chroot $__RootfsDir apt-get -y install $__UbuntuPackages
@@ -219,7 +263,7 @@ elif [[ -n $__LinuxCodeName ]]; then
umount $__RootfsDir/*
fi
if [[ "$__BuildArch" == "arm" && "$__LinuxCodeName" == "trusty" ]]; then
if [[ "$__BuildArch" == "arm" && "$__CodeName" == "trusty" ]]; then
pushd $__RootfsDir
patch -p1 < $__CrossDir/$__BuildArch/trusty.patch
patch -p1 < $__CrossDir/$__BuildArch/trusty-lttng-2.4.patch
+60 -36
View File
@@ -1,7 +1,11 @@
set(CROSS_ROOTFS $ENV{ROOTFS_DIR})
set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
set(CMAKE_SYSTEM_NAME Linux)
if(EXISTS ${CROSS_ROOTFS}/bin/freebsd-version)
set(CMAKE_SYSTEM_NAME FreeBSD)
else()
set(CMAKE_SYSTEM_NAME Linux)
endif()
set(CMAKE_SYSTEM_VERSION 1)
if(TARGET_ARCH_NAME STREQUAL "armel")
@@ -27,10 +31,17 @@ elseif(TARGET_ARCH_NAME STREQUAL "arm64")
elseif(TARGET_ARCH_NAME STREQUAL "x86")
set(CMAKE_SYSTEM_PROCESSOR i686)
set(TOOLCHAIN "i686-linux-gnu")
elseif (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
set(CMAKE_SYSTEM_PROCESSOR "x86_64")
set(triple "x86_64-unknown-freebsd11")
else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
@@ -39,50 +50,53 @@ if(TARGET_ARCH_NAME STREQUAL "armel")
endif()
endif()
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
if("$ENV{__DistroRid}" MATCHES "android.*")
if(TARGET_ARCH_NAME STREQUAL "arm")
set(ANDROID_ABI armeabi-v7a)
elseif(TARGET_ARCH_NAME STREQUAL "arm64")
set(ANDROID_ABI arm64-v8a)
endif()
endforeach()
endmacro()
# extract platform number required by the NDK's toolchain
string(REGEX REPLACE ".*\\.([0-9]+)-.*" "\\1" ANDROID_PLATFORM "$ENV{__DistroRid}")
set(ANDROID_TOOLCHAIN clang)
set(FEATURE_EVENT_TRACE 0) # disable event trace as there is no lttng-ust package in termux repository
set(CMAKE_SYSTEM_LIBRARY_PATH "${CROSS_ROOTFS}/usr/lib")
set(CMAKE_SYSTEM_INCLUDE_PATH "${CROSS_ROOTFS}/usr/include")
# include official NDK toolchain script
include(${CROSS_ROOTFS}/../build/cmake/android.toolchain.cmake)
elseif (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
# we cross-compile by instructing clang
set(CMAKE_C_COMPILER_TARGET ${triple})
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
else()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
endif()
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_param(CROSS_LINK_FLAGS "-m32")
add_link_options(-m32)
endif()
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$" AND NOT "$ENV{__DistroRid}" MATCHES "android.*")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
@@ -90,7 +104,17 @@ endif()
if(TARGET_ARCH_NAME MATCHES "^(arm|armel)$")
add_compile_options(-mthumb)
add_compile_options(-mfpu=vfpv3)
if (NOT DEFINED CLR_ARM_FPU_TYPE)
set (CLR_ARM_FPU_TYPE vfpv3)
endif (NOT DEFINED CLR_ARM_FPU_TYPE)
add_compile_options (-mfpu=${CLR_ARM_FPU_TYPE})
if (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
set (CLR_ARM_FPU_CAPABILITY 0x7)
endif (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
add_definitions (-DCLR_ARM_FPU_CAPABILITY=${CLR_ARM_FPU_CAPABILITY})
if(TARGET_ARCH_NAME STREQUAL "armel")
add_compile_options(-mfloat-abi=softfp)
if(DEFINED TIZEN_TOOLCHAIN)
@@ -103,7 +127,7 @@ elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths
# Set LLDB include and library paths for builds that need lldb.
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
@@ -131,7 +155,7 @@ if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+22 -8
View File
@@ -1,13 +1,14 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16",
$verbosity = "m"
$versionEndpoint = 'https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16',
$verbosity = 'minimal',
$toolpath = $null
)
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
function InstallDarcCli ($darcVersion, $toolpath) {
$darcCliPackageName = 'microsoft.dotnet.darc'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
@@ -23,11 +24,24 @@ function InstallDarcCli ($darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
if (-not $toolpath) {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity --tool-path '$toolpath'"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
}
InstallDarcCli $darcVersion
try {
InstallDarcCli $darcVersion $toolpath
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Darc' -Message $_
ExitWithExitCode 1
}
+21 -7
View File
@@ -2,8 +2,8 @@
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
verbosity=m
versionEndpoint='https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16'
verbosity='minimal'
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
@@ -20,6 +20,10 @@ while [[ $# > 0 ]]; do
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
@@ -52,17 +56,27 @@ function InstallDarcCli {
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
fi
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
}
InstallDarcCli
+12 -11
View File
@@ -1,26 +1,27 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet"
[string] $verbosity = 'minimal',
[string] $architecture = '',
[string] $version = 'Latest',
[string] $runtime = 'dotnet',
[string] $RuntimeSourceFeed = '',
[string] $RuntimeSourceFeedKey = ''
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
if ($architecture -and $architecture.Trim() -eq 'x86') {
$installdir = Join-Path $installdir 'x86'
}
InstallDotNet $installdir $version $architecture $runtime $true
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+45 -5
View File
@@ -11,9 +11,13 @@ while [[ -h "$source" ]]; do
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
@@ -29,20 +33,56 @@ while [[ $# > 0 ]]; do
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
echo "Invalid argument: $1"
usage
Write-PipelineTelemetryError -Category 'Build' -Message "Invalid argument: $1"
exit 1
;;
esac
shift
done
. "$scriptroot/tools.sh"
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv*l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
dotnetRoot="$repo_root/.dotnet"
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
+9 -2
View File
@@ -2,5 +2,12 @@ param(
[string] $token
)
Write-Host "##vso[task.setvariable variable=VSS_NUGET_ACCESSTOKEN]$token"
Write-Host "##vso[task.setvariable variable=VSS_NUGET_URI_PREFIXES]https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/"
. $PSScriptRoot\pipeline-logging-functions.ps1
# Write-PipelineSetVariable will no-op if a variable named $ci is not defined
# Since this script is only ever called in AzDO builds, just universally set it
$ci = $true
Write-PipelineSetVariable -Name 'VSS_NUGET_ACCESSTOKEN' -Value $token -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'VSS_NUGET_URI_PREFIXES' -Value 'https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/' -IsMultiJobVariable $false
+25 -26
View File
@@ -3,39 +3,39 @@ Param(
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $darcVersion, # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
Write-PipelineTelemetryError -Category 'Arcade' -Message "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
$ErrorActionPreference = 'Stop'
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
Write-Host 'Installing darc...'
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
CheckExitCode 'Running darc-init'
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$engCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
$nativeToolBaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
$installBin = Join-Path $graphvizInstallDir 'bin'
Write-Host "Installing dot..."
Write-Host 'Installing dot...'
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
@@ -51,37 +51,36 @@ try {
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = @( "--github-pat", "$gitHubPat", "--azdev-pat", "$azdoPat", "--password", "$barToken" )
$baseOptions = @( '--github-pat', "$gitHubPat", '--azdev-pat', "$azdoPat", '--password', "$barToken" )
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += @( "--include-toolset" )
Write-Host 'Toolsets will be included in the graph...'
$baseOptions += @( '--include-toolset' )
}
Write-Host "Generating standard dependency graph..."
Write-Host 'Generating standard dependency graph...'
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
CheckExitCode "Generating normal dependency graph"
CheckExitCode 'Generating normal dependency graph'
Write-Host "Generating flat dependency graph and graphviz file..."
Write-Host 'Generating flat dependency graph and graphviz file...'
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
CheckExitCode "Generating flat and graphviz dependency graph"
CheckExitCode 'Generating flat and graphviz dependency graph'
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
CheckExitCode "Generating graphviz image"
CheckExitCode 'Generating graphviz image'
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
Write-Host 'This might be a toolset repo which includes only toolset dependencies. ' -NoNewline -ForegroundColor Yellow
Write-Host 'Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again...' -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
} finally {
Pop-Location
Pop-Location
}
+20 -16
View File
@@ -35,7 +35,7 @@ File path to global.json file
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $BaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external',
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
@@ -45,26 +45,27 @@ Param (
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName 'global.json'
}
Set-StrictMode -version 2.0
$ErrorActionPreference="Stop"
$ErrorActionPreference='Stop'
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
. $PSScriptRoot\pipeline-logging-functions.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$Verbose = $VerbosePreference -Eq 'Continue'
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$EngCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
$InstallBin = Join-Path $NativeBaseDir 'bin'
$InstallerPath = Join-Path $EngCommonBaseDir 'install-tool.ps1'
# Process tools list
Write-Host "Processing $GlobalJsonFile"
@@ -74,7 +75,7 @@ try {
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
Select-Object -Expand 'native-tools' -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
@@ -112,18 +113,21 @@ try {
}
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host 'Native tools bootstrap failed'
exit 1
}
}
else {
Write-Host "No native tools defined in global.json"
Write-Host 'No native tools defined in global.json'
exit 0
}
@@ -131,18 +135,18 @@ try {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host 'Native tools are available from ' (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message 'Native tools install directory does not exist, installation failed'
exit 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message $_
ExitWithExitCode 1
}
+41 -9
View File
@@ -12,6 +12,7 @@ retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/pipeline-logging-functions.sh
. $scriptroot/native/common-library.sh
while (($# > 0)); do
@@ -33,6 +34,14 @@ while (($# > 0)); do
force=true
shift 1
;;
--donotabortonfailure)
donotabortonfailure=true
shift 1
;;
--donotdisplaywarnings)
donotdisplaywarnings=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
@@ -51,6 +60,8 @@ while (($# > 0)); do
echo " - (default) %USERPROFILE%/.netcoreeng/native"
echo ""
echo " --clean Switch specifying not to install anything, but cleanup native asset folders"
echo " --donotabortonfailure Switch specifiying whether to abort native tools installation on failure"
echo " --donotdisplaywarnings Switch specifiying whether to display warnings during native tools installation on failure"
echo " --force Clean and then install tools"
echo " --help Print help and exit"
echo ""
@@ -91,6 +102,7 @@ if [[ -z $install_directory ]]; then
fi
install_bin="${native_base_dir}/bin"
installed_any=false
ReadGlobalJsonNativeTools
@@ -102,8 +114,8 @@ else
for tool in "${!native_assets[@]}"
do
tool_version=${native_assets[$tool]}
installer_name="install-$tool.sh"
installer_command="$native_installer_dir/$installer_name"
installer_path="$native_installer_dir/install-$tool.sh"
installer_command="$installer_path"
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
@@ -117,11 +129,29 @@ else
installer_command+=" --clean"
fi
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
exit 1
if [[ -a $installer_path ]]; then
$installer_command
if [[ $? != 0 ]]; then
if [[ $donotabortonfailure = true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
exit 1
fi
else
$installed_any = true
fi
else
if [[ $donotabortonfailure == true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
exit 1
fi
fi
done
fi
@@ -134,8 +164,10 @@ if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
echo "Native tools install directory does not exist, installation failed" >&2
exit 1
if [[ $installed_any = true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Native tools install directory does not exist, installation failed"
exit 1
fi
fi
exit 0
+13 -15
View File
@@ -6,9 +6,8 @@ param(
[switch] $IsFeedPrivate
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
@@ -21,7 +20,7 @@ function SetupCredProvider {
)
# Install the Cred Provider NuGet plugin
Write-Host "Setting up Cred Provider NuGet plugin in the agent..."
Write-Host 'Setting up Cred Provider NuGet plugin in the agent...'
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
@@ -29,18 +28,18 @@ function SetupCredProvider {
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host "Installing plugin..."
Write-Host 'Installing plugin...'
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-Host "CredProvider plugin was not installed correctly!"
Write-PipelineTelemetryError -Category 'Arcade' -Message 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
}
else {
Write-Host "CredProvider plugin was installed correctly!"
Write-Host 'CredProvider plugin was installed correctly!'
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
@@ -49,7 +48,7 @@ function SetupCredProvider {
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-Host "NuGet.config file not found in repo's root!"
Write-PipelineTelemetryError -Category 'Build' -Message 'NuGet.config file not found in repo root!'
ExitWithExitCode 1
}
@@ -81,7 +80,7 @@ function SetupCredProvider {
}
else
{
Write-Host "No internal endpoints found in NuGet.config"
Write-Host 'No internal endpoints found in NuGet.config'
}
}
@@ -99,7 +98,7 @@ function InstallDotNetSdkAndRestoreArcade {
& $dotnet restore $restoreProjPath
Write-Host "Arcade SDK restored!"
Write-Host 'Arcade SDK restored!'
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
@@ -113,23 +112,22 @@ function InstallDotNetSdkAndRestoreArcade {
try {
Push-Location $PSScriptRoot
if ($Operation -like "setup") {
if ($Operation -like 'setup') {
SetupCredProvider $AuthToken
}
elseif ($Operation -like "install-restore") {
elseif ($Operation -like 'install-restore') {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-Host "Unknown operation '$Operation'!"
Write-PipelineTelemetryError -Category 'Arcade' -Message "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
}
finally {
Pop-Location
Pop-Location
}
+2 -2
View File
@@ -30,7 +30,7 @@ function SetupCredProvider {
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
echo "CredProvider plugin was not installed correctly!"
Write-PipelineTelemetryError -category 'Build' 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
@@ -42,7 +42,7 @@ function SetupCredProvider {
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
echo "NuGet.config file not found in repo's root!"
Write-PipelineTelemetryError -category 'Build' "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
+2 -3
View File
@@ -1,6 +1,6 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
@@ -18,9 +18,8 @@ try {
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+5 -5
View File
@@ -34,7 +34,7 @@ function ExpandZip {
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
echo Unable to remove '$output_directory'>&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to remove '$output_directory'"
return 1
fi
fi
@@ -45,7 +45,7 @@ function ExpandZip {
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
echo "Unable to extract '$zip_path'" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to extract '$zip_path'"
return 1
fi
@@ -117,7 +117,7 @@ function DownloadAndExtract {
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to download '$uri' to '$temp_tool_path'."
return 1
fi
@@ -125,7 +125,7 @@ function DownloadAndExtract {
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to extract '$temp_tool_path' to '$installDir'."
return 1
fi
@@ -148,7 +148,7 @@ function NewScriptShim {
fi
if [[ ! -f $tool_file_path ]]; then
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Specified tool file path:'$tool_file_path' does not exist"
return 1
fi
+2 -2
View File
@@ -101,7 +101,7 @@ fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
@@ -110,7 +110,7 @@ fi
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
+2 -2
View File
@@ -101,7 +101,7 @@ fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
@@ -110,7 +110,7 @@ fi
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
+7 -5
View File
@@ -46,6 +46,8 @@ Param (
[int] $RetryWaitTimeInSeconds = 30
)
. $PSScriptRoot\..\pipeline-logging-functions.ps1
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
@@ -93,7 +95,7 @@ try {
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-Error "Installation failed"
Write-PipelineTelemetryError "Installation failed" -Category "NativeToolsetBootstrapping"
exit 1
}
}
@@ -103,7 +105,7 @@ try {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Error "$ToolName was not found in $ToolFilePath."
Write-Host "$ToolName was not found in $ToolFilePath."
exit 1
}
@@ -117,14 +119,14 @@ try {
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-Error "Generate shim failed"
Write-PipelineTelemetryError "Generate shim failed" -Category "NativeToolsetBootstrapping"
return 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "NativeToolsetBootstrapping" -Message $_
exit 1
}
+20 -1
View File
@@ -6,7 +6,7 @@
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd</HelixPreCommands>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd;set PYTHONPATH=%HELIX_WORKITEM_PAYLOAD%\scripts%3B%HELIX_WORKITEM_PAYLOAD%</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
@@ -99,4 +99,23 @@
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<HelixWorkItem Include="Crossgen System.Private.Xml.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Private.Xml.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen System.Linq.Expressions.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Linq.Expressions.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.VisualBasic.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.VisualBasic.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.CSharp.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.CSharp.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
</ItemGroup>
</Project>
+11 -3
View File
@@ -9,12 +9,12 @@ Param(
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="coreclr corefx",
[string] $RunCategories="Libraries Runtime",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode"
[string] $Configurations="CompilationMode=$CompilationMode RunKind=$Kind"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
@@ -49,9 +49,17 @@ if ($Internal) {
$HelixSourcePrefix = "official"
}
$CommonSetupArguments="--frameworks $Framework --queue $Queue --build-number $BuildNumber --build-configs $Configurations"
# FIX ME: This is a workaround until we get this from the actual pipeline
$CommonSetupArguments="--channel master --queue $Queue --build-number $BuildNumber --build-configs $Configurations --architecture $Architecture"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
#This grabs the LKG version number of dotnet and passes it to our scripts
$VersionJSON = Get-Content global.json | ConvertFrom-Json
$DotNetVersion = $VersionJSON.tools.dotnet
$SetupArguments = "--dotnet-versions $DotNetVersion $SetupArguments"
if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"
+10 -3
View File
@@ -13,9 +13,9 @@ build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="coreclr corefx"
run_categories="Libraries Runtime"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations=
configurations="CompliationMode=$compilation_mode RunKind=$kind"
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
@@ -164,9 +164,16 @@ if [[ "$internal" == true ]]; then
fi
fi
common_setup_arguments="--frameworks $framework --queue $queue --build-number $build_number --build-configs $configurations"
common_setup_arguments="--channel master --queue $queue --build-number $build_number --build-configs $configurations --architecture $architecture"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
# Get the tools section from the global.json.
# This grabs the LKG version number of dotnet and passes it to our scripts
dotnet_version=`cat global.json | python3 -c 'import json,sys;obj=json.load(sys.stdin);print(obj["tools"]["dotnet"])'`
setup_arguments="--dotnet-versions $dotnet_version $setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
payload_directory=
workitem_directory=$source_directory
+39 -31
View File
@@ -12,6 +12,7 @@ $script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
@@ -25,49 +26,55 @@ function Write-PipelineTelemetryError {
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
[switch]$AsOutput,
[switch]$Force)
$PSBoundParameters.Remove("Category") | Out-Null
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
$PSBoundParameters.Remove("Message") | Out-Null
$PSBoundParameters.Add("Message", $Message)
$PSBoundParameters.Remove('Category') | Out-Null
if($Force -Or ((Test-Path variable:ci) -And $ci)) {
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
}
$PSBoundParameters.Remove('Message') | Out-Null
$PSBoundParameters.Add('Message', $Message)
Write-PipelineTaskError @PSBoundParameters
}
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
if(!$ci) {
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force
)
if(!$Force -And (-Not (Test-Path variable:ci) -Or !$ci)) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
if(-not $PSBoundParameters.ContainsKey('Type')) {
}
$PSBoundParameters.Remove('Force') | Out-Null
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
@@ -80,7 +87,7 @@ function Write-PipelineTaskError {
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if($ci) {
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
@@ -95,7 +102,8 @@ function Write-PipelineTaskError {
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if($ci) {
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
@@ -231,4 +239,4 @@ function Write-LogIssue {
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
}
+9 -2
View File
@@ -2,6 +2,7 @@
function Write-PipelineTelemetryError {
local telemetry_category=''
local force=false
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
@@ -11,6 +12,9 @@ function Write-PipelineTelemetryError {
telemetry_category=$2
shift
;;
-force|-f)
force=true
;;
-*)
function_args+=("$1 $2")
shift
@@ -22,19 +26,22 @@ function Write-PipelineTelemetryError {
shift
done
if [[ "$ci" != true ]]; then
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
if [[ $force == true ]]; then
function_args+=("-force")
fi
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ "$ci" != true ]]; then
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
@@ -2,26 +2,26 @@ param(
[Parameter(Mandatory=$true)][int] $BuildId,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
. $PSScriptRoot\post-build-utils.ps1
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that the channel we are going to promote the build to exist
$channelInfo = Get-MaestroChannel -ChannelId $ChannelId
if (!$channelInfo) {
Write-Host "Channel with BAR ID $ChannelId was not found in BAR!"
Write-PipelineTelemetryCategory -Category 'PromoteBuild' -Message "Channel with BAR ID $ChannelId was not found in BAR!"
ExitWithExitCode 1
}
# Get info about which channels the build has already been promoted to
# Get info about which channel(s) the build has already been promoted to
$buildInfo = Get-MaestroBuild -BuildId $BuildId
if (!$buildInfo) {
Write-Host "Build with BAR ID $BuildId was not found in BAR!"
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "Build with BAR ID $BuildId was not found in BAR!"
ExitWithExitCode 1
}
@@ -39,10 +39,10 @@ try {
Assign-BuildToChannel -BuildId $BuildId -ChannelId $ChannelId
Write-Host "done."
Write-Host 'done.'
}
catch {
Write-Host "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
Write-Host $_
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
ExitWithExitCode 1
}
@@ -0,0 +1,25 @@
param(
[Parameter(Mandatory=$true)][string] $PromoteToChannels, # List of channels that the build should be promoted to
[Parameter(Mandatory=$true)][array] $AvailableChannelIds # List of channel IDs available in the YAML implementation
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that every channel that Maestro told to promote the build to
# is available in YAML
$PromoteToChannelsIds = $PromoteToChannels -split "\D" | Where-Object { $_ }
foreach ($id in $PromoteToChannelsIds) {
if (($id -ne 0) -and ($id -notin $AvailableChannelIds)) {
Write-PipelineTaskError -Type 'warning' -Message "Channel $id is not present in the post-build YAML configuration!"
}
}
Write-Host 'done.'
}
catch {
Write-Host $_
Write-PipelineTelemetryError -Category 'CheckChannelConsistency' -Message "There was an error while trying to check consistency of Maestro default channels for the build and post-build YAML configuration."
ExitWithExitCode 1
}
@@ -1,45 +0,0 @@
param(
[Parameter(Mandatory=$true)][int] $BarBuildId, # ID of the build which assets should be downloaded
[Parameter(Mandatory=$true)][string] $DropLocation, # Where the assets should be downloaded to
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken, # Token used to access Maestro API
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com", # Maestro API URL
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16" # Version of Maestro API to use
)
. $PSScriptRoot\post-build-utils.ps1
try {
Write-Host "Installing DARC ..."
. $PSScriptRoot\..\darc-init.ps1
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-PipelineTaskError "Something failed while running 'darc-init.ps1'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
# For now, only use a dry run.
# Ideally we would change darc to enable a quick request that
# would check whether the file exists that you can download it,
# and that it won't conflict with other files.
# https://github.com/dotnet/arcade/issues/3674
# Right now we can't remove continue-on-error because we ocassionally will have
# dependencies that have no associated builds (e.g. an old dependency).
# We need to add an option to baseline specific dependencies away, or add them manually
# to the BAR.
darc gather-drop --non-shipping `
--dry-run `
--continue-on-error `
--id $BarBuildId `
--output-dir $DropLocation `
--bar-uri $MaestroApiEndpoint `
--password $MaestroApiAccessToken `
--latest-location
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
+5 -6
View File
@@ -6,20 +6,19 @@ param(
[Parameter(Mandatory=$true)][string] $ToolDestinationPath # Where the validation tool should be downloaded to
)
. $PSScriptRoot\post-build-utils.ps1
try {
$url = "https://raw.githubusercontent.com/NuGet/NuGetGallery/jver-verify/src/VerifyMicrosoftPackage/verify.ps1"
. $PSScriptRoot\post-build-utils.ps1
New-Item -ItemType "directory" -Path ${ToolDestinationPath} -Force
$url = 'https://raw.githubusercontent.com/NuGet/NuGetGallery/3e25ad135146676bcab0050a516939d9958bfa5d/src/VerifyMicrosoftPackage/verify.ps1'
New-Item -ItemType 'directory' -Path ${ToolDestinationPath} -Force
Invoke-WebRequest $url -OutFile ${ToolDestinationPath}\verify.ps1
& ${ToolDestinationPath}\verify.ps1 ${PackagesPath}\*.nupkg
}
catch {
Write-PipelineTaskError "NuGet package validation failed. Please check error logs."
Write-Host $_
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NuGetValidation' -Message $_
ExitWithExitCode 1
}
+16 -15
View File
@@ -1,16 +1,17 @@
# Most of the functions in this file require the variables `MaestroApiEndPoint`,
# `MaestroApiVersion` and `MaestroApiAccessToken` to be globally available.
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
. $PSScriptRoot\..\tools.ps1
function Create-MaestroApiRequestHeaders([string]$ContentType = "application/json") {
function Create-MaestroApiRequestHeaders([string]$ContentType = 'application/json') {
Validate-MaestroVars
$headers = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
@@ -50,14 +51,6 @@ function Get-MaestroSubscriptions([string]$SourceRepository, [int]$ChannelId) {
return $result
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Validate-MaestroVars
@@ -66,24 +59,32 @@ function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Invoke-WebRequest -Method Post -Uri $apiEndpoint -Headers $apiHeaders | Out-Null
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Validate-MaestroVars {
try {
Get-Variable MaestroApiEndPoint -Scope Global | Out-Null
Get-Variable MaestroApiVersion -Scope Global | Out-Null
Get-Variable MaestroApiAccessToken -Scope Global | Out-Null
if (!($MaestroApiEndPoint -Match "^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$")) {
Write-PipelineTaskError "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
if (!($MaestroApiEndPoint -Match '^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
ExitWithExitCode 1
}
if (!($MaestroApiVersion -Match "^[0-9]{4}-[0-9]{2}-[0-9]{2}$")) {
Write-PipelineTaskError "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
if (!($MaestroApiVersion -Match '^[0-9]{4}-[0-9]{2}-[0-9]{2}$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
ExitWithExitCode 1
}
}
catch {
Write-PipelineTaskError "Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script."
Write-PipelineTelemetryError -Category 'MaestroVars' -Message 'Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script.'
Write-Host $_
ExitWithExitCode 1
}
@@ -1,26 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $ReleaseConfigsPath # Full path to ReleaseConfigs.txt asset
)
. $PSScriptRoot\post-build-utils.ps1
try {
$Content = Get-Content $ReleaseConfigsPath
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
Write-PipelineSetVariable -Name 'BARBuildId' -Value $BarId
Write-PipelineSetVariable -Name 'InitialChannels' -Value "$Channels"
Write-PipelineSetVariable -Name 'IsStableBuild' -Value $IsStableBuild
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
+21 -21
View File
@@ -34,9 +34,9 @@ $ValidatePackage = {
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
Write-Host -NoNewLine 'Validating ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
@@ -58,7 +58,7 @@ $ValidatePackage = {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
if ($FileName.EndsWith('.resources.dll')) {
return
}
@@ -96,7 +96,7 @@ $ValidatePackage = {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match "github" -or $Uri.Host -match "githubusercontent")) {
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match 'github' -or $Uri.Host -match 'githubusercontent')) {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
@@ -143,19 +143,19 @@ $ValidatePackage = {
}
if ($FailedFiles -eq 0) {
Write-Host "Passed."
Write-Host 'Passed.'
return 0
}
else {
Write-Host "$PackagePath has broken SourceLink links."
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$PackagePath has broken SourceLink links."
return 1
}
}
function ValidateSourceLinkLinks {
if ($GHRepoName -ne "" -and !($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
if (!($GHRepoName -Match "^[^\s-]+-[^\s]+$")) {
Write-PipelineTaskError "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
if ($GHRepoName -ne '' -and !($GHRepoName -Match '^[^\s\/]+/[^\s\/]+$')) {
if (!($GHRepoName -Match '^[^\s-]+-[^\s]+$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
ExitWithExitCode 1
}
else {
@@ -163,14 +163,14 @@ function ValidateSourceLinkLinks {
}
}
if ($GHCommit -ne "" -and !($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-PipelineTaskError "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
if ($GHCommit -ne '' -and !($GHCommit -Match '^[0-9a-fA-F]{40}$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
ExitWithExitCode 1
}
if ($GHRepoName -ne "" -and $GHCommit -ne "") {
$RepoTreeURL = -Join("http://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
if ($GHRepoName -ne '' -and $GHCommit -ne '') {
$RepoTreeURL = -Join('http://api.github.com/repos/', $GHRepoName, '/git/trees/', $GHCommit, '?recursive=1')
$CodeExtensions = @('.cs', '.vb', '.fs', '.fsi', '.fsx', '.fsscript')
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
@@ -188,8 +188,8 @@ function ValidateSourceLinkLinks {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL . Execution will proceed without caching."
}
}
elseif ($GHRepoName -ne "" -or $GHCommit -ne "") {
Write-Host "For using the http caching mechanism both GHRepoName and GHCommit should be informed."
elseif ($GHRepoName -ne '' -or $GHCommit -ne '') {
Write-Host 'For using the http caching mechanism both GHRepoName and GHCommit should be informed.'
}
if (Test-Path $ExtractPath) {
@@ -217,18 +217,18 @@ function ValidateSourceLinkLinks {
$ValidationFailures = 0
foreach ($Job in @(Get-Job)) {
$jobResult = Wait-Job -Id $Job.Id | Receive-Job
if ($jobResult -ne "0") {
if ($jobResult -ne '0') {
$ValidationFailures++
}
}
if ($ValidationFailures -gt 0) {
Write-PipelineTaskError " $ValidationFailures package(s) failed validation."
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$ValidationFailures package(s) failed validation."
ExitWithExitCode 1
}
}
function InstallSourcelinkCli {
$sourcelinkCliPackageName = "sourcelink"
$sourcelinkCliPackageName = 'sourcelink'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
@@ -239,7 +239,7 @@ function InstallSourcelinkCli {
}
else {
Write-Host "Installing SourceLink CLI version $sourcelinkCliVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $sourcelinkCliPackageName --version $sourcelinkCliVersion --verbosity "minimal" --global
}
}
@@ -250,8 +250,8 @@ try {
ValidateSourceLinkLinks
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'SourceLink' -Message $_
ExitWithExitCode 1
}
+35 -36
View File
@@ -4,10 +4,6 @@ param(
[Parameter(Mandatory=$true)][string] $DotnetSymbolVersion # Version of dotnet symbol to use
)
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
@@ -23,19 +19,19 @@ function FirstMatchingSymbolDescriptionOrDefault {
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
$SymbolPath = $SymbolsPath + '\' + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
$PdbPath = $SymbolPath.Replace($Extension, '.pdb')
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
$NGenPdb = $SymbolPath.Replace($Extension, '.ni.pdb')
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
$SODbg = $SymbolPath.Replace($Extension, '.so.dbg')
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$DylibDwarf = $SymbolPath.Replace($Extension, '.dylib.dwarf')
$dotnetSymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetSymbolExe = Resolve-Path "$dotnetSymbolExe\dotnet-symbol.exe"
@@ -43,19 +39,19 @@ function FirstMatchingSymbolDescriptionOrDefault {
& $dotnetSymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
return 'PDB'
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
return 'NGen PDB'
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
return 'DBG for SO'
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
return 'Dwarf for Dylib'
}
elseif (Test-Path $SymbolPath) {
return "Module"
return 'Module'
}
else {
return $null
@@ -74,7 +70,7 @@ function CountMissingSymbols {
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
$RelevantExtensions = @('.dll', '.exe', '.so', '.dylib')
# How many files are missing symbol information
$MissingSymbols = 0
@@ -82,38 +78,38 @@ function CountMissingSymbols {
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath 'Symbols'
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
if ($_.FullName -Match '\\ref\\') {
Write-Host "`t Ignoring reference assembly file " $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--microsoft-symbol-server' $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--internal-server' $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
Write-Host -NoNewLine "`t Checking file " $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
Write-Host 'No symbols found on MSDL or SymWeb!'
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
Write-Host 'No symbols found on MSDL!'
}
else {
Write-Host "No symbols found on SymWeb!"
Write-Host 'No symbols found on SymWeb!'
}
}
}
@@ -132,27 +128,27 @@ function CheckSymbolsAvailable {
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
if ($FileName -Match 'Microsoft\.DotNet\.Darc\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
elseif ($FileName -Match 'Microsoft\.DotNet\.Maestro\.Tasks\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-PipelineTaskError "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
}
Write-Host
@@ -160,7 +156,7 @@ function CheckSymbolsAvailable {
}
function InstallDotnetSymbol {
$dotnetSymbolPackageName = "dotnet-symbol"
$dotnetSymbolPackageName = 'dotnet-symbol'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
@@ -171,19 +167,22 @@ function InstallDotnetSymbol {
}
else {
Write-Host "Installing dotnet-symbol version $dotnetSymbolVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $dotnetSymbolPackageName --version $dotnetSymbolVersion --verbosity "minimal" --global
}
}
try {
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
InstallDotnetSymbol
CheckSymbolsAvailable
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message $_
ExitWithExitCode 1
}
+50 -43
View File
@@ -2,56 +2,63 @@ param(
[Parameter(Mandatory=$true)][string] $SourceRepo,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
. $PSScriptRoot\post-build-utils.ps1
try {
. $PSScriptRoot\post-build-utils.ps1
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
if (!$subscriptions) {
Write-Host "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
if (!$subscriptions) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: $subscription.id"
[void]$subscriptionsToTrigger.Add($subscription.id)
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: ${$subscription.id}"
[void]$subscriptionsToTrigger.Add($subscription.id)
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host 'done.'
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message 'At least one subscription failed to be triggered...'
ExitWithExitCode 1
}
else {
Write-Host 'All subscriptions were triggered successfully!'
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host "done."
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-Host "At least one subscription failed to be triggered..."
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message $_
ExitWithExitCode 1
}
else {
Write-Host "All subscriptions were triggered successfully!"
}
+9 -10
View File
@@ -1,8 +1,8 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $configuration = "Debug",
[string] $configuration = 'Debug',
[string] $task,
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[string] $msbuildEngine = $null,
[switch] $restore,
[switch] $prepareMachine,
@@ -32,7 +32,7 @@ function Print-Usage() {
}
function Build([string]$target) {
$logSuffix = if ($target -eq "Execute") { "" } else { ".$target" }
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
@@ -46,33 +46,32 @@ function Build([string]$target) {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
if ($task -eq "") {
Write-Host "Missing required parameter '-task <value>'" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Missing required parameter '-task <value>'" -ForegroundColor Red
Print-Usage
ExitWithExitCode 1
}
$taskProject = GetSdkTaskProject $task
if (!(Test-Path $taskProject)) {
Write-Host "Unknown task: $task" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Unknown task: $task" -ForegroundColor Red
ExitWithExitCode 1
}
if ($restore) {
Build "Restore"
Build 'Restore'
}
Build "Execute"
Build 'Execute'
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+83 -69
View File
@@ -5,7 +5,7 @@ Param(
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ("artifacts")), # Required: the directory where build artifacts are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ('artifacts')), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
@@ -22,79 +22,93 @@ Param(
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel="Standard", # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string] $GuardianLoggerLevel='Standard', # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
try {
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path 'tools' 'guardian.cmd'))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Invalid Guardian CLI Location.'
ExitWithExitCode 1
}
& $(Join-Path $PSScriptRoot 'init-sdl.ps1') -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory '.gdn'
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not onboard to TSA -- not all required values ($TsaCodebaseName, $TsaNotificationEmail, $TsaCodebaseAdmin, $TsaBugAreaPath) were specified.'
ExitWithExitCode 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Update baseline'
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-publish failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not publish to TSA -- not all required values ($TsaBranchName, $BuildNumber) were specified.'
ExitWithExitCode 1
}
}
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path "tools" "guardian.cmd"))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
exit 1
}
& $(Join-Path $PSScriptRoot "init-sdl.ps1") -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory ".gdn"
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not onboard to TSA -- not all required values ($$TsaCodebaseName, $$TsaNotificationEmail, $$TsaCodebaseAdmin, $$TsaBugAreaPath) were specified."
exit 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Update baseline"
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-publish failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not publish to TSA -- not all required values ($$TsaBranchName, $$BuildNumber) were specified."
exit 1
}
}
+50 -47
View File
@@ -3,54 +3,12 @@ param(
[Parameter(Mandatory=$true)][string] $ExtractPath # Full path to directory where the packages will be extracted
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
$disableConfigureToolsetImport = $true
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Extracting" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
}
finally {
$zip.Dispose()
}
}
function ExtractArtifacts {
function ExtractArtifacts {
if (!(Test-Path $InputPath)) {
Write-Host "Input Path does not exist: $InputPath"
ExitWithExitCode 0
@@ -67,11 +25,56 @@ $ExtractPackage = {
}
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Extracting ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
finally {
$zip.Dispose()
}
}
Measure-Command { ExtractArtifacts }
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+29 -13
View File
@@ -1,16 +1,23 @@
Param(
[string] $GuardianCliLocation,
[string] $Repository,
[string] $BranchName="master",
[string] $BranchName='master',
[string] $WorkingDirectory,
[string] $AzureDevOpsAccessToken,
[string] $GuardianLoggerLevel="Standard"
[string] $GuardianLoggerLevel='Standard'
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
@@ -21,11 +28,10 @@ $uri = "https://dev.azure.com/dnceng/internal/_apis/git/repositories/sdl-tool-cf
$zipFile = "$WorkingDirectory/gdn.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
$gdnFolder = (Join-Path $WorkingDirectory ".gdn")
Try
{
$gdnFolder = (Join-Path $WorkingDirectory '.gdn')
try {
# We try to download the zip; if the request fails (e.g. the file doesn't exist), we catch it and init guardian instead
Write-Host "Downloading gdn folder from internal config repostiory..."
Write-Host 'Downloading gdn folder from internal config repostiory...'
Invoke-WebRequest -Headers @{ "Accept"="application/zip"; "Authorization"="Basic $encodedPat" } -Uri $uri -OutFile $zipFile
if (Test-Path $gdnFolder) {
# Remove the gdn folder if it exists (it shouldn't unless there's too much caching; this is just in case)
@@ -33,19 +39,29 @@ Try
}
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipFile, $WorkingDirectory)
Write-Host $gdnFolder
} Catch [System.Net.WebException] {
ExitWithExitCode 0
} catch [System.Net.WebException] { } # Catch and ignore webexception
try {
# if the folder does not exist, we'll do a guardian init and push it to the remote repository
Write-Host "Initializing Guardian..."
Write-Host 'Initializing Guardian...'
Write-Host "$GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel"
& $GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian init failed with exit code $LASTEXITCODE."
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian init failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We create the mainbaseline so it can be edited later
Write-Host "$GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline"
& $GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian baseline failed with exit code $LASTEXITCODE."
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian baseline failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
& $(Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Initialize gdn folder"
}
& $(Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Initialize gdn folder'
ExitWithExitCode 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+57 -39
View File
@@ -1,51 +1,69 @@
Param(
[string] $Repository,
[string] $BranchName="master",
[string] $BranchName='master',
[string] $GdnFolder,
[string] $AzureDevOpsAccessToken,
[string] $PushReason
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP "sdl"
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-Error "Git clone failed with exit code $LASTEXITCODE."
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) ".gdn"
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host "git add ."
git add .
if ($LASTEXITCODE -ne 0) {
Write-Error "Git add failed with exit code $LASTEXITCODE."
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-Error "Git commit failed with exit code $LASTEXITCODE."
}
Write-Host "git push"
git push
if ($LASTEXITCODE -ne 0) {
Write-Error "Git push failed with exit code $LASTEXITCODE."
}
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP 'sdl'
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
# Return to the original directory
Pop-Location
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git clone failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) '.gdn'
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host 'git add .'
git add .
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git add failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git commit failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host 'git push'
git push
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git push failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# Return to the original directory
Pop-Location
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+53 -39
View File
@@ -5,55 +5,69 @@ Param(
[string] $GdnFolder,
[string[]] $ToolsList,
[string] $UpdateBaseline,
[string] $GuardianLoggerLevel="Standard",
[string] $GuardianLoggerLevel='Standard',
[string[]] $CrScanAdditionalRunConfigParams,
[string[]] $PoliCheckAdditionalRunConfigParams
)
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder "r"
$ValidPath = Test-Path $GuardianCliLocation
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
exit 1
}
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder 'r'
$ValidPath = Test-Path $GuardianCliLocation
$configParam = @("--config")
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq "credscan") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
}
if ($tool -eq "policheck") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Invalid Guardian CLI Location."
ExitWithExitCode 1
}
$configParam+=$gdnConfigFile
}
$configParam = @('--config')
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq 'credscan') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
if ($tool -eq 'policheck') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
$configParam+=$gdnConfigFile
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+37 -7
View File
@@ -1,4 +1,5 @@
parameters:
enable: 'false' # Whether the SDL validation job should execute or not
overrideParameters: '' # Optional: to override values for parameters.
additionalParameters: '' # Optional: parameters that need user specific values eg: '-SourceToolsList @("abc","def") -ArtifactToolsList @("ghi","jkl")'
# There is some sort of bug (has been reported) in Azure DevOps where if this parameter is named
@@ -6,25 +7,54 @@ parameters:
# This can also be remedied by the caller (post-build.yml) if it does not use a nested parameter
sdlContinueOnError: false # optional: determines whether to continue the build if the step errors;
dependsOn: '' # Optional: dependencies of the job
artifactNames: '' # Optional: patterns supplied to DownloadBuildArtifacts
# Usage:
# artifactNames:
# - 'BlobArtifacts'
# - 'Artifacts_Windows_NT_Release'
jobs:
- job: Run_SDL
dependsOn: ${{ parameters.dependsOn }}
displayName: Run SDL tool
condition: eq( ${{ parameters.enable }}, 'true')
variables:
- group: DotNet-VSTS-Bot
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
name: Hosted VS2017
steps:
- checkout: self
clean: true
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: current
downloadType: specific files
matchingPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if ne(parameters.artifactNames, '') }}:
- ${{ each artifactName in parameters.artifactNames }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: ${{ artifactName }}
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if eq(parameters.artifactNames, '') }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: specific files
itemPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
+4
View File
@@ -24,6 +24,8 @@ parameters:
enablePublishBuildAssets: false
enablePublishTestResults: false
enablePublishUsingPipelines: false
mergeTestResults: false
testRunTitle: $(AgentOsName)-$(BuildConfiguration)-xunit
name: ''
preSteps: []
runAsPublic: false
@@ -194,6 +196,8 @@ jobs:
testResultsFormat: 'xUnit'
testResultsFiles: '*.xml'
searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults/$(_BuildConfig)'
testRunTitle: ${{ parameters.testRunTitle }}
mergeTestResults: ${{ parameters.mergeTestResults }}
continueOnError: true
condition: always()
@@ -37,6 +37,12 @@ jobs:
- name: _BuildConfig
value: ${{ parameters.configuration }}
- group: Publish-Build-Assets
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
@@ -2,37 +2,53 @@ parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: NetCore_30_Internal_Servicing_Publishing
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Core 3.0 Internal Servicing Publishing
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.InternalServicing_30_Channel_Id))
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
displayName: Download Build Assets
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
@@ -57,39 +73,48 @@ stages:
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
/p:PublishToMSDL=false
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.InternalServicing_30_Channel_Id))
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
@@ -128,15 +153,20 @@ stages:
/p:ChecksumsAzureAccountKey=$(InternalChecksumsBlobFeedKey)
/p:InstallersTargetStaticFeed=$(InternalInstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(InternalInstallersBlobFeedKey)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:PublishToMSDL=false
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
- template: ../../steps/publish-logs.yml
parameters:
ChannelId: ${{ variables.InternalServicing_30_Channel_Id }}
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -4,36 +4,65 @@ parameters:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
# If the channel name is empty, no links will be generated
akaMSChannelName: ''
stages:
- stage: NetCore_Release31_Publish
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Core 3.1 Release Publishing
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicRelease_31_Channel_Id))
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
@@ -48,37 +77,47 @@ stages:
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicRelease_31_Channel_Id))
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
- name: ArtifactsCategory
value: ${{ coalesce(variables._DotNetArtifactsCategory, '.NETCore') }}
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
@@ -98,7 +137,7 @@ stages:
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:ArtifactsCategory=$(ArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
@@ -118,15 +157,22 @@ stages:
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:LatestLinkShortUrlPrefix=dotnet/'${{ parameters.akaMSChannelName }}'
/p:AkaMSClientId=$(akams-client-id)
/p:AkaMSClientSecret=$(akams-client-secret)
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
- template: ../../steps/publish-logs.yml
parameters:
ChannelId: ${{ variables.PublicRelease_31_Channel_Id }}
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,97 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
stages:
- stage: NetCore_3_Tools_Validation_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET 3 Tools - Validation Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NETCore_3_Tools_Validation_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetValidationArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.NETCore_3_Tools_Validation_Channel_Id }}
@@ -1,132 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stages:
- stage: NetCore_3_Tools_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET 3 Tools Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_3_Tools_Channel_Id))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_3_Tools_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.NetCore_3_Tools_Channel_Id }}
@@ -1,132 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stages:
- stage: NetCore_Dev31_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Core 3.1 Dev Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicDevRelease_31_Channel_Id))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicDevRelease_31_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.PublicDevRelease_31_Channel_Id }}
@@ -1,132 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stages:
- stage: NetCore_Dev5_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Core 5 Dev Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_5_Dev_Channel_Id))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_5_Dev_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.NetCore_5_Dev_Channel_Id }}
@@ -1,132 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stages:
- stage: NetCore_Release30_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Core 3.0 Release Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicRelease_30_Channel_Id))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.PublicRelease_30_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3-transport/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.PublicRelease_30_Channel_Id }}
@@ -1,132 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stages:
- stage: NetCore_Tools_Latest_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Tools - Latest Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job:
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_Tools_Latest_Channel_Id))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_Tools_Latest_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.NetCore_Tools_Latest_Channel_Id }}
@@ -1,97 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
stages:
- stage: PVR_Publish
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: .NET Tools - Validation Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- group: DotNet-Blob-Feed
- group: AzureDevOps-Artifact-Feeds-Pats
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', variables.NetCore_Tools_Validation_Channel_Id))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetValidationArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:PublishToAzureDevOpsNuGetFeeds=true
/p:AzureDevOpsStaticShippingFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ variables.NetCore_Tools_Validation_Channel_Id }}
@@ -1,29 +1,31 @@
variables:
- group: Publish-Build-Assets
- group: AzureDevOps-Artifact-Feeds-Pats
- group: DotNet-Blob-Feed
- group: DotNet-DotNetCli-Storage
- group: DotNet-MSRC-Storage
- group: Publish-Build-Assets
# .NET Core 3.1 Dev
- name: PublicDevRelease_31_Channel_Id
value: 128
# .NET Core 5 Dev
- name: NetCore_5_Dev_Channel_Id
# .NET 5 Dev
- name: Net_5_Dev_Channel_Id
value: 131
# .NET Tools - Validation
- name: NetCore_Tools_Validation_Channel_Id
# .NET Eng - Validation
- name: Net_Eng_Validation_Channel_Id
value: 9
# .NET Tools - Latest
- name: NetCore_Tools_Latest_Channel_Id
# .NET Eng - Latest
- name: Net_Eng_Latest_Channel_Id
value: 2
# .NET 3 Tools - Validation
- name: NETCore_3_Tools_Validation_Channel_Id
# .NET 3 Eng - Validation
- name: NET_3_Eng_Validation_Channel_Id
value: 390
# .NET 3 Tools - Latest
# .NET 3 Eng
- name: NetCore_3_Tools_Channel_Id
value: 344
@@ -39,6 +41,18 @@ variables:
- name: PublicRelease_31_Channel_Id
value: 129
# General Testing
- name: GeneralTesting_Channel_Id
value: 529
# .NET Core 3.1 Blazor Features
- name: NetCore_31_Blazor_Features_Channel_Id
value: 531
# .NET Core Experimental
- name: NetCore_Experimental_Channel_Id
value: 562
# Whether the build is internal or not
- name: IsInternalBuild
value: ${{ and(ne(variables['System.TeamProject'], 'public'), contains(variables['Build.SourceBranch'], 'internal')) }}
@@ -76,3 +90,10 @@ variables:
value: https://dotnetclimsrc.blob.core.windows.net/dotnet/index.json
- name: InternalInstallersBlobFeedKey
value: $(dotnetclimsrc-access-key)
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
@@ -1,23 +0,0 @@
parameters:
ChannelId: 0
jobs:
- job: gatherDrop
displayName: Gather Drop
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.ChannelId }}))
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Darc gather-drop
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/darc-gather-drop.ps1
arguments: -BarBuildId $(BARBuildId)
-DropLocation $(Agent.BuildDirectory)/Temp/Drop/
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
+311 -104
View File
@@ -8,6 +8,7 @@ parameters:
enable: false
continueOnError: false
params: ''
artifactNames: ''
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
@@ -21,155 +22,361 @@ parameters:
publishDependsOn:
- Validate
# Channel ID's instantiated in this file.
# When adding a new channel implementation the call to `check-channel-consistency.ps1`
# needs to be updated with the new channel ID
NetEngLatestChannelId: 2
NetEngValidationChannelId: 9
NetDev5ChannelId: 131
GeneralTestingChannelId: 529
NETCoreToolingDevChannelId: 548
NETCoreToolingReleaseChannelId: 549
NETInternalToolingChannelId: 551
NETCoreExperimentalChannelId: 562
NetEngServicesIntChannelId: 678
NetEngServicesProdChannelId: 679
Net5Preview2ChannelId: 738
Net5Preview3ChannelId: 739
NetCoreSDK313xxChannelId: 759
NetCoreSDK313xxInternalChannelId: 760
stages:
- stage: Validate
dependsOn: ${{ parameters.validateDependsOn }}
displayName: Validate
variables:
- template: common-variables.yml
jobs:
- ${{ if eq(parameters.enableNugetValidation, 'true') }}:
- job:
displayName: NuGet Validation
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- template: setup-maestro-vars.yml
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- job:
displayName: Post-build Checks
dependsOn: setupMaestroVars
variables:
- name: InitialChannels
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'] ]
- name: PromoteToMaestroChannelId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Maestro Channels Consistency
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/check-channel-consistency.ps1
arguments: -PromoteToChannels "$(InitialChannels)[$(PromoteToMaestroChannelId)]"
-AvailableChannelIds ${{parameters.NetEngLatestChannelId}},${{parameters.NetEngValidationChannelId}},${{parameters.NetDev5ChannelId}},${{parameters.GeneralTestingChannelId}},${{parameters.NETCoreToolingDevChannelId}},${{parameters.NETCoreToolingReleaseChannelId}},${{parameters.NETInternalToolingChannelId}},${{parameters.NETCoreExperimentalChannelId}},${{parameters.NetEngServicesIntChannelId}},${{parameters.NetEngServicesProdChannelId}},${{parameters.Net5Preview2ChannelId}},${{parameters.Net5Preview3ChannelId}},${{parameters.NetCoreSDK313xxChannelId}},${{parameters.NetCoreSDK313xxInternalChannelId}}
- ${{ if eq(parameters.enableSigningValidation, 'true') }}:
- job:
displayName: Signing Validation
variables:
- template: common-variables.yml
- group: AzureDevOps-Artifact-Feeds-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- job:
displayName: NuGet Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableNugetValidation }}, 'true')
pool:
vmImage: 'windows-2019'
variables:
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
condition: eq(variables['IsInternalBuild'], 'true')
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- job:
displayName: Signing Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSigningValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.signingValidationAdditionalParameters }}
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- ${{ if eq(parameters.enableSourceLinkValidation, 'true') }}:
- job:
displayName: SourceLink Validation
variables:
- template: common-variables.yml
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
${{ parameters.signingValidationAdditionalParameters }}
- ${{ if eq(parameters.SDLValidationParameters.enable, 'true') }}:
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
- template: ../steps/publish-logs.yml
parameters:
StageLabel: 'Validation'
JobLabel: 'Signing'
- template: \eng\common\templates\post-build\channels\netcore-dev-5.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
- template: \eng\common\templates\post-build\channels\netcore-dev-31.yml
- job:
displayName: SourceLink Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSourceLinkValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
enable: ${{ parameters.SDLValidationParameters.enable }}
dependsOn: setupMaestroVars
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
artifactNames: ${{ parameters.SDLValidationParameters.artifactNames }}
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev5_Publish'
channelName: '.NET 5 Dev'
akaMSChannelName: 'net5/dev'
channelId: ${{ parameters.NetDev5ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\netcore-tools-latest.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview2_Publish'
channelName: '.NET 5 Preview 2'
akaMSChannelName: 'net5/preview2'
channelId: ${{ parameters.Net5Preview2ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\netcore-tools-validation.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview3_Publish'
channelName: '.NET 5 Preview 3'
akaMSChannelName: 'net5/preview3'
channelId: ${{ parameters.Net5Preview3ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\netcore-3-tools-validation.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
- template: \eng\common\templates\post-build\channels\netcore-3-tools.yml
parameters:
dependsOn: ${{ parameters.publishDependsOn }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
- template: \eng\common\templates\post-build\channels\netcore-release-30.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Latest_Publish'
channelName: '.NET Eng - Latest'
akaMSChannelName: 'eng/daily'
channelId: ${{ parameters.NetEngLatestChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\netcore-release-31.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Validation_Publish'
channelName: '.NET Eng - Validation'
akaMSChannelName: 'eng/validation'
channelId: ${{ parameters.NetEngValidationChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\netcore-internal-30.yml
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'General_Testing_Publish'
channelName: 'General Testing'
akaMSChannelName: 'generaltesting'
channelId: ${{ parameters.GeneralTestingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Dev_Publishing'
channelName: '.NET Core Tooling Dev'
channelId: ${{ parameters.NETCoreToolingDevChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Release_Publishing'
channelName: '.NET Core Tooling Release'
channelId: ${{ parameters.NETCoreToolingReleaseChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NET_Internal_Tooling_Publishing'
channelName: '.NET Internal Tooling'
channelId: ${{ parameters.NETInternalToolingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Experimental_Publishing'
channelName: '.NET Core Experimental'
channelId: ${{ parameters.NETCoreExperimentalChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Int_Publish'
channelName: '.NET Eng Services - Int'
channelId: ${{ parameters.NetEngServicesIntChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Prod_Publish'
channelName: '.NET Eng Services - Prod'
channelId: ${{ parameters.NetEngServicesProdChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_SDK_313xx_Publishing'
channelName: '.NET Core SDK 3.1.3xx'
channelId: ${{ parameters.NetCoreSDK313xxChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_SDK_313xx_Internal_Publishing'
channelName: '.NET Core SDK 3.1.3xx Internal'
channelId: ${{ parameters.NetCoreSDK313xxInternalChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-symbols/nuget/v3/index.json'
@@ -1,25 +0,0 @@
parameters:
ChannelId: 0
jobs:
- job:
displayName: Promote Build
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.ChannelId }}))
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: ChannelId
value: ${{ parameters.ChannelId }}
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/promote-build.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId $(ChannelId)
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,11 +1,20 @@
jobs:
- job: setupMaestroVars
displayName: Setup Maestro Vars
variables:
- template: common-variables.yml
- name: BuildId
value: $[ coalesce(variables.BARBuildId, 0) ]
- name: PromoteToChannelId
value: $[ coalesce(variables.PromoteToMaestroChannelId, 0) ]
pool:
vmImage: 'windows-2019'
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
displayName: Download Release Configs
condition: eq(variables.PromoteToChannelId, 0)
inputs:
buildType: current
artifactName: ReleaseConfigs
@@ -14,5 +23,59 @@ jobs:
name: setReleaseVars
displayName: Set Release Configs Vars
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/setup-maestro-vars.ps1
arguments: -ReleaseConfigsPath '$(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt'
targetType: inline
script: |
try {
if ($Env:PromoteToChannelId -eq 0) {
$Content = Get-Content $(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
$AzureDevOpsProject = $Env:System_TeamProject
$AzureDevOpsBuildDefinitionId = $Env:System_DefinitionId
$AzureDevOpsBuildId = $Env:Build_BuildId
$PromoteToMaestroChannelId = 0
}
else {
$buildApiEndpoint = "${Env:MaestroApiEndPoint}/api/builds/${Env:BARBuildId}?api-version=${Env:MaestroApiVersion}"
$apiHeaders = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$apiHeaders.Add('Accept', 'application/json')
$apiHeaders.Add('Authorization',"Bearer ${Env:MAESTRO_API_TOKEN}")
$buildInfo = try { Invoke-WebRequest -Method Get -Uri $buildApiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
$BarId = $Env:BARBuildId
$Channels = 'None'
#TODO: Fix this once this issue is done: https://github.com/dotnet/arcade/issues/3834
$IsStableBuild = 'False'
$AzureDevOpsProject = $buildInfo.azureDevOpsProject
$AzureDevOpsBuildDefinitionId = $buildInfo.azureDevOpsBuildDefinitionId
$AzureDevOpsBuildId = $buildInfo.azureDevOpsBuildId
$PromoteToMaestroChannelId = $Env:PromoteToMaestroChannelId
}
Write-Host "##vso[task.setvariable variable=BARBuildId;isOutput=true]$BarId"
Write-Host "##vso[task.setvariable variable=InitialChannels;isOutput=true]$Channels"
Write-Host "##vso[task.setvariable variable=IsStableBuild;isOutput=true]$IsStableBuild"
Write-Host "##vso[task.setvariable variable=AzDOProjectName;isOutput=true]$AzureDevOpsProject"
Write-Host "##vso[task.setvariable variable=AzDOPipelineId;isOutput=true]$AzureDevOpsBuildDefinitionId"
Write-Host "##vso[task.setvariable variable=AzDOBuildId;isOutput=true]$AzureDevOpsBuildId"
Write-Host "##vso[task.setvariable variable=PromoteToMaestroChannelId;isOutput=true]$PromoteToMaestroChannelId"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
exit 1
}
env:
MAESTRO_API_TOKEN: $(MaestroApiAccessToken)
@@ -5,9 +5,9 @@ steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/promote-build.ps1
filePath: $(Build.SourcesDirectory)/eng/common/post-build/add-build-to-channel.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
-MaestroApiVersion $(MaestroApiVersion)
@@ -0,0 +1,23 @@
parameters:
StageLabel: ''
JobLabel: ''
steps:
- task: Powershell@2
displayName: Prepare Binlogs to Upload
inputs:
targetType: inline
script: |
New-Item -ItemType Directory $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
Move-Item -Path $(Build.SourcesDirectory)/artifacts/log/Debug/* $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
continueOnError: true
condition: always()
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/PostBuildLogs'
PublishLocation: Container
ArtifactName: PostBuildLogs
continueOnError: true
condition: always()
@@ -23,6 +23,7 @@ parameters:
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
HelixBaseUri: 'https://helix.dot.net/' # optional -- sets the Helix API base URI (allows targeting int)
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
@@ -55,6 +56,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
@@ -85,6 +87,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
+169 -75
View File
@@ -5,7 +5,7 @@
[bool]$ci = if (Test-Path variable:ci) { $ci } else { $false }
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { "Debug" }
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { 'Debug' }
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
@@ -24,7 +24,7 @@
[bool]$restore = if (Test-Path variable:restore) { $restore } else { $true }
# Adjusts msbuild verbosity level.
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { "minimal" }
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { 'minimal' }
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
[bool]$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { !$ci }
@@ -41,21 +41,23 @@
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.ps1
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { "v1" }
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { 'v1' }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @('msbuild', 'dotnet', 'vbcscompiler') }
$disableConfigureToolsetImport = if (Test-Path variable:disableConfigureToolsetImport) { $disableConfigureToolsetImport } else { $null }
set-strictmode -version 2.0
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
function Create-Directory([string[]] $path) {
if (!(Test-Path $path)) {
New-Item -path $path -force -itemType "Directory" | Out-Null
New-Item -path $path -force -itemType 'Directory' | Out-Null
}
}
@@ -96,7 +98,10 @@ function Exec-Process([string]$command, [string]$commandArgs) {
}
}
function InitializeDotNetCli([bool]$install) {
# createSdkLocationFile parameter enables a file being generated under the toolset directory
# which writes the sdk's location into. This is only necessary for cmd --> powershell invocations
# as dot sourcing isn't possible.
function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
if (Test-Path variable:global:_DotNetInstallDir) {
return $global:_DotNetInstallDir
}
@@ -119,7 +124,7 @@ function InitializeDotNetCli([bool]$install) {
# Find the first path on %PATH% that contains the dotnet.exe
if ($useInstalledDotNetCli -and (-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -eq $null)) {
$dotnetCmd = Get-Command "dotnet.exe" -ErrorAction SilentlyContinue
$dotnetCmd = Get-Command 'dotnet.exe' -ErrorAction SilentlyContinue
if ($dotnetCmd -ne $null) {
$env:DOTNET_INSTALL_DIR = Split-Path $dotnetCmd.Path -Parent
}
@@ -132,13 +137,13 @@ function InitializeDotNetCli([bool]$install) {
if ((-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -ne $null) -and (Test-Path(Join-Path $env:DOTNET_INSTALL_DIR "sdk\$dotnetSdkVersion"))) {
$dotnetRoot = $env:DOTNET_INSTALL_DIR
} else {
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
if (-not (Test-Path(Join-Path $dotnetRoot "sdk\$dotnetSdkVersion"))) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
ExitWithExitCode 1
}
}
@@ -146,6 +151,24 @@ function InitializeDotNetCli([bool]$install) {
$env:DOTNET_INSTALL_DIR = $dotnetRoot
}
# Creates a temporary file under the toolset dir.
# The following code block is protecting against concurrent access so that this function can
# be called in parallel.
if ($createSdkLocationFile) {
do {
$sdkCacheFileTemp = Join-Path $ToolsetDir $([System.IO.Path]::GetRandomFileName())
}
until (!(Test-Path $sdkCacheFileTemp))
Set-Content -Path $sdkCacheFileTemp -Value $dotnetRoot
try {
Rename-Item -Force -Path $sdkCacheFileTemp 'sdk.txt'
} catch {
# Somebody beat us
Remove-Item -Path $sdkCacheFileTemp
}
}
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
# It also ensures that VS msbuild will use the downloaded sdk targets.
@@ -161,21 +184,54 @@ function InitializeDotNetCli([bool]$install) {
}
function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
$installScript = Join-Path $dotnetRoot 'dotnet-install.ps1'
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1" -OutFile $installScript
$maxRetries = 5
$retries = 1
$uri = "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1"
while($true) {
try {
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
break
}
catch {
Write-Host "Failed to download '$uri'"
Write-Error $_.Exception.Message -ErrorAction Continue
}
if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
throw "Unable to download file in $maxRetries attempts."
}
}
}
return $installScript
}
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = "") {
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = '') {
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) {
function InstallDotNet([string] $dotnetRoot,
[string] $version,
[string] $architecture = '',
[string] $runtime = '',
[bool] $skipNonVersionedFiles = $false,
[string] $runtimeSourceFeed = '',
[string] $runtimeSourceFeedKey = '') {
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
@@ -186,10 +242,32 @@ function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $archit
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Failed to install dotnet cli (exit code '$lastExitCode')."
ExitWithExitCode $lastExitCode
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from public location."
# Only the runtime can be installed from a custom [private] location.
if ($runtime -and ($runtimeSourceFeed -or $runtimeSourceFeedKey)) {
if ($runtimeSourceFeed) { $installParameters.AzureFeed = $runtimeSourceFeed }
if ($runtimeSourceFeedKey) {
$decodedBytes = [System.Convert]::FromBase64String($runtimeSourceFeedKey)
$decodedString = [System.Text.Encoding]::UTF8.GetString($decodedBytes)
$installParameters.FeedCredential = $decodedString
}
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from custom location '$runtimeSourceFeed'."
ExitWithExitCode 1
}
} else {
ExitWithExitCode 1
}
}
}
@@ -210,16 +288,16 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { '15.9' }
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
$msbuildCmd = Get-Command "msbuild.exe" -ErrorAction SilentlyContinue
$msbuildCmd = Get-Command 'msbuild.exe' -ErrorAction SilentlyContinue
if ($msbuildCmd -ne $null) {
# Workaround for https://github.com/dotnet/roslyn/issues/35793
# Due to this issue $msbuildCmd.Version returns 0.0.0.0 for msbuild.exe 16.2+
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split(@('-', '+'))[0])
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split([char[]]@('-', '+'))[0])
if ($msbuildVersion -ge $vsMinVersion) {
return $global:_MSBuildExe = $msbuildCmd.Path
@@ -239,7 +317,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
InitializeVisualStudioEnvironmentVariables $vsInstallDir $vsMajorVersion
} else {
if (Get-Member -InputObject $GlobalJson.tools -Name "xcopy-msbuild") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'xcopy-msbuild') {
$xcopyMSBuildVersion = $GlobalJson.tools.'xcopy-msbuild'
$vsMajorVersion = $xcopyMSBuildVersion.Split('.')[0]
} else {
@@ -247,9 +325,12 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
$xcopyMSBuildVersion = "$vsMajorVersion.$($vsMinVersion.Minor).0-alpha"
}
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
$vsInstallDir = $null
if ($xcopyMSBuildVersion.Trim() -ine "none") {
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
}
if ($vsInstallDir -eq $null) {
throw "Unable to find Visual Studio that has required version and components installed"
throw 'Unable to find Visual Studio that has required version and components installed'
}
}
@@ -273,7 +354,7 @@ function InstallXCopyMSBuild([string]$packageVersion) {
}
function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
$packageName = "RoslynTools.MSBuild"
$packageName = 'RoslynTools.MSBuild'
$packageDir = Join-Path $ToolsDir "msbuild\$packageVersion"
$packagePath = Join-Path $packageDir "$packageName.$packageVersion.nupkg"
@@ -289,7 +370,7 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
Unzip $packagePath $packageDir
}
return Join-Path $packageDir "tools"
return Join-Path $packageDir 'tools'
}
#
@@ -306,32 +387,37 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
if (Get-Member -InputObject $GlobalJson.tools -Name "vswhere") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'vswhere') {
$vswhereVersion = $GlobalJson.tools.vswhere
} else {
$vswhereVersion = "2.5.2"
$vswhereVersion = '2.5.2'
}
$vsWhereDir = Join-Path $ToolsDir "vswhere\$vswhereVersion"
$vsWhereExe = Join-Path $vsWhereDir "vswhere.exe"
$vsWhereExe = Join-Path $vsWhereDir 'vswhere.exe'
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host "Downloading vswhere"
Invoke-WebRequest "https://github.com/Microsoft/vswhere/releases/download/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
Write-Host 'Downloading vswhere'
try {
Invoke-WebRequest "https://netcorenativeassets.blob.core.windows.net/resource-packages/external/windows/vswhere/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
}
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild", "-products", "*")
$args = @('-latest', '-prerelease', '-format', 'json', '-requires', 'Microsoft.Component.MSBuild', '-products', '*')
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
if (Get-Member -InputObject $vsRequirements -Name 'version') {
$args += '-version'
$args += $vsRequirements.version
}
if (Get-Member -InputObject $vsRequirements -Name "components") {
if (Get-Member -InputObject $vsRequirements -Name 'components') {
foreach ($component in $vsRequirements.components) {
$args += "-requires"
$args += '-requires'
$args += $component
}
}
@@ -357,27 +443,27 @@ function InitializeBuildTool() {
# Initialize dotnet cli if listed in 'tools'
$dotnetRoot = $null
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
$dotnetRoot = InitializeDotNetCli -install:$restore
}
if ($msbuildEngine -eq "dotnet") {
if ($msbuildEngine -eq 'dotnet') {
if (!$dotnetRoot) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "/global.json must specify 'tools.dotnet'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "/global.json must specify 'tools.dotnet'."
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
$buildTool = @{ Path = Join-Path $dotnetRoot 'dotnet.exe'; Command = 'msbuild'; Tool = 'dotnet'; Framework = 'netcoreapp2.1' }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
ExitWithExitCode 1
}
@@ -386,15 +472,15 @@ function InitializeBuildTool() {
function GetDefaultMSBuildEngine() {
# Presence of tools.vs indicates the repo needs to build using VS msbuild on Windows.
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
if (Get-Member -InputObject $GlobalJson.tools -Name 'vs') {
return 'vs'
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
return 'dotnet'
}
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
ExitWithExitCode 1
}
@@ -403,9 +489,9 @@ function GetNuGetPackageCachePath() {
# Use local cache on CI to ensure deterministic build,
# use global cache in dev builds to avoid cost of downloading packages.
if ($useGlobalNuGetCache) {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile ".nuget\packages"
$env:NUGET_PACKAGES = Join-Path $env:UserProfile '.nuget\packages'
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot ".packages"
$env:NUGET_PACKAGES = Join-Path $RepoRoot '.packages'
}
}
@@ -418,7 +504,7 @@ function GetSdkTaskProject([string]$taskName) {
}
function InitializeNativeTools() {
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
if (-Not (Test-Path variable:DisableNativeToolsetInstalls) -And (Get-Member -InputObject $GlobalJson -Name "native-tools")) {
$nativeArgs= @{}
if ($ci) {
$nativeArgs = @{
@@ -447,14 +533,14 @@ function InitializeToolset() {
}
if (-not $restore) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Toolset version $toolsetVersion has not been restored."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 1
}
$buildTool = InitializeBuildTool
$proj = Join-Path $ToolsetDir "restore.proj"
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "ToolsetRestore.binlog") } else { "" }
$proj = Join-Path $ToolsetDir 'restore.proj'
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'ToolsetRestore.binlog') } else { '' }
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Set-Content $proj
@@ -476,7 +562,7 @@ function ExitWithExitCode([int] $exitCode) {
}
function Stop-Processes() {
Write-Host "Killing running build processes..."
Write-Host 'Killing running build processes...'
foreach ($processName in $processesToStopOnExit) {
Get-Process -Name $processName -ErrorAction SilentlyContinue | Stop-Process
}
@@ -493,7 +579,7 @@ function MSBuild() {
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if ($ci -and $buildTool.Tool -eq "dotnet") {
if ($ci -and $buildTool.Tool -eq 'dotnet') {
dotnet nuget locals http-cache -c
$env:NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS = 20
@@ -504,7 +590,7 @@ function MSBuild() {
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
$path = Join-Path $path (Join-Path $buildTool.Framework 'Microsoft.DotNet.Arcade.Sdk.dll')
$args += "/logger:$path"
}
@@ -519,12 +605,12 @@ function MSBuild() {
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
Write-PipelineTaskError -Message "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Binary log must be enabled in CI build.'
ExitWithExitCode 1
}
if ($nodeReuse) {
Write-PipelineTaskError -Message "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Node reuse must be disabled in CI build.'
ExitWithExitCode 1
}
}
@@ -534,10 +620,10 @@ function MSBuild-Core() {
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
$cmdArgs += ' /warnaserror /p:TreatWarningsAsErrors=true'
}
else {
$cmdArgs += " /p:TreatWarningsAsErrors=false"
$cmdArgs += ' /p:TreatWarningsAsErrors=false'
}
foreach ($arg in $args) {
@@ -549,7 +635,7 @@ function MSBuild-Core() {
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-PipelineTaskError -Message "Build failed."
Write-PipelineTelemetryError -Category 'Build' -Message 'Build failed.'
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
@@ -564,12 +650,12 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
if ($arg.StartsWith('/bl:', "OrdinalIgnoreCase")) {
return $arg.Substring('/bl:'.Length)
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
if ($arg.StartsWith('/binaryLogger:', 'OrdinalIgnoreCase')) {
return $arg.Substring('/binaryLogger:'.Length)
}
}
}
@@ -579,14 +665,14 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
. $PSScriptRoot\pipeline-logging-functions.ps1
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$ToolsetDir = Join-Path $ArtifactsDir "toolset"
$ToolsDir = Join-Path $RepoRoot ".tools"
$LogDir = Join-Path (Join-Path $ArtifactsDir "log") $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir "tmp") $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot "global.json") | ConvertFrom-Json
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$ArtifactsDir = Join-Path $RepoRoot 'artifacts'
$ToolsetDir = Join-Path $ArtifactsDir 'toolset'
$ToolsDir = Join-Path $RepoRoot '.tools'
$LogDir = Join-Path (Join-Path $ArtifactsDir 'log') $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir 'tmp') $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot 'global.json') | ConvertFrom-Json
# true if global.json contains a "runtimes" section
$globalJsonHasRuntimes = if ($GlobalJson.tools.PSObject.Properties.Name -Match 'runtimes') { $true } else { $false }
@@ -602,7 +688,15 @@ Write-PipelineSetVariable -Name 'TMP' -Value $TempDir
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
if (!$disableConfigureToolsetImport) {
$configureToolsetScript = Join-Path $EngRoot 'configure-toolset.ps1'
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
if ((Test-Path variable:failOnConfigureToolsetError) -And $failOnConfigureToolsetError) {
if ((Test-Path variable:LastExitCode) -And ($LastExitCode -ne 0)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'configure-toolset.ps1 returned a non-zero exit code'
ExitWithExitCode $LastExitCode
}
}
}
}
+69 -20
View File
@@ -41,7 +41,7 @@ fi
# Configures warning treatment in msbuild.
warn_as_error=${warn_as_error:-true}
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
@@ -77,11 +77,11 @@ function ResolvePath {
function ReadGlobalVersion {
local key=$1
local line=`grep -m 1 "$key" "$global_json_file"`
local line=$(awk "/$key/ {print; exit}" "$global_json_file")
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Error: Cannot find \"$key\" in $global_json_file"
Write-PipelineTelemetryError -category 'Build' "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
@@ -172,7 +172,7 @@ function InstallDotNetSdk {
function InstallDotNet {
local root=$1
local version=$2
GetDotNetInstallScript "$root"
local install_script=$_GetDotNetInstallScript
@@ -191,11 +191,55 @@ function InstallDotNet {
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from public location (exit code '$exit_code')."
if [[ -n "$runtimeArg" ]]; then
local runtimeSourceFeed=''
if [[ -n "${6:-}" ]]; then
runtimeSourceFeed="--azure-feed $6"
fi
local runtimeSourceFeedKey=''
if [[ -n "${7:-}" ]]; then
decodedFeedKey=`echo $7 | base64 --decode`
runtimeSourceFeedKey="--feed-credential $decodedFeedKey"
fi
if [[ -n "$runtimeSourceFeed" || -n "$runtimeSourceFeedKey" ]]; then
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from custom location '$runtimeSourceFeed' (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
ExitWithExitCode $exit_code
fi
fi
}
}
function with_retries {
local maxRetries=5
local retries=1
echo "Trying to run '$@' for maximum of $maxRetries attempts."
while [[ $((retries++)) -le $maxRetries ]]; do
"$@"
if [[ $? == 0 ]]; then
echo "Ran '$@' successfully."
return 0
fi
timeout=$((2**$retries-1))
echo "Failed to execute '$@'. Waiting $timeout seconds before next attempt ($retries out of $maxRetries)." 1>&2
sleep $timeout
done
echo "Failed to execute '$@' for $maxRetries times." 1>&2
return 1
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
@@ -208,13 +252,13 @@ function GetDotNetInstallScript {
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script" || {
with_retries curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
wget -q -O "$install_script" "$install_script_url" || {
else
with_retries wget -v -O "$install_script" "$install_script_url" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
@@ -229,11 +273,11 @@ function InitializeBuildTool {
if [[ -n "${_InitializeBuildTool:-}" ]]; then
return
fi
InitializeDotNetCli $restore
# return values
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildToolCommand="msbuild"
_InitializeBuildToolFramework="netcoreapp2.1"
}
@@ -252,6 +296,9 @@ function GetNuGetPackageCachePath {
}
function InitializeNativeTools() {
if [[ -n "${DisableNativeToolsetInstalls:-}" ]]; then
return
fi
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
@@ -294,14 +341,14 @@ function InitializeToolset {
if [[ "$binary_log" == true ]]; then
bl="/bl:$log_dir/ToolsetRestore.binlog"
fi
echo '<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' > "$proj"
MSBuild-Core "$proj" $bl /t:__WriteToolsetLocation /clp:ErrorsOnly\;NoSummary /p:__ToolsetLocationOutputFile="$toolset_location_file"
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Invalid toolset path: $toolset_build_proj"
Write-PipelineTelemetryError -category 'Build' "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
@@ -351,12 +398,12 @@ function MSBuild {
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
Write-PipelineTaskError "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
Write-PipelineTaskError "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
@@ -370,7 +417,7 @@ function MSBuild-Core {
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
Write-PipelineTaskError "Build failed (exit code '$exit_code')."
Write-PipelineTelemetryError -category 'Build' "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
@@ -391,7 +438,7 @@ temp_dir="$artifacts_dir/tmp/$configuration"
global_json_file="$repo_root/global.json"
# determine if global.json contains a "runtimes" entry
global_json_has_runtimes=false
dotnetlocal_key=`grep -m 1 "runtimes" "$global_json_file"` || true
dotnetlocal_key=$(awk "/runtimes/ {print; exit}" "$global_json_file") || true
if [[ -n "$dotnetlocal_key" ]]; then
global_json_has_runtimes=true
fi
@@ -413,9 +460,11 @@ Write-PipelineSetVariable -name "Temp" -value "$temp_dir"
Write-PipelineSetVariable -name "TMP" -value "$temp_dir"
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
if [ -z "${disable_configure_toolset_import:-}" ]; then
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
Vendored Submodule
+1
Submodule external/bdwgc added at 6e8db07c8d.
+1 -1
+1 -1
+1 -1
+1 -1
-1
Submodule external/llvm deleted from 29f36fd315.
Loaded 100 of 2041 files, more files were not shown because too many files have changed in this diff. Show more