Compare commits

..
306 Commits
Author SHA1 Message Date
monojenkins 3a07bd426d [2018-08] [arm64] Correct exception filter stack frame size (#14769)
[2018-08] [arm64] Correct exception filter stack frame size

Fixes: https://github.com/mono/mono/issues/14170
Fixes: https://github.com/xamarin/xamarin-android/issues/3112

The `call_filter()` function generated by `mono_arch_get_call_filter()`
was overwriting a part of the previous stack frame because it was not
creating a large enough new stack frame for itself.  This had been
working by luck before the switch to building the runtime with Android
NDK r19.

I used a local build of [xamarin/xamarin-android/d16-1@87a80b][0] to
verify that this change resolved both of the issues linked above.  I
also confirmed that I was able to reintroduce the issues in my local
environment by removing the change and rebuilding.

After this change, the generated `call_filter()` function now starts by
reserving sufficient space on the stack to hold a 64-bit value at the
`ctx_offset` location.  The 64-bit `ctx` value is saved to the
`ctx_offset` location (offset 344 in the new stack frame) shortly
afterwards:

    stp	x29, x30, [sp,#-352]!
    mov	x29, sp
    str	x0, [x29,#344]

As expected, the invocation of `call_filter()` now no longer modifies
the top of the previous stack frame.

Top of the stack before `call_filter()`:

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

Top of the stack after `call_filter()` (unchanged):

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

<details>
<summary>Additional background information</summary>

### The original lucky, "good" behavior for [mono/mono/2018-08@725ba2a built with Android NDK r14][1]

 1. The `resume` parameter for `mono_handle_exception_internal()` is
    held in register `w23`.

 2. That register is saved into the current stack frame at offset 20 by
    a `str` instruction:

           0x00000000000bc1bc <+3012>:	str	w23, [sp,#20]

 3. `handle_exception_first_pass()` invokes `call_filter()` via a `blr`
    instruction:

        2279							filtered = call_filter (ctx, ei->data.filter);
           0x00000000000bc60c <+4116>:	add	x9, x22, x24, lsl #6
           0x00000000000bc610 <+4120>:	ldr	x8, [x8,#3120]
           0x00000000000bc614 <+4124>:	ldr	x1, [x9,#112]
           0x00000000000bc618 <+4128>:	add	x0, sp, #0x110
           0x00000000000bc61c <+4132>:	blr	x8

    Before the `blr` instruction, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x0a8b31d8	0x00000071
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

 4. `call_filter()` runs.  This function is generated by
    `mono_arch_get_call_filter()` and starts with:

           stp	x29, x30, [sp,#-336]!
           mov	x29, sp
           str	x0, [x29,#344]

    Note in particular how the first line subtracts 336 from `sp` to
    start a new stack frame, but the third line writes to a position
    that is 344 bytes beyond that, back within the *previous* stack
    frame.

 5. After the invocations of `call_filter()` and
    `handle_exception_first_pass()`, `w23` is restored from the stack:

           0x00000000000bc820 <+4648>:	ldr	w23, [sp,#20]

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

    Notice how `call_filter()` has overwritten bytes 8 through 15 of the
    stack frame.  In this original lucky scenario, this does not affect
    the value restored into register `w23` because that value starts at
    byte 20.

 6. `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000bb960 <+872>:	cbz	w23, 0xbb9c4 <mono_handle_exception_internal+972>

    Since `w23` is `0`, `mono_handle_exception_internal()` correctly
    continues into the `else` branch.

### The bad behavior for [mono/mono/2018-08@725ba2a built with Android NDK r19][2] works slightly differently

 1. As before, the local `resume` parameter starts in register `w23`.
 2. This time, the register is saved into the stack frame at offset 12
    (instead of 20):

           0x00000000000bed7c <+3200>:	str	w23, [sp,#12]

 3. As before, `handle_exception_first_pass()` invokes `call_filter()`
    via a `blr` instruction.

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

 4. `call_filter()` runs as before.  And the first few instructions of
    that function are the same as before:

        stp	x29, x30, [sp,#-336]!
        mov	x29, sp
        str	x0, [x29,#344]

 5. `w23` is again restored from the stack, but this time from offset 12:

           0x00000000000bf2c0 <+4548>:	ldr	w23, [sp,#12]

    The problem is that `call_filter()` has again overwritten bytes 8
    through 15 of the stack frame:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

    So after the `ldr` instruction, `w23` has an incorrect value of
    `0x7f` rather than the correct value `0`.

 6. As before, `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000be430 <+820>:	cbz	w23, 0xbe834 <mono_handle_exception_internal+1848>

    But this time because `w23` is *not* zero, execution continues
    *incorrectly* into the `if` branch rather than the `else` branch.
    The result is that `ji` is set to `0` rather than a valid
    `(MonoJitInfo *)` value.  This incorrect value for `ji` leads to a
    null pointer dereference, as seen in the bug reports.

[0]: https://github.com/xamarin/xamarin-android/tree/xamarin-android-9.3.0.22
[1]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1432/Azure/
[2]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1436/Azure/

</details>

Backport of #14757.

/cc @lambdageek @brendanzagaeski
2019-06-03 14:26:18 -04:00
monojenkins 223ea7ef92 Bump API snapshot submodule 2019-05-10 11:09:54 +02:00
Marek Safar 913c11def5 [Facades] Include missing enums in System.Memory 2019-05-10 11:09:54 +02:00
Thays Grazia 6843da5c57 A deadlock happens when during a process_suspend there is a pending invoke and then when it's calling the invoke there is another call to process_suspend in the same thread. 2019-05-10 10:29:55 +02:00
Ankit Jain 167f04c902 Bump msbuild to track mono-2018-08 (#14396)
* Bump msbuild to track mono-2018-08

* Update nuget version
   (Thanks for @lewing)
2019-05-08 15:04:30 -04:00
Ankit Jain 082e1a2346 Bump msbuild to track mono-2018-08 2019-04-30 18:19:54 +02:00
Alexander Köplinger 18fde9f8c2 Fix Windows MSI build
(cherry picked from commit 8aaf420efe)
2019-04-29 13:53:21 -04:00
Alexander Köplinger ea8ae3e8e0 More Windows MSI fixes
(cherry picked from commit e0326df7c6)
(cherry picked from commit 04dc2f342c)
2019-04-29 13:53:11 -04:00
Alexander Köplinger d1b7bf2c47 Fix Windows MSI build, msbuild is now in Current instead of 15.0
(cherry picked from commit 0ae04b8e80)
2019-04-29 13:52:55 -04:00
monojenkins 3cb36842fc [corlib] Disable four tests in TypeTest (#14213)
TypeTest.MakeGenericType_BadUserType
TypeTest.MakeGenericType_NestedUserDefinedType
TypeTest.MakeGenericType_UserDefinedType
TypeTest.TestMakeGenericType_UserDefinedType_DotNet20SP1

These tests pass when run with the interpreter on xamarin-macios.
They were disabled on master with https://github.com/mono/mono/commit/dc9bc9e0e6462739872117afc59e46fbc780d981
2019-04-25 11:13:51 +02:00
Zoltan Varga 6c9921e64b [jit] Avoid an assert when gsharedvt enums are boxed.
Fixes https://github.com/mono/mono/issues/13297.
2019-04-24 11:17:17 +02:00
Ankit Jain 4d857ac249 Bump msbuild to track mono-2018-08 (#14068)
To pick up update for the NuGetSdkResolver: https://github.com/mono/msbuild/pull/92
2019-04-17 00:35:05 +02:00
Ankit Jain a2d28226d4 [nuget] Bump to nuget 5.0.0 2019-04-12 08:14:22 +02:00
Ankit Jain 592c75cf1f Bump msbuild to track mono-2018-08-test (#13927)
Bump to get the msbuild 16 update
<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-04-12 08:09:15 +02:00
Zoltan Varga 3f5ec6d95c [jit] Make dlmalloc use mono_valloc so it picks up the MAP_JIT flag on osx.
Fixes https://github.com/mono/mono/issues/13878.
2019-04-11 18:02:31 +02:00
monojenkins bf4e8d2ae1 [debugger-agent] Ignore thread_end event if thread is detached (#13937)
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set.  In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.

Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace

```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
  * frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
    frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
    frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
    frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
    frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
    frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
    frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
    frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
    frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
    frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
    frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
    frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
    frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
    frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
    frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
2019-04-09 10:37:47 -04:00
Alexander Köplinger ca30169e75 Bump bockbuild to get GTK fix (#13912) (#13919)
https://github.com/mono/bockbuild/pull/93
2019-04-08 15:57:49 +02:00
Alexander Köplinger 7fb1361513 [MacSDK] Bump bockbuild for fix to #13804 2019-04-08 11:52:41 +02:00
Alexander Kyte f236105056 [2018-08] [merp] Fix image size reported (#13886)
Fixes #13834

Backport of #13847.
2019-04-06 02:03:25 +02:00
Maxim Lipnin 4b020ca06d Fix transition offset logic (#13553)
I'm not sure about the correctness of this workaround but it works on macOS for the repro from https://github.com/mono/mono/issues/9664#issuecomment-472260968

Transition bounds used by time zone info represent some local date-time but actually they are of Unspecified date-time kind. 
When we try to get transition offset we compare some current date-time converted to UTC with appropriate transition bounds.
The fix is to use UTC for both current date-time and transition bounds during comparing.

Fixes https://github.com/mono/mono/issues/9664
2019-04-02 19:22:21 +02:00
Zoltan Varga e475d80db3 [llvm] Fix the computation of size of gshared instances in another place.
Fixes https://github.com/mono/mono/issues/13610.
2019-03-29 13:40:12 +01:00
Aleksey Kliger (λgeek) c87a95ea23 [llvm] Fix the computation of size of gshared instances in emit_args_to_vtype (). (#13675)
Fixes https://github.com/mono/mono/issues/13610.
2019-03-26 14:54:01 -04:00
Bernhard Urban 75282c430b [amd64] use 32bit variant of lea_membase for 32bit operations
Consider
```
 xor %r15, %r15       // %r15 = 0x0
 dec %r15d            // %r15 = 0xffff_ffff
 lea 0x1(%r15), %rdx  // %rdx = 0x1_0000_0000 but should be 0x0
```

instead `lea 0x1(%r15), %edx` should be generated.
2019-03-26 14:51:56 +01:00
Thays Grazia 5756d04e1d [debugger] Adding a comment to make it clear for the next debuggers. 2019-03-22 19:59:07 +01:00
Thays Grazia 484ce49a0b [debugger] Fixing two crashes while debugging an Android app.
-> Doing stepping into in VSM in some situation the frame saved in TLS is not synchronised with what is really been executed in the main thread. This makes the debugger-agent crashes because it tries to get variable info in some memory that is not available anymore.
	-> To fix it I forced the update of stack when CMD_THREAD_GET_FRAME_INFO is called.

-> Doing step over in Visual Studio for Windows, if you have the threads debugger window enabled, VSW calls frame_commands for each thread that is showed, and if the thread is not really_suspended it tries to get variable info from a memory that is not available anymore because the thread is not suspended yet.
	-> To fix it I don't send variable info of a frame if the thread is not really_suspended and doesn't have an async_state valid.

#Fixes 12843
2019-03-22 19:59:07 +01:00
Thomas Lehner 6a81d1dd5f [System] Fix a race condition in HttpRequestChannel.
Concurrent request completions could cause following exception:

    FATAL UNHANDLED EXCEPTION: System.ArgumentOutOfRangeException:
        Index was out of range. Must be non-negative and less than
        the size of the collection.
        Parameter name: index
       at System.Collections.Generic.List`1[T].RemoveAt (System.Int32 index) [0x00009] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Collections.Generic.List`1[T].Remove (T item) [0x0000c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.Cleanup () [0x00011] in <28c4695205ef47378f5370502b066592>:0
       at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.set_IsCompleted (System.Boolean value) [0x00034] in <28c4695205ef47378f5370502b066592>:0
       at System.ServiceModel.Channels.HttpRequestChannel+HttpChannelRequestAsyncResult.Complete (System.Exception ex) [0x0001a] in <28c4695205ef47378f5370502b066592>:0
       at System.ServiceModel.Channels.HttpRequestChannel+<>c__DisplayClass7_1.<BeginProcessRequest>b__0 (System.IAsyncResult r) [0x000bd] in <28c4695205ef47378f5370502b066592>:0

    This stack was captured after some changes in AwaitTaskContinuation.RunOrScheduleAction.
    Unfortunately the current implementation of AwaitTaskContinuation.RunOrScheduleAction re-throws the above
    exception with a wrong stack:

       at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.FinishStageTwo () [0x000a1] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.Finish (System.Boolean bUserDelegateExecuted) [0x00003] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.TrySetException (System.Object exceptionObject) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetException (System.Exception exception) [0x00029] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Net.HttpWebRequest+<RunWithTimeoutWorker>d__241`1[T].MoveNext () [0x000e0] in <3e9b3e26c4694baab3f689687ad40612>:0
       at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.InvokeMoveNext (System.Object stateMachine) [0x00000] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00024] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00023] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x0004f] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x0003d] in <4a8a0eaa024f41829d3435c07e5c7edb>:0
       at System.Net.ServicePointScheduler+<WaitAsync>d__45.MoveNext () [0x000f7] in <3e9b3e26c4694baab3f689687ad40612>:0
       at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.InvokeMoveNext (System.Object stateMachine) [0x00000] in <4a8a0

    On an Android Xamarin installation this causes an app crash. Since no user code is active the exception cannot be caught.
2019-03-21 11:28:57 +01:00
Alexander Kyte fdb26b0a44 [merp] Fix anon allocation, use MAP_PRIVATE 2019-03-20 13:57:41 +01:00
Alexander Kyte 82547c4a12 [crash] Fallback to MAP_ANONYMOUS for allocator
This is helpful by allowing the crash reporter to work even while
unable to write to the working directory.
2019-03-19 16:34:56 +01:00
Alexander Köplinger 693806ceda Bump bockbuild for GTK fix 2019-03-19 07:58:33 +01:00
Jo Shields 9cb3348c31 Append .1 to version number 2019-03-14 16:18:00 -04:00
Jo Shields 948b450991 Include libgdiplus 5.6.1
(cherry picked from commit 5888bf9acc)
2019-03-14 16:14:40 -04:00
Alexis Christoforides 60e60904ac [MacSDK] Bump Nuget to 4.8.2 2019-03-12 22:37:15 +01:00
Alexander Köplinger 89d7665e28 [2018-08] Bump bockbuild to get GTK fix (#13329)
See https://github.com/mono/bockbuild/pull/90
2019-03-06 23:36:48 +01:00
Zoltan Varga c47ab60cc4 [llvm] Emit target layout/target triple into bitcode files. (#13327)
Fixes https://github.com/mono/mono/issues/12307 for 2018-08.
2019-03-06 13:38:02 -05:00
monojenkins 7ad1871886 [2018-08] Clean up .o files from System.Native during make dist (#13256)
* Clean up .o files from System.Native during `make dist`

Closes: https://github.com/mono/mono/issues/11501

* Delete more useless binary cruft from `make dist`
2019-03-02 03:02:37 +01:00
Alexander Köplinger 56906e2376 Bump api-snapshot submodule
Gets in the API Diff change for https://github.com/mono/mono/commit/5dd1c19b53dd43ba55b6ea649dc3809b0ae4f057
2019-03-01 21:08:40 +01:00
Alexander Köplinger 6ed7152fc8 [sdks] Use throttle() instead of lock()
The former works in freestyle builds.

(cherry picked from commit 98e0f2697d)
2019-03-01 14:49:30 +01:00
Alexander Köplinger 68aad3b0eb [ci] Use dl.internalx.com when building on private Jenkins
(cherry picked from commit 9d4aa950aa)
2019-02-26 11:21:29 +01:00
Martin Baulig 5dd1c19b53 Make System.dll internals visible to Mono.Android. 2019-02-25 14:14:46 +01:00
Alexander Köplinger 29eb176dd9 Bump bockbuild for GTK fix
See https://github.com/mono/bockbuild/pull/84
2019-02-20 17:30:36 +01:00
Maxim Lipnin 45e519e9df use for loop to access item values and avoid InvalidCastException which occurs because VirtualDirectoryMappingCollection enumerator returns key and it's not expected 2019-02-19 23:12:00 +01:00
Alexander Kyte de14ec6c78 [2018-08] Add more detailed reporting for exceptions (#13012)
* [crash] Add more detailed reporting for exceptions

This bumps the crash reporting file format to version 0.0.3.

* [crash] Fix exception stack walker to consider captured_traces

Previously we were missing these frames from the captured_traces fields
of MonoException and MonoStackTrace.
2019-02-15 11:10:55 -05:00
Alexander Köplinger b041669258 [ci] Bump bockbuild to get fontconfig download url fix 2019-02-11 16:47:52 +01:00
monojenkins d2e33f5f5c [2018-08] [interp] Add method to seq_points table after we finish registering it (#12906)
* [interp] Add method to seq_points table after we finish registering it

When adding a breakpoint (mono_de_set_breakpoint) we search in this table for seq_point information for the method. If we find the information we expect that the method is already compiled and has the metadata initialized when actually adding the breakpoint (set_bp_in_method). We were crashing if the jinfo field of the interp method was not yet initialized.

Fixes https://github.com/xamarin/xamarin-macios/issues/5381

* [interp] Move profiler callback when we are actually done

Just in case.
2019-02-11 09:09:00 -05:00
monojenkins 32f1873983 [interp] Fix ldelema (#12930)
The ldelema IL instruction provided the element class while the Address intrinsic provided the array class.

Fixes https://github.com/xamarin/xamarin-macios/issues/5363
2019-02-11 09:08:13 -05:00
Alexander Köplinger bbf2394619 [ci] Support building the OSX .pkg on private Jenkins
(cherry picked from commit 7f5125179d)
2019-02-09 20:22:09 +01:00
monojenkins 98810acba1 [runtime] Fix building older runtimes with newer system Mono (#12574)
What was happening was that this was building with a Mono.Cecil from
the build profile, but was trying to load the system Mono.Cecil.

Those versions mismatch when building 2018-08 with a nightly mono that is 5.23.0.

Rather than messing with the MONO_PATH and risking shadowing something
important there, I chose to copy the Mono.Cecil files into this
"lib/build/tmp" directory that we are using for the emitted executable.
It is loaded by the system Mono as the correct Mono.Cecil to use.

This fixes the build.
2019-02-09 00:10:34 +01:00
monojenkins d946a25079 [2018-08] [runtime] Fix mono_pe_file_map logging (#12897)
Since it's being used when the image is created, it actually doesn't need to be signal safe. This restores the previous behavior. 

Backport of #12588.
2019-02-09 00:06:01 +01:00
Ludovic Henry 4d2b7f8b01 Merge pull request #12858 from monojenkins/backport-pr-12856-to-2018-08
[2018-08] [Registry] Prevent crash due to background thread facing problems in the file system
2019-02-08 15:44:49 -05:00
monojenkins 89fe6abeef [arm64] encode offset within stack slot for r4 in gsharedvt descriptor (#12885)
Follow up fix for https://github.com/mono/mono/pull/11741

Fixes https://github.com/mono/mono/issues/12831
2019-02-08 09:30:02 -05:00
Miguel de Icaza 9d21a2bb15 [Registry] Prevent crash due to background thread facing problems in the file system 2019-02-07 16:59:52 +00:00
Marek Safar 163f45d81c [corlib] Fix GetFrames_AsyncCalls test not to block 2019-02-05 17:45:58 +01:00
Maxim Lipnin 208d5a5f1b Avoid recursive extracting redundant frames from captured traces. 2019-02-04 13:41:24 +01:00
monojenkins bcc1b83e82 Add back missing stack frames to mono MERP reports (#12741)
Changeset https://github.com/mono/mono/commit/e1461ebc855977667f71d1804fd96eed2a155aff
introduced a regression where only the topmost frame of managed crashes are reported.

This change adds back the missing stack frames.
2019-02-01 19:59:48 -05:00
Bernhard Urban 7a3ce9849f Bump corert
Commit list for mono/corert:

* mono/corert@e229c9448 [Number] make double 0.0 cmp more portable (#30)

Diff: https://github.com/mono/corert/compare/6c9c4d9cffe2031acb788718646fbc013632c9ea...e229c9448cb262ceb230dfab64a8237140c1dc2d
2019-02-01 15:07:36 +01:00
Bernhard Urban 13204572a4 [arm/ios] workaround for faulty vcmp.f64 insn
Fixes https://github.com/mono/mono/issues/11965
2019-02-01 15:07:02 +01:00
Vlad Brezae 2359cba40e [coop] Fix deadlock when unwinding
At the moment of a safepoint, the runtime can be holding the loader lock. Fixes gsharing-valuetype-layout.exe timeouts.
2019-01-29 13:06:44 -05:00
Alexander Kyte a4956c837e [2018-08] Native Crash Stability Fix Batch (#12565)
* [crash] Support extra merp params

* [runtime] Make infrastructure for merp tests

* [runtime] Fix dumping when crash happens without sigctx

* [runtime] Disable crashy (886/1000 runs) stacktrace walker

* [crash] Remove usage of allocating build/os info functions

* [crash] Remove often-crashing g_free on native crash path

* [crash] Add crash_reporter checked build

We add a checked build mode that asserts when mono mallocs inside of
the crash reporter. It makes risky allocations into assertions. It's
useful for automated testing because the double-abort often represents
itself as an indefinite hang. If it happens before the thread dumping
supervisor process is started, or after it ends, the crash reporter
hangs.

* [crash] Remove reliance on nested SIGABRT/double-fault (broken on OSX)

* [crash] Fix top-level handling of double faults/assertions

* [runtime] Make fatal unwinding errors return into handled error paths

* [crash] Change dumper logging for better info

* [runtime] Fix handling of segfault on sgen thread

Threads without domains that get segfaults will end up in
this handler. It's not safe to call this function with a NULL domain.

See crash below:

```
* thread #1, name = 'tid_307', queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x10eff40f8)
  * frame #0: 0x000000010e1510d9 mono-sgen`mono_threads_summarize_execute(ctx=0x0000000000000000, out=0x0000001000000000, hashes=0x0000100000100000, silent=4096, mem="", provided_size=2199023296512) at threads.c:6414
    frame #1: 0x000000010e152092 mono-sgen`mono_threads_summarize(ctx=0x000000010effda00, out=0x000000010effdba0, hashes=0x000000010effdb90, silent=0, signal_handler_controller=1, mem=0x0000000000000000, provided_size=0) at threads.c:6508
    frame #2: 0x000000010df7c69f mono-sgen`dump_native_stacktrace(signal="SIGSEGV", ctx=0x000000010effef48) at mini-posix.c:1026
    frame #3: 0x000000010df7c37f mono-sgen`mono_dump_native_crash_info(signal="SIGSEGV", ctx=0x000000010effef48, info=0x000000010effeee0) at mini-posix.c:1147
    frame #4: 0x000000010de720a9 mono-sgen`mono_handle_native_crash(signal="SIGSEGV", ctx=0x000000010effef48, info=0x000000010effeee0) at mini-exceptions.c:3227
    frame #5: 0x000000010dd6ac0d mono-sgen`mono_sigsegv_signal_handler_debug(_dummy=11, _info=0x000000010effeee0, context=0x000000010effef48, debug_fault_addr=0xffffffffffffffff) at mini-runtime.c:3574
    frame #6: 0x000000010dd6a8d3 mono-sgen`mono_sigsegv_signal_handler(_dummy=11, _info=0x000000010effeee0, context=0x000000010effef48) at mini-runtime.c:3612
    frame #7: 0x00007fff73dbdf5a libsystem_platform.dylib`_sigtramp + 26
    frame #8: 0x0000000110bb81c1
    frame #9: 0x000000011085ffe1
    frame #10: 0x000000010dd6d4f3 mono-sgen`mono_jit_runtime_invoke(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, exc=0x00007ffee1ea9f08, error=0x00007ffee1eaa250) at mini-runtime.c:3215
    frame #11: 0x000000010e11509d mono-sgen`do_runtime_invoke(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, exc=0x0000000000000000, error=0x00007ffee1eaa250) at object.c:2977
    frame #12: 0x000000010e10d961 mono-sgen`mono_runtime_invoke_checked(method=0x00007faae4f01fe8, obj=0x0000000000000000, params=0x00007ffee1eaa180, error=0x00007ffee1eaa250) at object.c:3145
    frame #13: 0x000000010e11aa58 mono-sgen`do_exec_main_checked(method=0x00007faae4f01fe8, args=0x000000010f0003e8, error=0x00007ffee1eaa250) at object.c:5042
    frame #14: 0x000000010e118803 mono-sgen`mono_runtime_exec_main_checked(method=0x00007faae4f01fe8, args=0x000000010f0003e8, error=0x00007ffee1eaa250) at object.c:5138
    frame #15: 0x000000010e118856 mono-sgen`mono_runtime_run_main_checked(method=0x00007faae4f01fe8, argc=2, argv=0x00007ffee1eaa760, error=0x00007ffee1eaa250) at object.c:4599
    frame #16: 0x000000010de1db2f mono-sgen`mono_jit_exec_internal(domain=0x00007faae4f00860, assembly=0x00007faae4c02ab0, argc=2, argv=0x00007ffee1eaa760) at driver.c:1298
    frame #17: 0x000000010de1d95d mono-sgen`mono_jit_exec(domain=0x00007faae4f00860, assembly=0x00007faae4c02ab0, argc=2, argv=0x00007ffee1eaa760) at driver.c:1257
    frame #18: 0x000000010de2257f mono-sgen`main_thread_handler(user_data=0x00007ffee1eaa6a0) at driver.c:1375
    frame #19: 0x000000010de20852 mono-sgen`mono_main(argc=3, argv=0x00007ffee1eaa758) at driver.c:2551
    frame #20: 0x000000010dd56d7e mono-sgen`mono_main_with_options(argc=3, argv=0x00007ffee1eaa758) at main.c:50
    frame #21: 0x000000010dd5638d mono-sgen`main(argc=3, argv=0x00007ffee1eaa758) at main.c:406
    frame #22: 0x00007fff73aaf015 libdyld.dylib`start + 1
    frame #23: 0x00007fff73aaf015 libdyld.dylib`start + 1
  thread #2, name = 'SGen worker'
    frame #0: 0x000000010e2afd77 mono-sgen`mono_get_hazardous_pointer(pp=0x0000000000000178, hp=0x000000010ef87618, hazard_index=0) at hazard-pointer.c:208
    frame #1: 0x000000010e0b28e1 mono-sgen`mono_jit_info_table_find_internal(domain=0x0000000000000000, addr=0x00007fff73bffa16, try_aot=1, allow_trampolines=1) at jit-info.c:304
    frame #2: 0x000000010dd6aa5f mono-sgen`mono_sigsegv_signal_handler_debug(_dummy=11, _info=0x000070000fb81c58, context=0x000070000fb81cc0, debug_fault_addr=0x000000010e28fb20) at mini-runtime.c:3540
    frame #3: 0x000000010dd6a8d3 mono-sgen`mono_sigsegv_signal_handler(_dummy=11, _info=0x000070000fb81c58, context=0x000070000fb81cc0) at mini-runtime.c:3612
    frame #4: 0x00007fff73dbdf5a libsystem_platform.dylib`_sigtramp + 26
    frame #5: 0x00007fff73bffa17 libsystem_kernel.dylib`__psynch_cvwait + 11
    frame #6: 0x00007fff73dc8589 libsystem_pthread.dylib`_pthread_cond_wait + 732
    frame #7: 0x000000010e28d76d mono-sgen`mono_os_cond_wait(cond=0x000000010e44c9d8, mutex=0x000000010e44c998) at mono-os-mutex.h:168
    frame #8: 0x000000010e28df4f mono-sgen`get_work(worker_index=0, work_context=0x000070000fb81ee0, do_idle=0x000070000fb81ed4, job=0x000070000fb81ec8) at sgen-thread-pool.c:165
    frame #9: 0x000000010e28d2cb mono-sgen`thread_func(data=0x0000000000000000) at sgen-thread-pool.c:196
    frame #10: 0x00007fff73dc7661 libsystem_pthread.dylib`_pthread_body + 340
    frame #11: 0x00007fff73dc750d libsystem_pthread.dylib`_pthread_start + 377
    frame #12: 0x00007fff73dc6bf9 libsystem_pthread.dylib`thread_start + 13
  thread #3, name = 'Finalizer'
    frame #0: 0x00007fff73bf6246 libsystem_kernel.dylib`semaphore_wait_trap + 10
    frame #1: 0x000000010e1d9c0a mono-sgen`mono_os_sem_wait(sem=0x000000010e43e400, flags=MONO_SEM_FLAGS_ALERTABLE) at mono-os-semaphore.h:84
    frame #2: 0x000000010e1d832d mono-sgen`mono_coop_sem_wait(sem=0x000000010e43e400, flags=MONO_SEM_FLAGS_ALERTABLE) at mono-coop-semaphore.h:41
    frame #3: 0x000000010e1da787 mono-sgen`finalizer_thread(unused=0x0000000000000000) at gc.c:920
    frame #4: 0x000000010e152919 mono-sgen`start_wrapper_internal(start_info=0x0000000000000000, stack_ptr=0x000070000fd85000) at threads.c:1178
    frame #5: 0x000000010e1525b6 mono-sgen`start_wrapper(data=0x00007faae4f31bd0) at threads.c:1238
    frame #6: 0x00007fff73dc7661 libsystem_pthread.dylib`_pthread_body + 340
    frame #7: 0x00007fff73dc750d libsystem_pthread.dylib`_pthread_start + 377
    frame #8: 0x00007fff73dc6bf9 libsystem_pthread.dylib`thread_start + 13
  thread #4
    frame #0: 0x00007fff73c0028a libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #1: 0x00007fff73dc7009 libsystem_pthread.dylib`_pthread_wqthread + 1035
    frame #2: 0x00007fff73dc6be9 libsystem_pthread.dylib`start_wqthread + 13
(lldb)
```

* [crash] Add signal-safe mmap/file "allocator"

* [crash] Remove use of static memory from dumper

* [runtime] Reduce print buffer size for lockless printer.

Each frame that prints ends up increased by the size of buff.
In practice, clang often fails to deduplicate some of these buffers,
leading to 30k-big stackframes.

It was noticed by a series of hard-to-diagnose segfaults on stacks that
looked otherwise fine during the crash reporting stress test.

This change fixes this, making stacks a 1/10th of the size. It doesn't
seem to break the crash reporter messages anywhere (may need to shrink
other "max name length" fields), and it's not mission-critical anywhere
else.

* [crash] Use async-safe file memory for dumper internals

* [crash] Add memory barriers around merp configuration

* [crash] Use signal-safe printers on all native crash paths

* [crash] Move gdb/lldb lookup to startup

* [runtime] Move MOSTLY_ASYNC_SAFE_FPRINTF to eglib

* [runtime] Fix all callsites of MOSTLY_ASYNC_SAFE_PRINTF

* [runtime] Fix all callsites of MOSTLY_ASYNC_SAFE_FPRINTF

* [crash] Switch to signal-safe exit function

* [crash] Make dumper enum in managed

* [runtime] Add more information to managed frame

* [crash] Make async_safe printers inlined

* [runtime] Move basic pe_file functionality into proclib

* [crash] Fix handling of thread attributes

* [crash] Place hashes into the json file for all threads

* [crash] Fix 2018-08 CI, disable test
2019-01-25 14:17:41 +01:00
Martin Baulig d4d4fb7987 [eglib]Fix memory leak in g_find_program_in_path(). #11615. 2019-01-24 20:01:44 +01:00
Martin Baulig 9939c1a656 [System]Fix race condition in SystemDependencyProvider.Initialize(). #12538. 2019-01-22 09:48:25 +01:00
Alexander Kyte b0c4390d60 [runtime] Don't probe aot cache on android 2019-01-18 08:42:13 +01:00
Kirill Osenkov 725ba2a252 Fix https://github.com/mono/mono/issues/12461.
After .NET 4.5 .NET Framework has changed the System.ComponentModel.Composition ReflectionModelServices.IsExportFactoryImportDefinition to not throw.

Updating this to match the .NET Framework behavior.
2019-01-17 16:14:09 +01:00
Marek Habersack 1dee62626d [2018-08 Android,AOT] Support dumping AOT offsets with Android NDK r17+ (#12454) (#12465)
Backport of: https://github.com/mono/mono/commit/eebe4b52857140ba9dea872c6249960273d222f2 from https://github.com/mono/mono/pull/12454

The latest Android NDK releases no longer ship with per-platform C headers.
Instead they ship with a set of unified headers and require the presence of the
`__ANDROID_API__` macro in order to select the target platform.
The AOT offset dumper doesn't support this new layout and this commit changes
that by adding support for the unified headers (without removing support for the
older NDKs) thus fixing the following error when trying to dump the offsets with
newer NDK:

    Processing triple: aarch64-v8a-linux-android
    Error parsing '[...]/mono/mono/metadata/metadata-cross-helpers.c'
    [...]/mono/mono/metadata/metadata-cross-helpers.c(5,10): fatal: 'stdio.h' file not found
    Error parsing '[...]/mono/mono/mini/mini-cross-helpers.c'
    [...]/mono/mono/mini/mini-cross-helpers.c(7,10): fatal: 'stdio.h' file not found
2019-01-17 13:32:43 +01:00
Zoltan Varga 146c8191fb [x86] Set the LLVM data layout. 2019-01-16 13:03:07 +01:00
Zoltan Varga 5d3077c7bc [arm] Set the LLVM data layout, without it, opt seems to make incorrect optimizations.
Fixes https://github.com/mono/mono/issues/12307.
2019-01-16 13:03:07 +01:00
Thays Grazia 5266e6a8f1 [2018-08][debugger][backport] Fix crash when there is a generic struct with a field that is an enumerator (#12410)
* [Debugger] Fix crash when there is a generic struct with a field that is an enumerator. (#12368)

* [Debugger]  Debugger crashes when inside a class, there is an internal struct, with a field that is an enumerator.
files.myBucket.GetEnumerator().get_Current().Key
Fixes #10735

* [Debugger] Debugger crashes when there is a generic struct with a field that is an enumerator.
Example: files.get_Current().Key
A unit test that reproduces this crash was added too.
Fixes #10735

* Removing the extra space.

* Merging changes from dtest-app.cs

* Adding extra line.
2019-01-15 18:23:01 +01:00
monojenkins 0d988bc696 Bump API snapshot submodule 2019-01-14 18:21:36 +01:00
Maxim Lipnin e6285d015b Bump corefx 2019-01-14 18:21:36 +01:00
Alexander Kyte 51c4f45402 [2018-08] [merp] Batch of fixes (#12095)
This contains the fixes and tests I'm going to call "service pack 1" for ease of grouping

#11884
#11855
#11636
#11909
#12036
2019-01-11 12:08:37 +01:00
Bernhard Urban 3bf37058fd [interp] use interp_lmf wrapper for interp2native on platforms with wonky sigcontext (#12348)
On some platforms (e.g. AppleTV) SigContext is available, but isn't as
precise as we would like to have it. For example, it doesn't save
callee-saved registers and thus aren't restored properly.

Unfortunately, this also means we can't use the fastpath for
interp2native, so we disable it on those platforms.
2019-01-11 11:40:21 +01:00
Marek Safar 25a8be9111 Bump CoreFX to bring a fix for #12093 2019-01-10 14:55:46 +01:00
Marek Habersack 29e0e642da [Android] Fix build with NDK r18 (#11016) (#12291)
Xamarin.Android is looking to switch to using the latest Android NDK, r18.
However, there are a few build issues in Mono and BTLS which need to be
addressed before that happens.

  * BTLS fails to build some ARM assembly sources with NDK r18's clang.
    https://github.com/mono/boringssl/pull/12 fixes this issue

  * BTLS uses obsolete Android cmake toolchain file.
    Add a configure option to set alternative toolchain file.

  * NDK r18 minimu supported API level is 16.
    This commit makes the API level configurable and sets the minimum platform
    to API 16

  * NDK r18 makes changes to declaration of `mmap` which break Mono build.
    Detect NDK version in `mono-compiler.h` and act accordingly (do not
    redeclare `mmap`)

  * NDK r18 doesn't have `cpp` and SDKs build assumes it's present.
    Detect whether `cpp` is present and, if not, use `compiler -E` instead.

  * NDK r18 uses unified headers exclusively and requires __ANDROID_API__ to be
    set in order to present the correct set of native APIs.
    SDKs now define the macro when the `ANDROID_NEW_NDK` Make variable is set to
    anything else than `no`.
2019-01-07 14:31:07 +01:00
Alexis Christoforides 7b44d5719e [tests] Remove an invalid assertion on System.IO.DriveInfo
All fixed drives are not guaranteed to be ready
2019-01-07 08:22:51 -05:00
Egor Bogatov 3fad9a7a1c Update DateTimeTest.cs 2019-01-07 10:19:42 +01:00
Egor Bogatov 6b6fd890ee Update mono/metadata/culture-info-tables.h 2019-01-07 10:19:42 +01:00
EgorBo 8f50f96e8f Add tests 2019-01-07 10:19:42 +01:00
EgorBo 1edf795ae0 ko-* specific AM/PM designators 2019-01-07 10:19:42 +01:00
monojenkins 9671417e21 [2018-08] [Network] Fix CVE 2018-8292 on Mac OS X. (#12153)
* [Network] Fix CVE 2018-8292 on Mac OS X.

This commit ensures that CVE 2018-8292 is fixed by not setting
auto-redirect in the first request done by CFNetwork. CFNetwork has the
same bug, but we can work around it by performing the first request,
checking the response and, if it is a rediret, follow it without the
Authorization headers.

* No need to close the obc, it can be reused.

* Small typo.

* Fix error while chaging the request :/

* Minor style changes.
2019-01-03 15:41:54 +01:00
Zoltan Varga 047d0abd5c [llvm] Add support for OP_RCNEQ.
Fixes https://github.com/mono/mono/issues/12131.
2018-12-22 11:31:39 +01:00
Cody Russell 237c5fedc7 [2018-08] [GTK] Bump bockbuild for VSTS #737323 fix to GtkNSView. (#12164)
* [GTK] Bump bockbuild for VSTS #737323 fix to GtkNSView.

This is a 2018-08 backport of #11972
2018-12-21 16:27:00 +01:00
Ankit Jain bac9fc1f88 [2018-08] Bump msbuild to track xplat-master (#12115)
Backport of #12071.
2018-12-21 15:05:23 +01:00
monojenkins 88eafb72e4 [2018-08] Bump to latest SQLite for Mac bundle (#12159)
Bump Mac bundle from SQLite 3.9.2 to 3.26.0

Backport of #12118.
2018-12-20 16:11:23 +01:00
monojenkins 3f799cc3c7 [exceptions] Verbose logging for cfa_reg assertion (#11916)
[2018-08] [exceptions] Verbose logging for cfa_reg assertion

Historically very common and hard to reproduce assertion, which, in some cases, could be fixed just from this logging. Log example :

```
Unset cfa_reg in method <0x40011220 - (null) trampoline>. Memory around ip (0x40011220):
0x40011210  41 89 07 4c 8b 3c 24 48 83 c4 08 c3 00 00 00 00  A..L.<$H........
0x40011220  48 83 c7 10 b8 00 12 01 40 ff e0 00 00 00 00 00  H.......@.......
0x40011230  00 00 00 00 e8 c7 ef fe ff 08 f8 34 e0 55 55 55  ...........4.UUU
0x40011240  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
* Assertion: should not be reached at unwind.c:664
```




Backport of #11888.

/cc @BrzVlad
2018-12-19 20:45:40 -05:00
Zoltan Varga 0d9e781cb6 [jit] Fix a buffer overflow in mini_get_gsharedvt_out_sig_wrapper_signature ().
Fixes https://github.com/mono/mono/issues/12130.
2018-12-19 21:53:18 +01:00
Maxim Lipnin 35a1ed626d Fix IndexOutOfRangeException in MethodInfo.ReturnParameter.IsDefined(type) 2018-12-19 21:47:27 +01:00
Ankit Jain 03b9354ed9 [MacSDK] Bump NuGet to 4.8.1 2018-12-19 14:02:20 +01:00
monojenkins f919fb58c4 Fix https://github.com/mono/mono/issues/11898 (#11904)
There are two identical sounding functions:

mono_string_handle_to_utf8:
  What we usually use. Sounds reasonable. Usually is reasonable.
  Allocates with malloc.

mono_string_to_utf8str_handle:
  What should be used here. Allocates with CoTaskMemAlloc.
  So it can be freed with marshal_free => CoTaskMemFree.

i.e. The allocator and the freeer need to agree.
2018-12-03 19:39:56 -05:00
monojenkins 95fbc22436 Fix race condition in XmlCharType.Instance (#11828) 2018-12-03 19:19:31 -05:00
Zoltan Varga bcda004855 [aot] Emit runtime invoke wrappers for up to 40 parameters for bitcode, newly added tests depend on it.
Fixes https://github.com/mono/mono/issues/11874.
2018-12-03 14:50:04 +01:00
monojenkins f0e466618d [crash] Add managed exception class name (#11821) 2018-11-30 13:28:10 -08:00
Egor Bogatov 23f2024af8 [2018-08] Fix Encoding serialization issue (#11807)
Manual backport of https://github.com/mono/mono/pull/11661
2018-11-28 13:21:17 +01:00
Bernhard Urban 414cafac93 [interp] Don't include internal frames in stack trace (#11793)
For handled exceptions. On unhandled exceptions we leave them for debugging purposes.

Fixes exception18.cs
2018-11-27 14:33:07 -08:00
Alexander Köplinger 68746a81f2 [ci] Disable log compression in pipeline builds for now - second attempt
(cherry picked from commit 94d4055773)
2018-11-26 22:05:59 +01:00
monojenkins 1f62222fa6 [mini] use AOT trampolines in interp mixed mode (#11781)
[2018-08] [mini] use AOT trampolines in interp mixed mode

Fixes this on mscorlib/mixed mode on Xamarin.iOS:
```
[FAIL] NonExceptional.Serialization_Deserialization : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt. 
[FAIL] NonExceptional.Serialize_Deserialize_FixedDateRule : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt. 
[FAIL] NonExceptional.Serialize_Deserialize_FloatingDateRule : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt. 
[FAIL] SerializationTests.Serialization_Deserialization : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt. 
[FAIL] TimeZoneTest.CurrentTimeZone_SerializationRoundtrip : System.Runtime.Serialization.SerializationException : An error occurred while deserializing the object. The serialized data is corrupt. 
```


Backport of #11767.

/cc @lewurm
2018-11-26 13:55:55 -05:00
Alexander Köplinger d0184b6970 [ci] Disable log compression in pipeline builds for now
It is incompatible with JEP-210 at the moment.

(cherry picked from commit fb4b611636)
2018-11-26 17:49:44 +01:00
Aleksey Kliger a4c0912ecf [loader] ignore 'internalcall' impl attribute on 'abstract' methods
Otherwise we emit a noisy warning when doing a lookup of the method.

Fixes https://github.com/mono/mono/issues/11479
2018-11-26 10:38:22 -05:00
Zoltan Varga 7b9d9b3f1b [arm64] Fix passing r4/r8 arguments on the stack in the gsharedvt trampoline.
Fixes https://github.com/mono/mono/issues/11613.
2018-11-20 12:00:04 +01:00
Alexander Köplinger e3a4484af7 [2018-08] Add .NET 4.7.2 reference assemblies (#11733)
Backport of https://github.com/mono/mono/pull/11391 to 2018-08
2018-11-19 16:58:34 +01:00
monojenkins 99b6da7944 [interp] attempt to intrinsify again after a method is resolved regarding generics (#11717)
[2018-08] [interp] attempt to intrinsify again after a method is resolved regarding generics

discovered when running https://github.com/Clancey/gMusic



Backport of #11712.

/cc @luhenry @lewurm
2018-11-16 14:05:43 -05:00
monojenkins 2f57669e30 [2018-08] [aot] image is usable without debug flag when interpreter is used (#11709)
Backport of #11705.
2018-11-16 12:10:05 +01:00
monojenkins 0206338c1e [2018-08] [crash] Remove duplicated il_offset (#11702)
This code is duplicated completely from the code directly above it (github doesn't show in this diff)

Backport of #11700.
2018-11-15 23:23:30 +01:00
monojenkins b20dad6656 [runtime] Fix the size calculation in mono_debug_add_method (). (#11687)
Backport of #11672.
2018-11-15 16:11:52 +01:00
Bernhard Urban 0f81533ae3 [arm64] Pass small stack args correctly on linux. Fixes pinvoke.exe with llvm+full aot. (#11682)
Backport of #10674. LLVM is already properly bumped on this branch.
2018-11-15 16:09:09 +01:00
monojenkins 81f3c4763e [2018-08] Fix starting process with empty environment incorrectly inherits all variables (#11645)
This pull request fixes starting a process with an empty environment variable collection in `ProcessStartInfo`. As per the issue it fixes a regression that it used to work but currently an empty environment variable collection works like it was never touched.

I also made a minor fix to remove the `StringBuilder` because a simple `string.Concat` does the job without the `StringBuilder` allocation and overhead.

I've included unit tests. Please let me know if you'd like any changes.

Fixes #8766

Backport of #11630.
2018-11-15 16:05:33 +01:00
Alexander Köplinger 6a9ee132c1 [ci] Clean acceptance-tests/external before build
Otherwise it might have left-over files from previous builds since it
uses our custom submodule system which isn't catched by Jenkins cleaning.

(cherry picked from commit 49a5af74f7)
(cherry picked from commit b429e290e8)
2018-11-15 16:03:56 +01:00
monojenkins b8ebc791fc [ios] cosmetic fix for WatchOS Makefile (#11668)
It's not really used, and the directory `[...]/SDKs/WatchOS.sdk` always
exists.
2018-11-14 16:42:06 -05:00
Bernhard Urban 32d4b6f22a [llvm] bump for armhf callingconv fix (#11609)
Commit list for mono/llvm:

* mono/llvm@c97510286a [mono] respect hardfloat/softloat setting in ARM ABI (#16)
@ mono/llvm@8415fd85ce Fix the mono calling convention on arm64+linux.
@ mono/llvm@117a508c0c Merge pull request #15 from mono/android-eh
@ mono/llvm@7dd82303e5 Emit .fnstart/.fnend directives on arm ELF platforms in the mono EH writer since the ARM backends depends on ARMException doing it, and ARMException is not ran when --disable-gnu-eh-frame is given.

Diff: https://github.com/mono/llvm/compare/81376d161cb6c2230834b2dc096ccbdae8d93020...c97510286a58f9aaa116fcfdb8b693d5f61910d2


Backport of https://github.com/mono/mono/pull/11607
2018-11-08 13:38:01 -05:00
monojenkins 925e0c0d0c [ios] remove DISABLE_POLICY_EVIDENCE (#11581)
[2018-08] [ios] remove DISABLE_POLICY_EVIDENCE

Originally it was introduced with db3591b
It seems like only the icall definitions were disabled for device
runtimes (which aren't shipped there anyway), however the icalls
themselves are _included_ in the device runtime. Not sure what the
rationale was behind this. Since the original change is very old, I
think it just happened to be like that today.  When running with the
interpreter we need the definitions at runtime too (via the optional
icall table that is linked in, so no additional cost for the fullAOT
runtime due to this change).

Backport of https://github.com/mono/mono/pull/11565

/cc @lewurm
2018-11-06 19:55:30 -05:00
Alex Earl 65cd16c227 Bump roslyn-binaries submodule to pull in System.Xml.dll file name fix in vbc.rsp (#11573)
(cherry picked from commit 5d5d8fc7ea)
2018-11-06 21:42:36 +01:00
Alexander Köplinger b5891e25ac [ci] Fix Windows .msi build
(cherry picked from commit c8d59d1d1d)
2018-11-06 10:41:13 +01:00
monojenkins d4d1d1ec24 [bcl] avoid false pinning in GCTest (#11536)
[2018-08] [bcl] avoid false pinning in GCTest

Backport of #11521.

/cc @luhenry @lewurm

Description:
needed for mscorlibtests on Xamarin.iOS with interpreter
2018-11-05 21:45:33 -05:00
monojenkins 3837f0c276 [interp] accept typed-by-ref as return type from a JIT call (#11533) 2018-11-05 21:01:12 -05:00
monojenkins 2e5c0d8f1e [interp] transform delegate invoke in method body too (#11551) 2018-11-05 20:58:29 -05:00
Alexander Köplinger 60c21beaeb [ci] Switch to new syntax of Azure Storage plugin
(cherry picked from commit de2cb5fdde)
2018-11-05 18:20:31 +01:00
Martin Baulig 3c2b25ce23 [System]lock protect SystemCertificateProvider.EnsureInitialized(). 2018-11-01 23:05:27 +01:00
Bernhard Urban 5fedefe43e [tests] reduce recursion depth
so it doesn't trigger a stack overflow on Xamarin.iOS with interpreter on a debug build (`CFLAGS=-O0`).
2018-10-31 11:56:58 +01:00
Alexander Kyte 53a21a0e84 [crash] Fix async setting for crash reporter
We were previously only setting it from the icall. The icall was
therefore fine, while invocations associated with actual dumps
caused crashes like this:

```

thread #10, name = 'Thread Pool Worker'
    frame #0: 0x00007fff978dac22 libsystem_kernel.dylib`__psynch_mutexwait + 10
    frame #1: 0x00007fff979c5dfa libsystem_pthread.dylib`_pthread_mutex_lock_wait + 100
    frame #2: 0x00007fff979c3519 libsystem_pthread.dylib`_pthread_mutex_lock_slow + 285
    frame #3: 0x0000000108e0d4a9 mono`mono_loader_lock + 73
    frame #4: 0x0000000108dc9106 mono`mono_class_create_from_typedef + 182
    frame #5: 0x0000000108dc1f2c mono`mono_class_get_checked + 92
    frame #6: 0x0000000108e21da2 mono`mono_metadata_parse_type_internal + 1378
    frame #7: 0x0000000108e25e11 mono`mono_metadata_parse_mh_full + 1233
    frame #8: 0x0000000108ca4f79 mono`mono_debug_add_aot_method + 121
    frame #9: 0x0000000108cc6b1e mono`decode_exception_debug_info + 5918
    frame #10: 0x0000000108cc5047 mono`mono_aot_find_jit_info + 1367
    frame #11: 0x0000000108e082cc mono`mono_jit_info_table_find_internal + 204
    frame #12: 0x0000000108cdb035 mono`mini_jit_info_table_find_ext + 69
    frame #13: 0x0000000108cdad5c mono`mono_find_jit_info_ext + 124
    frame #14: 0x0000000108cdc3a5 mono`unwinder_unwind_frame + 229
    frame #15: 0x0000000108cdbade mono`mono_walk_stack_full + 798
    frame #16: 0x0000000108cda1a4 mono`mono_summarize_managed_stack + 100
    frame #17: 0x0000000108e6dc03 mono`mono_threads_summarize_execute + 1347
    frame #18: 0x0000000108e6df88 mono`mono_threads_summarize + 360

```
2018-10-31 08:30:16 +01:00
monojenkins 04bdf8852b [2018-08] [crash] Fix merp invocation (#11468)
Backport of #11435.

/cc @alexanderkyte 

Description:
The change which made the handler return (rather than crashing the
runtime) did not consider this control flow path that causes the sigterm
signal handler to trigger the merp dumper if we return.

We now always return.

This was previously so that a random sigterm could be used to trigger a
dump. This isn't really used, and won't work right now anyways
(we don't register it until right before we dump). My fix is to remove
it, and to make the signal handler only work as part of the dumper
machinery.

@monojenkins skip
2018-10-30 22:53:24 +01:00
monojenkins 8df60a29bf [delegates] do not use CEE_CALLVIRT for non-virtual methods (#11451)
[2018-08] [delegates] do not use CEE_CALLVIRT for non-virtual methods

Backport of #11433.

/cc @lewurm 

Description:
2018-10-30 12:45:19 -04:00
monojenkins d90fbe3a0a [llvm] Fix passing vtypes received by value as arguments, they need to be copied. Fixes https://github.com/mono/mono/issues/11378. (#11429) 2018-10-30 11:13:29 -04:00
Aleksey Kliger ea740d2bc8 Disable hybrid suspend default for 2018-08
Earlier in the 2018-08 branch, Mono on desktop Linux, OSX and Windows would
default to hybrid suspend.  Turn that off for now (revert to preemptive suspend
by default).

Continue hybrid suspend work on 2018-10 and master.
2018-10-30 10:09:47 +01:00
monojenkins 0522076f0b [interp] throw overflow exception when {float,double}.NaN is converted to an int type (#11411)
A comparison with NaN is always `false`.
2018-10-28 18:05:05 -04:00
monojenkins 6034c589df [interp] enable more BCL tests (#11399)
[2018-08] [interp] enable more BCL tests

Backport of #11381.

/cc @lewurm 

Description:
probably won't pass on all configurations, I wanna find out on which.
2018-10-26 17:55:10 -04:00
Marek Habersack aebff6ce01 [SDKS, LLVM] Disable detection and use of libxml2 (#11340)
Mono doesn't use libxml2 and neither do the LLVM bits used by Mono when building
the AOT cross-compilers. Presence of `-lxml2` on the command line breaks 32-bit
builds on 64-bit Linux hosts:

    libtool: link: cc -shared  -fPIC -DPIC  -Wl,--whole-archive ./.libs/libmini.a ./.libs/libmono-ee-interp.a ./.libs/libmono-dbg.a ../../mono/metadata/.libs/libmonoruntimesgen.a ../../mono/sgen/.libs/libmonosgen.a ../../mono/utils/.libs/libmonoutils.a ../../mono/eglib/.libs/libeglib.a -Wl,--no-whole-archive  -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMBitReader -lLLVMCore -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -L$HOME/xamarin-android/external/mono/sdks/out/llvm-llvm32/lib -lz -ltinfo -lxml2 -lstdc++ -lm -lrt -ldl -lpthread  -O0 -ggdb3 -m32 -g -mno-tls-direct-seg-refs -Wl,-export-dynamic   -Wl,-soname -Wl,libmonosgen-2.0.so.1 -o .libs/libmonosgen-2.0.so.1.0.0
    mv -f .deps/libmini_la-aot-runtime-wasm.Tpo .deps/libmini_la-aot-runtime-wasm.Plo
    /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libxml2.so when searching for -lxml2
    /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libxml2.a when searching for -lxml2
    /usr/bin/ld: cannot find -lxml2
    collect2: error: ld returned 1 exit status
    make[7]: *** [Makefile:1495: libmonosgen-2.0.la] Error 1

Installation of `libxml2-dev` (on Ubuntu/Debian + friends) for the x86
architecture (or x86_64 architecture on 32-bit host) would be possible but then
it would break the 64-bit build because the 64-bit version of the library would
be absent. It would be a problem if Mono+LLVM actually used xml2, but since it
doesn't we can just disable the feature completely.

(cherry picked from commit 0d68ffd1ce)
2018-10-26 00:00:12 +02:00
Bernhard Urban 818a27a386 [2018-08] [aot] add unbox_arbitrary_trampoline for fullaot/interp mixed mode (#11363)
[2018-08] [aot] add unbox_arbitrary_trampoline for fullaot/interp mixed mode

Backport of https://github.com/mono/mono/pull/11321
2018-10-25 16:35:25 -04:00
Marek Safar b5e8de9182 [2018-08][corlib] Adds back accidentaly removed legacy COM API to Binder (#11368)
* [corlib] Adds back accidentaly removed legacy COM API to Binder

* Bump API snapshot submodule

* [csproj] Update project files
2018-10-25 15:03:49 +02:00
monojenkins 205b8129d5 [tests] lower recursion depth of PerformNoPinAction (#11323)
[2018-08] [tests] lower recursion depth of PerformNoPinAction

Backport of #11065.

/cc @lewurm 

Description:
so it doesn't overflow the stack when running on macOS with `clang -O0`.
2018-10-24 05:55:33 -04:00
Jay Krell c2c764dfc6 [2018-08] Replace mono_object_get_class with mono_object_class to avoid gc transitions. (#11338)
Replacement for #11104.
Backport of #11077.
2018-10-24 03:56:57 -04:00
Aleksey Kliger (λgeek) b778499454 [wasm] Add missing include (#11337)
Fixup for
https://github.com/mono/mono/commit/5756ba4b46f43d4f1c61c0d7dbbcf27a271f24ab
2018-10-24 02:11:20 -04:00
Marek Safar b7ad1c6923 Bump corefx 2018-10-23 20:34:45 +02:00
Aleksey Kliger (λgeek) 7a7768ff51 [marshal] Fixup missed mono_class_is_assignable_from (#11313)
Backport PR https://github.com/mono/mono/pull/11106 missed a call
2018-10-22 23:44:31 +02:00
Jay Krell fb7328e644 mono_class_is_assignable_from to mono_class_is_assignable_from_internal to skip GC mode machinery FIXME mono_class_is_assignable_from_checked 2018-10-22 18:29:03 +02:00
Jay Krell d88635a373 mono_class_enum_basetype to mono_class_enum_basetype_internal to skip GC mode machinery. 2018-10-22 18:28:07 +02:00
Aleksey Kliger (λgeek) 465b71e99d [runtime] Don't init classes in ves_icall_RuntimeTypeHandle_is_subclass_of
We must not get a TLE if referenced types are in an assembly that can't be
loaded.

Backported fix of https://github.com/mono/mono/issues/11123
2018-10-22 18:25:32 +02:00
Aleksey Kliger (λgeek) 5756ba4b46 [2018-08][marshal] Emit GC Safe transitions around foreign internal calls. (#11180)
* [threads] Make mono_add_internal_call external only

The runtime should use mono_add_internal_call_internal or
mono_add_internal_call_with_flags.

The thread suspend mechanism needs to know if an icall is added via the legacy
mono_add_internal_call API (which is not coop-aware, and so the registered
icalls must run in GC Safe mode), or if it is added by the runtime or by a
cooperative client (currently either the profiler or the System.Native PAL)
which knows to add GC transitions and safepoints and not to block indefinitely.

* [marshal] Factor out GCSafeTransitionBuilder

Make a local builder for creating the GC Safe transition calls for a method wraper.

* [sgen] Add coop GC transitions in mono_gc_toggleref_add

Also mark it external only. It's not used inside the runtime.

* [marshal] Emit GC Safe transitions around foreign icalls.

A foreign icall is added using mono_add_internal_call and that is not coop GC
aware.  (The runtime uses mono_add_internal_call_with_flags or
mono_add_internal_call_internal)

Under hybrid suspend, foreign icalls will run in GC Safe mode and transition to
GC Unsafe only when the call back into the runtime or invoke managed code.

Fixes https://github.com/mono/mono/issues/11138

* [runtime] Add new API call mono_dangerous_add_raw_internal_call

Under hybrid suspend, this adds an icall that is assumed to run in GC Unsafe
mode.  As such it has additional requirements for correct operation: it must
not run loops without periodically polling the runtime, and it must not perform
blocking operations such as blocking I/O or taking locks without manually
switching to GC Safe mode.
2018-10-22 18:17:50 +02:00
monojenkins d7dcd456b3 [interp] use unsigned conversion for nuint (#11284)
[2018-08] [interp] use unsigned conversion for nuint

Backport of #11259.

/cc @lewurm 

Description:
Fixes `MutableDataTest.Constructor` on Xamarin.iOS with interpreter.

monotouch tests are looking good now with interp-only:
<img width="906" alt="screenshot 2018-10-19 at 11 22 07" src="https://user-images.githubusercontent.com/75403/47209686-3d7b2780-d391-11e8-887f-9e1f39de2026.png">
😬
2018-10-21 08:25:34 -04:00
monojenkins 145155d8cf [aot] Reenable recursion checking when initing shared got entries (#11296)
[2018-08] [aot] Reenable recursion checking when initing shared got entries

Backport of #11258.

/cc @BrzVlad 

Description:
Broken in https://github.com/mono/mono/pull/11190
<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2018-10-20 20:35:28 -04:00
Marek Safar 423928c21d Bump mcs boostrap version 2018-10-20 09:25:17 +01:00
monojenkins b83fc43c32 [2018-08] [arm64] Remove the limitation on the number of nullable arguments for dyncalls. (#11241)
* [arm64] Remove the limitation on the number of nullable arguments for dyncalls.

Fixes https://github.com/xamarin/xamarin-macios/issues/4984.

* [amd64] Remove the limitation on the number of nullable arguments for dyncalls.

* [jit] Add tests for dyncalls with lots of nullable arguments.

* fix merge conflict
2018-10-19 23:49:49 +02:00
Ludovic Henry 9bc88897b7 [corlib] Make sure necessary resources are alongside the test assembly (#11270) 2018-10-19 15:38:58 -04:00
monojenkins 74788c7505 [2018-08] [llvm] Make sure OP_LSHL_IMM is transformed to a 64 bit shift even if the input is 32 bit, which can happen because the JIT IR is not regular. Fixes bug-17590.exe on arm64+fullaot+llvm. (#11264)
* [llvm] Make sure OP_LSHL_IMM is transformed to a 64 bit shift even if the input is 32 bit, which can happen because the JIT IR is not regular. Fixes bug-17590.exe on arm64+fullaot+llvm.

* [llvm] Fix a crash when trying to pass a vtype of size 0 on arm64. Fixes generic-constrained.2.exe.
2018-10-19 18:25:18 +02:00
Alexander Köplinger a4b6322c4f [2018-08] Bump corefx 2018-10-19 18:21:52 +02:00
monojenkins 934cf3c65a [WinForms] Propagate the flags from DrawTextInternal to MeasureTextInternal (#11261)
Flags to DrawText never reached MeasureText therefore strings with '&' were measured incorrect when drawn with flag TextFormatFlags.NoPrefix.
Fixes #6352
2018-10-19 18:08:51 +02:00
Ludovic Henry bc9d709e70 [sdks] Package what's missing for XA (#11256)
They require the additional files:
 - mono/eglib/eglib-config.h: that's to build `libmonodroid`
 - mcs/build/Consts.cs: that's to build `Mono.Posix` and `Mono.Data.Sqlite`
2018-10-19 06:57:56 -04:00
monojenkins 31a6b877f6 [pipeline] Ensure only 1 build per-product is executed per-node (#11249) 2018-10-18 22:09:45 +02:00
Vlad Brezae 4b343e98f3 [interp] Implement interpreter entry trampolines on amd64 (#10978) (#11164)
* [interp] Avoid unnecessary allocations when pinvoke void functions

* [amd64] Refactor interp pinvoke context computation

* [amd64] Interp trampoline for interpreter entry

* [aot] Don't include interp_in_wrapper when we use entry trampoline

These wrappers take a lot of space in the image.
2018-10-18 15:55:01 -04:00
monojenkins 2bc1ed09a3 [aot] Ensure shared got entries are initialized before loading methods (#11226)
Executing aot code that uses them would crash. Fixes random csc crashes.
2018-10-18 15:48:32 -04:00
Alexander Kyte 010e741bac [2018-08] Crash Reporter V2 (#11161)
* [runtime] Add icall to format crashes to disk in uniform way

* [crash] More gentle error handling around stack walking

* [crash] Make self-summarizier not need managed thread state

* [crash] Remove reliance on sgen/gchandles

* [crash] Refactor dumper mechanism

* [crash] Make threads not die during thread dump

* [crash] Add support for registering/dumping native libraries

* [crash] Add silent option to dumper

* [crash] Print error when unable to dump thread

* [crash] Add icalls to directly trigger telemetry-style walks

* [crash] Increase maximum frame depth

* [crash] Add managed method names when privacy disabled

* [crash] Add tests for concurrent dumping

* [crash] Add support for concurrent global dump requests

* [crash] Switch from CAS+Sleep to semaphore for dumper

Locks and GC-Safe regions are both unsafe in a signal handler
context. CAS primitives and the sleep syscall was used before.
Semaphores appear to be safe, enabling the following cleanup.

* [runtime] Make find_method use stack memory for dumper-facing entry point

* [crash] Enable install_state_summarizer on all platforms

* [crash] Dump managed stacks from controlling thread

* [crash] Don't use allocating method name formatter

* [crash] Copy safer strings with crash privacy disabled

* [crash] Fix string races with DumpTotal icall

* [crash] Fix space in crash file name

* [crash] Fix lifetime and size checks on mono-state memory

* [crash] Fix icall declarations for new style

* [runtime] Make sequence point lookup lockless

* [crash] Fix managed exception dumper icall format

* [crash] Print half-completed dump when space runs out

* [crash] Bump static memory for crash string to 500k

Note:
- This memory isn't accessed until we're crashing
- On any modern machine, unused static memory is lazily allocated. When
we access it, the fault will trigger the MMU to map memory for us.
- This is guaranteed to be reentrant and safe. mmap (the alternative) is
surely fairly stable on many systems, but this is not guaranteed. The
documentation pages for mmap specifically claim it as not safe in these async
contexts.

* [crash] Make whitelist lookup lockless

* [crash] Split unmanaged ptrs into method ip + offset
2018-10-18 20:23:40 +02:00
Alexis Christoforides ec1a736266 [corlib] Ensure that Console Streams are thread-safe
The Console class wraps the standard system stream reader/writers on construction so as to expose them as thread-safe. However, the Set-Out/In/Err methods in referencesource do not do the same when they replace default targets.

CoreFX code does: https://github.com/mono/corefx/blob/1748f6492e8b96842f76e1a9e7b5d7e8ccfca480/src/System.Console/src/System/Console.cs\#L412-L444

Possible fix for https://github.com/mono/mono/issues/11146
2018-10-18 10:58:03 -04:00
EgorBo a9b1acbee1 bump corefx 2018-10-18 10:53:26 -04:00
EgorBo 81326b8896 [2018-08] bump mono 2018-10-18 10:53:26 -04:00
Alexander Köplinger 04bde6eb75 [ci] Move OSX .pkg build to a separate bot pool
Allows us to better utilize the bockbuild cache.

(cherry picked from commit 229a4673d4)
2018-10-18 16:43:17 +02:00
monojenkins 2653d3eeda [pkg] Add preinstall which removes existing Mono of the same version (#11208)
Fixes https://github.com/mono/mono/issues/11202
2018-10-17 22:38:24 +02:00
Martin Baulig 47e556d4cf [2018-08][System]: Reenable native Apple Certificate support on WatchOS. (#11173) (#11199)
* [System]: Reenable native Apple Certificate support on WatchOS. (#11173)

During the code refactoring this summer, support for native apple certificates
has been dropped on watchOS.

* add `apple-certificates.sources` and move all certificate-related code from
  `appletls.sources`.

* move non-network related code from `MacProxy` into a new `AppleTypes`.

* add the `MONO_FEATURE_APPLE_X509` magic to `System/Makefile` where it was missing from.

(cherry picked from commit b61a3ffb02)

* [csproj] Update project files
2018-10-17 21:07:12 +02:00
Ludovic Henry 9f72a46b41 [sdks] Fix Archive Jenkins job (#11217) 2018-10-17 14:42:42 -04:00
Ludovic Henry 2712292c56 [sdks] Add Groovy script to build the SDKs archives for the products (#11192) 2018-10-16 12:25:51 -04:00
Ludovic Henry 839f49d3b2 [sdks] futimens and futimes symbols are missing on anything earlier than 10.13 (#11176)
This breaks when LLVM is built on 10.13+ and the runtimes are built on 10.12.
2018-10-16 08:14:41 -04:00
Ludovic Henry 657f915eb6 [2018-08] [sdks] Create archive targets for pre-building on CI (#10932)
* [sdks] Remove redundant --enable-cxx

* [sdks] Shorten `-m32`/`-m64` parameters detection for runtimes

* [sdks] Build BCL profiles without configure flags

* [sdks] Add BclTemplate to build BCL profiles

This is to avoid building all the profiles in a single `sdks/out/bcl` directory which will be shared between android, ios and wasm.

* [sdks] Add download/pack capabilities for LLVM

* [sdks] Create archive targets for pre-building on CI

* [sdks] Use archive-{android,ios,wasm} on CI

* [sdks] Run archive-llvm-llvm{,win}{32,64} on CI

* [sdks] Ensure LLVM is cloned before trying to download

* [sdks] Simplify call to BclTemplate

* [mcs] Make directory dependency as order-only

We would otherwise always rebuild the target since the directory is updated whenever something is added to the directory. This is a common Makefile technique.

* [sdks] Build tests for BCL targets
2018-10-12 16:13:26 -04:00
Ludovic Henry bd1f64acc3 [sdks] Add cmake toolchain file for LLVM MXE builds (#11055)
We couldn't compile llvmwin{32,64} on Linux because we would try to use {i686,x86_64}-w64-mingw-cmake but it wouldn't exist.
2018-10-12 10:55:12 -04:00
Ludovic Henry 7805ccc43d [sdks] Fix llvm-llvm32 build (#11117) 2018-10-12 10:10:33 -04:00
Alexander Köplinger 04a6e8294e [2018-08] Bump xunit (#11085) 2018-10-11 11:09:31 -04:00
Alexander Köplinger f001e691ca [2018-08] Bump msbuild (#11087) 2018-10-11 09:39:58 -04:00
Aleksey Kliger 7001d39bb2 Always use InvariantCulture calendars in X509Certificate on Mobile
Allow the linker to remove UmAlQuraCalendar and HijriCalendar on mobile devices

Match the CoreFX X509Certificate changes: https://github.com/mono/corefx/commit/4eb1fc2abd2ffa70e402d22415d0209b21725a10

Fixes https://github.com/mono/mono/issues/10448
2018-10-09 15:46:15 -07:00
Zoltan Varga e257971d3c [llvm] Remove an incorrect assert. 2018-10-08 14:20:23 -04:00
monojenkins 4d41b0560f [2018-08] [jit] Fix a constant propagation problem on 32 bit with iconst+conv.i8, we would be setting inst_imm even for long opcodes. (#10955)
* [jit] Fix a constant propagation problem on 32 bit with iconst+conv.i8, we would be setting inst_imm even for long opcodes.

* [llvm] Fix the special casing of OP_LCOMPARE_IMM, its not needed.

* Use TARGET_SIZEOF_VOID_P.
2018-10-08 10:12:53 +02:00
Martin Baulig 8ecede131d Use 'OnlyFaulted' only. 2018-10-08 10:12:16 +02:00
Martin Baulig 577ac4fd91 Add TaskContinuationOptions.OnlyOnFaulted | TaskContinuationOptions.OnlyOnCanceled as suggested by Marek. 2018-10-08 10:12:16 +02:00
Martin Baulig dfd13979ad [System]Make sure HttpWebRequest observes exceptions on timeout. #10488.
`HttpWebRequest.RunWithTimeoutWorker()` needs to observe the worker task's exception
on timeout to prevent it from being thrown via the `TaskScheduler.UnobservedTaskException`
event.  Fixes #10488.
2018-10-08 10:12:16 +02:00
Marek Safar cae86185c2 Bump corefx 2018-10-07 14:00:04 +02:00
Bernhard Urban 05c480e635 [2018-08] [interp] fixes for native type (#10998)
* [interp] fix op_implicit/op_explicit conversions for native types

Conversion operators have been mostly broken in the interpreter, for
example conversion from nint to float didn't work. While some
conversions are easy to intrinsify (like nint -> I8), there are harder
cases like nint -> decimal. The JIT actually falls back to the managed
implementation, so let's do this in the interpreter as well.

However the managed implementation expects a pointer to the value type,
i.e. the container type System.nfloat instead of float/double primitives.
In such cases the value is going to be stored in a local and then the
address of its storage is pushed on the stack before invoking the
managed implementation of the conversion.

Added test cases for scenarios observed in Xamarin.iOS.

* [interp] support ntype.CompareTo ()

* [interp] support ntype.Equals ()

* [interp] support nfloat.*Infinity

* [interp] support ntype.ToString ()

* [interp] introduce float R4 stack type
2018-10-05 11:27:42 -04:00
Egor Bogatov 5c70d5b946 [2018-08] Bump corefx (#10979)
* bump corefx

* bump corefx
2018-10-05 15:44:25 +02:00
Alexander Köplinger 2b620cc54b [ci] Always pass sha1 from Mac to Windows packaging build
Before we only did this for PR builds but it turns out it's needed for
normal master/branch builds too.

In situations where there are two Mac builds at roughly the same time
the downstream Windows builds would otherwise both refer to the same/latest
commit since they just checkout the latest ref on the branch.

Now we reset the Windows build to the sha1 which is passed to it.

(cherry picked from commit b13145bb12)
2018-10-05 00:28:25 +02:00
Alexander Köplinger 8d220f3913 [corlib] Simplify confusing "SSPI" error message and align with corefx (#10976)
Right now if anything goes wrong in the security stack you get a message like this:

```
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
```

This is confusing since SSPI is a Windows-only thing and is not used on Mono.
We inherited this error message string from referencesource.

Change the string to something more generic which also aligns us with CoreFX:
https://github.com/dotnet/corefx/blob/a10890f4ffe0fadf090c922578ba0e606ebdd16c/src/System.Net.Security/src/Resources/Strings.resx#L210-L212

(cherry picked from commit 42e347ee47)
2018-10-05 00:22:37 +02:00
Rolf Bjarne Kvinge 54700cf89f [sdks/ios] Build libMonoPosixHelper for device architectures, so that zlib-helper.o is created. (#10921) (#10971)
xamarin-macios currently builds zlib-helper.o using mono's source code.

It would be preferrable to use a version built by mono, and since
zlib-helper.o is a byproduct of building libMonoPosixHelper, we need to build
libMonoPosixHelper (we already build it for simulator architectures), even
though we don't need/use libMonoPosixHelper itself.
2018-10-04 11:23:47 -04:00
monojenkins d7ff201b56 Put xunit tests in mcs/class/lib/PROFILE/tests too (#10966)
So they are next to their NUnit counterparts.

Move xunit remote executor app to tests dir too and compile it when
compiling the test assembly instead of doing it before running the test.

We need to use an absolute path for the xunit remote executor and it
needs to be in Windows format on Windows since we pass it as an env var
to the xunit tests.
2018-10-04 11:06:09 -04:00
monojenkins 0a27b0f72d [2018-08] Use Mono.Android as a reference for netstandard facade (#10973)
Backport of #10849.

/cc @marek-safar @EgorBo

Description:
see https://github.com/mono/mono/pull/10835
2018-10-04 13:07:49 +02:00
Marek Safar 1644a1a01d Bump linker 2018-10-03 18:07:21 +02:00
monojenkins 3e89a56e50 Merge pull request #10906 from monojenkins/backport-pr-10899-to-2018-08
[2018-08] [runtime] Disable stack guard for main thread on osx

Backport of #10899.

/cc @luhenry @BrzVlad

Description of #10899:
On macOS Mojave, it seems that changing the mapping of stack pages for main thread can lead to corruption bugs.

https://github.com/mono/mono/issues/10802



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2018-10-03 18:05:34 +02:00
Vlad Brezae 9f01ca1a9f [runtime] Fix undefined pthread_main_np (#10930)
It is exported only when _DARWIN_C_SOURCE is defined.
2018-10-03 10:37:45 +03:00
Maxim Lipnin 8b099949ad [Reflection] Fix issue with finding types in module using an asterisk as filter criteria 2018-10-03 08:45:15 +02:00
Aleksey Kliger f21a48dad5 [w32handle] Add coop state transitions in mono_w32handle_lock_handles
If another thread locked at least one of the other handles, and there is a
cooperative GC STW happening, this function will loop forever.

Add GC Safe transitions around the call to sleep to give this thread a chance
to suspend.

Fixes https://github.com/mono/mono/issues/10863
2018-10-02 17:17:07 -04:00
Martin Baulig 2cc940be99 Use passwordless import in the X509Certificate(byte[]) constructor.
The `X509Certificate(byte[])` constructor should call `CertificateProvider.Import(byte[])`
to use the native apple implementation if possible.

(cherry picked from commit 8b3ca0dbbe0af1436d33d74bb2ca00b3cbfc28b8)
2018-10-02 17:21:18 +02:00
Alexander Köplinger 25cfd22997 [packaging] Remove installation of openssl via brew on macOS (#10874)
We're now using an msbuild version which uses .NET Core 2.x which no longer needs openssl.

(cherry picked from commit 634e726bdf)
2018-10-02 13:13:52 +02:00
Vlad Brezae 03a769f787 [runtime] Disable stack guard for main thread on osx
On macOS Mojave, it seems that changing the mapping of stack pages for main thread can lead to corruption bugs.

https://github.com/mono/mono/issues/10802
2018-10-01 20:30:02 +00:00
Ankit Jain ced11fba7b Bump msbuild to track xplat-master (#10668)
* Bump msbuild to track xplat-master

* Print mono version, to debug CI failures
2018-09-28 11:56:19 +02:00
monojenkins aed3298ae2 [sdks] One more update to get XA PR builds working on Linux (#10856)
It turns out some Debian versions identify themselves as `Debian GNU/Linux`, add
this string to the list of distributions which have mingw. This *should* fix the
PR builds (fingers crossed)
2018-09-27 21:31:45 -04:00
monojenkins 425b46eca7 [csproj] Update project files 2018-09-27 19:21:14 -04:00
Marek Safar d507bd8929 [System.Core] Removes no longer needed reference 2018-09-27 19:21:14 -04:00
monojenkins bc97d1e742 [merp] Fix return value handling of posix_spawn (#10843) 2018-09-27 10:32:35 -04:00
monojenkins d2ae599e2c [sdks] Debian Linux doesn't need to build MXE (#10820)
Debian and its derivatives include full mingw chain (both 64 and 32-bit) and so
they don't need to build the toolchain at all. However, the current detection of
Linux flavor takes into account only Ubuntu and all other distros are configured
to *require* MXE but they do *not* build them.

This commit fixes detection of distros which provide the mingw toolchain in a
way that makes it easy to extend the set of compatible distributions as needed.
In order to mark a distribution as one with MinGW one has to add the `:Distro:`
string to the `LINUX_WITH_MINGW` make variable in `mxe.mk`. The `Distro` string
is whatever the `determine-linux-flavor.sh` script returns.
2018-09-27 10:30:12 -04:00
monojenkins 005c804cb2 [SDKS] Build host runtime with correct bitness (#10741)
When building 32-bit binaries on a 64-bit system and vice versa, it is necessary
to pass the `-m32` or `-m64` compiler flags, respectively. Failure to do so will
result not only in linking problems (different binary architecture) but also in
problems when using some system headers which are sensitive to the architecture.
For instance, `ucontext.h` contains register set definitions for different
architectures and building Mono runtime with a 64-bit compiler targetting 32-bit
architecture results in this error:

   .../mono/mono/utils/mono-sigcontext.h:173:74: error: ‘REG_EIP’ undeclared (first use in this function); did you mean ‘REG_RIP’?
      #define UCONTEXT_REG_EIP(ctx) (((ucontext_t*)(ctx))->uc_mcontext.gregs [REG_EIP])

The issue here is that the compiler defines the `__x86_64__` macro which causes
`ucontext.h` to choose declarations for amd64 (which does not use `REG_EIP` but
has `REG_RIP` instead) while Mono runtime assumes its building for a 32-bit host
and uses 32-bit register names. Passing `-m32` to the compiler will fix the
issue as now the compiler will define the `__i386__` macro and all parties will
agree where they are and what they want.

This commit uses a rather baroque statement when constructing compiler flags to
build the Mono runtime to check whether the host triple targets 32 or 64-bit
host and adds the appropriate flag.
2018-09-25 12:49:30 -04:00
Marek Safar 96d8a05b06 [System.Net.Http] Allow empty value in ProductHeaderValue
Fixes #10227
2018-09-24 14:31:04 +02:00
monojenkins 5df25e7d88 [crash] Fix summarize_frame assertion (#10728) 2018-09-21 10:33:34 -04:00
Marek Safar 65c1b941f9 [corlib] Makes remoting test more robust (and disable it on mobile) 2018-09-20 19:54:10 +02:00
EgorBo e6be71c516 bump corefx 2018-09-20 15:42:26 +02:00
monojenkins a65b4f5548 [mono-api-html] Render something for fields with marshalling info. (#10699)
Fixes this warning when comparing Xamarin.iOS.dll:

    Comparison resulting in no changes (src: Public, HasFieldMarshal dst: Public) :
    <field name="EdgeTessellationFactor" attrib="4102" fieldtype="System.UInt16[]" />
    <field name="EdgeTessellationFactor" attrib="6" fieldtype="System.UInt16[]" />
2018-09-19 17:16:35 -04:00
monojenkins 0a85488fde [offset-tool] Error out when parsing fails. (#10695) 2018-09-19 17:15:45 -04:00
monojenkins b4b0d0fa35 [System]Fix WebResponseStream.ReadAllAsyncInner() return value. (#10587)
We need to call `ToArray()` instead of `GetBuffer()` to truncate the buffer
to the amount of bytes that we actually read.
2018-09-19 17:15:30 -04:00
monojenkins 2a9ad9b4d4 [mono-api-html] Fix rendering of nuint. (#10665) 2018-09-19 12:17:41 -04:00
monojenkins 1a309a7b45 Implement IEnumerable for ConditionalWeakTable (#10656) 2018-09-19 12:16:41 -04:00
monojenkins 49b5001dfd [2018-08] [interp] add safepoints and enable hybrid suspend as default (#10685)
* [interp] add safepoints and enable hybrid suspend as default

* [interp] consolidate checkpoint and safepoint into one opcode
2018-09-19 12:14:29 -04:00
Zoltan Varga 2bceefe4e6 [sdb] Avoid calling into the debuggee from VirtualMachine.GetObject () while holding the objects_lock. 2018-09-19 10:01:47 +02:00
monojenkins 56bed71f11 Merge pull request #10621 from lewurm/2018-08-interp-remove-coop-attach-insn
[2018-08] [interp] move thread attach to interpreter entry

Backport of https://github.com/mono/mono/pull/10579
2018-09-18 05:45:11 -04:00
Bernhard Urban 62412a804c [ci] disable System.Core-xunit for interp, does not work on this branch yet 2018-09-17 20:12:29 +02:00
Vlad Brezae ef105562be [interp] Fix missing InvalidProgramException (#10095)
* [interp] Throw proper exception instead of warning

Fix code causing wrong stack state.

* [interp] Compile method for errors when needed

Delegate creation doesn't require compiling the method.

* [interp] Enable System.Core xunit tests
2018-09-17 14:44:46 +02:00
monojenkins 2510fd9253 [2018-08] Fix MonoTests.System.Linq.ParallelEnumerableTests.TestGroupBy (Linux/amd64/FullAOT/LLVM). (#10633)
mono_aot_get_unbox_trampoline: amd64 hardcode unbox trampoline size to 5 at compile-time.
This fixes: MonoTests.System.Linq.ParallelEnumerableTests.TestGroupBy frequently crashing on FullAOT/LLVM/Linux/amd64 CI.

https://github.com/mono/mono/issues/10441
https://github.com/mono/mono/issues/9554
https://github.com/mono/mono/issues/9046

The actual size varies between 6 and 9 but we can just pretend the last instruction has size 1 (it is 2 or 5; calling it 5 can be fatal, calling it 2 is no better than 1).

Otherwise, a trampoline can overlap the next function, jit info lookup of the next function can fail, then gsharedvt wrapper omited, and crash.

The afflicted function in this case was
System_Linq_Parallel_WrapperEqualityComparer_1_T_GSHAREDVT_GetHashCode_System_Linq_Parallel_Wrapper_1_T_GSHAREDVT.

You can see it with debugprint: https://github.com/mono/mono/pull/10582.

There are many other possible fixes.
Such as runtime disassembly, hardcode in runtime instead of compiler, produce static trampinfo, omit trampinfo entirely, compute minimum size.

See https://github.com/mono/mono/commit/d589f3b64fca77c5c8bb5a8582eea5c0670312d7.
2018-09-16 13:10:49 -07:00
Marek Safar b76a034c12 [2018-06][Facades] Removes duplicate entry 2018-09-14 16:48:29 +02:00
Bernhard Urban d4f52d1116 [interp/jit] fix memory leak in pinvoke wrapper allocation 2018-09-14 15:32:26 +02:00
Bernhard Urban c3f4e28057 [interp] fix memory corruption around pinvoke wrapper allocation 2018-09-14 15:32:20 +02:00
Bernhard Urban b402c5bcf3 [interp] do not propagate uninitialized ThreadContext to JIT TLS 2018-09-14 15:31:28 +02:00
Bernhard Urban 4504185ee9 [interp] move thread attach to interpreter entry
We want to attach a thread before any TLS operation is done by the
interpreter. In the JIT this happens inside the native-to-managed
wrapper via `mono_threads_attach_coop ()`. Unfortunately, this is too
late for the interpreter as it already requires a working TLS before
even calling into the wrapper.

Also, fixup naming of locals in the native-to-managed wrapper.
2018-09-14 15:31:25 +02:00
monojenkins 63de341bb8 [arm64] Fix the alignment of valuetypes passed after small primitive types on IOS. (#10590)
Fixes https://github.com/xamarin/xamarin-macios/issues/4347.
2018-09-14 08:23:31 -04:00
monojenkins 9946fef5a3 [runtime] Fix Marshal.SizeOf (typeof (void)). (#10600)
Fixes https://github.com/mono/mono/issues/10303.
2018-09-14 08:21:34 -04:00
monojenkins 2e3c7693c9 [2018-08] Drop JIT attach: use coop attach in mini, aot and interp (#10551)
* [metadata] Hack - get mono_use_interpreter from metadata

Don't use this, it's a mistake.  The granularity is all wrong, for one: in the
future the interpreter may not be an all-or-nothing deal.

* [marshal] Don't emit CEE_MONO_JIT_ATTACH except for interpreter

Always use the mono_threads_coop_attach/mono_threads_coop_detach pair.

* [aot] Drop MONO_PATCH_INFO_JIT_THREAD_ATTACH

We always emit the mono_threads_attach_coop icalls now.

Bump MONO_AOT_FILE_VERSION

* [mini] mark mono_jit_thread_attach external only

It's used by Xamarin.Mac and possibly other embedders.

Interpreter still used mono_jit_thread_attach_interp until it's fixed to work
with coop.

There shouldn't be any other callers.

* [interp] Add a thread attach EE callback

Use it to set the interpreter thread context on the JIT tls value

* [marshal] Always emit mono_threads_attach_coop instead of CEE_MONO_JIT_ATTACH

Use the coop icall for the interpreter, too.

* [interp] Drop MINT_MONO_JIT_ATTACH and detach opcodes

* [metadata] Drop CEE_MONO_JIT_ATTACH and detach opcodes

* Revert "[metadata] Hack - get mono_use_interpreter from metadata"

This reverts commit c3d2f488b824dcdfa261f8292013c805740f353f.

* [mini] Drop unused mini_jit_thread_attach_interp

It was a temporary workaround for the interpreter; no longer needed.

* [coop] Switch to GC Safe in mono_jit_thread_attach.

It's an external-only function that's not called by the runtime on any
threads.  So the only calls are explicit uses by Mono API clients.  Leave the
thread in GC Safe mode and leave it to the n2m invoke wrappers or MONO_API
entry points to switch to GC Unsafe.

* [interp] Add MINT_MONO_THREADS_ATTACH_COOP opcode

Because this icall can be called from detached threads, don't go through
do_icall () which needs initialized tls keys

* fixup for c++ build

* fixup indentation and comments

* fixup C++ a bit more

* [mini] Bump MONO_EE_API_VERSION

* Revert "[interp] Add a thread attach EE callback"

This reverts commit 8a343d6ce15f0475a62936f28264f638ebf0344f.

* [interp] sync jittls context before entering EH machinery

* [interp] get context refactor (#10561)

* [interp] Add declaration for set_context (#10575)
2018-09-13 10:00:35 +02:00
monojenkins 228cab7601 [csproj] Update project files 2018-09-13 09:37:40 +02:00
monojenkins c12aa07fd7 Bump API snapshot submodule 2018-09-13 09:37:40 +02:00
Marek Safar ee57ffc058 [Facades] Adds System.Buffers mobile facade
and sets up facades without corresponding nuget _._ for conflict
resolution done by msbuild
2018-09-13 09:37:40 +02:00
monojenkins 7298849995 Treat EXDEV from clonefile the same way as ENOTSUP. (#10529)
Backport of #10528.

/cc @akoeplinger
2018-09-12 13:56:39 -04:00
Zoltan Varga eac547723c [arm64] Add dyncall support for ArgVtypeByRefOnStack arguments.
Fixes https://github.com/mono/mono/issues/10483.
2018-09-11 08:56:42 +02:00
Johan Lorensson 947193b9e2 Fix Mono Windows cross compiler using mono LLVM release_60 branch. (#10413) (#10503)
* Fix Mono Windows cross compiler using mono LLVM release_60 branch.

For scenario where llvm-config.exe can't be run (none WSL/CygWin build target)
Build will fallback to hard coded libraries not working with mono LLVM release_60
branch.

Added an exact mirror of what llvm-config.exe is returning for different components
on mono LLVM master and mono LLVM release_60 branch.

The selected codegen libraries (passed in as extra_libs) was not handled in any cases
so added them into the build script as well.

* Split parameter between lvm codegen libs and llvm extra libs.

Need to explicit pass -lz in case where configure have detected zlib availability
and llvm-config.exe can't be executed. In cases where zlib is not available, default
in mingw installs, this check make sure link step won't fail due to missing library.

* Fix inversion of aarch64codegen libs.

* Enable package-android-cross-{arm,arm64,x86,x86_64}-win builds on CI.
2018-09-07 16:38:40 -04:00
monojenkins 522cd7ce2a [corlib] Don't decrement ConditionalWeakTable::size on remove. (#10494)
We can't decrement the size on removal as we use tombstones.
When using tombstones, the invariant to keep is on the number of null entries as those
are used as search stop-marks.

Without this change, it's possible that a long series of add/remove operations could leave the array with no null entries,
just tombstones and filled entries. In that case, search would degenerate into a linear search that would look into all array elements.

This change fixes worst-case perf of not-found searches to be bound to the load factor.

Additionally, clear the array to null entries, otherwise we fall into the above trap even more easily.
2018-09-07 16:37:38 -04:00
Alexander Köplinger 1bdb478a71 [ci] Fix run-test-mac-sdk.sh build
0dbcdd5 added CFLAGS etc in front of the command but that's not something the TESTCMD can execute.
We shouldn't pass CFLAGS etc at all for this since that is handled by bockbuild.
2018-09-07 17:22:58 +02:00
monojenkins 1518670974 Fix DeflateStream handling of zero-length reads from underlying stream. (#10493) 2018-09-07 12:08:16 +02:00
monojenkins b5f8508f6f [llvmonly] Fix a regression caused by 55950a0608. (#9976) 2018-09-06 18:56:27 -04:00
monojenkins ea17477aac [2018-08] Ensure the finalization in DeflateStream is performed in the correct order. (#10492)
* Ensure the finalization of SafeHandle and GCHandle in DeflateStreamNative is performed in the correct order.

* Do not access BaseStream from finalizer.
2018-09-06 18:54:18 -04:00
monojenkins ed5b463839 [interp] white list nfloat::IsNaN () (#10385) 2018-09-06 18:49:18 -04:00
monojenkins 60f295e675 [2018-08] [exceptions] ftnptr_eh_callback test that stresses n2m transitions (#10387)
* [exceptions] ftnptr_eh_callback test that stresses n2m transitions

on Xamarin.iOS it happens that the ftnptr_eh_callback calls into managed
code, that is yet another native-to-managed transition.

* [coop] Use unbalanced transition to GC Unsafe in mono_raise_exception

Since `mono_raise_exception_deprecated` never returns, the transition won't be
balanced.  This fixes a cookie-counting problem (assert on mono shutdown) in
coop and hybrid checked builds using the new test case:

    MONO_THREADS_SUSPEND=coop MONO_CHECK_MODE=thread,gc mono mono/tests/install_eh_callback.exe
2018-09-06 18:48:58 -04:00
monojenkins 0ab763464d [2018-08] Truncate MachineName (and UserDomainName) at first period (#10418)
Backport of #9843.

* Truncate MachineName (and UserDomainName) at first period

Unix machines often put their FQDN as the host name, and domain
name is used for things such as yp. As such, truncate the FQDN to
just the hostname, as other Unix utilities seem to do.

Matches the .NET Core and .NET Framework behaviour, and MS seems to
require this for a future project. Fixes #9839.

* Add a unit test for bug

* Formatting tweaks
2018-09-04 13:20:20 +02:00
monojenkins 2ef3349e76 Update framework assemblies list in assembly.c (#10233)
Backport of #10214.
Fixes https://github.com/mono/mono/issues/9973
2018-09-04 13:16:39 +02:00
monojenkins bf580871f2 [2018-08] [corlib] Bring back String.Concat with __arglist (#10459)
Backport of #10452.
Fixes #9996
Tested repro project from the link on a local PostgresSQL server and EntityFramework packages.

See https://github.com/aspnet/EntityFramework6/blob/master/src/EntityFramework/Core/Objects/ELinq/MethodCallTranslator.cs#L1482-L1497
2018-09-04 13:14:38 +02:00
Ludovic Henry c19ca0ded5 [2018-08] [sdks] Add cross-{arm,arm64,x86,x86_64}{,-win} runtimes for XA + Unify provisioning of LLVM for both Android and iOS. (#10427)
* [sdks] Add CrossRuntimeTemplate

* [sdks] Add cross-{arm,arm64,x86,x86_64}{,-win} runtimes for XA

* [sdks] Factor various variables for RuntimeTemplate

* [sdks] Use CrossRuntimeTemplate for iOS

* [sdks] Unify provisioning of LLVM for both Android and iOS.

A global CFLAGS would be set in `run-jenkins.sh`, and that would screw up with LLVM. The global CFLAGS makes sense as long as we are only building Mono, but that's not the case with LLVM anymore.

* [sdks] Use MonoRuntime template for desktop-x86_64

* [sdks] Remove android-armeabi, it's been removed from XA already and the target has been deprecated in the NDK
2018-09-02 07:05:51 -04:00
monojenkins 51ea240a35 [sdb] Disable an assert to avoid breaking debugging. (#9983)
Fixes https://github.com/mono/mono/issues/8848.
Fixes https://github.com/mono/mono/issues/9262.
Fixes https://github.com/mono/mono/issues/8627.
2018-09-01 09:54:27 -04:00
monojenkins 3e238c836c [2018-08] Use clonefile instead of copyfile on macOS. (#10119)
* Use clonefile instead of copyfile on macOS.

* Address PR review comments.

* Address PR review comments (if -> assert).

* RTLD_DEFAULT - > libc_handle
2018-09-01 09:54:07 -04:00
monojenkins 802091af08 [coop] Fixup dump_threads () legend to match thread state enum values (#10386) 2018-09-01 09:52:41 -04:00
monojenkins 4dbe2ced4e [sgen] Fix string size inconsistency between alloc/scan (#10408)
This could lead to crashes when having strings with size at the LOS boundary.

https://github.com/mono/mono/issues/10309
2018-09-01 09:52:21 -04:00
monojenkins 2d50ef119b [MacSDK] Go back to incluging opt/llc after change in LLVM install process (#10433)
Fixes https://github.com/mono/mono/issues/10243
2018-09-01 10:27:23 +03:00
monojenkins eae59de800 [System]Fix MonoBtlsProvider.CheckValidationResult () logic. (#10397)
Backport of #10396.
2018-08-31 17:12:29 +02:00
monojenkins ad0854917e [System]Make `MobileAuthenticatedStream's read buffer large enough to hold a full TLS record. (#10394) 2018-08-31 16:56:50 +02:00
monojenkins 0e91bc8dd9 Merge pull request #10346 from monojenkins/backport-pr-10224-to-2018-08
[2018-08] [interp] fix ftnptr_eh_callback

Backport of #10224.

/cc @lewurm
2018-08-30 20:05:03 -04:00
monojenkins 6d2a0eb4aa [2018-08] [mini] fix memory dumping on devices (#10349)
[2018-08] [mini] fix memory dumping on devices

Backport of #10316.

/cc @lewurm
2018-08-30 09:40:23 -04:00
Johan Lorensson 80ade74200 Merge pull request #10358 from monojenkins/backport-pr-10321-to-2018-08
[2018-08] Fix Windows LLVM mono cross compiler build.
2018-08-30 13:36:53 +02:00
Alexander Köplinger 9d6252eb65 Follow up to bump target requirement to OSX 10.9 Mavericks
After another grep with different search terms I found a few more matches.
2018-08-30 12:44:44 +03:00
Alexander Köplinger b5f8d09d91 Bump target requirement to OSX 10.9 Mavericks
Fixes #9581
2018-08-30 12:44:44 +03:00
Ludovic Henry a2649d5d40 [sdks] Allow for overwriting llvm-{llvm32,llvm64} CC, CXX, CFLAGS and CXXFLAGS (#10362) 2018-08-29 22:12:39 -04:00
lateralusX 44a01472b1 Fix Windows LLVM mono cross compiler build.
This commit re-enables Windows LLVM mono cross compiler build + adjust to
new changes done in mono LLVM build system.
2018-08-29 13:28:07 +00:00
monojenkins 75f66645c4 [2018-08] [System]: MonoTlsStream is now IDisposable and cleans up on error. (#10335)
Backport of #10254.

/cc @marek-safar
2018-08-29 14:01:25 +02:00
monojenkins 756f617d2b [2018-08] Disable tailcall of gsharedvt. (#10324)
Backport of #10323.

/cc @marek-safar
2018-08-29 13:06:02 +02:00
Bernhard Urban 9256a02e8d [interp] turn off optimizations for callers of INTERP_PUSH_LMF_WITH_CTX 2018-08-29 09:03:28 +00:00
Bernhard Urban 207b7b6525 [interp] attach LMF with MonoContext in interp2native transition
this is needed so that EH can resume into the interpreter when the
exception was raised via mono_raise_exception from native code.
2018-08-29 09:03:28 +00:00
Bernhard Urban 10b2571905 [exceptions] pop lmf stack before executing ftnptr_eh_callback 2018-08-29 09:03:28 +00:00
Filip Navara e8415ffe23 Reintroduce fix for TextBoxTest.Bug82749. We need to save the user requested control bounds outside of ExplicitBounds since ExplicitBounds are used by table layout engine. 2018-08-27 21:32:04 +02:00
Filip Navara 805dfec75d Add test case. 2018-08-27 21:32:04 +02:00
Filip Navara b7635aed96 Fix changing TextBox font before the Control handle has been created. 2018-08-27 21:32:04 +02:00
Filip Navara f99f8aeacd Fix TextBox height fixup to work properly inside TableLayout. 2018-08-27 21:32:04 +02:00
Filip Navara 2a71a71c7b Fix layout of scrollbars in combobox. 2018-08-27 21:32:04 +02:00
Zoltan Varga eedc49b402 [llvm] Emit volatile loads in emit_volatile_load (), otherwise llvm might optimize the loads away.
Fixes https://github.com/mono/mono/issues/10035.
2018-08-27 14:20:18 +02:00
monojenkins 4151ad78f1 [llvm] Use curl as fallback for wget (#10265) 2018-08-24 11:38:23 +03:00
Marek Habersack 056528177e [eglib] Fix URI to file name conversion
Hexadecimal ASCII to decimal integer conversion must be 10-based.
2018-08-24 09:22:12 +02:00
Alexander Kyte c1910c80f4 Merge pull request #10241 from alexanderkyte/2018-08-merp
[2018-08] Add pipeline to convert managed exception into telemetry format
2018-08-23 23:26:10 -04:00
Aleksey Kliger 1054aba62a [mini] Don't regen Makefile.am if llvm/llvm_config.mk changes
The llvm/llvm_config.mk makefile fragment just includes make (not automake)
variable definitions.  So there is no need to regenerate the mini/Makefile.am
file if the fragment updates - the include will pick it up.
2018-08-23 17:47:03 +02:00
monojenkins d0fed2af14 [sdks] Use separate temporary dirs for llvm36/llvm. (#9975) 2018-08-23 15:59:05 +03:00
Alexander Kyte 85a7f7aca1 [runtime] Add interpreter support for walking Exception stacks 2018-08-22 15:05:00 -04:00
Alexander Kyte 6a783bb78c [runtime] Stub out exception summarizer when crash reporting is disabled 2018-08-22 15:04:31 -04:00
Alexander Kyte dd97b32959 [runtime] Consistently zero summarizer state memory 2018-08-22 15:04:22 -04:00
Alexander Kyte 67ed6f3152 [runtime] Create more ergonomic API for Exc to Telemetry 2018-08-22 15:04:11 -04:00
Alexander Kyte 22738e7d30 [runtime] Add test for managed telemetry reporting
To test the proprietary bits, you must first have a copy of MERP
installed somewhere.

Then run the following test:

using System;
using System.Reflection;

public class MinimalTelemetry {

	static MinimalTelemetry ()
	{
		var merp_path = "/xxx/xxx/xxx/Microsoft Error Reporting.app";
		var monoType = Type.GetType ("Mono.Runtime", false);
		var m = monoType.GetMethod("EnableMicrosoftTelemetry", BindingFlags.NonPublic | BindingFlags.Static);
		var m_params = new object[] {"com.xamarin.MinimalTelemetry", "Test.Xam.MinimalTelemetry", "1337001", merp_path, "AppleAppCrash", "/Library/Frameworks/Mono.framework/Versions/Current/bin/mono"};
		m.Invoke(null, m_params);

		AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(HandleException);
	}

	public static void HandleException (object sender, UnhandledExceptionEventArgs e)
	{
		var exc = e.ExceptionObject as Exception;

		var monoType = Type.GetType ("Mono.Runtime", false);
		var send = monoType.GetMethod("SendExceptionToTelemetry", BindingFlags.NonPublic | BindingFlags.Static);
		var send_params = new object[] {exc};
		send.Invoke(null, send_params);

		// Now the MERP GUI should pop up
		// Click "More Information"
		// You should see something like this:
		// {
		//  "payload" : {
		//  "protocol_version" : "0.0.1",
		//  "configuration" : {
		//    "version" : "5.19.0 (managed_telemetry_pipeline/d342c73e320 Wed Aug 15 14:40:40 EDT 2018)",
		//    "tlc" : "normal",
		//    "sigsgev" : "altstack",
		//    "notifications" : "kqueue",
		//    "architecture" : "amd64",
		//    "disabled_features" : "none",
		//    "smallconfig" : "disabled",
		//    "bigarrays" : "disabled",
		//    "softdebug" : "enabled",
		//    "interpreter" : "enabled",
		//    "llvm_support" : "disabled",
		//    "suspend" : "hybrid"
		//  },
		//  "memory" : {
		//    "Resident Size" : "16306176",
		//    "Virtual Size" : "4491988992",
		//    "minor_gc_time" : "0",
		//    "major_gc_time" : "0",
		//    "minor_gc_count" : "0",
		//    "major_gc_count" : "0",
		//    "major_gc_time_concurrent" : "0"
		//  },
		//  "threads" : [
		//    {
		//      "is_managed" : false,
		//      "managed_thread_ptr" : "0x0",
		//      "thread_info_addr" : "0x0",
		//      "native_thread_id" : "0x0",
		//      "managed_frames" : [
		//        {
		//          "is_managed" : "true",
		//          "guid" : "4F72FB1B-1E99-47BA-91CE-9515783DC174",
		//          "token" : "0x6000004",
		//          "native_offset" : "0x6a",
		//          "il_offset" : "0x00000"
		//        }
		//      ]
		//    }
		//  ]},
		//  "parameters" : {
		//    "ApplicationBundleId:" : "Test.Xam.MinimalTelemetry",
		//    "ApplicationVersion:" : "1337001",
		//    "ApplicationBitness:" : "x64",
		//    "ApplicationName:" : "com.xamarin.MinimalTelemetry",
		//    "BlameModuleName:" : "Mono Exception",
		//    "BlameModuleVersion:" : "5.19.0 (managed_telemetry_pipeline/d342c73e320 Wed Aug 15 14:40:40 EDT 2018)",
		//    "BlameModuleOffset:" : "0x0",
		//    "ExceptionType:" : "0x02000000",
		//    "StackChecksum:" : "0x60114aa",
		//    "StackHash:" : "0x6011514",
		//    "OSVersion:" : "17.4.0",
		//    "LanguageID:" : "0x7f",
		//    "SystemManufacturer:" : "apple",
		//    "SystemModel:" : "MacBookPro14,3",
		//    "EventType:" : "AppleAppCrash"
		//    }
		//  }

	}

	public static void Main (string[] args)
	{
		//AppDomain.CurrentDomain.LoadFrom ("something that's not a valid path")
		throw new Exception ("MinimalTelemetrybar MinimalTelemetrybar");
	}
}
2018-08-22 15:04:00 -04:00
Alexander Kyte ffae0deeba [runtime] Zero out stack memory for mono-state dumper 2018-08-22 15:03:45 -04:00
Alexander Kyte b072bc6749 [runtime] Add initial support for Exc -> Telemetry
In order to allow managed crash reporting using the existing telemetry
format, I have added initial support for converting a MonoException into
a json blob for telemetry.
2018-08-22 15:03:28 -04:00
Alexander Kyte 82ac95b94e [runtime] Expose merp sender to icall 2018-08-22 14:25:53 -04:00
monojenkins faac90015c An implementation of the ChainStatus property for Btls. (#10215)
When we execute the certificate verification callback we just don't test for success and set status11. Now we get the error (if any) and map the error code to those defined by ChainStatusFlags and add it to a list of X509ChainStatus objects for that chain. Now when get_ChainStatus is called we convert that list to an array of X509ChainStatus and return it.
2018-08-22 11:45:37 +02:00
monojenkins 4368bb5d0a [2018-08] [hybrid-suspend] Two phase STW (#10221)
* [threads] Add enums for suspend phases and for begin-suspend results

* [coop] Two phase suspend

If we're using hybrid suspend, initially request suspension for threads, but
don't preemptively suspend blocking threads.  In the mop-up phase, peek at the
state and if they are still blocking_suspend_requested, preemptively suspend
them.

In full coop and in full preemptive, just use a single phase.

This should fix hybrid suspension deadlocks due to preemptively-suspending
threads taking locks that prevent cooperatively-suspending threads from
reaching a safepoint by leaving all threads running until all
cooperatively-suspending threads come to a safepoint. At that point we can
preemptively suspend the remaining threads.

Fixes https://github.com/mono/mono/issues/9921

* [tests] renable flaky hybrid suspend tests on PRs

* [coop] Thread state machine helper peek_blocking_suspend_requested
2018-08-22 11:13:20 +02:00
monojenkins 101736c438 [2018-08] [coop] Check async suspend status in ReqSuspendInitSuspendBlocking case (#10162)
* [coop] Check async suspend status in ReqSuspendInitSuspendBlocking case

So the issue is that calling begin_suspend_for_blocking_thread under hybrid
suspend will perform a preemptive suspension of the victim thread.

The preemptive suspend doesn't always succeed.
In particular, thread_state_init_from_sigctx or thread_state_init_from_handle
could return FALSE and into->suspend_can_continue will be set to FALSE.
When the thread_state_init_* functions return FALSE, they also set the
MonoThreadUnwindState:valid to FALSE.

So it is important to check the value of MonoThreadInfo:suspend_can_continue
before proceeding with a suspension.

As a result in suspend_sync in the ReqSuspendInitSuspendRunning case we call
check_async_suspend which checks (for preemptively suspended threads) whethere
suspend_can_continue is true.  If it isn't, we resume the victim thread and
loop once more in suspend_sync_nolock and try again.

We didn't have a check_async_suspend call in the ReqSuspendInitSuspendBlocking
case, so a thread could be considered suspended even if we couldn't capture a
valid MonoThreadUnwindState for it.

That breaks the assertion the call to mono_thread_info_get_last_managed in
async_suspend_critical which needs a valid MonoThreadUnwindState.

Fixes https://github.com/mono/mono/issues/10031

* fixup - remove coop assertion in check_async_suspend

result for coop could be iether BeginSuspendOkCooperative or
BeginSuspendOkNoWait depending on whether the thread was running (Ok) or
blocking (OkNoWait)

* [coop] suspend_can_continue if async raced with self-suspend (osx + win32)

Companion to 006d6ce097 from earlier this year
which applied this fix for POSIX signal based suspend.  This is the
corresponding OSX and Win32 fix.

In hybrid suspend, in the case where a suspend initiator needs to preemptively
suspend a thread, but the thread self-suspended while after the suspend
initiator put it into the blocking_suspend_requested state, we want to allow
the suspend to continue.  We do have to call the syscall to undo the native
async suspend, but after that, we can leave the victim to continue waiting for
a resume.

This should not have any effect on full cooperative or full preemptive suspend
- in the full coop case we never call async suspend, in the full preemptive
case the thread never self-suspends in blocking.
2018-08-21 14:14:41 -04:00
monojenkins bd20434769 Properly inflate pointer types in inflate_generic_type. Generic pointer types are encountered with 'unmanaged' constraint. (#10195)
Backport of #10145.
2018-08-21 00:57:56 +02:00
monojenkins 84a848a68f [2018-08] Enable mono-state-based telemetry for crashing on CI (#9963)
[2018-08] Enable mono-state-based telemetry for crashing on CI

Backport of #8654.

/cc @marek-safar
2018-08-20 18:55:52 +02:00
Alexander Köplinger 91eaa0832c Bump nunitlite to get NUnit2 xml output fix and failure on file not found fix (#10188)
See mono/NUnitLite#11 and mono/NUnitLite#12

(cherry picked from commit 8f67fb9fb1)
2018-08-20 16:56:49 +02:00
monojenkins 94bb56808f Fix process leak in process enumeration. Dispose rejected ones. (#10186)
This https://github.com/mono/mono/issues/10143 and fix is from there.

I considered nulling the array elements as I go, and then the tail,
to help the GC, but does it help actually? I suspect not.
2018-08-20 14:39:04 +02:00
monojenkins f236586422 [System.Xml.Linq] Fix namespace conflict with new Xamarin.Mac namespace in test code. (#10182)
macOS 10.14 introduces a Network framework, which means Xamarin.Mac now has a Network namespace.

This conflicts with 'using Network = System.Net':

> mono/mcs/class/System.Xml.Linq/Test/System.Xml.Schema/ExtensionsTest.cs(97,27): error CS0576: Namespace '<global namespace>' contains a definition conflicting with alias 'Network'

so just remove the using and use the full 'System.Net' namespace instead.
2018-08-20 12:08:54 +02:00
monojenkins 2a490e383b [sdks] Fix local testing for Android (#10176) 2018-08-19 12:11:19 -04:00
monojenkins 59277c72b7 [2018-08] [reflection] Assembly.LoadFile should throw ArgumentException for relative paths (#10082)
* [reflection] Assembly.LoadFile should throw ArgumentException for relative paths

Fixes https://github.com/mono/mono/issues/9943

* [mcs/tests] Use full path in the compiler-tester

Otherwise the call to LoadFile will fail

* [mono/tests] Use full path in LoadFile in test-multi-netmodule-4-exe

* [mono/tests] Use the test runner for a couple of tests

* [fullaot] Disable test-multi-netmodule-4-exe and custom-attr-errors.exe

* [tests] Use Path.GetFullPath and Path.Combine in AssemblyTest

Fix windows test failures
2018-08-17 09:27:05 -04:00
monojenkins d0e57fbac4 [sdks] Fix typo in ios template (#10131)
[2018-08] [sdks] Fix typo in ios template

Backport of #10130.

/cc @lambdageek
2018-08-16 14:10:17 +02:00
monojenkins 226c6a8b54 [cxx] One more MONO_API ahead of MONO_RT_EXTERNAL_ONLY. (#9965) 2018-08-15 10:39:52 -07:00
monojenkins b11c4232fd [2018-08] [cxx] MONO_API before MONO_RT_EXTERNAL_ONLY per 58efe94192. (#9966)
* [cxx] MONO_API before MONO_RT_EXTERNAL_ONLY per 58efe94192.

* [cxx] MONO_API before MONO_RT_EXTERNAL_ONLY per 58efe94192.
2018-08-15 09:37:50 -07:00
monojenkins 29d91bd6cf [2018-08] [reflection] Implement RuntimeType.IsSubclassOf in native (#10073)
* [reflection] Implement RuntimeType.IsSubclassOf in native

Fixes a performance regression https://github.com/mono/mono/issues/10000

* fixup - byref and reflexive IsSubclassOf
2018-08-15 18:36:37 +02:00
monojenkins 3e8a282bc3 [bcl] Disable a test which fails with linking. (#10107)
Fixes https://github.com/xamarin/xamarin-macios/issues/4617.
2018-08-15 01:08:55 +02:00
Alexander Köplinger ba38f97879 Add .gitignore for mono/tests/assembly-load-reference 2018-08-14 13:51:28 +02:00
monojenkins 64e9b1a983 [mini] Don't assert on MONO_PATCH_INFO_GC_SAFE_POINT_FLAG if coop is disabled (#9979)
If an AOT library is loaded that was compiled with safepoints, but currently
we're running with preemptive suspend (e.g. MONO_THREADS_SUSPEND=preemptive is
set), don't assert.  `mono_polling_required` will always be false, so the
safepoint will be ignored.

Fixes https://github.com/mono/mono/issues/9947
2018-08-13 16:23:55 +02:00
monojenkins 2e16537134 [ci] Disable sgen-new-threads-collect.exe until #9959 is addressed (#10029)
The test times out due to an issue in hyrbid suspend.  Turn off the test
temporarily while we work on a solution.

https://github.com/mono/mono/issues/9959
https://github.com/mono/mono/issues/9921
2018-08-10 22:35:12 +02:00
monojenkins 6a844e7677 [w32file] Fix typo (#10003) 2018-08-09 18:01:20 -04:00
monojenkins cc3d5f0374 [coop] Thread.JoinInternal should not call mono_error_set_pending_exception (#9971)
[2018-08] [coop] Thread.JoinInternal should not call mono_error_set_pending_exception

Backport of #9948.

/cc @lambdageek
2018-08-09 19:25:19 +02:00
monojenkins 2245249f6d [2018-08] [Facades] Adds System.Threading.Tasks.Extensions for mobile profiles (#9978)
* [Facades] Adds System.Threading.Tasks.Extensions for mobile profile

* Bump API snapshot submodule

* [csproj] Update project files
2018-08-09 00:20:09 +02:00
Alexander Köplinger f123d67e04 Merge pull request #9923 from monojenkins/backport-pr-9858-to-2018-08
[2018-08] [loader] Determine assembly load context from path
2018-08-08 11:45:30 +02:00
monojenkins d551bcc1f2 Remove extern "C" from MONO_LLVM_INTERNAL. Consider also putting it at start of prototype instead of end, like MONO_API. (#9957) 2018-08-08 11:40:44 +02:00
monojenkins b606864198 [2018-08] [threads] switch to GC Safe around pthread calls in mono_thread_internal_set_priority (#9917)
* [threads] switch to GC Safe around pthread calls in mono_thread_internal_set_priority

   Observed pthread_getschedparam taking a lock on OSX

* [win32] Capture GetLastError before MONO_EXIT_GC_SAFE
2018-08-07 21:46:02 -04:00
monojenkins 0242e31892 Add missing return from the function. (#9932)
Was accidentally removed in 3780114620.
2018-08-07 12:54:05 +02:00
monojenkins 5de08e1822 [sdks] Rename LLVM_VERSION make var to LLVM_RELEASE, LLVM_VERSION is already used by versions.mk. (#9933) 2018-08-07 12:53:47 +02:00
Aleksey Kliger 55ff5b426f [tests] AOT assembly-load-reference tests 2018-08-06 23:03:42 +00:00
Aleksey Kliger 9c01d63b45 [tests] Test loading references from LoadFrom and LoadFile contexts
The LoadFrom assembly loading context and the individual assembly loading
context (aka LoadFile aka "no context") behave differently when requesting a
referenced assembly - for LoadFile we expect the runtime to probe the appdomain
base, the GAC + MONO_PATH but not the basedir of the requesting assembly.  For
LoadFrom the basedir of the requesting assembly _is_ included.  So the tests
here try all the possible variations.
2018-08-06 23:03:42 +00:00
Aleksey Kliger 031968ad2e [loader] Use appdomain base dir and loadfrom asmctx hooks
1. If an assembly filename is in the GAC, always open the assembly in default
context.
2. If an assembly filename is in the appdomain search path (base directory or
private bin path) always open the assembly in default context.
3. If an assembly filename is in the base dir of the requesting assembly and
the requesting assembly was in LoadFrom context, always open the assembly in
LoadFrom context.

Fixes https://github.com/mono/mono/issues/9753
and Fixes https://github.com/mono/mono/issues/9542
2018-08-06 23:03:42 +00:00
Aleksey Kliger 079aff5529 [loader] Add mono_install_asmctx_from_path_hook
Add a hook that can be used to check whether a given path belongs to some
assembly load context.

For example if a given path is in the application base of the current
appdomain, an appdomain hook would set out_asmctx to MONO_ASMCTX_DEFAULT and
return TRUE.
2018-08-06 23:03:42 +00:00
Aleksey Kliger 721092cc06 [utils] Add mono_path_filename_in_basedir 2018-08-06 23:03:42 +00:00
Aleksey Kliger 9f65ed3311 [loader] Pass requesting assembly to mono_assembly_open_predicate
In LoadFrom and LoadFile icalls, pass the currently executing assembly as the
requesting assembly.
2018-08-06 23:03:42 +00:00
Aleksey Kliger cf4eb293da [loader] Assemblies in individual asmctx can reference assemblies in application base
If an assembly was loaded with LoadFile, it can reference assemblies from the
application base, the GAC or MONO_PATH.

Fixes https://github.com/mono/mono/issues/8575
2018-08-06 23:03:42 +00:00
Aleksey Kliger 1f44a5e5a0 [loader] Consult asmctx of executing assembly for Assembly.Load
if an assembly in LoadFrom context calls Assembly.Load, the given assembly name
should be tried in the basedir of the executing assembly, and the resulting
loaded assembly should be in LoadFrom context.
2018-08-06 23:03:42 +00:00
Aleksey Kliger 6f31677959 [runtime] Add mono_runtime_get_caller_no_system_or_reflection
Exposes an already existing stackwalk that we use to find out who called a
corlib API.
2018-08-06 23:03:42 +00:00
Jo Shields 42e523a21e Merge pull request #9905 from directhex/2018-08
Bump version to 5.18.0
2018-08-06 14:36:24 -04:00
Jo Shields 1c5f8bc400 Bump version to 5.18.0 2018-08-06 11:07:36 -04:00
6805 changed files with 292180 additions and 632956 deletions

No files matched your search

+4 -7
View File
@@ -1,6 +1,3 @@
# diff settings
*.cs diff=csharp
# ensure LF endings on all checkouts
configure.ac crlf=input
config.rpath crlf=input
@@ -16,11 +13,11 @@ mkinstalldirs crlf=input
*.h crlf
*.cs crlf
*.il crlf
*.sln crlf
*.*proj* crlf
# don't do anything to line-endings. Let CRLFs/LFs go into the repo, and CRLF/LFs on checkout
*.bat -crlf
# don't do anything to line-endings. Let CRLFs go into the repo, and CRLF on checkout
*.bat -crlf
*.sln -crlf
*.*proj* -crlf
*.xml -crlf
# CRLF Handling
-161
View File
@@ -1,161 +0,0 @@
# Mono Code Owners File
#
# GitHub uses this file to determine who to assign for reviews
# on pull requests. Please keep this file alphabetically sorted.
#
# References:
#
# https://github.com/blog/2392-introducing-code-owners
# https://help.github.com/articles/about-codeowners
# These owners will be the default owners for everything in the repo.
* @marek-safar
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger @kg
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/data @directhex @akoeplinger
/data/gdb @lewurm @vargaz
/data/lldb @lewurm @vargaz
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp @lewurm
/docs/sources/mono-api-profiler.html @BrzVlad
/external @akoeplinger @marek-safar
/external/api-snapshot @marek-safar @akoeplinger
/external/binary-reference-assemblies @marek-safar @akoeplinger
/external/bockbuild @akoeplinger @alexischr
/external/boringssl @baulig
/external/cecil @marek-safar @akoeplinger
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/linker @marek-safar
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
/libgc @BrzVlad @vargaz @naricc
/llvm @vargaz @SamMonoRT @imhameed
/man @marek-safar @miguel
/man/*prof* @BrzVlad
/mcs @marek-safar @akoeplinger @egorbo
/mcs/build @marek-safar @akoeplinger
/mcs/class @steveisok @akoeplinger
/mcs/class/corlib/System.Reflection*/ @vargaz @lambdageek
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/Mono.Btls.Interface @baulig @steveisok
/mcs/class/Mono.Data.Tds @egorbo
/mcs/class/Mono.Debugger.Soft @vargaz @thaystg
/mcs/class/Mono.Options @jonpryor @marek-safar
/mcs/class/Mono.Profiler.Log @lambdageek
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @steveisok
/mcs/class/System/Mono.AppleTls @baulig @rolf
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @steveisok
/mcs/class/System/Mono.Security.Interface @baulig @steveisok
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/System.Data* @egorbo @steveisok
/mcs/errors @marek-safar @lewurm
/mcs/ilasm @marek-safar @lewurm
/mcs/jay @marek-safar @lewurm
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar @lewurm
/mcs/tools @marek-safar @akoeplinger @steveisok
/mcs/tools/gensources @kg @akoeplinger
/mcs/tools/resx2sr @baulig
/mcs/tools/upload-to-sentry @directhex @akoeplinger
/mono/arch @vargaz @lewurm
/mono/btls @baulig @marek-safar
/mono/dis @lambdageek @vargaz
/mono/eglib @vargaz @lambdageek
/mono/benchmark @SamMonoRT @lewurm
/mono/metadata @vargaz @lambdageek @thaystg
/mono/metadata/*profiler* @lambdageek
/mono/metadata/handle* @lambdageek @vargaz
/mono/metadata/monitor* @brzvlad @vargaz
/mono/metadata/sgen* @brzvlad @vargaz @naricc
/mono/metadata/thread* @lateralusX @lambdageek
/mono/metadata/w32* @lateralusX @lambdageek
/mono/metadata/*-win* @lateralusX @lambdageek
/mono/metadata/appdomain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/assembly* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/domain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/loader* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lewurm @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz @BrzVlad
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
/mono/native @baulig @egorbo @alexischr @marek-safar
/mono/profiler @BrzVlad @lambdageek
/mono/sgen @BrzVlad @lambdageek
/mono/utils @vargaz @lewurm @lambdageek
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
/mono/utils/mono-mem* @lewurm @vargaz
/mono/utils/mono-merp* @alexischr @lambdageek
/mono/utils/mono-state* @alexischr @lambdageek
/mono/utils/mono-threads* @lambdageek @vargaz
/mono/utils/*-win* @lateralusX @lambdageek
/msvc @lateralusX @kg @akoeplinger
/msvc/scripts @akoeplinger @kg
/netcore @marek-safar @akoeplinger @egorbo @vargaz @steveisok
/packaging @alexischr @akoeplinger @directhex
/packaging/MacSDK/msbuild.py @radical @akoeplinger
/runtime @akoeplinger @marek-safar
/samples @lambdageek @lewurm
/samples/profiler @BrzVlad @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
/scripts/ci @akoeplinger @directhex
/sdks @marek-safar @lewurm @vargaz @lambdageek
/sdks/builds/android* @alexischr @jonpryor
/sdks/builds/mxe* @alexischr @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
/sdks/builds/wasm* @vargaz @kjpou1 @lewing
/sdks/wasm @vargaz @kjpou1 @kg @lewing
/sdks/ios @vargaz @rolfbjarne
/sdks/android @alexischr @jonpryor
/support @lambdageek @vargaz
/tools @lambdageek @vargaz
/tools/locale-builder @marek-safar @egorbo
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/sgen @brzvlad @vargaz @naricc
+1 -18
View File
@@ -24,6 +24,7 @@
*.tlh
*.bak
*.ilk
*.log
*.lib
*.sbr
*.sdf
@@ -88,7 +89,6 @@ stamp-h.in
*~
*.swp
*.o
*.dylib
.deps
.libs
@@ -128,24 +128,7 @@ docs/perlmod*
compile
mono.h
mono-*.tar.*
mono_crash*
tmpinst-dir.stamp
msvc/scripts/inputs/
extensions-config.h
*.dylib.dSYM
*.mlpd
llvm/build/
llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
tools/offsets-tool-py/offtool/
tools/offsets-tool-py/clang/__pycache__
##############################################################################
# Arcade output directories
##############################################################################
.packages
artifacts
.dotnet
+1 -13
View File
@@ -49,6 +49,7 @@
[submodule "external/corert"]
path = external/corert
url = git://github.com/mono/corert.git
branch = mono-2018-08
[submodule "external/xunit-binaries"]
path = external/xunit-binaries
url = git://github.com/mono/xunit-binaries.git
@@ -58,16 +59,3 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+8 -7
View File
@@ -3,8 +3,11 @@
{
"name": "Mac",
"includePath": [
"${default}",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1",
"/usr/local/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include",
"/usr/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
"${workspaceRoot}/mono/eglib",
@@ -14,8 +17,11 @@
"intelliSenseMode": "clang-x64",
"browse": {
"path": [
"${default}",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1",
"/usr/local/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include",
"/usr/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
"${workspaceRoot}/mono/eglib",
@@ -25,7 +31,6 @@
"databaseFilename": ""
},
"macFrameworkPath": [
"${default}",
"/System/Library/Frameworks",
"/Library/Frameworks"
]
@@ -33,7 +38,6 @@
{
"name": "Linux",
"includePath": [
"${default}",
"/usr/include",
"/usr/local/include",
"${workspaceRoot}",
@@ -45,7 +49,6 @@
"intelliSenseMode": "clang-x64",
"browse": {
"path": [
"${default}",
"/usr/include",
"/usr/local/include",
"${workspaceRoot}",
@@ -60,7 +63,6 @@
{
"name": "Win32",
"includePath": [
"${default}",
"C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
@@ -74,7 +76,6 @@
"intelliSenseMode": "msvc-x64",
"browse": {
"path": [
"${default}",
"C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/include/*",
"${workspaceRoot}",
"${workspaceRoot}/mono",
+136
View File
@@ -0,0 +1,136 @@
# Mono Code Owners File
#
# GitHub uses this file to determine who to assign for reviews
# on pull requests. Please keep this file alphabetically sorted.
#
# References:
#
# https://github.com/blog/2392-introducing-code-owners
# https://help.github.com/articles/about-codeowners
# These owners will be the default owners for everything in the repo.
* @luhenry @marek-safar
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger
/acceptance-tests/profiler-stress* @alexrp
/data @marek-safar @akoeplinger @luhenry
/data/gdb @vargaz
/data/lldb @lewurm @vargaz
/docs @luhenry @marek-safar
/docs/sources/mono-api-profiler.html @alexrp
/external @marek-safar @luhenry
/external/binary-reference-assemblies @marek-safar @akoeplinger
/external/bockbuild @akoeplinger @alexischr
/external/boringssl @baulig
/external/nuget-buildtasks @radical @akoeplinger
/libgc @BrzVlad @vargaz @luhenry
/llvm @vargaz @luhenry
/man @marek-safar @miguel @luhenry
/man/*prof* @alexrp
/mcs @marek-safar @luhenry
/mcs/build @marek-safar @akoeplinger
/mcs/class @marek-safar @akoeplinger
/mcs/class/corlib/System.Reflection*/ @vargaz @lambdageek
/mcs/class/Mono.Btls.Interface @baulig
/mcs/class/Mono.Data.Tds @egorbo
/mcs/class/Mono.Debugger.Soft @vargaz
/mcs/class/Mono.Options @jonpryor
/mcs/class/Mono.Profiler.Log @alexrp
/mcs/class/Mono.Security/Mono.Security/Interface @baulig
/mcs/class/System/Mono.AppleTls @baulig
/mcs/class/System/Mono.Btls @baulig
/mcs/class/System/Mono.Net.Security @baulig
/mcs/class/System/Mono.Security.Interface @baulig
/mcs/class/System.Data* @egorbo
/mcs/errors @marek-safar
/mcs/ilasm @marek-safar
/mcs/jay @marek-safar
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar
/mcs/tools @marek-safar @akoeplinger
/mono @luhenry
/mono/btls @baulig
/mono/dis @lambdageek @vargaz
/mono/eglib @kumpera @vargaz @luhenry
/mono/metadata @vargaz @kumpera
/mono/metadata/*verify* @kumpera
/mono/metadata/*profiler* @alexrp
/mono/metadata/monitor* @brzvlad
/mono/metadata/sgen* @brzvlad
/mono/metadata/threads* @luhenry @kumpera
/mono/metadata/threadpool* @luhenry
/mono/metadata/w32* @luhenry
/mono/metadata/*-win* @lateralusX @luhenry
/mono/mini @vargaz @kumpera
/mono/mini/*cfgdump* @lewurm
/mono/mini/*exceptions* @lewurm
/mono/mini/*ppc* @lewurm
/mono/mini/*type-check* @lewurm
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @alexrp
/mono/mini/debugger-agent.c @DavidKarlas
/mono/profiler @alexrp @kumpera
/mono/sgen @BrzVlad @kumpera
/mono/utils @kumpera @vargaz
/mono/utils/atomic* @alexrp
/mono/utils/mono-hwcap* @alexrp
/mono/utils/mono-mem* @alexrp
/mono/utils/mono-threads* @luhenry @kumpera
/mono/utils/*-win* @lateralusX @kumpera
/msvc @lateralusX @luhenry
/msvc/*profiler* @alexrp
/msvc/scripts @akoeplinger @kg
/packaging/ @alexischr @akoeplinger @luhenry @directhex
/packaging/MacSDK/msbuild.py @radical @akoeplinger
/runtime @akoeplinger @marek-safar @luhenry
/samples @luhenry
/samples/profiler @alexrp
/samples/size @alexrp
/scripts @akoeplinger @luhenry
/scripts/ci @akoeplinger @alexischr
/scripts/ci/run-test-profiler-stress-tests.sh @alexrp
/sdks @marek-safar @luhenry
/sdks/builds/android* @luhenry @jonpryor
/sdks/builds/mxe* @luhenry @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
/sdks/wasm @kumpera @lrz
/sdks/ios @vargaz @rolfbjarne
/sdks/android @luhenry @jonpryor
/support @luhenry @vargaz
/tools @kumpera @luhenry
/tools/checker @tritao
/tools/crash-bisector @kumpera
/tools/locale-builder @marek-safar
/tools/monograph @vargaz
/tools/nuget-hash-extractor @marek-safar
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @kumpera @lambdageek
/tools/sgen @brzvlad
+2
View File
@@ -28,6 +28,8 @@ Mono itself, and does not end up in the redistributable part of Mono:
* mcs/jay: 4-clause BSD licensed
* mcs/nunit24: MS-PL
* mcs/class/I18N/mklist.sh, tools/cvt.sh: GNU GPLv2
Runtime Code
+9 -33
View File
@@ -11,29 +11,15 @@ else
tools_dir = tools
endif
if ENABLE_MSVC
build_with_msvc = msvc
build_without_msvc =
else
build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
SUBDIRS = mk llvm mono runtime netcore
else
update_submodules = update_submodules
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
endif
SUBDIRS = mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) msvc $(docs_dir) acceptance-tests
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore netcore/corerun
DIST_SUBDIRS = m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools msvc docs acceptance-tests
all: $(update_submodules)
all: update_submodules
SUBMODULE_ERROR='Could not recursively update all git submodules. You may experience compilation problems if some submodules are out of date'
update_submodules:
@cd $(srcdir) && scripts/update_submodules.sh
@$(srcdir)/scripts/update_submodules.sh
.PHONY: update_submodules
@@ -46,10 +32,9 @@ EXTRA_DIST= \
winconfig.h \
code_of_conduct.md \
external \
netcore/Makefile \
mcs/class/referencesource
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_RUNTIME=false
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_MCS=false EXTERNAL_RUNTIME=false
# Distribute the 'mcs' tree too
GIT_DIR ?= $(srcdir)/.git
@@ -65,11 +50,8 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.Plo'`
rm -f `find $(top_distdir)/external -name '\.dirstamp'`
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*'`
rm -rf "$(top_distdir)/external/linker/linker/Tests"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
@@ -84,15 +66,9 @@ if BITCODE
BITCODE_CHECK=yes
endif
if DEFAULT_TESTS
CI_TEST_SCRIPT=$(srcdir)/scripts/ci/run-test-default.sh
else
CI_TEST_SCRIPT=$(srcdir)/scripts/ci/run-test-$(TEST_PROFILE).sh
endif
.PHONY: check-ci
check-ci:
MONO_LLVMONLY=$(BITCODE_CHECK) $(CI_TEST_SCRIPT)
MONO_LLVMONLY=$(BITCODE_CHECK) $(srcdir)/scripts/ci/run-test-$(TEST_PROFILE).sh
.PHONY: validate do-build-mono-mcs mcs-do-clean mcs-do-tests
validate: do-build-mono-mcs
-8
View File
@@ -1,8 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
+13 -19
View File
@@ -3,7 +3,7 @@ create cross platform applications. It is an open source
implementation of Microsoft's .NET Framework based on the ECMA
standards for C# and the Common Language Runtime.
The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.org/)
The Mono project is part of the [.NET Foundation](http://www.dotnetfoundation.org/)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/mono/mono?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
@@ -30,8 +30,7 @@ The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.o
| Windows | i386 | [![windows-i386][17]][18] |
| CentOS | s390x (cs) | [![centos-s390x][19]][20] |
| Debian 9 | ppc64el (cs) | [![debian-9-ppc64el][21]][22]|
| AIX 6.1 | ppc64 (cs) | [![aix-ppc64][23]][24] |
| FreeBSD 12 | amd64 (cs) | [![freebsd-amd64][25]][26] |
| AIX | ppc64 (cs) | [![aix-ppc64][23]][24] |
_(cs) = community supported architecture_
@@ -59,8 +58,6 @@ _(cs) = community supported architecture_
[22]: https://jenkins.mono-project.com/job/test-mono-mainline-community-chroot/label=debian-9-ppc64el
[23]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64/badge/icon
[24]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64
[25]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64/badge/icon
[26]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64
Compilation and Installation
============================
@@ -69,15 +66,15 @@ Building the Software
---------------------
Please see our guides for building Mono on
[Mac OS X](https://www.mono-project.com/docs/compiling-mono/mac/),
[Linux](https://www.mono-project.com/docs/compiling-mono/linux/) and
[Windows](https://www.mono-project.com/docs/compiling-mono/windows/).
[Mac OS X](http://www.mono-project.com/docs/compiling-mono/mac/),
[Linux](http://www.mono-project.com/docs/compiling-mono/linux/) and
[Windows](http://www.mono-project.com/docs/compiling-mono/windows/).
Note that building from Git assumes that you already have Mono installed,
so please download and [install the latest Mono release](https://www.mono-project.com/download/)
so please download and [install the latest Mono release](http://www.mono-project.com/download/)
before trying to build from Git. This is required because the Mono build
relies on a working Mono C# compiler to compile itself
(also known as [bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(compilers))).
(also known as [bootstrapping](http://en.wikipedia.org/wiki/Bootstrapping_(compilers))).
If you don't have a working Mono installation
---------------------------------------------
@@ -206,9 +203,9 @@ Contributing to Mono
====================
Before submitting changes to Mono, please review the [contribution
guidelines](https://www.mono-project.com/community/contributing/).
guidelines](http://www.mono-project.com/community/contributing/).
Please pay particular attention to the [Important
Rules](https://www.mono-project.com/community/contributing/#important-rules)
Rules](http://www.mono-project.com/community/contributing/#important-rules)
section.
Reporting bugs
@@ -218,7 +215,7 @@ To submit bug reports, please [open an issue on the mono GitHub repo](https://gi
Please use the search facility to ensure the same bug hasn't already
been submitted and follow our
[guidelines](https://www.mono-project.com/community/bugs/make-a-good-bug-report/)
[guidelines](http://www.mono-project.com/community/bugs/make-a-good-bug-report/)
on how to make a good bug report.
Configuration Options
@@ -309,9 +306,6 @@ faster under the Xen virtualization system.
* `--with-large-heap=yes,no` - Enable support for GC heaps larger than 3GB.
* This only applies only to the Boehm garbage collector, the SGen garbage
collector does not use this configuration option.
* This defaults to `no`.
* `--enable-small-config=yes,no` - Enable some tweaks to reduce memory usage
@@ -441,7 +435,7 @@ for Mono. The LLVM code generator and optimizer will be
used instead of Mono's built-in code generator for both
Just in Time and Ahead of Time compilations.
* See https://www.mono-project.com/docs/advanced/mono-llvm/ for the
* See http://www.mono-project.com/docs/advanced/mono-llvm/ for the
full details and up-to-date information on this feature.
* You will need to have an LLVM built that Mono can link
@@ -510,7 +504,7 @@ also, see the man page.
* This configures the Mono compiler to generate code
suitable to be used by Google's Native Client:
https://code.google.com/p/nativeclient/
http://code.google.com/p/nativeclient/
* Currently this is used with Mono's AOT engine as
Native Client does not support JIT engines yet.
@@ -584,7 +578,7 @@ file for information about Microsoft's patent grant.
Mono Trademark Use Policy
=========================
The use of trademarks and logos for Mono can be found [here](https://www.dotnetfoundation.org/legal/mono-tm).
The use of trademarks and logos for Mono can be found [here](http://www.dotnetfoundation.org/legal/mono-tm).
Maintaining the Class Library Solution Files
============================================
-185
View File
@@ -1,185 +0,0 @@
Mono uses third-party libraries or other resources that may be
distributed under licenses different than the Mono software.
Attributions and license notices for test cases originally authored by
third parties can be found in the respective test directories.
In the event that we accidentally failed to list a required notice, please
bring it to our attention. Post an issue or email us:
[email protected]
The attached notices are provided for information only.
License notice for Mono
-----------------------
Note: multiple license notices are provided that occure in source code.
They are equivalent forms of the the MIT license from various copyright holders.
// Dual licensed under the terms of the MIT X11 or GNU GPL
// Copyright 2003-2011 Novell, Inc.
// Copyright 2011 Xamarin Inc
/**
*
* Copyright 2001-2003 Ximian, Inc
* Copyright 2003-2010 Novell, Inc.
* Copyright (C) 2012 Xamarin Inc
*
* Licensed under the MIT license. See LICENSE file in the project root for full license information.
*/
/******************************************************************************
* The MIT License
* Copyright (c) 2003 Novell Inc. www.novell.com
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the Software), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*******************************************************************************/
License notice for Apple
-------------------------
Content: https://github.com/mono/mono/blob/435f968102d0820ca65d63e1e741e4130b4d1303/support/ios/net/route.h
License: https://opensource.apple.com/apsl/
License for GCC
---------------
License: http://gcc.gnu.org/onlinedocs/gcc-4.6.4/libstdc++/manual/manual/license.html
Content: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini
Example: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini/decompose.c#L967
GCC RUNTIME LIBRARY EXCEPTION
Version 3.1, 31 March 2009
Copyright (C) 2009 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
This GCC Runtime Library Exception ("Exception") is an additional
permission under section 7 of the GNU General Public License, version
3 ("GPLv3"). It applies to a given file (the "Runtime Library") that
bears a notice placed by the copyright holder of the file stating that
the file is governed by GPLv3 along with this Exception.
When you use GCC to compile a program, GCC may combine portions of
certain GCC header files and runtime libraries with the compiled
program. The purpose of this Exception is to allow compilation of
non-GPL (including proprietary) programs to use, in this way, the
header files and runtime libraries covered by this Exception.
0. Definitions.
A file is an "Independent Module" if it either requires the Runtime
Library for execution after a Compilation Process, or makes use of an
interface provided by the Runtime Library, but is not otherwise based
on the Runtime Library.
"GCC" means a version of the GNU Compiler Collection, with or without
modifications, governed by version 3 (or a specified later version) of
the GNU General Public License (GPL) with the option of using any
subsequent versions published by the FSF.
"GPL-compatible Software" is software whose conditions of propagation,
modification and use would permit combination with GCC in accord with
the license of GCC.
"Target Code" refers to output from any compiler for a real or virtual
target processor architecture, in executable form or suitable for
input to an assembler, loader, linker and/or execution
phase. Notwithstanding that, Target Code does not include data in any
format that is used as a compiler intermediate representation, or used
for producing a compiler intermediate representation.
The "Compilation Process" transforms code entirely represented in
non-intermediate languages designed for human-written code, and/or in
Java Virtual Machine byte code, into Target Code. Thus, for example,
use of source code generators and preprocessors need not be considered
part of the Compilation Process, since the Compilation Process can be
understood as starting with the output of the generators or
preprocessors.
A Compilation Process is "Eligible" if it is done using GCC, alone or
with other GPL-compatible software, or if it is done without using any
work based on GCC. For example, using non-GPL-compatible Software to
optimize any GCC intermediate representations would not qualify as an
Eligible Compilation Process.
1. Grant of Additional Permission.
You have permission to propagate a work of Target Code formed by
combining the Runtime Library with Independent Modules, even if such
propagation would otherwise violate the terms of GPLv3, provided that
all Target Code was generated by Eligible Compilation Processes. You
may then convey such a combination under terms of your choice,
consistent with the licensing of the Independent Modules.
2. No Weakening of GCC Copyleft.
The availability of this Exception does not imply any general
presumption that third-party software is unaffected by the copyleft
requirements of the license of GCC.
Hopefully that text is self-explanatory. If it isn't, you need to speak to your lawyer, or the Free Software Foundation.
License notice for Silicon Graphics and HP
------------------------------------------
/*
* Copyright (c) 1996-1997
* Silicon Graphics Computer Systems, Inc.
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Silicon Graphics makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*
* Copyright (c) 2002
* Hewlett-Packard Company
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Hewlett-Packard Company makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*/
/*
* This implements standard-conforming allocators that interact with
* the garbage collector. Gc_alloctor<T> allocates garbage-collectable
* objects of type T. Traceable_allocator<T> allocates objects that
* are not temselves garbage collected, but are scanned by the
* collector for pointers to collectable objects. Traceable_alloc
* should be used for explicitly managed STL containers that may
* point to collectable objects.
*
* This code was derived from an earlier version of the GNU C++ standard
* library, which itself was derived from the SGI STL implementation.
*/
Content: https://github.com/mono/mono/blob/d58dfa185af1d48fb93efdef1d046762f214f0a7/libgc/include/gc_allocator.h
-1
View File
@@ -4,4 +4,3 @@ Makefile
*.dll
*.mdb
coreclr-testlist.txt
microbench-perf.sh
@@ -0,0 +1,14 @@
using System;
using System.IO;
using System.Reflection;
namespace System.Runtime.Loader
{
public static class AssemblyExtensions
{
public static Type[] GetTypes(this Assembly assembly)
{
return assembly.GetTypes ();
}
}
}
+1 -3
View File
@@ -3,10 +3,9 @@ BENCHMARKER_PATH=$(ACCEPTANCE_TESTS_PATH)/benchmarker
ROSLYN_PATH=$(ACCEPTANCE_TESTS_PATH)/roslyn
CORECLR_PATH=$(ACCEPTANCE_TESTS_PATH)/coreclr
MSTESTSUITE_PATH=$(ACCEPTANCE_TESTS_PATH)/ms-test-suite
DEBIANSHOOTOUTMONO_PATH=$(ACCEPTANCE_TESTS_PATH)/DebianShootoutMono
CLEANFILES = *.dll *.exe *.mdb
EXTRA_DIST=README.md SUBMODULES.json microbench.mk versions.mk profiler-stress.mk roslyn.mk coreclr.mk ms-test-suite.mk
EXTRA_DIST=README.md SUBMODULES.json versions.mk profiler-stress.mk roslyn.mk coreclr.mk ms-test-suite.mk
CLASS=$(mcs_topdir)/class/lib/$(DEFAULT_PROFILE)
TOOLS_CLASS=$(mcs_topdir)/class/lib/build
@@ -18,7 +17,6 @@ ILASM = $(TOOLS_RUNTIME) $(TOOLS_CLASS)/ilasm.exe
XUNIT = $(RUNTIME) $(abs_top_builddir)/external/xunit-binaries/xunit.console.exe
include versions.mk
include microbench.mk
include profiler-stress.mk
include roslyn.mk
include coreclr.mk
+3 -11
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"rev": "380eec412868464a132cdcd3a45ef457fab6e060",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
@@ -10,7 +10,7 @@
{
"name": "coreclr",
"url": "git://github.com/mono/coreclr.git",
"rev": "90f7060935732bb624e1f325d23f63072433725f",
"rev": "aa62c0382ae79562293f3ea1a94ef0f16f80f498",
"remote-branch": "origin/mono",
"branch": "mono",
"directory": "coreclr"
@@ -18,7 +18,7 @@
{
"name": "ms-test-suite",
"url": "[email protected]:xamarin/ms-test-suite.git",
"rev": "39b1b222e8a3690a1b8ea018ce54d2a826f5246e",
"rev": "67f29dfc0741b5311dd746c75760963a2915e648",
"remote-branch": "origin/master",
"branch": "master",
"directory": "ms-test-suite"
@@ -30,13 +30,5 @@
"remote-branch": "origin/master",
"branch": "master",
"directory": "benchmarker"
},
{
"name": "DebianShootoutMono",
"url": "https://github.com/alexanderkyte/DebianShootoutMono.git",
"rev": "3fde2ced806c1fe7eed81120a40d99474fa009f0",
"remote-branch": "origin/release_11_15_2018",
"branch": "release_11_15_2018",
"directory": "DebianShootoutMono"
}
]
+4 -2
View File
@@ -3175,6 +3175,7 @@ CORECLR_COREMANGLIB_TEST_CS_SRC= \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/inattribute/inattributector.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/layoutkind/layoutkindauto.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/layoutkind/layoutkindsequential.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/marshal/marshalgetlastwin32error.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/marshal/marshalreadint641.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/marshal/marshalsizeof1.cs \
$(CORECLR_PATH)/tests/src/CoreMangLib/cti/system/runtime/interopservices/marshalasattribute/marshalasattributearraysubtype.cs \
@@ -3688,8 +3689,9 @@ CORECLR_STRESSTEST_RUNNER_CS_SRC= \
$(CORECLR_PATH)/tests/src/GC/Stress/Framework/RFLogging.cs \
$(CORECLR_PATH)/tests/src/GC/Stress/Framework/DetourHelpers.cs \
$(CORECLR_PATH)/tests/src/GC/Stress/Framework/LoaderClass.cs \
GCStressTests/AssemblyLoadContext.cs
GCStressTests/AssemblyLoadContext.cs \
GCStressTests/AssemblyExtensions.cs
CORECLR_TESTLIBRARY_CS_SRC = \
$(CORECLR_PATH)/tests/src/Common/CoreCLRTestLibrary/TestFramework.cs \
$(CORECLR_PATH)/tests/src/Common/CoreCLRTestLibrary/Utilities.cs \
-17
View File
@@ -1,17 +0,0 @@
#! /bin/sh
r='@mono_build_root@'
aotpattern="--aot="
# if this is an aot invoke
if [[ $@ =~ $aotpattern ]];
then
# just aot the code
echo "$r/runtime/mono-wrapper $@"
exec $r/runtime/mono-wrapper $@
else
# else run it under perf
echo "$MONO_PERF_BINARY record -o $r/acceptance-tests/perf.data -v -s -g -- $r/mono/mini/mono-sgen $@"
exec $MONO_PERF_BINARY record -o $r/acceptance-tests/perf.data -v -s -g $r/mono/mini/mono-sgen $@
fi
-119
View File
@@ -1,119 +0,0 @@
check-microbench: DebianShootoutMono.stamp
@$(MAKE) test-run-microbench
DebianShootoutMono.stamp:
@$(MAKE) validate-DebianShootoutMono RESET_VERSIONS=1
@$(MAKE) prepare-dlls
@touch $@
abs_top_srcdir = $(abspath $(top_srcdir))
TEST_EXE_PATH=$(abs_top_srcdir)/acceptance-tests/external/DebianShootoutMono/release/
NET_4_X_RUNTIME=MONO_PATH=$(TEST_EXE_PATH):$(abs_top_srcdir)/mcs/class/lib/net_4_x $(abs_top_srcdir)/runtime/mono-wrapper
FULL_AOT_RUNTIME=MONO_PATH=$(abs_top_srcdir)/mcs/class/lib/testing_aot_full $(abs_top_srcdir)/runtime/mono-wrapper
PERF_BINARY=$(if $(MONO_PERF_BINARY),$(MONO_PERF_BINARY),perf)
PERF_RUNTIME=$(abs_top_srcdir)/acceptance-tests/microbench-perf.sh
define BenchmarkDotNetTemplate
run-microbench-$(1):: DebianShootoutMono.stamp
MONO_BENCH_AOT_RUN="$(AOT_RUN_FLAGS)" \
MONO_BENCH_AOT_BUILD="$(AOT_BUILD_FLAGS)" \
MONO_BENCH_EXECUTABLE="$(abs_top_srcdir)/runtime/mono-wrapper" \
MONO_BENCH_PATH="$(abs_top_srcdir)/mcs/class/lib/$(TEST_PROFILE)" \
MONO_BENCH_INPUT="$(2)" \
$(NET_4_X_RUNTIME) \
$(TEST_EXE_PATH)/DebianShootoutMono.exe $(1)
test-run-microbench:: run-microbench-$(1)
run-microbench-debug-$(1):: DebianShootoutMono.stamp
echo MONO_PATH="$(abs_top_srcdir)/mcs/class/lib/$(TEST_PROFILE)" $(abs_top_srcdir)/runtime/mono-wrapper $(AOT_BUILD_FLAGS) $(TEST_EXE_PATH)/DebianShootoutMono.exe
MONO_BENCH_INPUT="$(2)" \
MONO_PATH="$(abs_top_srcdir)/mcs/class/lib/$(TEST_PROFILE)" \
$(abs_top_srcdir)/runtime/mono-wrapper $(AOT_RUN_FLAGS) $(TEST_EXE_PATH)/DebianShootoutMono.exe Run $(1)
test-run-microbench-debug:: run-microbench-debug-$(1)
if HOST_LINUX
run-microbench-profiled-$(1):: microbench-results/$(1).perf.data
microbench-results/$(1).perf.data: DebianShootoutMono.stamp
mkdir -p microbench-results
MONO_PERF_BINARY="$(PERF_BINARY)" \
MONO_BENCH_EXECUTABLE="$(PERF_RUNTIME)" \
MONO_BENCH_AOT_RUN="$(AOT_RUN_FLAGS)"\
MONO_BENCH_AOT_BUILD="$(AOT_BUILD_FLAGS)"\
MONO_BENCH_PATH="$(abs_top_srcdir)/mcs/class/lib/$(TEST_PROFILE)" \
MONO_BENCH_INPUT="$(2)" \
$(NET_4_X_RUNTIME) \
$(TEST_EXE_PATH)/DebianShootoutMono.exe $(1) $(2)
mv perf.data microbench-results/$(1).perf.data
microbench-results/$(1).tmp.perf: microbench-results/$(1).perf.data
$(PERF_BINARY) script -i microbench-results/$(1).perf.data > microbench-results/$(1).tmp.perf
microbench-results/$(1).perf-flame.svg: microbench-results/$(1).tmp.perf
cat microbench-results/$(1).tmp.perf | ./external/DebianShootoutMono/FlameGraph/stackcollapse-perf.pl > microbench-results/$(1).perf-folded
./external/DebianShootoutMono/FlameGraph/flamegraph.pl microbench-results/$(1).perf-folded > microbench-results/$(1).perf-flame.svg
rm microbench-results/$(1).tmp.perf
rm microbench-results/$(1).perf-folded
MONO_PERF_FLAGS=--show-cpu-utilization -n --hierarchy -T $(MONO_PERF_ADDITIONAL_FLAGS)
microbench-results/$(1).perf.report: microbench-results/$(1).perf.data
$(PERF_BINARY) report -i microbench-results/$(1).perf.data $(MONO_PERF_FLAGS) > microbench-results/$(1).perf.report
test-run-microbench-profiled:: run-microbench-profiled-$(1)
test-run-microbench-publish-collect:: microbench-results/$(1).perf.data microbench-results/$(1).perf.report microbench-results/$(1).perf-flame.svg
endif
endef
if HOST_LINUX
test-run-microbench-perf-check:
$(PERF_BINARY) record -a -o perf.data -- echo "testing"
rm perf.data
microbench-results/perf-data.zip:
zip microbench-results/perf-data.zip microbench-results/*.perf.data
rm microbench-results/*.perf.data
perf-report: microbench-results/perf-data.zip
perf-report-total: test-run-microbench-publish-collect
@$(MAKE) perf-report
endif
.PHONY: prepare-dlls
if FULL_AOT_TESTS
prepare-dlls:
$(FULL_AOT_RUNTIME) $(AOT_BUILD_FLAGS) $(TEST_EXE_PATH)/*.{dll,exe}
else
prepare-dlls:
endif
FIXTURE_DIR=$(abs_top_srcdir)/acceptance-tests/external/DebianShootoutMono/fixtures
$(eval $(call BenchmarkDotNetTemplate,Mandelbrot,))
$(eval $(call BenchmarkDotNetTemplate,RegexRedux,$(FIXTURE_DIR)/regexredux-input.txt))
$(eval $(call BenchmarkDotNetTemplate,KNucleotide,$(FIXTURE_DIR)/knucleotide-input.txt))
$(eval $(call BenchmarkDotNetTemplate,BinaryTrees,))
$(eval $(call BenchmarkDotNetTemplate,NBodyTest,))
$(eval $(call BenchmarkDotNetTemplate,SpectralNorm,))
$(eval $(call BenchmarkDotNetTemplate,Fannkuchredux,))
$(eval $(call BenchmarkDotNetTemplate,Fasta,))
$(eval $(call BenchmarkDotNetTemplate,RevComp,$(FIXTURE_DIR)/revcomp-input.txt))
#$(eval $(call BenchmarkDotNetTemplate,GistBenchmark,$(MONO_BENCH_GIST_URL))) broken in BDN, bug filed
+1 -3
View File
@@ -1,7 +1,5 @@
check-roslyn:
@echo "Runnning roslyn tests using mono from PATH:"
mono --version
@$(MAKE) validate-roslyn RESET_VERSIONS=1
cd $(ROSLYN_PATH); \
./build.sh --restore --build --test --mono || exit; \
./mono-testing.sh "$(XUNIT)" || exit; \
echo "done"
-10
View File
@@ -7,7 +7,6 @@ $(eval $(call ValidateVersionTemplate,benchmarker,BENCHMARKER))
$(eval $(call ValidateVersionTemplate,roslyn,ROSLYN))
$(eval $(call ValidateVersionTemplate,coreclr,CORECLR))
$(eval $(call ValidateVersionTemplate,ms-test-suite,MSTESTSUITE))
$(eval $(call ValidateVersionTemplate,DebianShootoutMono,DEBIANSHOOTOUTMONO))
# Bump the given submodule to the revision given by the REV make variable
# If COMMIT is 1, commit the change
@@ -15,7 +14,6 @@ bump-benchmarker: __bump-benchmarker
bump-roslyn: __bump-version-roslyn
bump-coreclr: __bump-version-coreclr
bump-ms-test-suite: __bump-version-ms-test-suite
bump-DebianShootoutMono: __bump-version-DebianShootoutMono
# Bump the given submodule to the branch given by the BRANCH/REMOTE_BRANCH make variables
# If COMMIT is 1, commit the change
@@ -23,7 +21,6 @@ bump-branch-benchmarker: __bump-branch-benchmarker
bump-branch-roslyn: __bump-branch-roslyn
bump-branch-coreclr: __bump-branch-coreclr
bump-branch-ms-test-suite: __bump-branch-ms-test-suite
bump-branch-DebianShootoutMono: __bump-branch-DebianShootoutMono
# Bump the given submodule to its current GIT version
# If COMMIT is 1, commit the change
@@ -31,7 +28,6 @@ bump-current-benchmarker: __bump-current-benchmarker
bump-current-roslyn: __bump-current-version-roslyn
bump-current-coreclr: __bump-current-version-coreclr
bump-current-ms-test-suite: __bump-current-version-ms-test-suite
bump-current-DebianShootoutMono: __bump-current-version-DebianShootoutMono
commit-bump-benchmarker:
$(MAKE) bump-benchmarker COMMIT=1
@@ -45,9 +41,6 @@ commit-bump-coreclr:
commit-bump-ms-test-suite:
$(MAKE) bump-ms-test-suite COMMIT=1
commit-bump-DebianShootoutMono:
$(MAKE) bump-DebianShootoutMono COMMIT=1
commit-bump-current-benchmarker:
$(MAKE) bump-current-benchmarker COMMIT=1
@@ -59,6 +52,3 @@ commit-bump-current-coreclr:
commit-bump-current-ms-test-suite:
$(MAKE) bump-current-ms-test-suite COMMIT=1
commit-bump-current-DebianShootoutMono:
$(MAKE) bump-current-DebianShootoutMono COMMIT=1
+6 -18
View File
@@ -120,7 +120,7 @@ for PARAM; do
done
if test x$has_ext_mod = xtrue; then
pushd $top_srcdir../mono-extensions/scripts
pushd ../mono-extensions/scripts
sh ./prepare-repo.sh $ext_mod_args || exit 1
popd
else
@@ -150,29 +150,17 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
if test -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
echo "MONO_EXTRA_CONFIGURE_FLAGS is $MONO_EXTRA_CONFIGURE_FLAGS"
fi
host_conf_flag=
build_uname_all=`(uname -a) 2>/dev/null`
case "$build_uname_all" in
CYGWIN*)
if [[ "$@" != *"--host="* ]]; then
echo "Missing --host parameter, configure using ./configure --host=i686-w64-mingw32 or --host=x86_64-w64-mingw32"
echo "Falling back using --host=x86_64-w64-mingw32 as default."
host_conf_flag="--host=x86_64-w64-mingw32"
fi
;;
esac
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings $host_conf_flag" #--enable-iso-c
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings" #--enable-iso-c
if test x$NOCONFIGURE = x; then
echo Running $srcdir/configure $conf_flags "$@" ...
+1471 -1227
View File
File diff suppressed because it is too large. Load diff
+423 -1672
View File
File diff suppressed because it is too large. Load diff
+3 -3
View File
@@ -7,7 +7,7 @@ EXTRA_DIST = \
config.in \
browscap.ini mono.supp mono.d README \
mono.pc.in mono-2.pc.in monosgen-2.pc.in mint.pc.in dotnet.pc.in dotnet35.pc.in wcf.pc.in monodoc.pc.in \
mono-cairo.pc.in mono-options.pc.in cecil.pc.in \
mono-nunit.pc.in mono-cairo.pc.in mono-options.pc.in cecil.pc.in \
mono-lineeditor.pc.in system.web.extensions_1.0.pc.in system.web.extensions.design_1.0.pc.in\
dtrace-prelink.sh system.web.mvc.pc.in system.web.mvc2.pc.in system.web.mvc3.pc.in aspnetwebstack.pc.in \
reactive.pc.in xbuild12.pc \
@@ -25,10 +25,10 @@ else
SGENPCFILE=
endif
pkgconfig_DATA= mono.pc mono-2.pc dotnet.pc dotnet35.pc wcf.pc mono-cairo.pc mono-options.pc cecil.pc monodoc.pc mono-lineeditor.pc system.web.extensions_1.0.pc \
pkgconfig_DATA= mono.pc mono-2.pc dotnet.pc dotnet35.pc wcf.pc mono-nunit.pc mono-cairo.pc mono-options.pc cecil.pc monodoc.pc mono-lineeditor.pc system.web.extensions_1.0.pc \
system.web.extensions.design_1.0.pc system.web.mvc.pc system.web.mvc2.pc system.web.mvc3.pc aspnetwebstack.pc reactive.pc xbuild12.pc $(SGENPCFILE)
DISTCLEANFILES= mono-2.pc mono.pc mint.pc dotnet.pc dotnet35.pc wcf.pc mono-cairo.pc mono-options.pc cecil.pc monodoc.pc mono-lineeditor.pc system.web.extensions_1.0.pc \
DISTCLEANFILES= mono-2.pc mono.pc mint.pc dotnet.pc dotnet35.pc wcf.pc mono-nunit.pc mono-cairo.pc mono-options.pc cecil.pc monodoc.pc mono-lineeditor.pc system.web.extensions_1.0.pc \
system.web.extensions.design_1.0.pc system.web.mvc.pc system.web.mvc2.pc system.web.mvc3.pc aspnetwebstack.pc reactive.pc $(SGENPCFILE) mono-sgen-gdb.py
mono_DATA = config \
+2 -4
View File
@@ -6,14 +6,12 @@
<dllmap dll="libintl" name="bind_textdomain_codeset" target="@LIBC@" os="solaris"/>
<dllmap dll="libintl" target="@INTL@" os="!windows"/>
<dllmap dll="i:libxslt.dll" target="libxslt@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="@ODBC@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libodbc@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libiodbc.dylib" os="osx"/>
<dllmap dll="oci" target="libclntsh@libsuffix@" os="!windows"/>
<dllmap dll="db2cli" target="libdb2_36@libsuffix@" os="!windows"/>
<dllmap dll="MonoPosixHelper" target="$mono_libdir/libMonoPosixHelper@libsuffix@" os="!windows" />
<dllmap dll="System.Native" target="$mono_libdir/@MONO_NATIVE_LIBRARY_NAME@@libsuffix@" os="!windows" />
<dllmap dll="System.Net.Security.Native" target="$mono_libdir/@MONO_NATIVE_LIBRARY_NAME@@libsuffix@" os="!windows" />
<dllmap dll="System.Security.Cryptography.Native.Apple" target="$mono_libdir/@MONO_NATIVE_LIBRARY_NAME@@libsuffix@" os="osx" />
<dllmap dll="System.Native" target="$mono_libdir/libmono-system-native@libsuffix@" os="!windows" />
<dllmap dll="libmono-btls-shared" target="$mono_libdir/libmono-btls-shared@libsuffix@" os="!windows" />
<dllmap dll="i:msvcrt" target="@LIBC@" os="!windows"/>
<dllmap dll="i:msvcrt.dll" target="@LIBC@" os="!windows"/>
Executable → Regular
View File
File mode changed.
+13 -13
View File
@@ -1,9 +1,8 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
print('%c thread #%i' % ('*' if current_thread else ' ', thread.idx))
print '%c thread #%i' % ('*' if current_thread else ' ', thread.idx)
if current_thread:
selected_frame = thread.GetSelectedFrame()
@@ -15,16 +14,17 @@ def print_frames(thread, num_frames, current_thread):
if function_name == "interp_exec_method_full":
try:
s = 'frame->imethod->method'
methoddesc = frame.EvaluateExpression('(char*) mono_method_full_name (' + s + ', 1)').summary[1:-1]
klassname = frame.EvaluateExpression('(char*) ' + s + '->klass->name').summary[1:-1]
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
ipoffset -= frame.EvaluateExpression('rtm->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
var = '%s::%s @ %d %s || %s' % (klassname, methodname, ipoffset, insn, frame)
except Exception as e:
traceback.print_exc()
print "DBG: execfail:" + str(e)
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -32,7 +32,7 @@ def print_frames(thread, num_frames, current_thread):
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
var = 'transforming %s::%s || %s' % (klassname, methodname, frame)
except Exception as e:
print("DBG: transformfail:" + str(e))
print "DBG: transformfail:" + str(e)
elif pc[0] == '0':
try:
framestr = frame.EvaluateExpression('(char*)mono_pmip((void*)%s)' % pc).summary[1:-1]
@@ -40,7 +40,7 @@ def print_frames(thread, num_frames, current_thread):
except:
pass
print(' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var))
print ' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var)
def monobt(debugger, command, result, dict):
opts = {'all_bt': False, 'num_frames': None}
@@ -50,20 +50,20 @@ def monobt(debugger, command, result, dict):
elif command.isdigit():
opts['num_frames'] = int(command)
elif command != '':
print('error: monobt [<number>|all]')
print 'error: monobt [<number>|all]'
return
target = debugger.GetSelectedTarget()
process = target.process
if not process.IsValid():
print('error: invalid process')
print 'error: invalid process'
return
if opts['all_bt']:
for thread in process.threads:
print_frames(thread, len(thread), process.selected_thread == thread)
print('')
print ''
else:
thread = process.selected_thread
num_frames = len(thread) if opts['num_frames'] is None else opts['num_frames']
@@ -76,4 +76,4 @@ def __lldb_init_module (debugger, dict):
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f monobt.monobt monobt')
debugger.HandleCommand('command alias mbt monobt')
print('"monobt" command installed')
print '"monobt" command installed'
+9
View File
@@ -0,0 +1,9 @@
prefix=${pcfiledir}/../..
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
Name: Mono NUnit
Description: Mono's version of NUnit
Version: @VERSION@
Libs: -r:${libdir}/mono/4.5/nunit.core.dll -r:${libdir}/mono/4.5/nunit.core.interfaces.dll -r:${libdir}/mono/4.5/nunit.core.extensions.dll -r:${libdir}/mono/4.5/nunit.framework.dll -r:${libdir}/mono/4.5/nunit.framework.extensions.dll -r:${libdir}/mono/4.5/nunit.mocks.dll -r:${libdir}/mono/4.5/nunit.util.dll -r:${libdir}/mono/4.5/nunit-console-runner.dll
+2 -3
View File
@@ -52,8 +52,7 @@ EXTRA_DIST = \
-include $(mcs_topdir)/build/config.make
DOC_PROFILE:=net_4_x
CLASS=$(mcs_topdir)/class/lib/$(DOC_PROFILE)
CLASS=$(mcs_topdir)/class/lib/$(DEFAULT_PROFILE)
dist-hook:
$(mkdir_p) $(distdir)/sources
@@ -69,7 +68,7 @@ clean-local:
-rm -f deploy/* convert.exe* AgilityPack.dll*
-rm -f $(ASSEMBLED_DOCS)
TOOL_MAKE=$(MAKE) -f $(srcdir)/docs.make topdir=$(srcdir)/../mcs srcdir=$(srcdir) PROFILE=$(DOC_PROFILE)
TOOL_MAKE=$(MAKE) -f $(srcdir)/docs.make topdir=$(srcdir)/../mcs srcdir=$(srcdir)
monoapi.zip: monoapi.tree
-23
View File
@@ -18,27 +18,6 @@ convert.exe: $(srcdir)/convert.cs AgilityPack.dll
AgilityPack.dll:
$(CSCOMPILE) -r:$(topdir)/class/lib/$(PROFILE)/mscorlib.dll -r:$(topdir)/class/lib/$(PROFILE)/System.dll -r:$(topdir)/class/lib/$(PROFILE)/System.Xml.dll -target:library -out:$@ $(srcdir)/HtmlAgilityPack/*.cs
ifdef MCS_MODE
monoapi.zip:
touch $@
monoapi.tree: monoapi.zip
touch $@
mono-tools.zip:
touch $@
mono-tools.tree: mono-tools.zip
touch $@
mono-file-formats.zip:
touch $@
mono-file-formats.tree: mono-file-formats.zip
touch $@
else
monoapi.zip: monoapi.tree
@test -f $@ || { rm -f $< && $(MAKE) $<; }
@@ -57,7 +36,5 @@ mono-file-formats.zip: mono-file-formats.tree
mono-file-formats.tree: $(srcdir)/mono-file-formats.config $(srcdir)/docs.make
$(MDOC) assemble -o mono-file-formats -f man $<
endif
.doc-stamp:
-138
View File
@@ -1,138 +0,0 @@
# Mono RISC-V Port
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documentation
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
## Setting up a cross environment
### Debian/Ubuntu packages
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Next, install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
```
Finally, build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ make linux
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
will have to build the managed libraries and tools through a native build of
Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 ./mono/mini/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
```
## Things to be done
Things that need to be done beyond the basic 64-bit port, in no particular
order:
* Complete the soft float port.
* Complete the 32-bit port.
* Add unwind info to trampolines.
* Implement AOT support.
* Implement interpreter support.
* Implement LLVM support.
* Implement SDB support.
* Implement `dyn_call` support.
* Ensure all runtime tests pass.
* Ensure all corlib tests pass.
* Set up CI on Jenkins.
-6
View File
@@ -109,12 +109,6 @@ typedef struct {
<h3>Write Barriers</h3>
<p>SGen is a concurrent and generational GC, features which require
tracking changes to the state of the heap. This is achieved through
write barriers. Whenever native code is changing the state of the
heap by storing references into another managed object, it needs to
do it using this write barrier API.
<h4><a name="api:mono_gc_wbarrier_arrayref_copy">mono_gc_wbarrier_arrayref_copy</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_nostore">mono_gc_wbarrier_generic_nostore</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_store">mono_gc_wbarrier_generic_store</a></h4>
-27
View File
@@ -1,27 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>39d545dc14bc4d5a546e5e2f58d1ebaca72fcd61</Sha>
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.19567.2">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>928efaa081ffd17b37f6c662fee44775f2d0b90f</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.19567.2">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>928efaa081ffd17b37f6c662fee44775f2d0b90f</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
-14
View File
@@ -1,14 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>5.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
</Project>
-2
View File
@@ -1,2 +0,0 @@
@echo off
powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0Build.ps1""" -restore -build -test -sign -pack -publish -ci %*"
-158
View File
@@ -1,158 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SymbolToolPath # Full path to directory where dotnet symbol-tool was installed
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $TargetServerParam, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $SymbolsPath
)
$FileName = [System.IO.Path]::GetFileName($FullPath)
$Extension = [System.IO.Path]::GetExtension($FullPath)
# Those below are potential symbol files that the `dotnet symbol` might
# return. Which one will be returned depend on the type of file we are
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
}
elseif (Test-Path $SymbolPath) {
return "Module"
}
else {
return $null
}
}
function CountMissingSymbols {
param(
[string] $PackagePath # Path to a NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
# How many files are missing symbol information
$MissingSymbols = 0
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
# Makes easier to reference `symbol tool`
Push-Location $SymbolToolPath
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
}
else {
Write-Host "No symbols found on SymWeb!"
}
}
}
}
Pop-Location
return $MissingSymbols
}
function CheckSymbolsAvailable {
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-Error "Missing symbols for $Status modules in the package $FileName"
}
Write-Host
}
}
CheckSymbolsAvailable
-11
View File
@@ -1,11 +0,0 @@
@{
IncludeRules=@('PSAvoidUsingCmdletAliases',
'PSAvoidUsingWMICmdlet',
'PSAvoidUsingPositionalParameters',
'PSAvoidUsingInvokeExpression',
'PSUseDeclaredVarsMoreThanAssignments',
'PSUseCmdletCorrectly',
'PSStandardDSCFunctionsInResource',
'PSUseIdenticalMandatoryParametersForDSC',
'PSUseIdenticalParametersForDSC')
}
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PushToStaticFeed task that will read the build asset
manifest and publish the assets described in the manifest to
informed target feeds.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(MSBuildThisFileDirectory)DefaultVersions.props" Condition="Exists('$(MSBuildThisFileDirectory)DefaultVersions.props')" />
<!--
This won't be necessary once we solve this issue:
https://github.com/dotnet/arcade/issues/2266
-->
<Import Project="$(MSBuildThisFileDirectory)ArtifactsCategory.props" Condition="Exists('$(MSBuildThisFileDirectory)ArtifactsCategory.props')" />
<Import Project="$(NuGetPackageRoot)microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<Target Name="PublishToFeed">
<Error Condition="'$(ArtifactsCategory)' == ''" Text="ArtifactsCategory: The artifacts' category produced by the build wasn't provided." />
<Error Condition="'$(AccountKeyToStaticFeed)' == ''" Text="AccountKeyToStaticFeed: Account key for target feed wasn't provided." />
<Error Condition="'$(ManifestsBasePath)' == ''" Text="Full path to asset manifests directory wasn't provided." />
<Error Condition="'$(BlobBasePath)' == '' AND '$(PackageBasePath)' == ''" Text="A valid full path to BlobBasePath of PackageBasePath is required." />
<ItemGroup>
<!-- Include all manifests found in the manifest folder. -->
<ManifestFiles Include="$(ManifestsBasePath)*.xml" />
</ItemGroup>
<Error Condition="'@(ManifestFiles)' == ''" Text="No manifest file was found in the provided path: $(ManifestsBasePath)" />
<!--
For now the type of packages being published will be informed for the whole build.
Eventually this will be specified on a per package basis:
TODO: https://github.com/dotnet/arcade/issues/2266
-->
<PropertyGroup>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCORE'">https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCOREVALIDATION'">https://dotnetfeed.blob.core.windows.net/arcade-validation/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORETOOLING'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ENTITYFRAMEWORKCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETEXTENSIONS'">https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORECLR'">https://dotnetfeed.blob.core.windows.net/dotnet-coreclr/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORESDK'">https://dotnetfeed.blob.core.windows.net/dotnet-sdk/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSINTERNAL'">https://dotnetfeed.blob.core.windows.net/dotnet-tools-internal/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSET'">https://dotnetfeed.blob.core.windows.net/dotnet-toolset/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'WINDOWSDESKTOP'">https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'NUGETCLIENT'">https://dotnetfeed.blob.core.windows.net/nuget-nugetclient/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETENTITYFRAMEWORK6'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframework6/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETBLAZOR'">https://dotnetfeed.blob.core.windows.net/aspnet-blazor/index.json</TargetStaticFeed>
</PropertyGroup>
<Error
Condition="'$(TargetStaticFeed)' == ''"
Text="'$(ArtifactsCategory)' wasn't recognized as a valid artifact category. Valid categories are: '.NetCore' and '.NetCoreValidation'" />
<!-- Iterate publishing assets from each manifest file. -->
<PushArtifactsInManifestToFeed
ExpectedFeedUrl="$(TargetStaticFeed)"
AccountKey="$(AccountKeyToStaticFeed)"
BARBuildId="$(BARBuildId)"
MaestroApiEndpoint="$(MaestroApiEndpoint)"
BuildAssetRegistryToken="$(BuildAssetRegistryToken)"
Overwrite="$(OverrideAssetsWithSameName)"
PassIfExistingItemIdentical="$(PassIfExistingItemIdentical)"
MaxClients="$(MaxParallelUploads)"
UploadTimeoutInMinutes="$(MaxUploadTimeoutInMinutes)"
AssetManifestPath="%(ManifestFiles.Identity)"
BlobAssetsBasePath="$(BlobBasePath)"
PackageAssetsBasePath="$(PackageBasePath)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
</ItemGroup>
</Project>
-84
View File
@@ -1,84 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PublishSymbols tasks to publish symbols to MSDL and SymWeb.
Parameters:
- PDBArtifactsDirectory : Full path to directory containing PDB files to be published.
- BlobBasePath : Full path containing *.symbols.nupkg packages to be published.
- DotNetSymbolServerTokenMsdl : PAT to access MSDL.
- DotNetSymbolServerTokenSymWeb : PAT to access SymWeb.
- DotNetSymbolExpirationInDays : Expiration days for published packages. Default is 3650.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(NuGetPackageRoot)microsoft.symboluploader.build.task\$(SymbolUploaderVersion)\build\PublishSymbols.targets" />
<Target Name="PublishSymbols">
<ItemGroup>
<FilesToPublishToSymbolServer Include="$(PDBArtifactsDirectory)\*.pdb"/>
<PackagesToPublishToSymbolServer Include="$(BlobBasePath)\*.symbols.nupkg"/>
<!--
These packages from Arcade-Services include some native libraries that
our current symbol uploader can't handle. Below is a workaround until
we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
-->
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Darc.*" />
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Maestro.Tasks.*" />
</ItemGroup>
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymWeb Condition="'$(PublishToSymWeb)' == ''">true</PublishToSymWeb>
<PublishToMSDL Condition="'$(PublishToMSDL)' == ''">true</PublishToMSDL>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
<Message
Importance="High"
Text="No symbol package(s) were found to publish."
Condition="$(PublishToSymbolServer) == false" />
<!-- Symbol Uploader: MSDL -->
<Message Importance="High" Text="Publishing symbol packages to MSDL ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenMsdl)"
SymbolServerPath="https://microsoftpublicsymbols.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer) and $(PublishToMSDL)"/>
<!--
Symbol Uploader: SymWeb
Watson, VS insertion testings and the typical internal dev usage require SymWeb.
Currently we need to call the task twice (https://github.com/dotnet/core-eng/issues/3489).
-->
<Message Importance="High" Text="Publishing symbol packages to SymWeb ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenSymWeb)"
SymbolServerPath="https://microsoft.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer) and $(PublishToSymWeb)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.SymbolUploader.Build.Task" Version="$(SymbolUploaderVersion)" />
</ItemGroup>
</Project>
-28
View File
@@ -1,28 +0,0 @@
# Don't touch this folder
uuuuuuuuuuuuuuuuuuuu
u" uuuuuuuuuuuuuuuuuu "u
u" u$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
$ $$$" ... "$... ...$" ... "$$$ ... "$$$ $
$ $$$u `"$$$$$$$ $$$ $$$$$ $$ $$$ $$$ $
$ $$$$$$uu "$$$$ $$$ $$$$$ $$ """ u$$$ $
$ $$$""$$$ $$$$ $$$u "$$$" u$$ $$$$$$$$ $
$ $$$$....,$$$$$..$$$$$....,$$$$..$$$$$$$$ $
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$" u"
"u """""""""""""""""" u"
""""""""""""""""""""
!!! Changes made in this directory are subject to being overwritten by automation !!!
The files in this directory are shared by all Arcade repos and managed by automation. If you need to make changes to these files, open an issue or submit a pull request to https://github.com/dotnet/arcade first.
-127
View File
@@ -1,127 +0,0 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile ${Env:BUILD_SOURCESDIRECTORY}/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-Host "Couldn't find the file NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
$doc.Save($filename)
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $BUILD_SOURCESDIRECTORY/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
echo "Couldn't find the file NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
# Ensure dotnet3-internal and dotnet3-internal-transport is in the packageSources
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" NuGet.config
fi
# Ensure dotnet3-internal and dotnet3-internal-transport is in the packageSources
grep -i "<add key=\"dotnet3-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" NuGet.config
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
PackageSources+=('dotnet3-internal')
PackageSources+=('dotnet3-internal-transport')
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" NuGet.config
fi
done
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
- PackageBasePath : Directory containing all files that need to be validated.
- SignCheckVersion : Version of SignCheck package to be used.
- SignValidationExclusionList : ItemGroup containing exclusion list to be forwarded to SignCheck.
- EnableJarSigningCheck : Whether .jar files should be validated.
- EnableStrongNameCheck : Whether strong name check should be performed.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<!--
From 'Signing.props' we import $(SignValidationExclusionList)
-->
<Import Project="$(MSBuildThisFileDirectory)Signing.props" Condition="Exists('$(MSBuildThisFileDirectory)Signing.props')" />
<Target Name="ValidateSigning">
<PropertyGroup>
<SignCheckToolPath>$(NuGetPackageRoot)Microsoft.DotNet.SignCheck\$(SignCheckVersion)\tools\Microsoft.DotNet.SignCheck.exe</SignCheckToolPath>
<SignCheckInputDir>$(PackageBasePath)</SignCheckInputDir>
<SignCheckLog>signcheck.log</SignCheckLog>
<SignCheckErrorLog>signcheck.errors.log</SignCheckErrorLog>
<SignCheckExclusionsFile>signcheck.exclusions.txt</SignCheckExclusionsFile>
</PropertyGroup>
<ItemGroup>
<!--
Documentation for these arguments is available here:
https://github.com/dotnet/arcade/tree/master/src/SignCheck
-->
<SignCheckArgs Include="--recursive" />
<SignCheckArgs Include="--traverse-subfolders" />
<SignCheckArgs Include="--file-status AllFiles" />
<SignCheckArgs Include="--log-file $(SignCheckLog)" />
<SignCheckArgs Include="--error-log-file $(SignCheckErrorLog)" />
<SignCheckArgs Include="--input-files $(SignCheckInputDir)" />
<SignCheckArgs Include="--exclusions-file $(SignCheckExclusionsFile)" Condition="'@(SignValidationExclusionList)' != ''" />
<SignCheckArgs Include="--verify-jar" Condition="'$(EnableJarSigningCheck)' == 'true'" />
<SignCheckArgs Include="--verify-strongname" Condition="'$(EnableStrongNameCheck)' == 'true'" />
</ItemGroup>
<WriteLinesToFile
File="$(SignCheckExclusionsFile)"
Lines="@(SignValidationExclusionList)"
Condition="'@(SignValidationExclusionList)' != ''"
Overwrite="true"
Encoding="Unicode"/>
<!--
IgnoreExitCode='true' because the tool doesn't return '0' on success.
-->
<Exec
Command="&quot;$(SignCheckToolPath)&quot; @(SignCheckArgs, ' ')"
IgnoreExitCode='true'
ConsoleToMsBuild="false"
StandardErrorImportance="high" />
<Error
Text="Signing validation failed. Check $(SignCheckErrorLog) for more information."
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
<Message
Text="##vso[artifact.upload containerfolder=LogFiles;artifactname=LogFiles]{SignCheckErrorLog}"
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.SignCheck" Version="$(SignCheckVersion)" />
</ItemGroup>
</Project>
-184
View File
@@ -1,184 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SourceLinkToolPath, # Full path to directory where dotnet SourceLink CLI was installed
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
$RepoTreeURL = -Join("https://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
Measure-Command { ValidateSourceLinkLinks }
-148
View File
@@ -1,148 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string][Alias('c')]$configuration = "Debug",
[string]$platform = $null,
[string] $projects,
[string][Alias('v')]$verbosity = "minimal",
[string] $msbuildEngine = $null,
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch][Alias('r')]$restore,
[switch] $deployDeps,
[switch][Alias('b')]$build,
[switch] $rebuild,
[switch] $deploy,
[switch][Alias('t')]$test,
[switch] $integrationTest,
[switch] $performanceTest,
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
[switch] $help,
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot "restore-toolset.ps1"
if (Test-Path $script) {
. $script
}
}
function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
MSBuild $toolsetBuildProj `
$bl `
$platformArg `
/p:Configuration=$configuration `
/p:RepoRoot=$RepoRoot `
/p:Restore=$restore `
/p:DeployDeps=$deployDeps `
/p:Build=$build `
/p:Rebuild=$rebuild `
/p:Deploy=$deploy `
/p:Test=$test `
/p:Pack=$pack `
/p:IntegrationTest=$integrationTest `
/p:PerformanceTest=$performanceTest `
/p:Sign=$sign `
/p:Publish=$publish `
@properties
}
if ($clean) {
if(Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host "Artifacts directory deleted."
}
exit 0
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
exit 0
}
if ($ci) {
$binaryLog = $true
$nodeReuse = $false
}
if ($restore) {
InitializeNativeTools
}
Build
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
ExitWithExitCode 1
}
ExitWithExitCode 0
-216
View File
@@ -1,216 +0,0 @@
#!/usr/bin/env bash
# Stop script if unbound variable found (use ${var:-} if intentional)
set -u
# Stop script if command returns non-zero exit code.
# Prevents hidden errors caused by missing error code propagation.
set -e
usage()
{
echo "Common settings:"
echo " --configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
echo " --verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
echo " --binaryLog Create MSBuild binary log (short: -bl)"
echo " --help Print help and exit (short: -h)"
echo ""
echo "Actions:"
echo " --restore Restore dependencies (short: -r)"
echo " --build Build solution (short: -b)"
echo " --rebuild Rebuild solution"
echo " --test Run all unit tests in the solution (short: -t)"
echo " --integrationTest Run all integration tests in the solution"
echo " --performanceTest Run all performance tests in the solution"
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
echo " --projects <value> Project or solution file(s) to build"
echo " --ci Set when running on CI server"
echo " --prepareMachine Prepare machine for CI run, clean up processes after build"
echo " --nodeReuse <value> Sets nodereuse msbuild parameter ('true' or 'false')"
echo " --warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
echo ""
echo "Command line arguments not listed above are passed thru to msbuild."
echo "Arguments can also be passed in with a single hyphen."
}
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
restore=false
build=false
rebuild=false
test=false
integration_test=false
performance_test=false
pack=false
publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
binary_log=false
pipelines_log=false
projects=''
configuration='Debug'
prepare_machine=false
verbosity='minimal'
properties=''
while [[ $# > 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-help|-h)
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
;;
-verbosity|-v)
verbosity=$2
shift
;;
-binarylog|-bl)
binary_log=true
;;
-pipelineslog|-pl)
pipelines_log=true
;;
-restore|-r)
restore=true
;;
-build|-b)
build=true
;;
-rebuild)
rebuild=true
;;
-pack)
pack=true
;;
-test|-t)
test=true
;;
-integrationtest)
integration_test=true
;;
-performancetest)
performance_test=true
;;
-sign)
sign=true
;;
-publish)
publish=true
;;
-preparemachine)
prepare_machine=true
;;
-projects)
projects=$2
shift
;;
-ci)
ci=true
;;
-warnaserror)
warn_as_error=$2
shift
;;
-nodereuse)
node_reuse=$2
shift
;;
*)
properties="$properties $1"
;;
esac
shift
done
if [[ "$ci" == true ]]; then
pipelines_log=true
binary_log=true
node_reuse=false
fi
. "$scriptroot/tools.sh"
function InitializeCustomToolset {
local script="$eng_root/restore-toolset.sh"
if [[ -a "$script" ]]; then
. "$script"
fi
}
function Build {
InitializeToolset
InitializeCustomToolset
if [[ ! -z "$projects" ]]; then
properties="$properties /p:Projects=$projects"
fi
local bl=""
if [[ "$binary_log" == true ]]; then
bl="/bl:\"$log_dir/Build.binlog\""
fi
MSBuild $_InitializeToolset \
$bl \
/p:Configuration=$configuration \
/p:RepoRoot="$repo_root" \
/p:Restore=$restore \
/p:Build=$build \
/p:Rebuild=$rebuild \
/p:Test=$test \
/p:Pack=$pack \
/p:IntegrationTest=$integration_test \
/p:PerformanceTest=$performance_test \
/p:Sign=$sign \
/p:Publish=$publish \
$properties
ExitWithExitCode 0
}
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
fi
if [[ "$restore" == true ]]; then
InitializeNativeTools
fi
Build
-16
View File
@@ -1,16 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $SOURCE until the file is no longer a symlink
while [[ -h $source ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where
# the symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/build.sh" --restore --build --test --pack --publish --ci $@
@@ -1,41 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)
## Specify the toolchain
set(TOOLCHAIN "arm-linux-androideabi")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
add_compile_options(-mfloat-abi=soft)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/arm-linux-androideabi/)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
@@ -1,42 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
## Specify the toolchain
set(TOOLCHAIN "aarch64-linux-android")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
## Needed for Android or bionic specific conditionals
add_compile_options(-D__ANDROID__)
add_compile_options(-D__BIONIC__)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
-3
View File
@@ -1,3 +0,0 @@
# Debian (sid) # UNSTABLE
deb http://ftp.debian.org/debian/ sid main contrib non-free
deb-src http://ftp.debian.org/debian/ sid main contrib non-free
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
@@ -1,71 +0,0 @@
From e72c9d7ead60e3317bd6d1fade995c07021c947b Mon Sep 17 00:00:00 2001
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 7 May 2015 13:25:04 -0400
Subject: [PATCH] Fix: building probe providers with C++ compiler
Robert Daniels wrote:
> > I'm attempting to use lttng userspace tracing with a C++ application
> > on an ARM platform. I'm using GCC 4.8.4 on Linux 3.14 with the 2.6
> > release of lttng. I've compiled lttng-modules, lttng-ust, and
> > lttng-tools and have been able to get a simple test working with C
> > code. When I attempt to run the hello.cxx test on my target it will
> > segfault.
>
>
> I spent a little time digging into this issue and finally discovered the
> cause of my segfault with ARM C++ tracepoints.
>
> There is a struct called 'lttng_event' in ust-events.h which contains an
> empty union 'u'. This was the cause of my issue. Under C, this empty union
> compiles to a zero byte member while under C++ it compiles to a one byte
> member, and in my case was four-byte aligned which caused my C++ code to
> have the 'cds_list_head node' offset incorrectly by four bytes. This lead
> to an incorrect linked list structure which caused my issue.
>
> Since this union is empty, I simply removed it from the struct and everything
> worked correctly.
>
> I don't know the history or purpose behind this empty union so I'd like to
> know if this is a safe fix. If it is I can submit a patch with the union
> removed.
That's a very nice catch!
We do not support building tracepoint probe provider with
g++ yet, as stated in lttng-ust(3):
"- Note for C++ support: although an application instrumented with
tracepoints can be compiled with g++, tracepoint probes should be
compiled with gcc (only tested with gcc so far)."
However, if it works fine with this fix, then I'm tempted to take it,
especially because removing the empty union does not appear to affect
the layout of struct lttng_event as seen from liblttng-ust, which must
be compiled with a C compiler, and from probe providers compiled with
a C compiler. So all we are changing is the layout of a probe provider
compiled with a C++ compiler, which is anyway buggy at the moment,
because it is not compatible with the layout expected by liblttng-ust
compiled with a C compiler.
Reported-by: Robert Daniels <[email protected]>
Signed-off-by: Mathieu Desnoyers <[email protected]>
---
include/lttng/ust-events.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/usr/include/lttng/ust-events.h b/usr/include/lttng/ust-events.h
index 328a875..3d7a274 100644
--- a/usr/include/lttng/ust-events.h
+++ b/usr/include/lttng/ust-events.h
@@ -407,8 +407,6 @@ struct lttng_event {
void *_deprecated1;
struct lttng_ctx *ctx;
enum lttng_ust_instrumentation instrumentation;
- union {
- } u;
struct cds_list_head node; /* Event list in session */
struct cds_list_head _deprecated2;
void *_deprecated3;
--
2.7.4
-97
View File
@@ -1,97 +0,0 @@
diff -u -r a/usr/include/urcu/uatomic/generic.h b/usr/include/urcu/uatomic/generic.h
--- a/usr/include/urcu/uatomic/generic.h 2014-03-28 06:04:42.000000000 +0900
+++ b/usr/include/urcu/uatomic/generic.h 2017-02-13 10:35:21.189927116 +0900
@@ -65,17 +65,17 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- return __sync_val_compare_and_swap_1(addr, old, _new);
+ return __sync_val_compare_and_swap_1((uint8_t *) addr, old, _new);
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- return __sync_val_compare_and_swap_2(addr, old, _new);
+ return __sync_val_compare_and_swap_2((uint16_t *) addr, old, _new);
#endif
case 4:
- return __sync_val_compare_and_swap_4(addr, old, _new);
+ return __sync_val_compare_and_swap_4((uint32_t *) addr, old, _new);
#if (CAA_BITS_PER_LONG == 64)
case 8:
- return __sync_val_compare_and_swap_8(addr, old, _new);
+ return __sync_val_compare_and_swap_8((uint64_t *) addr, old, _new);
#endif
}
_uatomic_link_error();
@@ -100,20 +100,20 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- __sync_and_and_fetch_1(addr, val);
+ __sync_and_and_fetch_1((uint8_t *) addr, val);
return;
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- __sync_and_and_fetch_2(addr, val);
+ __sync_and_and_fetch_2((uint16_t *) addr, val);
return;
#endif
case 4:
- __sync_and_and_fetch_4(addr, val);
+ __sync_and_and_fetch_4((uint32_t *) addr, val);
return;
#if (CAA_BITS_PER_LONG == 64)
case 8:
- __sync_and_and_fetch_8(addr, val);
+ __sync_and_and_fetch_8((uint64_t *) addr, val);
return;
#endif
}
@@ -139,20 +139,20 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- __sync_or_and_fetch_1(addr, val);
+ __sync_or_and_fetch_1((uint8_t *) addr, val);
return;
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- __sync_or_and_fetch_2(addr, val);
+ __sync_or_and_fetch_2((uint16_t *) addr, val);
return;
#endif
case 4:
- __sync_or_and_fetch_4(addr, val);
+ __sync_or_and_fetch_4((uint32_t *) addr, val);
return;
#if (CAA_BITS_PER_LONG == 64)
case 8:
- __sync_or_and_fetch_8(addr, val);
+ __sync_or_and_fetch_8((uint64_t *) addr, val);
return;
#endif
}
@@ -180,17 +180,17 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- return __sync_add_and_fetch_1(addr, val);
+ return __sync_add_and_fetch_1((uint8_t *) addr, val);
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- return __sync_add_and_fetch_2(addr, val);
+ return __sync_add_and_fetch_2((uint16_t *) addr, val);
#endif
case 4:
- return __sync_add_and_fetch_4(addr, val);
+ return __sync_add_and_fetch_4((uint32_t *) addr, val);
#if (CAA_BITS_PER_LONG == 64)
case 8:
- return __sync_add_and_fetch_8(addr, val);
+ return __sync_add_and_fetch_8((uint64_t *) addr, val);
#endif
}
_uatomic_link_error();
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
@@ -1,11 +0,0 @@
deb http://deb.debian.org/debian buster main
deb-src http://deb.debian.org/debian buster main
deb http://deb.debian.org/debian-security/ buster/updates main
deb-src http://deb.debian.org/debian-security/ buster/updates main
deb http://deb.debian.org/debian buster-updates main
deb-src http://deb.debian.org/debian buster-updates main
deb http://deb.debian.org/debian buster-backports main contrib non-free
deb-src http://deb.debian.org/debian buster-backports main contrib non-free
@@ -1,12 +0,0 @@
deb http://deb.debian.org/debian stretch main
deb-src http://deb.debian.org/debian stretch main
deb http://deb.debian.org/debian-security/ stretch/updates main
deb-src http://deb.debian.org/debian-security/ stretch/updates main
deb http://deb.debian.org/debian stretch-updates main
deb-src http://deb.debian.org/debian stretch-updates main
deb http://deb.debian.org/debian stretch-backports main contrib non-free
deb-src http://deb.debian.org/debian stretch-backports main contrib non-free
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
@@ -1,3 +0,0 @@
# Debian (jessie) # Stable
deb http://ftp.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.debian.org/debian/ jessie main contrib non-free
@@ -1,44 +0,0 @@
#!/usr/bin/env bash
set -e
__ARM_SOFTFP_CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__TIZEN_CROSSDIR="$__ARM_SOFTFP_CrossDir/tizen"
if [[ -z "$ROOTFS_DIR" ]]; then
echo "ROOTFS_DIR is not defined."
exit 1;
fi
# Clean-up (TODO-Cleanup: We may already delete $ROOTFS_DIR at ./cross/build-rootfs.sh.)
# hk0110
if [ -d "$ROOTFS_DIR" ]; then
umount $ROOTFS_DIR/*
rm -rf $ROOTFS_DIR
fi
TIZEN_TMP_DIR=$ROOTFS_DIR/tizen_tmp
mkdir -p $TIZEN_TMP_DIR
# Download files
echo ">>Start downloading files"
VERBOSE=1 $__ARM_SOFTFP_CrossDir/tizen-fetch.sh $TIZEN_TMP_DIR
echo "<<Finish downloading files"
echo ">>Start constructing Tizen rootfs"
TIZEN_RPM_FILES=`ls $TIZEN_TMP_DIR/*.rpm`
cd $ROOTFS_DIR
for f in $TIZEN_RPM_FILES; do
rpm2cpio $f | cpio -idm --quiet
done
echo "<<Finish constructing Tizen rootfs"
# Cleanup tmp
rm -rf $TIZEN_TMP_DIR
# Configure Tizen rootfs
echo ">>Start configuring Tizen rootfs"
rm ./usr/lib/libunwind.so
ln -s libunwind.so.8 ./usr/lib/libunwind.so
ln -sfn asm-arm ./usr/include/asm
patch -p1 < $__TIZEN_CROSSDIR/tizen.patch
echo "<<Finish configuring Tizen rootfs"
-171
View File
@@ -1,171 +0,0 @@
#!/usr/bin/env bash
set -e
if [[ -z "${VERBOSE// }" ]] || [ "$VERBOSE" -ne "$VERBOSE" ] 2>/dev/null; then
VERBOSE=0
fi
Log()
{
if [ $VERBOSE -ge $1 ]; then
echo ${@:2}
fi
}
Inform()
{
Log 1 -e "\x1B[0;34m$@\x1B[m"
}
Debug()
{
Log 2 -e "\x1B[0;32m$@\x1B[m"
}
Error()
{
>&2 Log 0 -e "\x1B[0;31m$@\x1B[m"
}
Fetch()
{
URL=$1
FILE=$2
PROGRESS=$3
if [ $VERBOSE -ge 1 ] && [ $PROGRESS ]; then
CURL_OPT="--progress-bar"
else
CURL_OPT="--silent"
fi
curl $CURL_OPT $URL > $FILE
}
hash curl 2> /dev/null || { Error "Require 'curl' Aborting."; exit 1; }
hash xmllint 2> /dev/null || { Error "Require 'xmllint' Aborting."; exit 1; }
hash sha256sum 2> /dev/null || { Error "Require 'sha256sum' Aborting."; exit 1; }
TMPDIR=$1
if [ ! -d $TMPDIR ]; then
TMPDIR=./tizen_tmp
Debug "Create temporary directory : $TMPDIR"
mkdir -p $TMPDIR
fi
TIZEN_URL=http://download.tizen.org/releases/milestone/tizen
BUILD_XML=build.xml
REPOMD_XML=repomd.xml
PRIMARY_XML=primary.xml
TARGET_URL="http://__not_initialized"
Xpath_get()
{
XPATH_RESULT=''
XPATH=$1
XML_FILE=$2
RESULT=$(xmllint --xpath $XPATH $XML_FILE)
if [[ -z ${RESULT// } ]]; then
Error "Can not find target from $XML_FILE"
Debug "Xpath = $XPATH"
exit 1
fi
XPATH_RESULT=$RESULT
}
fetch_tizen_pkgs_init()
{
TARGET=$1
PROFILE=$2
Debug "Initialize TARGET=$TARGET, PROFILE=$PROFILE"
TMP_PKG_DIR=$TMPDIR/tizen_${PROFILE}_pkgs
if [ -d $TMP_PKG_DIR ]; then rm -rf $TMP_PKG_DIR; fi
mkdir -p $TMP_PKG_DIR
PKG_URL=$TIZEN_URL/$PROFILE/latest
BUILD_XML_URL=$PKG_URL/$BUILD_XML
TMP_BUILD=$TMP_PKG_DIR/$BUILD_XML
TMP_REPOMD=$TMP_PKG_DIR/$REPOMD_XML
TMP_PRIMARY=$TMP_PKG_DIR/$PRIMARY_XML
TMP_PRIMARYGZ=${TMP_PRIMARY}.gz
Fetch $BUILD_XML_URL $TMP_BUILD
Debug "fetch $BUILD_XML_URL to $TMP_BUILD"
TARGET_XPATH="//build/buildtargets/buildtarget[@name=\"$TARGET\"]/repo[@type=\"binary\"]/text()"
Xpath_get $TARGET_XPATH $TMP_BUILD
TARGET_PATH=$XPATH_RESULT
TARGET_URL=$PKG_URL/$TARGET_PATH
REPOMD_URL=$TARGET_URL/repodata/repomd.xml
PRIMARY_XPATH='string(//*[local-name()="data"][@type="primary"]/*[local-name()="location"]/@href)'
Fetch $REPOMD_URL $TMP_REPOMD
Debug "fetch $REPOMD_URL to $TMP_REPOMD"
Xpath_get $PRIMARY_XPATH $TMP_REPOMD
PRIMARY_XML_PATH=$XPATH_RESULT
PRIMARY_URL=$TARGET_URL/$PRIMARY_XML_PATH
Fetch $PRIMARY_URL $TMP_PRIMARYGZ
Debug "fetch $PRIMARY_URL to $TMP_PRIMARYGZ"
gunzip $TMP_PRIMARYGZ
Debug "unzip $TMP_PRIMARYGZ to $TMP_PRIMARY"
}
fetch_tizen_pkgs()
{
ARCH=$1
PACKAGE_XPATH_TPL='string(//*[local-name()="metadata"]/*[local-name()="package"][*[local-name()="name"][text()="_PKG_"]][*[local-name()="arch"][text()="_ARCH_"]]/*[local-name()="location"]/@href)'
PACKAGE_CHECKSUM_XPATH_TPL='string(//*[local-name()="metadata"]/*[local-name()="package"][*[local-name()="name"][text()="_PKG_"]][*[local-name()="arch"][text()="_ARCH_"]]/*[local-name()="checksum"]/text())'
for pkg in ${@:2}
do
Inform "Fetching... $pkg"
XPATH=${PACKAGE_XPATH_TPL/_PKG_/$pkg}
XPATH=${XPATH/_ARCH_/$ARCH}
Xpath_get $XPATH $TMP_PRIMARY
PKG_PATH=$XPATH_RESULT
XPATH=${PACKAGE_CHECKSUM_XPATH_TPL/_PKG_/$pkg}
XPATH=${XPATH/_ARCH_/$ARCH}
Xpath_get $XPATH $TMP_PRIMARY
CHECKSUM=$XPATH_RESULT
PKG_URL=$TARGET_URL/$PKG_PATH
PKG_FILE=$(basename $PKG_PATH)
PKG_PATH=$TMPDIR/$PKG_FILE
Debug "Download $PKG_URL to $PKG_PATH"
Fetch $PKG_URL $PKG_PATH true
echo "$CHECKSUM $PKG_PATH" | sha256sum -c - > /dev/null
if [ $? -ne 0 ]; then
Error "Fail to fetch $PKG_URL to $PKG_PATH"
Debug "Checksum = $CHECKSUM"
exit 1
fi
done
}
Inform "Initialize arm base"
fetch_tizen_pkgs_init standard base
Inform "fetch common packages"
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel libatomic
fetch_tizen_pkgs noarch linux-glibc-devel
Inform "fetch coreclr packages"
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l libcom_err libcom_err-devel zlib zlib-devel libopenssl libopenssl-devel krb5 krb5-devel libcurl libcurl-devel
Inform "Initialize standard unified"
fetch_tizen_pkgs_init standard unified
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l gssdp gssdp-devel tizen-release
@@ -1,50 +0,0 @@
lang en_US.UTF-8
keyboard us
timezone --utc Asia/Seoul
part / --fstype="ext4" --size=3500 --ondisk=mmcblk0 --label rootfs --fsoptions=defaults,noatime
rootpw tizen
desktop --autologinuser=root
user --name root --groups audio,video --password 'tizen'
repo --name=standard --baseurl=http://download.tizen.org/releases/milestone/tizen/unified/latest/repos/standard/packages/ --ssl_verify=no
repo --name=base --baseurl=http://download.tizen.org/releases/milestone/tizen/base/latest/repos/standard/packages/ --ssl_verify=no
%packages
tar
gzip
sed
grep
gawk
perl
binutils
findutils
util-linux
lttng-ust
userspace-rcu
procps-ng
tzdata
ca-certificates
### Core FX
libicu
libunwind
iputils
zlib
krb5
libcurl
libopenssl
%end
%post
### Update /tmp privilege
chmod 777 /tmp
####################################
%end
-18
View File
@@ -1,18 +0,0 @@
diff -u -r a/usr/lib/libc.so b/usr/lib/libc.so
--- a/usr/lib/libc.so 2016-12-30 23:00:08.284951863 +0900
+++ b/usr/lib/libc.so 2016-12-30 23:00:32.140951815 +0900
@@ -2,4 +2,4 @@
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
-GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.3 ) )
+GROUP ( libc.so.6 libc_nonshared.a AS_NEEDED ( ld-linux.so.3 ) )
diff -u -r a/usr/lib/libpthread.so b/usr/lib/libpthread.so
--- a/usr/lib/libpthread.so 2016-12-30 23:00:19.408951841 +0900
+++ b/usr/lib/libpthread.so 2016-12-30 23:00:39.068951801 +0900
@@ -2,4 +2,4 @@
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
-GROUP ( /lib/libpthread.so.0 /usr/lib/libpthread_nonshared.a )
+GROUP ( libpthread.so.0 libpthread_nonshared.a )
-137
View File
@@ -1,137 +0,0 @@
#!/usr/bin/env bash
set -e
__NDK_Version=r14
usage()
{
echo "Creates a toolchain and sysroot used for cross-compiling for Android."
echo.
echo "Usage: $0 [BuildArch] [ApiLevel]"
echo.
echo "BuildArch is the target architecture of Android. Currently only arm64 is supported."
echo "ApiLevel is the target Android API level. API levels usually match to Android releases. See https://source.android.com/source/build-numbers.html"
echo.
echo "By default, the toolchain and sysroot will be generated in cross/android-rootfs/toolchain/[BuildArch]. You can change this behavior"
echo "by setting the TOOLCHAIN_DIR environment variable"
echo.
echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"
echo "you can set the NDK_DIR environment variable to have this script use that installation of the NDK."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.21-arm64. This file is to replace '/etc/os-release', which is not available for Android."
exit 1
}
__ApiLevel=21 # The minimum platform for arm64 is API level 21
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
for i in "$@"
do
lowerI="$(echo $i | awk '{print tolower($0)}')"
case $lowerI in
-?|-h|--help)
usage
exit 1
;;
arm64)
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
;;
arm)
__BuildArch=arm
__AndroidArch=arm
__AndroidToolchain=arm-linux-androideabi
;;
*[0-9])
__ApiLevel=$i
;;
*)
__UnprocessedBuildArgs="$__UnprocessedBuildArgs $i"
;;
esac
done
# Obtain the location of the bash script to figure out where the root of the repo is.
__CrossDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__Android_Cross_Dir="$__CrossDir/android-rootfs"
__NDK_Dir="$__Android_Cross_Dir/android-ndk-$__NDK_Version"
__libunwind_Dir="$__Android_Cross_Dir/libunwind"
__lldb_Dir="$__Android_Cross_Dir/lldb"
__ToolchainDir="$__Android_Cross_Dir/toolchain/$__BuildArch"
if [[ -n "$TOOLCHAIN_DIR" ]]; then
__ToolchainDir=$TOOLCHAIN_DIR
fi
if [[ -n "$NDK_DIR" ]]; then
__NDK_Dir=$NDK_DIR
fi
echo "Target API level: $__ApiLevel"
echo "Target architecture: $__BuildArch"
echo "NDK location: $__NDK_Dir"
echo "Target Toolchain location: $__ToolchainDir"
# Download the NDK if required
if [ ! -d $__NDK_Dir ]; then
echo Downloading the NDK into $__NDK_Dir
mkdir -p $__NDK_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__Android_Cross_Dir
fi
if [ ! -d $__lldb_Dir ]; then
mkdir -p $__lldb_Dir
echo Downloading LLDB into $__lldb_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
fi
# Create the RootFS for both arm64 as well as aarch
rm -rf $__Android_Cross_Dir/toolchain
echo Generating the $__BuildArch toolchain
$__NDK_Dir/build/tools/make_standalone_toolchain.py --arch $__BuildArch --api $__ApiLevel --install-dir $__ToolchainDir
# Install the required packages into the toolchain
# TODO: Add logic to get latest pkg version instead of specific version number
rm -rf $__Android_Cross_Dir/deb/
rm -rf $__Android_Cross_Dir/tmp
mkdir -p $__Android_Cross_Dir/deb/
mkdir -p $__Android_Cross_Dir/tmp/$arch/
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu-dev_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob-dev_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support-dev_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma-dev_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind-dev_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb
echo Unpacking Termux packages
dpkg -x $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
cp -R $__Android_Cross_Dir/tmp/$__AndroidArch/data/data/com.termux/files/usr/* $__ToolchainDir/sysroot/usr/
# Generate platform file for build.sh script to assign to __DistroRid
echo "Generating platform file..."
echo "RID=android.21-arm64" > $__ToolchainDir/sysroot/android_platform
echo Now run:
echo CONFIG_DIR=\`realpath cross/android/$__BuildArch\` ROOTFS_DIR=\`realpath $__ToolchainDir/sysroot\` ./build.sh cross $__BuildArch skipgenerateversion skipnuget cmakeargs -DENABLE_LLDBPLUGIN=0
-234
View File
@@ -1,234 +0,0 @@
#!/usr/bin/env bash
usage()
{
echo "Usage: $0 [BuildArch] [LinuxCodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "BuildArch can be: arm(default), armel, arm64, x86"
echo "LinuxCodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine"
echo "--skipunmount - optional, will skip the unmount of rootfs folder."
exit 1
}
__LinuxCodeName=xenial
__CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__InitialDir=$PWD
__BuildArch=arm
__UbuntuArch=armhf
__UbuntuRepo="http://ports.ubuntu.com/"
__LLDB_Package="liblldb-3.9-dev"
__SkipUnmount=0
# base development support
__UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
# symlinks fixer
__UbuntuPackages+=" symlinks"
# CoreCLR and CoreFX dependencies
__UbuntuPackages+=" libicu-dev"
__UbuntuPackages+=" liblttng-ust-dev"
__UbuntuPackages+=" libunwind8-dev"
__AlpinePackages+=" gettext-dev"
__AlpinePackages+=" icu-dev"
__AlpinePackages+=" libunwind-dev"
__AlpinePackages+=" lttng-ust-dev"
# CoreFX dependencies
__UbuntuPackages+=" libcurl4-openssl-dev"
__UbuntuPackages+=" libkrb5-dev"
__UbuntuPackages+=" libssl-dev"
__UbuntuPackages+=" zlib1g-dev"
__AlpinePackages+=" curl-dev"
__AlpinePackages+=" krb5-dev"
__AlpinePackages+=" openssl-dev"
__AlpinePackages+=" zlib-dev"
__UnprocessedBuildArgs=
while :; do
if [ $# -le 0 ]; then
break
fi
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
-?|-h|--help)
usage
exit 1
;;
arm)
__BuildArch=arm
__UbuntuArch=armhf
__AlpineArch=armhf
__QEMUArch=arm
;;
arm64)
__BuildArch=arm64
__UbuntuArch=arm64
__AlpineArch=aarch64
__QEMUArch=aarch64
;;
armel)
__BuildArch=armel
__UbuntuArch=armel
__UbuntuRepo="http://ftp.debian.org/debian/"
__LinuxCodeName=jessie
;;
x86)
__BuildArch=x86
__UbuntuArch=i386
__UbuntuRepo="http://archive.ubuntu.com/ubuntu/"
;;
lldb3.6)
__LLDB_Package="lldb-3.6-dev"
;;
lldb3.8)
__LLDB_Package="lldb-3.8-dev"
;;
lldb3.9)
__LLDB_Package="liblldb-3.9-dev"
;;
lldb4.0)
__LLDB_Package="liblldb-4.0-dev"
;;
lldb5.0)
__LLDB_Package="liblldb-5.0-dev"
;;
lldb6.0)
__LLDB_Package="liblldb-6.0-dev"
;;
no-lldb)
unset __LLDB_Package
;;
trusty) # Ubuntu 14.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=trusty
fi
;;
xenial) # Ubuntu 16.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=xenial
fi
;;
zesty) # Ubuntu 17.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=zesty
fi
;;
bionic) # Ubuntu 18.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=bionic
fi
;;
jessie) # Debian 8
__LinuxCodeName=jessie
__UbuntuRepo="http://ftp.debian.org/debian/"
;;
stretch) # Debian 9
__LinuxCodeName=stretch
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
buster) # Debian 10
__LinuxCodeName=buster
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
tizen)
if [ "$__BuildArch" != "armel" ]; then
echo "Tizen is available only for armel."
usage;
exit 1;
fi
__LinuxCodeName=
__UbuntuRepo=
__Tizen=tizen
;;
alpine)
__LinuxCodeName=alpine
__UbuntuRepo=
;;
--skipunmount)
__SkipUnmount=1
;;
--rootfsdir|-rootfsdir)
shift
__RootfsDir=$1
;;
*)
__UnprocessedBuildArgs="$__UnprocessedBuildArgs $1"
;;
esac
shift
done
if [ "$__BuildArch" == "armel" ]; then
__LLDB_Package="lldb-3.5-dev"
fi
__UbuntuPackages+=" ${__LLDB_Package:-}"
if [ -z "$__RootfsDir" ] && [ ! -z "$ROOTFS_DIR" ]; then
__RootfsDir=$ROOTFS_DIR
fi
if [ -z "$__RootfsDir" ]; then
__RootfsDir="$__CrossDir/../../../.tools/rootfs/$__BuildArch"
fi
if [ -d "$__RootfsDir" ]; then
if [ $__SkipUnmount == 0 ]; then
umount $__RootfsDir/*
fi
rm -rf $__RootfsDir
fi
if [[ "$__LinuxCodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.7
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
rm -r $__ApkToolsDir
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__LinuxCodeName $__RootfsDir/etc/apt/sources.list
chroot $__RootfsDir apt-get update
chroot $__RootfsDir apt-get -f -y install
chroot $__RootfsDir apt-get -y install $__UbuntuPackages
chroot $__RootfsDir symlinks -cr /usr
if [ $__SkipUnmount == 0 ]; then
umount $__RootfsDir/*
fi
if [[ "$__BuildArch" == "arm" && "$__LinuxCodeName" == "trusty" ]]; then
pushd $__RootfsDir
patch -p1 < $__CrossDir/$__BuildArch/trusty.patch
patch -p1 < $__CrossDir/$__BuildArch/trusty-lttng-2.4.patch
popd
fi
elif [ "$__Tizen" == "tizen" ]; then
ROOTFS_DIR=$__RootfsDir $__CrossDir/$__BuildArch/tizen-build-rootfs.sh
else
echo "Unsupported target platform."
usage;
exit 1
fi
-119
View File
@@ -1,119 +0,0 @@
set(CROSS_ROOTFS $ENV{ROOTFS_DIR})
set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
if(TARGET_ARCH_NAME STREQUAL "armel")
set(CMAKE_SYSTEM_PROCESSOR armv7l)
set(TOOLCHAIN "arm-linux-gnueabi")
if("$ENV{__DistroRid}" MATCHES "tizen.*")
set(TIZEN_TOOLCHAIN "armv7l-tizen-linux-gnueabi/6.2.1")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "arm")
set(CMAKE_SYSTEM_PROCESSOR armv7l)
if(EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/armv6-alpine-linux-musleabihf)
set(TOOLCHAIN "armv6-alpine-linux-musleabihf")
else()
set(TOOLCHAIN "arm-linux-gnueabihf")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "arm64")
set(CMAKE_SYSTEM_PROCESSOR aarch64)
if(EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/aarch64-alpine-linux-musl)
set(TOOLCHAIN "aarch64-alpine-linux-musl")
else()
set(TOOLCHAIN "aarch64-linux-gnu")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
set(CMAKE_SYSTEM_PROCESSOR i686)
set(TOOLCHAIN "i686-linux-gnu")
else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
include_directories(SYSTEM ${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}/include/c++/)
include_directories(SYSTEM ${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}/include/c++/armv7l-tizen-linux-gnueabi)
endif()
endif()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
# Specify link flags
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_link_options(-m32)
endif()
# Specify compile options
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
endif()
if(TARGET_ARCH_NAME MATCHES "^(arm|armel)$")
add_compile_options(-mthumb)
add_compile_options(-mfpu=vfpv3)
if(TARGET_ARCH_NAME STREQUAL "armel")
add_compile_options(-mfloat-abi=softfp)
if(DEFINED TIZEN_TOOLCHAIN)
add_compile_options(-Wno-deprecated-declarations) # compile-time option
add_compile_options(-D__extern_always_inline=inline) # compile-time option
endif()
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-m32)
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths for builds that need lldb.
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
else() # arm/armel case
set(LLVM_CROSS_DIR "$ENV{LLVM_ARM_HOME}")
endif()
if(LLVM_CROSS_DIR)
set(WITH_LLDB_LIBS "${LLVM_CROSS_DIR}/lib/" CACHE STRING "")
set(WITH_LLDB_INCLUDES "${LLVM_CROSS_DIR}/include" CACHE STRING "")
set(LLDB_H "${WITH_LLDB_INCLUDES}" CACHE STRING "")
set(LLDB "${LLVM_CROSS_DIR}/lib/liblldb.so" CACHE STRING "")
else()
if(TARGET_ARCH_NAME STREQUAL "x86")
set(WITH_LLDB_LIBS "${CROSS_ROOTFS}/usr/lib/i386-linux-gnu" CACHE STRING "")
set(CHECK_LLVM_DIR "${CROSS_ROOTFS}/usr/lib/llvm-3.8/include")
if(EXISTS "${CHECK_LLVM_DIR}" AND IS_DIRECTORY "${CHECK_LLVM_DIR}")
set(WITH_LLDB_INCLUDES "${CHECK_LLVM_DIR}")
else()
set(WITH_LLDB_INCLUDES "${CROSS_ROOTFS}/usr/lib/llvm-3.6/include")
endif()
else() # arm/armel case
set(WITH_LLDB_LIBS "${CROSS_ROOTFS}/usr/lib/${TOOLCHAIN}" CACHE STRING "")
set(WITH_LLDB_INCLUDES "${CROSS_ROOTFS}/usr/lib/llvm-3.6/include" CACHE STRING "")
endif()
endif()
endif()
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ bionic main restricted universe
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ xenial main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ xenial main restricted universe
deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
-38
View File
@@ -1,38 +0,0 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16",
$verbosity = "m",
$toolpath = $null
)
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list -g
if ($toolList -like "*$darcCliPackageName*") {
& "$dotnet" tool uninstall $darcCliPackageName -g
}
# If the user didn't explicitly specify the darc version,
# query the Maestro API for the correct version of darc to install.
if (-not $darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
if (-not $toolpath) {
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
}
InstallDarcCli $darcVersion
-82
View File
@@ -1,82 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
verbosity=m
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--darcversion)
darcVersion=$2
shift
;;
--versionendpoint)
versionEndpoint=$2
shift
;;
--verbosity)
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ -z "$darcVersion" ]; then
darcVersion=$(curl -X GET "$versionEndpoint" -H "accept: text/plain")
fi
function InstallDarcCli {
local darc_cli_package_name="microsoft.dotnet.darc"
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
fi
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
}
InstallDarcCli
-2
View File
@@ -1,2 +0,0 @@
@echo off
powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0dotnet-install.ps1""" %*"
-29
View File
@@ -1,29 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet",
[string] $RuntimeSourceFeed = "",
[string] $RuntimeSourceFeedKey = ""
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
-88
View File
@@ -1,88 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
-version|-v)
shift
version="$1"
;;
-architecture|-a)
shift
architecture="$1"
;;
-runtime|-r)
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
echo "Invalid argument: $1"
exit 1
;;
esac
shift
done
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv7l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
. "$scriptroot/tools.sh"
dotnetRoot="$repo_root/.dotnet"
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
ExitWithExitCode 0
@@ -1,6 +0,0 @@
param(
[string] $token
)
Write-Host "##vso[task.setvariable variable=VSS_NUGET_ACCESSTOKEN]$token"
Write-Host "##vso[task.setvariable variable=VSS_NUGET_URI_PREFIXES]https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/"
-87
View File
@@ -1,87 +0,0 @@
Param(
[Parameter(Mandatory=$true)][string] $barToken, # Token generated at https://maestro-prod.westus2.cloudapp.azure.com/Account/Tokens
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
Write-Host "Installing dot..."
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
$darcExe = Resolve-Path "$darcExe\darc.exe"
Create-Directory $outputFolder
# Generate 3 graph descriptions:
# 1. Flat with coherency information
# 2. Graphviz (dot) file
# 3. Standard dependency graph
$graphVizFilePath = "$outputFolder\graphviz.txt"
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = @( "--github-pat", "$gitHubPat", "--azdev-pat", "$azdoPat", "--password", "$barToken" )
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += @( "--include-toolset" )
}
Write-Host "Generating standard dependency graph..."
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
CheckExitCode "Generating normal dependency graph"
Write-Host "Generating flat dependency graph and graphviz file..."
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
CheckExitCode "Generating flat and graphviz dependency graph"
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
CheckExitCode "Generating graphviz image"
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
} finally {
Pop-Location
}
-26
View File
@@ -1,26 +0,0 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup>
<Language>msbuild</Language>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<ItemGroup>
<HelixWorkItem Include="WorkItem" Condition="'$(WorkItemDirectory)' != ''">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand)</Command>
<Timeout Condition="'$(WorkItemTimeout)' != ''">$(WorkItemTimeout)</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup>
<XUnitProject Include="$(XUnitProjects.Split(';'))">
<Arguments />
</XUnitProject>
</ItemGroup>
</Project>
-3
View File
@@ -1,3 +0,0 @@
@echo off
powershell -NoProfile -NoLogo -ExecutionPolicy ByPass -command "& """%~dp0init-tools-native.ps1""" %*"
exit /b %ErrorLevel%
-148
View File
@@ -1,148 +0,0 @@
<#
.SYNOPSIS
Entry point script for installing native tools
.DESCRIPTION
Reads $RepoRoot\global.json file to determine native assets to install
and executes installers for those tools
.PARAMETER BaseUri
Base file directory or Url from which to acquire tool archives
.PARAMETER InstallDirectory
Directory to install native toolset. This is a command-line override for the default
Install directory precedence order:
- InstallDirectory command-line override
- NETCOREENG_INSTALL_DIRECTORY environment variable
- (default) %USERPROFILE%/.netcoreeng/native
.PARAMETER Clean
Switch specifying to not install anything, but cleanup native asset folders
.PARAMETER Force
Clean and then install tools
.PARAMETER DownloadRetries
Total number of retry attempts
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds
.PARAMETER GlobalJsonFile
File path to global.json file
.NOTES
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30,
[string] $GlobalJsonFile
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
}
Set-StrictMode -version 2.0
$ErrorActionPreference="Stop"
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
# Process tools list
Write-Host "Processing $GlobalJsonFile"
If (-Not (Test-Path $GlobalJsonFile)) {
Write-Host "Unable to find '$GlobalJsonFile'"
exit 0
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
$ToolVersion = $_.Value
$LocalInstallerArguments = @{ ToolName = "$ToolName" }
$LocalInstallerArguments += @{ InstallPath = "$InstallBin" }
$LocalInstallerArguments += @{ BaseUri = "$BaseUri" }
$LocalInstallerArguments += @{ CommonLibraryDirectory = "$EngCommonBaseDir" }
$LocalInstallerArguments += @{ Version = "$ToolVersion" }
if ($Verbose) {
$LocalInstallerArguments += @{ Verbose = $True }
}
if (Get-Variable 'Force' -ErrorAction 'SilentlyContinue') {
if($Force) {
$LocalInstallerArguments += @{ Force = $True }
}
}
if ($Clean) {
$LocalInstallerArguments += @{ Clean = $True }
}
Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$InstallerPath $($LocalInstallerArguments.Keys.ForEach({"-$_ '$($LocalInstallerArguments.$_)'"}) -join ' ')'"
& $InstallerPath @LocalInstallerArguments
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
$showNativeToolsWarning = $true
if ((Get-Variable 'DoNotDisplayNativeToolsInstallationWarnings' -ErrorAction 'SilentlyContinue') -and $DoNotDisplayNativeToolsInstallationWarnings) {
$showNativeToolsWarning = $false
}
if ($showNativeToolsWarning) {
Write-Warning $errMsg
}
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
exit 1
}
}
else {
Write-Host "No native tools defined in global.json"
exit 0
}
if ($Clean) {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
exit 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
}
-141
View File
@@ -1,141 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
base_uri='https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
install_directory=''
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/native/common-library.sh
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installdirectory)
install_directory=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --installdirectory Directory to install native toolset."
echo " This is a command-line override for the default"
echo " Install directory precedence order:"
echo " - InstallDirectory command-line override"
echo " - NETCOREENG_INSTALL_DIRECTORY environment variable"
echo " - (default) %USERPROFILE%/.netcoreeng/native"
echo ""
echo " --clean Switch specifying not to install anything, but cleanup native asset folders"
echo " --force Clean and then install tools"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --baseuri <value> Base URI for where to download native tools from"
echo " --downloadretries <value> Number of times a download should be attempted"
echo " --retrywaittimeseconds <value> Wait time between download attempts"
echo ""
exit 0
;;
esac
done
function ReadGlobalJsonNativeTools {
# Get the native-tools section from the global.json.
local native_tools_section=$(cat $global_json_file | awk '/"native-tools"/,/}/')
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=$( echo "$native_tools_list" | sed 's/\s//g' | sed 's/,/\n/g' )
local old_IFS=$IFS
while read -r line; do
# Lines are of the form: 'tool:version'
IFS=:
while read -r key value; do
native_assets[$key]=$value
done <<< "$line"
done <<< "$native_tools_list"
IFS=$old_IFS
return 0;
}
native_base_dir=$install_directory
if [[ -z $install_directory ]]; then
native_base_dir=$(GetNativeInstallDirectory)
fi
install_bin="${native_base_dir}/bin"
ReadGlobalJsonNativeTools
if [[ ${#native_assets[@]} -eq 0 ]]; then
echo "No native tools defined in global.json"
exit 0;
else
native_installer_dir="$scriptroot/native"
for tool in "${!native_assets[@]}"
do
tool_version=${native_assets[$tool]}
installer_name="install-$tool.sh"
installer_command="$native_installer_dir/$installer_name"
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
echo $installer_command
if [[ $force = true ]]; then
installer_command+=" --force"
fi
if [[ $clean = true ]]; then
installer_command+=" --clean"
fi
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
exit 1
fi
done
fi
if [[ $clean = true ]]; then
exit 0
fi
if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
echo "Native tools install directory does not exist, installation failed" >&2
exit 1
fi
exit 0
-135
View File
@@ -1,135 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $Operation,
[string] $AuthToken,
[string] $CommitSha,
[string] $RepoName,
[switch] $IsFeedPrivate
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables. This should ONLY be called from identified
# internal builds
function SetupCredProvider {
param(
[string] $AuthToken
)
# Install the Cred Provider NuGet plugin
Write-Host "Setting up Cred Provider NuGet plugin in the agent..."
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host "Installing plugin..."
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-Host "CredProvider plugin was not installed correctly!"
ExitWithExitCode 1
}
else {
Write-Host "CredProvider plugin was installed correctly!"
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-Host "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
}
$endpoints = New-Object System.Collections.ArrayList
$nugetConfigPackageSources = Select-Xml -Path $nugetConfigPath -XPath "//packageSources/add[contains(@key, 'darc-int-')]/@value" | foreach{$_.Node.Value}
if (($nugetConfigPackageSources | Measure-Object).Count -gt 0 ) {
foreach ($stableRestoreResource in $nugetConfigPackageSources) {
$trimmedResource = ([string]$stableRestoreResource).Trim()
[void]$endpoints.Add(@{endpoint="$trimmedResource"; password="$AuthToken"})
}
}
if (($endpoints | Measure-Object).Count -gt 0) {
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
$endpointCredentials = @{endpointCredentials=$endpoints} | ConvertTo-Json -Compress
# Create the environment variables the AzDo way
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $endpointCredentials -Properties @{
'variable' = 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS'
'issecret' = 'false'
}
# We don't want sessions cached since we will be updating the endpoints quite frequently
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data 'False' -Properties @{
'variable' = 'NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED'
'issecret' = 'false'
}
}
else
{
Write-Host "No internal endpoints found in NuGet.config"
}
}
#Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
$dotnetTempDir = "$RepoRoot\dotnet"
$dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
$dotnet = "$dotnetTempDir\dotnet.exe"
$restoreProjPath = "$PSScriptRoot\restore.proj"
Write-Host "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Out-File "$restoreProjPath"
& $dotnet restore $restoreProjPath
Write-Host "Arcade SDK restored!"
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
}
if (Test-Path -Path $dotnetTempDir) {
Remove-Item $dotnetTempDir -Recurse
}
}
try {
Push-Location $PSScriptRoot
if ($Operation -like "setup") {
SetupCredProvider $AuthToken
}
elseif ($Operation -like "install-restore") {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-Host "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
finally {
Pop-Location
}
-142
View File
@@ -1,142 +0,0 @@
#!/usr/bin/env bash
set -e
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables.
# This should ONLY be called from identified internal builds
function SetupCredProvider {
local authToken=$1
# Install the Cred Provider NuGet plugin
echo "Setting up Cred Provider NuGet plugin in the agent..."...
echo "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
local url="https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh"
echo "Writing the contents of 'installcredprovider.ps1' locally..."
local installcredproviderPath="installcredprovider.sh"
if command -v curl > /dev/null; then
curl $url > "$installcredproviderPath"
else
wget -q -O "$installcredproviderPath" "$url"
fi
echo "Installing plugin..."
. "$installcredproviderPath"
echo "Deleting local copy of 'installcredprovider.sh'..."
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
echo "CredProvider plugin was not installed correctly!"
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
fi
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
echo "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
local endpoints='['
local nugetConfigPackageValues=`cat "$nugetConfigPath" | grep "key=\"darc-int-"`
local pattern="value=\"(.*)\""
for value in $nugetConfigPackageValues
do
if [[ $value =~ $pattern ]]; then
local endpoint="${BASH_REMATCH[1]}"
endpoints+="{\"endpoint\": \"$endpoint\", \"password\": \"$authToken\"},"
fi
done
endpoints=${endpoints%?}
endpoints+=']'
if [ ${#endpoints} -gt 2 ]; then
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
local endpointCredentials="{\"endpointCredentials\": "$endpoints"}"
echo "##vso[task.setvariable variable=VSS_NUGET_EXTERNAL_FEED_ENDPOINTS]$endpointCredentials"
echo "##vso[task.setvariable variable=NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED]False"
else
echo "No internal endpoints found in NuGet.config"
fi
}
# Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
local dotnetTempDir="$repo_root/dotnet"
local dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
local restoreProjPath="$repo_root/eng/common/restore.proj"
echo "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
echo "<Project Sdk=\"Microsoft.DotNet.Arcade.Sdk\"/>" > "$restoreProjPath"
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
local res=`$dotnetTempDir/dotnet restore $restoreProjPath`
echo "Arcade SDK restored!"
# Cleanup
if [ "$restoreProjPath" ]; then
rm "$restoreProjPath"
fi
if [ "$dotnetTempDir" ]; then
rm -r $dotnetTempDir
fi
}
source="${BASH_SOURCE[0]}"
operation=''
authToken=''
repoName=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--operation)
operation=$2
shift
;;
--authtoken)
authToken=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ "$operation" = "setup" ]; then
SetupCredProvider $authToken
elif [ "$operation" = "install-restore" ]; then
InstallDotNetSdkAndRestoreArcade
else
echo "Unknown operation '$operation'!"
fi
@@ -1,4 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project>
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
</Project>
-27
View File
@@ -1,27 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) Microsoft. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information. -->
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net472</TargetFramework>
<ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
</PropertyGroup>
<ItemGroup>
<!-- Clear references, the SDK may add some depending on UsuingToolXxx settings, but we only want to restore the following -->
<PackageReference Remove="@(PackageReference)"/>
<PackageReference Include="Microsoft.DotNet.IBCMerge" Version="$(MicrosoftDotNetIBCMergeVersion)" Condition="'$(UsingToolIbcOptimization)' == 'true'" />
<PackageReference Include="Drop.App" Version="$(DropAppVersion)" ExcludeAssets="all" Condition="'$(UsingToolVisualStudioIbcTraining)' == 'true'"/>
</ItemGroup>
<PropertyGroup>
<RestoreSources></RestoreSources>
<RestoreSources Condition="'$(UsingToolIbcOptimization)' == 'true'">
https://devdiv.pkgs.visualstudio.com/_packaging/dotnet-core-internal-tooling/nuget/v3/index.json;
</RestoreSources>
<RestoreSources Condition="'$(UsingToolVisualStudioIbcTraining)' == 'true'">
$(RestoreSources);
https://devdiv.pkgs.visualstudio.com/_packaging/VS/nuget/v3/index.json;
</RestoreSources>
</PropertyGroup>
<!-- Repository extensibility point -->
<Import Project="$(RepositoryEngineeringDir)InternalTools.props" Condition="Exists('$(RepositoryEngineeringDir)InternalTools.props')" />
</Project>
-27
View File
@@ -1,27 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
[switch] $prepareMachine,
[Parameter(ValueFromRemainingArguments=$true)][String[]]$extraArgs
)
. $PSScriptRoot\tools.ps1
try {
if ($ci) {
$nodeReuse = $false
}
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity='minimal'
warn_as_error=true
node_reuse=true
prepare_machine=false
extra_args=''
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--verbosity)
verbosity=$2
shift 2
;;
--warnaserror)
warn_as_error=$2
shift 2
;;
--nodereuse)
node_reuse=$2
shift 2
;;
--ci)
ci=true
shift 1
;;
--preparemachine)
prepare_machine=true
shift 1
;;
*)
extra_args="$extra_args $1"
shift 1
;;
esac
done
. "$scriptroot/tools.sh"
if [[ "$ci" == true ]]; then
node_reuse=false
fi
MSBuild $extra_args
ExitWithExitCode 0
-389
View File
@@ -1,389 +0,0 @@
<#
.SYNOPSIS
Helper module to install an archive to a directory
.DESCRIPTION
Helper module to download and extract an archive to a specified directory
.PARAMETER Uri
Uri of artifact to download
.PARAMETER InstallDirectory
Directory to extract artifact contents to
.PARAMETER Force
Force download / extraction if file or contents already exist. Default = False
.PARAMETER DownloadRetries
Total number of retry attempts. Default = 5
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds. Default = 30
.NOTES
Returns False if download or extraction fail, True otherwise
#>
function DownloadAndExtract {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Uri,
[Parameter(Mandatory=$True)]
[string] $InstallDirectory,
[switch] $Force = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30
)
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$TempToolPath = CommonLibrary\Get-TempPathFilename -Path $Uri
# Download native tool
$DownloadStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries $DownloadRetries `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$Force `
-Verbose:$Verbose
if ($DownloadStatus -Eq $False) {
Write-Error "Download failed"
return $False
}
# Extract native tool
$UnzipStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$Force `
-Verbose:$Verbose
if ($UnzipStatus -Eq $False) {
# Retry Download one more time with Force=true
$DownloadRetryStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries 1 `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$True `
-Verbose:$Verbose
if ($DownloadRetryStatus -Eq $False) {
Write-Error "Last attempt of download failed as well"
return $False
}
# Retry unzip again one more time with Force=true
$UnzipRetryStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$True `
-Verbose:$Verbose
if ($UnzipRetryStatus -Eq $False)
{
Write-Error "Last attempt of unzip failed as well"
# Clean up partial zips and extracts
if (Test-Path $TempToolPath) {
Remove-Item $TempToolPath -Force
}
if (Test-Path $InstallDirectory) {
Remove-Item $InstallDirectory -Force -Recurse
}
return $False
}
}
return $True
}
<#
.SYNOPSIS
Download a file, retry on failure
.DESCRIPTION
Download specified file and retry if attempt fails
.PARAMETER Uri
Uri of file to download. If Uri is a local path, the file will be copied instead of downloaded
.PARAMETER Path
Path to download or copy uri file to
.PARAMETER Force
Overwrite existing file if present. Default = False
.PARAMETER DownloadRetries
Total number of retry attempts. Default = 5
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds Default = 30
#>
function Get-File {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Uri,
[Parameter(Mandatory=$True)]
[string] $Path,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30,
[switch] $Force = $False
)
$Attempt = 0
if ($Force) {
if (Test-Path $Path) {
Remove-Item $Path -Force
}
}
if (Test-Path $Path) {
Write-Host "File '$Path' already exists, skipping download"
return $True
}
$DownloadDirectory = Split-Path -ErrorAction Ignore -Path "$Path" -Parent
if (-Not (Test-Path $DownloadDirectory)) {
New-Item -path $DownloadDirectory -force -itemType "Directory" | Out-Null
}
if (Test-Path -IsValid -Path $Uri) {
Write-Verbose "'$Uri' is a file path, copying file to '$Path'"
Copy-Item -Path $Uri -Destination $Path
return $?
}
else {
Write-Verbose "Downloading $Uri"
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
while($Attempt -Lt $DownloadRetries)
{
try {
Invoke-WebRequest -UseBasicParsing -Uri $Uri -OutFile $Path
Write-Verbose "Downloaded to '$Path'"
return $True
}
catch {
$Attempt++
if ($Attempt -Lt $DownloadRetries) {
$AttemptsLeft = $DownloadRetries - $Attempt
Write-Warning "Download failed, $AttemptsLeft attempts remaining, will retry in $RetryWaitTimeInSeconds seconds"
Start-Sleep -Seconds $RetryWaitTimeInSeconds
}
else {
Write-Error $_
Write-Error $_.Exception
}
}
}
}
return $False
}
<#
.SYNOPSIS
Generate a shim for a native tool
.DESCRIPTION
Creates a wrapper script (shim) that passes arguments forward to native tool assembly
.PARAMETER ShimName
The name of the shim
.PARAMETER ShimDirectory
The directory where shims are stored
.PARAMETER ToolFilePath
Path to file that shim forwards to
.PARAMETER Force
Replace shim if already present. Default = False
.NOTES
Returns $True if generating shim succeeds, $False otherwise
#>
function New-ScriptShim {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ShimName,
[Parameter(Mandatory=$True)]
[string] $ShimDirectory,
[Parameter(Mandatory=$True)]
[string] $ToolFilePath,
[Parameter(Mandatory=$True)]
[string] $BaseUri,
[switch] $Force
)
try {
Write-Verbose "Generating '$ShimName' shim"
if (-Not (Test-Path $ToolFilePath)){
Write-Error "Specified tool file path '$ToolFilePath' does not exist"
return $False
}
# WinShimmer is a small .NET Framework program that creates .exe shims to bootstrapped programs
# Many of the checks for installed programs expect a .exe extension for Windows tools, rather
# than a .bat or .cmd file.
# Source: https://github.com/dotnet/arcade/tree/master/src/WinShimmer
if (-Not (Test-Path "$ShimDirectory\WinShimmer\winshimmer.exe")) {
$InstallStatus = DownloadAndExtract -Uri "$BaseUri/windows/winshimmer/WinShimmer.zip" `
-InstallDirectory $ShimDirectory\WinShimmer `
-Force:$Force `
-DownloadRetries 2 `
-RetryWaitTimeInSeconds 5 `
-Verbose:$Verbose
}
if ((Test-Path (Join-Path $ShimDirectory "$ShimName.exe"))) {
Write-Host "$ShimName.exe already exists; replacing..."
Remove-Item (Join-Path $ShimDirectory "$ShimName.exe")
}
& "$ShimDirectory\WinShimmer\winshimmer.exe" $ShimName $ToolFilePath $ShimDirectory
return $True
}
catch {
Write-Host $_
Write-Host $_.Exception
return $False
}
}
<#
.SYNOPSIS
Returns the machine architecture of the host machine
.NOTES
Returns 'x64' on 64 bit machines
Returns 'x86' on 32 bit machines
#>
function Get-MachineArchitecture {
$ProcessorArchitecture = $Env:PROCESSOR_ARCHITECTURE
$ProcessorArchitectureW6432 = $Env:PROCESSOR_ARCHITEW6432
if($ProcessorArchitecture -Eq "X86")
{
if(($ProcessorArchitectureW6432 -Eq "") -Or
($ProcessorArchitectureW6432 -Eq "X86")) {
return "x86"
}
$ProcessorArchitecture = $ProcessorArchitectureW6432
}
if (($ProcessorArchitecture -Eq "AMD64") -Or
($ProcessorArchitecture -Eq "IA64") -Or
($ProcessorArchitecture -Eq "ARM64")) {
return "x64"
}
return "x86"
}
<#
.SYNOPSIS
Get the name of a temporary folder under the native install directory
#>
function Get-TempDirectory {
return Join-Path (Get-NativeInstallDirectory) "temp/"
}
function Get-TempPathFilename {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Path
)
$TempDir = CommonLibrary\Get-TempDirectory
$TempFilename = Split-Path $Path -leaf
$TempPath = Join-Path $TempDir $TempFilename
return $TempPath
}
<#
.SYNOPSIS
Returns the base directory to use for native tool installation
.NOTES
Returns the value of the NETCOREENG_INSTALL_DIRECTORY if that environment variable
is set, or otherwise returns an install directory under the %USERPROFILE%
#>
function Get-NativeInstallDirectory {
$InstallDir = $Env:NETCOREENG_INSTALL_DIRECTORY
if (!$InstallDir) {
$InstallDir = Join-Path $Env:USERPROFILE ".netcoreeng/native/"
}
return $InstallDir
}
<#
.SYNOPSIS
Unzip an archive
.DESCRIPTION
Powershell module to unzip an archive to a specified directory
.PARAMETER ZipPath (Required)
Path to archive to unzip
.PARAMETER OutputDirectory (Required)
Output directory for archive contents
.PARAMETER Force
Overwrite output directory contents if they already exist
.NOTES
- Returns True and does not perform an extraction if output directory already exists but Overwrite is not True.
- Returns True if unzip operation is successful
- Returns False if Overwrite is True and it is unable to remove contents of OutputDirectory
- Returns False if unable to extract zip archive
#>
function Expand-Zip {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ZipPath,
[Parameter(Mandatory=$True)]
[string] $OutputDirectory,
[switch] $Force
)
Write-Verbose "Extracting '$ZipPath' to '$OutputDirectory'"
try {
if ((Test-Path $OutputDirectory) -And (-Not $Force)) {
Write-Host "Directory '$OutputDirectory' already exists, skipping extract"
return $True
}
if (Test-Path $OutputDirectory) {
Write-Verbose "'Force' is 'True', but '$OutputDirectory' exists, removing directory"
Remove-Item $OutputDirectory -Force -Recurse
if ($? -Eq $False) {
Write-Error "Unable to remove '$OutputDirectory'"
return $False
}
}
if (-Not (Test-Path $OutputDirectory)) {
New-Item -path $OutputDirectory -Force -itemType "Directory" | Out-Null
}
Add-Type -assembly "system.io.compression.filesystem"
[io.compression.zipfile]::ExtractToDirectory("$ZipPath", "$OutputDirectory")
if ($? -Eq $False) {
Write-Error "Unable to extract '$ZipPath'"
return $False
}
}
catch {
Write-Host $_
Write-Host $_.Exception
return $False
}
return $True
}
export-modulemember -function DownloadAndExtract
export-modulemember -function Expand-Zip
export-modulemember -function Get-File
export-modulemember -function Get-MachineArchitecture
export-modulemember -function Get-NativeInstallDirectory
export-modulemember -function Get-TempDirectory
export-modulemember -function Get-TempPathFilename
export-modulemember -function New-ScriptShim
-168
View File
@@ -1,168 +0,0 @@
#!/usr/bin/env bash
function GetNativeInstallDirectory {
local install_dir
if [[ -z $NETCOREENG_INSTALL_DIRECTORY ]]; then
install_dir=$HOME/.netcoreeng/native/
else
install_dir=$NETCOREENG_INSTALL_DIRECTORY
fi
echo $install_dir
return 0
}
function GetTempDirectory {
echo $(GetNativeInstallDirectory)temp/
return 0
}
function ExpandZip {
local zip_path=$1
local output_directory=$2
local force=${3:-false}
echo "Extracting $zip_path to $output_directory"
if [[ -d $output_directory ]] && [[ $force = false ]]; then
echo "Directory '$output_directory' already exists, skipping extract"
return 0
fi
if [[ -d $output_directory ]]; then
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
echo Unable to remove '$output_directory'>&2
return 1
fi
fi
echo "Creating directory: '$output_directory'"
mkdir -p $output_directory
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
echo "Unable to extract '$zip_path'" >&2
return 1
fi
return 0
}
function GetCurrentOS {
local unameOut="$(uname -s)"
case $unameOut in
Linux*) echo "Linux";;
Darwin*) echo "MacOS";;
esac
return 0
}
function GetFile {
local uri=$1
local path=$2
local force=${3:-false}
local download_retries=${4:-5}
local retry_wait_time_seconds=${5:-30}
if [[ -f $path ]]; then
if [[ $force = false ]]; then
echo "File '$path' already exists. Skipping download"
return 0
else
rm -rf $path
fi
fi
if [[ -f $uri ]]; then
echo "'$uri' is a file path, copying file to '$path'"
cp $uri $path
return $?
fi
echo "Downloading $uri"
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$uri" -sSL --retry $download_retries --retry-delay $retry_wait_time_seconds --create-dirs -o "$path" --fail
else
wget -q -O "$path" "$uri" --tries="$download_retries"
fi
return $?
}
function GetTempPathFileName {
local path=$1
local temp_dir=$(GetTempDirectory)
local temp_file_name=$(basename $path)
echo $temp_dir$temp_file_name
return 0
}
function DownloadAndExtract {
local uri=$1
local installDir=$2
local force=${3:-false}
local download_retries=${4:-5}
local retry_wait_time_seconds=${5:-30}
local temp_tool_path=$(GetTempPathFileName $uri)
echo "downloading to: $temp_tool_path"
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
return 1
fi
# Extract File
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
return 1
fi
return 0
}
function NewScriptShim {
local shimpath=$1
local tool_file_path=$2
local force=${3:-false}
echo "Generating '$shimpath' shim"
if [[ -f $shimpath ]]; then
if [[ $force = false ]]; then
echo "File '$shimpath' already exists." >&2
return 1
else
rm -rf $shimpath
fi
fi
if [[ ! -f $tool_file_path ]]; then
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
return 1
fi
local shim_contents=$'#!/usr/bin/env bash\n'
shim_contents+="SHIMARGS="$'$1\n'
shim_contents+="$tool_file_path"$' $SHIMARGS\n'
# Write shim file
echo "$shim_contents" > $shimpath
chmod +x $shimpath
echo "Finished generating shim '$shimpath'"
return $?
}
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake-test"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
exit 1
fi
exit 0
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
exit 1
fi
exit 0
-130
View File
@@ -1,130 +0,0 @@
<#
.SYNOPSIS
Install native tool
.DESCRIPTION
Install cmake native tool from Azure blob storage
.PARAMETER InstallPath
Base directory to install native tool to
.PARAMETER BaseUri
Base file directory or Url from which to acquire tool archives
.PARAMETER CommonLibraryDirectory
Path to folder containing common library modules
.PARAMETER Force
Force install of tools even if they previously exist
.PARAMETER Clean
Don't install the tool, just clean up the current install of the tool
.PARAMETER DownloadRetries
Total number of retry attempts
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds
.NOTES
Returns 0 if install succeeds, 1 otherwise
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ToolName,
[Parameter(Mandatory=$True)]
[string] $InstallPath,
[Parameter(Mandatory=$True)]
[string] $BaseUri,
[Parameter(Mandatory=$True)]
[string] $Version,
[string] $CommonLibraryDirectory = $PSScriptRoot,
[switch] $Force = $False,
[switch] $Clean = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30
)
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$Arch = CommonLibrary\Get-MachineArchitecture
$ToolOs = "win64"
if($Arch -Eq "x32") {
$ToolOs = "win32"
}
$ToolNameMoniker = "$ToolName-$Version-$ToolOs-$Arch"
$ToolInstallDirectory = Join-Path $InstallPath "$ToolName\$Version\"
$Uri = "$BaseUri/windows/$ToolName/$ToolNameMoniker.zip"
$ShimPath = Join-Path $InstallPath "$ToolName.exe"
if ($Clean) {
Write-Host "Cleaning $ToolInstallDirectory"
if (Test-Path $ToolInstallDirectory) {
Remove-Item $ToolInstallDirectory -Force -Recurse
}
Write-Host "Cleaning $ShimPath"
if (Test-Path $ShimPath) {
Remove-Item $ShimPath -Force
}
$ToolTempPath = CommonLibrary\Get-TempPathFilename -Path $Uri
Write-Host "Cleaning $ToolTempPath"
if (Test-Path $ToolTempPath) {
Remove-Item $ToolTempPath -Force
}
exit 0
}
# Install tool
if ((Test-Path $ToolInstallDirectory) -And (-Not $Force)) {
Write-Verbose "$ToolName ($Version) already exists, skipping install"
}
else {
$InstallStatus = CommonLibrary\DownloadAndExtract -Uri $Uri `
-InstallDirectory $ToolInstallDirectory `
-Force:$Force `
-DownloadRetries $DownloadRetries `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-Error "Installation failed"
exit 1
}
}
$ToolFilePath = Get-ChildItem $ToolInstallDirectory -Recurse -Filter "$ToolName.exe" | % { $_.FullName }
if (@($ToolFilePath).Length -Gt 1) {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Error "$ToolName was not found in $ToolFilePath."
exit 1
}
# Generate shim
# Always rewrite shims so that we are referencing the expected version
$GenerateShimStatus = CommonLibrary\New-ScriptShim -ShimName $ToolName `
-ShimDirectory $InstallPath `
-ToolFilePath "$ToolFilePath" `
-BaseUri $BaseUri `
-Force:$Force `
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-Error "Generate shim failed"
return 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
}
@@ -1,102 +0,0 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<WorkItemCommand>%HELIX_CORRELATION_PAYLOAD%\performance\scripts\benchmarks_ci.py --csproj %HELIX_CORRELATION_PAYLOAD%\performance\$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
<DotnetExe>%HELIX_CORRELATION_PAYLOAD%\performance\tools\dotnet\$(Architecture)\dotnet.exe</DotnetExe>
<Percent>%25%25</Percent>
<XMLResults>%HELIX_WORKITEM_ROOT%\testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
<BaseDirectory>$HELIX_CORRELATION_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)/performance</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'true'">
<BaseDirectory>$HELIX_WORKITEM_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT'">
<WorkItemCommand>$(PerformanceDirectory)/scripts/benchmarks_ci.py --csproj $(PerformanceDirectory)/$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<BaselineCoreRun>$(BaseDirectory)/Baseline_Core_Root/corerun</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>$(PerformanceDirectory)/src/tools/ResultsComparer/ResultsComparer.csproj</ResultsComparer>
<DotnetExe>$(PerformanceDirectory)/tools/dotnet/$(Architecture)/dotnet</DotnetExe>
<Percent>%25</Percent>
<XMLResults>$HELIX_WORKITEM_ROOT/testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(UseBaselineCoreRun)' == 'true'">
<BaselineCoreRunArgument>--corerun $(BaselineCoreRun)</BaselineCoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>
<PropertyGroup Condition="'$(_Framework)' != 'net461'">
<WorkItemCommand>$(WorkItemCommand) $(CliArguments)</WorkItemCommand>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<PropertyGroup>
<PartitionCount>5</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="$(BuildConfig).Partition0" Index="0" />
<Partition Include="$(BuildConfig).Partition1" Index="1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>
<PropertyGroup Condition="'$(Compare)' == 'true'">
<FailOnTestFailure>false</FailOnTestFailure>
</PropertyGroup>
<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults);$(FinalCommand)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
</Project>
@@ -1,106 +0,0 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $BaselineCoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp5.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="coreclr corefx",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)
$UseBaselineCoreRun = ($BaselineCoreRootDirectory -ne [string]::Empty)
$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
$WorkItemDirectory = (Join-Path $SourceDirectory "workitem")
$ExtraBenchmarkDotNetArguments = "--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
$Creator = $env:BUILD_DEFINITIONNAME
$PerfLabArguments = ""
$HelixSourcePrefix = "pr"
$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"
if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS5.Open"
}
if ($Compare) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf.Open"
$PerfLabArguments = ""
$ExtraBenchmarkDotNetArguments = ""
}
if ($Internal) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}
$CommonSetupArguments="--frameworks $Framework --queue $Queue --build-number $BuildNumber --build-configs $Configurations"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"
robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}
if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}
if ($UseBaselineCoreRun) {
$NewBaselineCoreRoot = (Join-Path $PayloadDirectory "Baseline_Core_Root")
Move-Item -Path $BaselineCoreRootDirectory -Destination $NewBaselineCoreRoot
}
$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory
# Set variables that we will need to have in future steps
$ci = $true
. "$PSScriptRoot\..\pipeline-logging-functions.ps1"
# Directories
Write-PipelineSetVariable -Name 'PayloadDirectory' -Value "$PayloadDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerformanceDirectory' -Value "$PerformanceDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'WorkItemDirectory' -Value "$WorkItemDirectory" -IsMultiJobVariable $false
# Script Arguments
Write-PipelineSetVariable -Name 'Python' -Value "py -3" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'ExtraBenchmarkDotNetArguments' -Value "$ExtraBenchmarkDotNetArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'SetupArguments' -Value "$SetupArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerfLabArguments' -Value "$PerfLabArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'BDNCategories' -Value "$RunCategories" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseBaselineCoreRun' -Value "$UseBaselineCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Compare' -Value "$Compare" -IsMultiJobVariable $false
# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Queue' -Value "$Queue" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'HelixSourcePrefix' -Value "$HelixSourcePrefix" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name '_BuildConfig' -Value "$Architecture.$Kind.$Framework" -IsMultiJobVariable $false
exit 0
-216
View File
@@ -1,216 +0,0 @@
#!/usr/bin/env bash
source_directory=$BUILD_SOURCESDIRECTORY
core_root_directory=
baseline_core_root_directory=
architecture=x64
framework=netcoreapp5.0
compilation_mode=tiered
repository=$BUILD_REPOSITORY_NAME
branch=$BUILD_SOURCEBRANCH
commit_sha=$BUILD_SOURCEVERSION
build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="coreclr corefx"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations=
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--sourcedirectory)
source_directory=$2
shift 2
;;
--corerootdirectory)
core_root_directory=$2
shift 2
;;
--baselinecorerootdirectory)
baseline_core_root_directory=$2
shift 2
;;
--architecture)
architecture=$2
shift 2
;;
--framework)
framework=$2
shift 2
;;
--compilationmode)
compilation_mode=$2
shift 2
;;
--repository)
repository=$2
shift 2
;;
--branch)
branch=$2
shift 2
;;
--commitsha)
commit_sha=$2
shift 2
;;
--buildnumber)
build_number=$2
shift 2
;;
--kind)
kind=$2
shift 2
;;
--runcategories)
run_categories=$2
shift 2
;;
--csproj)
csproj=$2
shift 2
;;
--internal)
internal=true
shift 1
;;
--compare)
compare=true
shift 1
;;
--configurations)
configurations=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --corerootdirectory <value> Directory where Core_Root exists, if running perf testing with --corerun"
echo " --architecture <value> Architecture of the testing being run"
echo " --configurations <value> List of key=value pairs that will be passed to perf testing infrastructure."
echo " ex: --configurations \"CompilationMode=Tiered OptimzationLevel=PGO\""
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --framework <value> The framework to run, if not running in master"
echo " --compliationmode <value> The compilation mode if not passing --configurations"
echo " --sourcedirectory <value> The directory of the sources. Defaults to env:BUILD_SOURCESDIRECTORY"
echo " --repository <value> The name of the repository in the <owner>/<repository name> format. Defaults to env:BUILD_REPOSITORY_NAME"
echo " --branch <value> The name of the branch. Defaults to env:BUILD_SOURCEBRANCH"
echo " --commitsha <value> The commit sha1 to run against. Defaults to env:BUILD_SOURCEVERSION"
echo " --buildnumber <value> The build number currently running. Defaults to env:BUILD_BUILDNUMBER"
echo " --csproj The relative path to the benchmark csproj whose tests should be run. Defaults to src\benchmarks\micro\MicroBenchmarks.csproj"
echo " --kind <value> Related to csproj. The kind of benchmarks that should be run. Defaults to micro"
echo " --runcategories <value> Related to csproj. Categories of benchmarks to run. Defaults to \"coreclr corefx\""
echo " --internal If the benchmarks are running as an official job."
echo ""
exit 0
;;
esac
done
if [ "$repository" == "dotnet/performance" ] || [ "$repository" == "dotnet-performance" ]; then
run_from_perf_repo=true
fi
if [ -z "$configurations" ]; then
configurations="CompliationMode=$compilation_mode"
fi
if [ -z "$core_root_directory" ]; then
use_core_run=false
fi
if [ -z "$baseline_core_root_directory" ]; then
use_baseline_core_run=false
fi
payload_directory=$source_directory/Payload
performance_directory=$payload_directory/performance
workitem_directory=$source_directory/workitem
extra_benchmark_dotnet_arguments="--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
perflab_arguments=
queue=Ubuntu.1804.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"
if [[ "$compare" == true ]]; then
extra_benchmark_dotnet_arguments=
perflab_arguments=
# No open queues for arm64
if [[ "$architecture" = "arm64" ]]; then
echo "Compare not available for arm64"
exit 1
fi
queue=Ubuntu.1804.Amd64.Tiger.Perf.Open
fi
if [[ "$internal" == true ]]; then
perflab_arguments="--upload-to-perflab-container"
helix_source_prefix="official"
creator=
extra_benchmark_dotnet_arguments=
if [[ "$architecture" = "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
queue=Ubuntu.1804.Amd64.Tiger.Perf
fi
fi
common_setup_arguments="--frameworks $framework --queue $queue --build-number $build_number --build-configs $configurations"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
payload_directory=
workitem_directory=$source_directory
performance_directory=$workitem_directory
setup_arguments="--perf-hash $commit_sha $common_setup_arguments"
else
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $performance_directory
docs_directory=$performance_directory/docs
mv $docs_directory $workitem_directory
fi
if [[ "$use_core_run" = true ]]; then
new_core_root=$payload_directory/Core_Root
mv $core_root_directory $new_core_root
fi
if [[ "$use_baseline_core_run" = true ]]; then
new_baseline_core_root=$payload_directory/Baseline_Core_Root
mv $baseline_core_root_directory $new_baseline_core_root
fi
ci=true
_script_dir=$(pwd)/eng/common
. "$_script_dir/pipeline-logging-functions.sh"
# Make sure all of our variables are available for future steps
Write-PipelineSetVariable -name "UseCoreRun" -value "$use_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "UseBaselineCoreRun" -value "$use_baseline_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "Architecture" -value "$architecture" -is_multi_job_variable false
Write-PipelineSetVariable -name "PayloadDirectory" -value "$payload_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerformanceDirectory" -value "$performance_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "WorkItemDirectory" -value "$workitem_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "Queue" -value "$queue" -is_multi_job_variable false
Write-PipelineSetVariable -name "SetupArguments" -value "$setup_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "Python" -value "$python3" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerfLabArguments" -value "$perflab_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "ExtraBenchmarkDotNetArguments" -value "$extra_benchmark_dotnet_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "BDNCategories" -value "$run_categories" -is_multi_job_variable false
Write-PipelineSetVariable -name "TargetCsproj" -value "$csproj" -is_multi_job_variable false
Write-PipelineSetVariable -name "RunFromPerfRepo" -value "$run_from_perf_repo" -is_multi_job_variable false
Write-PipelineSetVariable -name "Creator" -value "$creator" -is_multi_job_variable false
Write-PipelineSetVariable -name "HelixSourcePrefix" -value "$helix_source_prefix" -is_multi_job_variable false
Write-PipelineSetVariable -name "Kind" -value "$kind" -is_multi_job_variable false
Write-PipelineSetVariable -name "_BuildConfig" -value "$architecture.$kind.$framework" -is_multi_job_variable false
Write-PipelineSetVariable -name "Compare" -value "$compare" -is_multi_job_variable false
-234
View File
@@ -1,234 +0,0 @@
# Source for this file was taken from https://github.com/microsoft/azure-pipelines-task-lib/blob/11c9439d4af17e6475d9fe058e6b2e03914d17e6/powershell/VstsTaskSdk/LoggingCommandFunctions.ps1 and modified.
# NOTE: You should not be calling these method directly as they are likely to change. Instead you should be calling the Write-Pipeline* functions defined in tools.ps1
$script:loggingCommandPrefix = '##vso['
$script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"?
New-Object psobject -Property @{ Token = ';' ; Replacement = '%3B' }
New-Object psobject -Property @{ Token = "`r" ; Replacement = '%0D' }
New-Object psobject -Property @{ Token = "`n" ; Replacement = '%0A' }
New-Object psobject -Property @{ Token = "]" ; Replacement = '%5D' }
)
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Category,
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
$PSBoundParameters.Remove("Category") | Out-Null
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
$PSBoundParameters.Remove("Message") | Out-Null
$PSBoundParameters.Add("Message", $Message)
Write-PipelineTaskError @PSBoundParameters
}
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
if(!$ci) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Name,
[string]$Value,
[switch]$Secret,
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
'isOutput' = $IsMultiJobVariable
} -AsOutput:$AsOutput
}
}
function Write-PipelinePrependPath {
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
<########################################
# Private functions.
########################################>
function Format-LoggingCommandData {
[CmdletBinding()]
param([string]$Value, [switch]$Reverse)
if (!$Value) {
return ''
}
if (!$Reverse) {
foreach ($mapping in $script:loggingCommandEscapeMappings) {
$Value = $Value.Replace($mapping.Token, $mapping.Replacement)
}
} else {
for ($i = $script:loggingCommandEscapeMappings.Length - 1 ; $i -ge 0 ; $i--) {
$mapping = $script:loggingCommandEscapeMappings[$i]
$Value = $Value.Replace($mapping.Replacement, $mapping.Token)
}
}
return $Value
}
function Format-LoggingCommand {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Area,
[Parameter(Mandatory = $true)]
[string]$Event,
[string]$Data,
[hashtable]$Properties)
# Append the preamble.
[System.Text.StringBuilder]$sb = New-Object -TypeName System.Text.StringBuilder
$null = $sb.Append($script:loggingCommandPrefix).Append($Area).Append('.').Append($Event)
# Append the properties.
if ($Properties) {
$first = $true
foreach ($key in $Properties.Keys) {
[string]$value = Format-LoggingCommandData $Properties[$key]
if ($value) {
if ($first) {
$null = $sb.Append(' ')
$first = $false
} else {
$null = $sb.Append(';')
}
$null = $sb.Append("$key=$value")
}
}
}
# Append the tail and output the value.
$Data = Format-LoggingCommandData $Data
$sb.Append(']').Append($Data).ToString()
}
function Write-LoggingCommand {
[CmdletBinding(DefaultParameterSetName = 'Parameters')]
param(
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Area,
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Event,
[Parameter(ParameterSetName = 'Parameters')]
[string]$Data,
[Parameter(ParameterSetName = 'Parameters')]
[hashtable]$Properties,
[Parameter(Mandatory = $true, ParameterSetName = 'Object')]
$Command,
[switch]$AsOutput)
if ($PSCmdlet.ParameterSetName -eq 'Object') {
Write-LoggingCommand -Area $Command.Area -Event $Command.Event -Data $Command.Data -Properties $Command.Properties -AsOutput:$AsOutput
return
}
$command = Format-LoggingCommand -Area $Area -Event $Event -Data $Data -Properties $Properties
if ($AsOutput) {
$command
} else {
Write-Host $command
}
}
function Write-LogIssue {
[CmdletBinding()]
param(
[ValidateSet('warning', 'error')]
[Parameter(Mandatory = $true)]
[string]$Type,
[string]$Message,
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
$command = Format-LoggingCommand -Area 'task' -Event 'logissue' -Data $Message -Properties @{
'type' = $Type
'code' = $ErrCode
'sourcepath' = $SourcePath
'linenumber' = $LineNumber
'columnnumber' = $ColumnNumber
}
if ($AsOutput) {
return $command
}
if ($Type -eq 'error') {
$foregroundColor = $host.PrivateData.ErrorForegroundColor
$backgroundColor = $host.PrivateData.ErrorBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Red
$backgroundColor = [System.ConsoleColor]::Black
}
} else {
$foregroundColor = $host.PrivateData.WarningForegroundColor
$backgroundColor = $host.PrivateData.WarningBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Yellow
$backgroundColor = [System.ConsoleColor]::Black
}
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
function Write-PipelineTelemetryError {
local telemetry_category=''
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-category|-c)
telemetry_category=$2
shift
;;
-*)
function_args+=("$1 $2")
shift
;;
*)
message=$*
;;
esac
shift
done
if [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
local message_type="error"
local sourcepath=''
local linenumber=''
local columnnumber=''
local error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-ln)
linenumber=$2
shift
;;
-columnnumber|-cn)
columnnumber=$2
shift
;;
-errcode|-e)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
local message="##vso[task.logissue"
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
function Write-PipelineSetVariable {
if [[ "$ci" != true ]]; then
return
fi
local name=''
local value=''
local secret=false
local as_output=false
local is_multi_job_variable=true
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-name|-n)
name=$2
shift
;;
-value|-v)
value=$2
shift
;;
-secret|-s)
secret=true
;;
-as_output|-a)
as_output=true
;;
-is_multi_job_variable|-i)
is_multi_job_variable=$2
shift
;;
esac
shift
done
value=${value/;/%3B}
value=${value/\\r/%0D}
value=${value/\\n/%0A}
value=${value/]/%5D}
local message="##vso[task.setvariable variable=$name;isSecret=$secret;isOutput=$is_multi_job_variable]$value"
if [[ "$as_output" == true ]]; then
$message
else
echo "$message"
fi
}
function Write-PipelinePrependPath {
local prepend_path=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-path|-p)
prepend_path=$2
shift
;;
esac
shift
done
export PATH="$prepend_path:$PATH"
if [[ "$ci" == true ]]; then
echo "##vso[task.prependpath]$prepend_path"
fi
}
Loaded 100 of 6805 files, more files were not shown because too many files have changed in this diff. Show more