Compare commits

..
215 Commits
Author SHA1 Message Date
Ryan Lucia 476d72b9e3 [IO] Remove read-only logic in mono_w32_get_disk_free_space
This brings us in line with GetDiskFreeSpaceExW on Windows as well as .NET Core.
2019-10-15 10:43:27 +02:00
Aleksey Kliger (λgeek) bdc557612b [2019-06][merp] Don't install SIGTERM handler in EnableMicrosoftTelemetry (#17296)
* [merp] Put thread into async context before running summarizer

[merp] Put thread into async context before running summarizer

Followup work for https://github.com/mono/mono/pull/17090

In order to get managed method info for unmanaged stacktraces, we call
`mini_jit_info_table_find_ext` which calls `decode_exception_debug_info` which may
allocate if it's not in async context.

Do the switch in `mono_threads_summarize_execute` so that non-supervising threads
all switch when they run the `sigterm_signal_handler`.  `mono_threads_summarize`
already does it for the supervisor thread.

Fixes https://github.com/mono/mono/issues/17180

Backport of #17194.

/cc @lambdageek

* [merp] Don't call install_state_summarizer in EnableMicrosoftTelemetry

install_state_summarizer installs a SIGTERM handler. Unfortunately
sigterm_signal_handler assumes that it will only be called when a crash is
already in progress (it assumes it won't be the supervisor, and so it doesn't
setup the memory for capturing a crash report among other things).

* [merp] Register SIGTERM handler in dump_native_stacktrace

Previously it was installed by EnableMicrosoftTelemetry, which is too early.
Now we get here without a handler installed even when mono_merp_enabled returns true.
2019-10-14 17:24:44 +02:00
Alexander Köplinger 1e2d850ec4 [tests] Bump corefx to get Azure testhost change (#17275)
* Bump corefx to get Azure testhost change

See https://github.com/mono/corefx/pull/360

* Update ClientWebSocketTest as well

(cherry picked from commit d457346e06)

# Conflicts:
#	external/corefx
2019-10-11 09:41:33 +02:00
Alexander Köplinger 703e9a2fad [2019-06] Add drawing type converters to mobile profiles (#17241)
* [2019-06] Add drawing type converters to mobile profiles

Needed for https://github.com/xamarin/xamarin-android/issues/3726

* Bump API snapshot submodule

* [csproj] Update project files
2019-10-09 12:16:23 +02:00
Thays Grazia 4ec605fc9b If there is a perform_wait_callback in the stack there will be another catch generated by the method called in the owner thread, so we don't need to throw as unhandled exception, we can continue and find the next catch.
Fixes #17083

Backport of #17106.
2019-10-09 09:07:21 +02:00
monojenkins 061c225621 [merp] exit_status is 0 if we ran the uploader successfully (#17186)
Not 1 (TRUE)
2019-10-05 11:36:29 -04:00
Alexis Christoforides 07c23f2ca4 [crashing] Remove Mono signal handlers when starting to handle a crash
Fixes https://github.com/mono/mono/issues/16803

Backport of https://github.com/mono/mono/pull/17002
2019-10-02 09:35:59 +02:00
Egor Bogatov 5608fe0abb [2019-06] Update NETStandard 2.1 APIs (#17081)
Contributes to #17064
2019-09-27 17:42:14 +02:00
Steve Pfister a5b8d6fcb5 [2019-06] Fix time zone issue when jumping into DST (#17062)
There is an issue with jumping into DST for some time zones when the incorrect date-time offset is returned for date-time in UTC (which comes from DateTime.Now). The fix is to just check if the incoming date-time is in UTC

Also I added set of tests for some time zones but they verify jumping into DST in general.

Fixes https://github.com/mono/mono/issues/16395

Backport of https://github.com/mono/mono/pull/16430
2019-09-26 20:03:02 +02:00
Marek Safar 390e6e124f Revert "[metadata] Fix leaks when handling a few attributes (#16675) (#16851)"
This reverts commit a6b5187d76.
2019-09-26 18:49:35 +02:00
Alexander Köplinger 459dd927df [2019-06] Bump bockbuild to get https://github.com/mono/bockbuild/pull/121 2019-09-25 20:00:56 +02:00
Rolf Bjarne Kvinge beb9a1b182 [sdks] Remove the mac32 build.
Xamarin.Mac doesn't need nor use it anymore.

This also removes the last part of the mono archive that required Xcode 9.4.
2019-09-25 16:52:49 +02:00
Alexander Köplinger 747a919a06 [ci] Make ios/mac sdks archive URL more predictable
Requested by xamarin-macios team.

(cherry picked from commit 0a3c9c451b)
2019-09-24 15:26:55 +02:00
Alexander Köplinger 114013096e [ci] Build iOS/Mac Mono sdks archive using Xcode 11
This was requested by the xamarin-macios team.

(cherry picked from commit 6ad929c317)
2019-09-23 17:37:21 +02:00
EgorBo 10a24f3ea1 Implement WriteCore and ReadCore in DeflateStream 2019-09-23 16:48:16 +02:00
Rolf Bjarne Kvinge a925846b1f [offsets-tool] Install clang into the user-specific python directory. (#16933)
Because the system location isn't world writeable on Catalina.
2019-09-19 17:43:35 +02:00
Larry Ewing fe64a4765e [2019-06] Bump msbuild and sdk versions to 3.0.1xx latest (#16870)
* [2019-06] Bump msbuild and sdk versions to 3.0.1xx latest

* Bump msbuild again to pull in the build updates

* pull in roslyn-binaries fix

* One more bump

* Fix the roslyn hash

* [msbuild] Bump to pick up a build fix from https://github.com/mono/msbuild/pull/138

* Pull in final update from msbuild of nuget and sdk
NuGet 5.3.0-rtm.6251
Sdk 3.0.100-rc2.19467.3

* Pick up packaging fix in https://github.com/mono/msbuild/pull/141
2019-09-18 19:13:51 +02:00
monojenkins 7293597b90 [corlib] Fix building nunit-lite twice (#16910)
We sometimes saw the following build error in Mono SDKs builds on Jenkins:

```
[2019-09-16T01:03:30.709Z] CSC     [xammac] nunit-lite-console.exe
[2019-09-16T01:03:30.709Z] error CS0009: Metadata file '/Users/builder/jenkins/workspace/archive-mono/2019-08/mac/release/mcs/class/lib/xammac/nunitlite.dll' could not be opened -- PE image doesn't contain managed metadata.
```

Taking a closer look we found out that nunit-lite was trying to be built twice for the same profile which caused a race where the file was corrupted.

The reason for this is that the corlib Makefile defines two test assemblies: the normal PROFILE_corlib_test.dll and BinarySerializationOverVersionsTest.dll and
integrating the BinarySerializationOverVersionsTest.dll via the `test-vts` target incorrectly overwrote the `test` target instead of using `test-local`.

Backport of #16894.
2019-09-18 13:39:36 +02:00
Rolf Bjarne Kvinge 1648e88687 Rename bundle identifier for the various Mono.frameworks we create for Xamarin.iOS. Fixes xamarin/xamarin-macios#7005. (#16896)
This is the equivalent of
https://github.com/xamarin/xamarin-macios/commit/a9abc07cb357f9f012e3da66c70fa09d96bf02f7,
just for the Mono.frameworks in the mono archive.

Fixes https://github.com/xamarin/xamarin-macios/issues/7005.
2019-09-17 21:16:29 +02:00
Vlad Brezae a6b5187d76 [metadata] Fix leaks when handling a few attributes (#16675) (#16851)
Callers of mono_reflection_create_custom_attr_data_args_noalloc were leaking some of the returned information. Accessed attributes are FixedBufferAttribute and UnmanagedFunctionPointerAttribute.
2019-09-16 14:16:01 +03:00
Steve Pfister 7da9a041b3 [2019-06] Bump to mono/corefx@e79cf5b
Backport of https://github.com/mono/mono/pull/16774
2019-09-12 09:08:26 +02:00
monojenkins 2b7050bdf3 [2019-06] Add RenamedEvent* to FSW sources from CoreFX (#16758)
Add RenamedEvent* to FSW sources from CoreFX

Fixes #16486 - This was only partially fixed until now

* [csproj] Update project files

* Add test files which test these code paths on corefx
2019-09-11 12:29:51 -04:00
Larry Ewing 4f5ed502c6 [msbuild] pick up p4 versions 2019-09-10 09:35:15 +02:00
Larry Ewing f04ee2219d [2019-06][msbuid][roslyn] Bump msbuild and roslyn-binaries to pick up dotnet 3.0.100-p9 toolset 2019-09-10 09:35:15 +02:00
Thays Grazia 6b4b99e571 Vtable [i] can be null so this should be check before use it. Fixes #16712 2019-09-09 19:17:01 +02:00
monojenkins 7af64d1ebe [2019-06] [android] remove text relocations in AOT/arm32 (#16660)
* Revert "[aot] partial revert of #14043 (#14135)"

This reverts commit 223e0bd78d.

* Revert "[aot] change method table generation so that it works for larger assemblies on ARM (#14043)"

This reverts commit 9ff3b0d65e.
2019-09-04 15:31:19 -05:00
monojenkins 2c3aeaf378 [2019-06] [corlib] Add hack for Hebrew calendar data (#16642) 2019-09-03 14:09:37 -04:00
monojenkins 3216a3c1e3 [2019-06] Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5. (#16575)
* Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5.

* [monodis] Add a missing cattr parent case.

Backport of #16567.
2019-08-30 12:30:03 +02:00
Alexander Köplinger 7ef99c046c Build pkg-config as 64bit only, libtiff as universal and remove gtk-quartz-engine.py (#16548)
See https://github.com/mono/bockbuild/pull/116

(cherry picked from commit 309cf1bcee)

# Conflicts:
#	external/bockbuild
2019-08-29 20:37:00 +02:00
monojenkins 1604b9df9d [2019-06] Use the right FileSystemWatcher sources (#16555)
* Use the right FileSystemWatcher sources

The corefx code for FileSystemEventArgs has some changes to support
constructing one with a null filename.

Fixes #16486

* Remove the mono sources for the newly replaced files

* [csproj] Update project files
2019-08-29 14:35:35 +02:00
monojenkins fd75f0f487 [2019-06] Implement Read and WriteAsyncMemory in Mono DeflateStream
Resolves https://github.com/mono/mono/issues/16122

Since we put in the corefx version of GZipStream, it is expected that
our version of DeflateStream actually implements Write+ReadAsyncMemory.

This change does that by just calling the plain WriteAsync & ReadAsync
methods until we decide (or not) to pull in the corefx DeflateStream.

Backport of https://github.com/mono/mono/pull/16523.
2019-08-29 08:33:44 -04:00
Steve Pfister 99c1602fcd Allow file:// based Uri's to be used in HttpClient when the BaseAddress is set
Fixes https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/967582

Bumps the HttpClient part in mono/corefx at: https://github.com/mono/corefx/commit/ee3acec16b37bec5016da129e1e3840097ca629b

The test that would have caught this failure was disabled previously via:
https://github.com/mono/mono/commit/2f411750acfd2132f9b4970ef8cf11d9a0f62df9

(cherry picked from commit 580defd967)
2019-08-27 20:04:13 +02:00
Alexander Köplinger 16c2464e24 [acceptance-tests] Bump roslyn test repo to remove a problematic nuget feed
(cherry picked from commit 8d422436f5)
2019-08-27 19:48:42 +02:00
monojenkins 5ecff82230 [2019-06] [w32socket] Translate some errno codes to WSA_ errors imprecisely. (#16489)
* [w32socket] Translate some errno codes to WSA_ errors imprecisely.

These aren't a perfect match, but better than falling into the `default` case
where Mono asserts and crashes.

Addresses part of https://github.com/mono/mono/issues/16024

* [w32error] Add WSA_INVALID_PARAMETER and WSA_INVALID_HANDLE

Backport of #16331.
2019-08-27 01:31:53 +02:00
monojenkins 8966e775a5 [utils] Don't use MAP_32BIT on Apple platforms, fixes crash with XCode 11 beta6 (#16481)
As of XCode 11 beta6 the MacOSX SDK defines the MAP_32BIT symbol.

This causes the `mono_valloc()` function to try to use the `MAP_32BIT` flag for `mmap()`.

However as mentioned in the comment for the symbol in mman.h it seems to require a special entitlement which isn't available/documented anywhere yet.
This in turn causes the mmap call to fail presumably because we're missing that entitlement.

Instead we now skip setting this flag on Apple platforms to make `mmap()` behave like it did before.

Backport of https://github.com/mono/mono/pull/16441
2019-08-26 23:07:37 +02:00
Larry Ewing 3608a49541 [2019-06][msbuild] bump msbuild to pick up latest p9 versions and enable shared compilation again (#16445)
* [2019-06] bump msbuild to stop disabling shared compilation

* Pull in nuget update for preview 9
2019-08-24 09:51:03 +02:00
Ankit Jain 049d9ecb3c [msbuild] Bump to track mono-2019-06 (#16428)
* Bump msbuild to track mono-2019-06

.. to get changes from https://github.com/mono/msbuild/pull/130
which tracks dotnet `release/3.0.100-preview9`.

Also, bump Roslyn to 3.3.1-beta3-19421-04 .

* Bump the roslyn reference
2019-08-23 22:15:13 +02:00
monojenkins 66d623d118 Fix issue in SafeHandle.DangerousReleaseInternal() that caused #16034. (#16424)
Using the CoreCLR version of this line fixed the problem and my investigations
also revealed why.

The `SafePipeHandle` contains a reference to both the Socket as well as the
`SafeSocketHandle`, taking ownership of them.  If there are no additional
references to either of them, so that both are collected in the same GC pass,
their finalizer may be invoked in any order.

If the Socket gets finalized first, it's `SafeSocketHandle` will have a `_mstate`
of 8 (two references) at the beginning of `DangerousReleaseInternal()` - which
will set it to 6 (one reference plus disposed flag).

Then when the `SafePipeHandle` gets collected, it will call `DangerousRelease ()`
on it's `SafeSocketHandle`.  As mentioned above, it's `_mstate` will be 6.
Now we own the handle, so we'll decrease it's reference count by one and set the
`Closed` flag - the new `_mstate` should thus be 3.

However, using

    new_state = (old_state & RefCount_Mask) - RefCount_One;

would set it to 1 - thus effectively clearing the `Disposed' flag.  And since it's
reference count is now also zero, the next finalize pass will then throw.

The CoreCLR version of this class uses

    new_state = old_state - RefCount_One;

which fixes this.

Backport of #16244.
2019-08-23 15:34:57 +02:00
Alexander Köplinger 3e76198aa4 Bump corefx to get https://github.com/mono/corefx/pull/336 2019-08-23 14:44:43 +02:00
Steve Pfister fb26dfd84f [2019-06] Add Compiler Server Workaround for 32-bit Linux (#16422)
Currently, if the process is 32-bit, roslyn tries to determine how much
memory is available. To do that, it tries to pinvoke into GlobalMemoryStatusEx,
which is not supported on mono.

Without it, the compiler server bombs on https://github.com/dotnet/roslyn/blob/0e63260c5afb3fb5b74c357dd250e500172bcd63/src/Compilers/Server/VBCSCompiler/NamedPipeClientConnection.cs#L55-L5

Backport of https://github.com/mono/mono/pull/16405
2019-08-23 12:08:30 +02:00
monojenkins 6434153d8b [android] mono_dl_open_file: use g_file_test only on absolute paths (#16392)
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is
the name of a shared library we want to open is to speed up probing for
non-existent libraries.

See https://github.com/mono/mono/pull/12074

The problem is that if file is just a simple "libdl.so" then `dlopen (file)`
doesn't just look for it in the current working directory, it will probe some
other paths too.  (For example on desktop linux you'd also look in all the
directories in LD_LIBRARY_PATH).  So the g_file_test() call is not a robust way
to avoid calling dlopen if the filename is relative.

But it actually broke more things: https://github.com/xamarin/xamarin-android/issues/3388

When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try
prepending some paths that it knows about and we end up calling
`dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security
restrictions on what code can dlopen something from /system/lib with an
absolute path.  Eventually mono_lookup_pinvoke_call will go back to trying the
bare "libdl.so" which hits `g_file_test` and returns NULL.

The new code only does the file test if we pass it an absolute path, which
gives Bionic's dlopen a chance to deal with relative paths however it needs to.
2019-08-22 13:36:13 -04:00
Alexander Köplinger cc0632e4f3 Bump corefx to get https://github.com/mono/corefx/pull/332 2019-08-20 20:52:51 +02:00
monojenkins ff5e4b074f [2019-06] Add missing Dispose() call on error in MonoTlsStream.CreateStream(). (#16327)
* Add missing `Dispose()` call on error in `MonoTlsStream.CreateStream()`.

* Remove the `finally` block to make it more easy to read.

Backport of #16233.
2019-08-20 19:45:59 +02:00
Alexander Köplinger 45c7df08f4 [pkg] Bump bockbuild to get gtksharp bump (#16346) 2019-08-20 11:51:03 +02:00
monojenkins e6d44f7ce4 [2019-06] [corlib] Fix sharing mode validation rules (#16280)
Fixes https://github.com/mono/mono/issues/16032

When we switched to corefx File.Copy on unix, the behavior changed where it opened a source and destination stream in order to pass the safe handles to Interop.Sys.CopyFile. This exposed a flaw in our sharing validation rules in MonoIO.Open, which threw an exception in this scenario:

new FileStream("source.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite, 2048, FileOptions.DeleteOnClose);
new FileStream("source.txt", FileMode.Open, FileAccess.Read, FileShare.Read, 2048, FileOptions.None);

Simplified the FileShare check down to:

1. If the caller specifies FileShare.None, then it's an exclusive lock and no
   sharing is allowed.

2. If the caller specifies some other FileShare option, then FileShare.None
    cannot be specified.
2019-08-16 10:23:09 -04:00
monojenkins 4a85530472 (mono.1) Update the MONO_THREADS_SUSPEND description (#16238) 2019-08-14 19:24:21 +02:00
monojenkins 6c97670c23 [sre] Outline typebuilder_setup_fields loop body (#16203)
The loop body uses coop handles, so wrap it in HANDLE_FUNCTION_ENTER/HANDLE_FUNCTION_RETURN

Addresses part of #16179

Backport of #16184.
2019-08-14 12:32:00 +02:00
Alexander Köplinger 4988ad56ad [pkg] Bump libgdiplus to 6.0.2
(cherry picked from commit b60aaea791)
2019-08-13 12:13:48 +02:00
Sebastien Pouliot 4365dce8b7 [2019-06][sdk] Enable the SRE runtime code inside iOS builds
This will allow the interpreter to use System.Reflection.Emit.

Note: There is a small size increase (about 50kb) for an application
built in release mode with the LLVM backend (even if the interpreter
is now used).

Backport of https://github.com/mono/mono/pull/16151
2019-08-12 21:43:22 +02:00
Alexander Köplinger a7919897e8 Bump corefx to fix build break 2019-08-10 23:47:19 +02:00
Alexander Köplinger 9e02c677aa Bump corefx to get https://github.com/mono/corefx/pull/327 2019-08-10 16:20:04 +02:00
Alexander Köplinger 3c1ee41b12 Bump corefx to get https://github.com/mono/corefx/pull/323 2019-08-09 16:17:39 +02:00
monojenkins 63d98369c6 [2019-06] Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements. (#16108)
* Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements.

https://github.com/mono/mono/issues/16010

The current behavior for GetConstructorCandidates when the name is null, "",
or "*" is to return the set of constructors for the type. For "", .NET Core and
.NET Framework return 0 elements.  This PR adds a check for "" to match that
behavior.

* Removed unncessary changes

* Unnecessary spaces

* Made the if check a bit more readable
2019-08-09 11:56:56 +02:00
Alexis Christoforides 893099f23f [2019-06] [System] Guard against attempting to creating out-of-range spans in m… (#16123)
* [System] Guard against attempting to creating out-of-range spans in macOS FileSystemWatcher

Backport of #16049
2019-08-09 00:38:29 +02:00
Ankit Jain 5d9caa2373 [msbuild] Bump to track mono-2019-06 and update roslyn (#16095)
Prompted by mono/msbuild#123 :

This includes:

Update to upstream vs16.3 branch
SDK update to track dotnet release/3.0.100-preview8
Roslyn updated to 3.3.0-beta2-19381-14

* Work around remoting1.exe test issue with new Roslyn
2019-08-08 20:07:00 +02:00
Larry Ewing 8c035c2144 Pull in the unbroken msbuild 2019-08-07 22:53:27 -05:00
Larry Ewing 3683f7d2b8 Bump msbuild cd5b32d36ad51cb5c21621bb5cfe6a7452214eaa to disable compiler server 2019-08-07 11:34:20 -05:00
monojenkins 91adcb5cd1 [2019-06] [arm] respect I1 and I2 on gsharedvt out transition (#16017)
* [arm] respect I1 and I2 on gsharedvt out transition

In order to trigger the problem, we must call into gsharedvt code ("in" transition) and then escape the gsharedvt world ("out" transition). The bug happens in the latter. Also some other stars need to align (I haven't fully understood it), thus the repro is quite long.

The provided repro only triggers with LLVM, which we don't have enabled on CI yet: https://github.com/mono/mono/issues/15999

Fixes https://github.com/mono/mono/issues/15261
Fixes https://github.com/mono/mono/issues/15262
Fixes https://github.com/mono/mono/issues/15263
Fixes https://github.com/mono/mono/issues/15307


Backport of #16000.
2019-08-07 13:28:18 +02:00
Thays Grazia b454f5a9d0 allow emitting DIM 2019-08-07 09:25:31 +02:00
Aleksey Kliger df37100e76 [w32socket] Translate ELOOP and ENAMETOOLONG
Translate to WSAELOOP and WSAENAMETOOLONG, respectively.

See https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2
2019-08-07 09:23:10 +02:00
Egor Bogatov 7581fffa8a Forgot to check IsInvalid state 2019-08-06 19:29:54 +02:00
monojenkins d53c952790 [corlib] Ignore TimeZoneTest.TestCtors on Android GMT (#16038)
TimeZone.CurrentTimeZone.DaylightName returns an empty value on Android
when the system timezone is set to GMT.

Backport of #15987.
2019-08-06 14:03:00 +02:00
monojenkins 6fbdb372b8 [2019-06] Add finalizer to DeflateStream (#16008)
Fixes #15825

Backport of #15879.
2019-08-05 13:21:53 +02:00
Jo Shields 47a3d9204d Bump libgdiplus to new release (#15996)
(cherry picked from commit 9a7b6fd5ee)
2019-08-03 00:50:36 +02:00
monojenkins 5dc37d7ee6 [2019-06] [marshal] Free delegates with target that are passed to native code. (#15966)
* [marshal] Always use gchandles in delegate_hash_table

Makes the code easier to follow and it also fixes race from https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b with boehm.

* [marshal] Free delegates with target that are passed to native code.

For static method delegates, we have a unique delegate_trampoline that is shared among all delegates. We always keep alive the first static method delegate passed to native, by creating a normal gchandle to it and storing it in delegate_hash_table. For instance methods, each delegate will create a separate wrapper and these wrappers are never shared (which was wrongly assumed in https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b). We shuldn't keep the delegate alive and this commit reverts the behavior for delegate with instance methods introduced in the mentioned commit.

Fixes https://github.com/mono/mono/issues/15751

Backport of #15935.
2019-08-01 20:39:33 +02:00
monojenkins 8b837cb9ca [2019-06] Bump corefx (#15962)
This PR brings changes from mono/corefx#313.
Fixes #15887.

Backport of #15932.
2019-08-01 18:39:40 +02:00
monojenkins 84bc821cdc Fix Socket.ConnectAsync(SocketAsyncEventArgs) behavior. (#15949)
We need to clearly distinguish between synchronous and asynchronous completions
and avoid any ambiguities and race conditions when it comes to error handling.

On synchronous completion, we must not invoke the async completion delegate.

In `System.Net.Sockets/Socket.cs`, the internal `BeginSConnect()` and `BeginMConnect()`
now return a `bool` indicating whether or not an async operation is pending.

If any error happens prior to starting the async operation, then we shall call
`SocketAsyncResult.Complete(..., true)` and return `false`.  And since this is
a synchronous completion, it's `AsyncCallback` will not be invoked.

In `referencesource/System/net/System/Net/Sockets/_MultipleConnectAsync.cs`,
I compared the entire file against the CoreFX version and then copied the
`AttemptConnection()` method from their implementation to adapt their behavior.

The difference is that on synchronous completion, the `InternalConnectCallback(null, args)`
should be invoked.

Backport of #15947.
2019-08-01 12:33:37 +02:00
monojenkins f93add7b16 [threads] Enter GC Unsafe before pumping HP queue in unregister_thread (#15940)
[2019-06] [threads] Enter GC Unsafe before pumping HP queue in unregister_thread

If the thread info TLS key dtor runs `unregister_thread` from a foreign thread and we have work in the hazard pointer queue, we need to first switch to GC Unsafe mode because some of the free methods passed to `mono_thread_hazardous_try_free` need to be coop-aware.

Also, added checked mode assertions that `conc_table_free` and `jit_info_table_free` are called from GC Unsafe mode.

Fixes https://github.com/mono/mono/issues/15878


Backport of #15890.

/cc @lambdageek
2019-08-01 02:35:52 -04:00
monojenkins a5e772c4a1 [2019-06] [sdks] remove 32bit llvm (#15924)
* [sdks] remove 32bit llvm

* remove MonoAotOffsetsDumper leftover

* fix typo
2019-07-31 11:27:33 +02:00
monojenkins 0c44ef150f HANDLE_FUNCTION_RETURN_VAL parameter should not use handles in the current frame. (#15880)
Backport of #15873.
2019-07-30 21:05:36 +02:00
Alexander Köplinger 9df9494bcf [sdks] Disable WASM build on 2019-06 2019-07-30 21:02:35 +02:00
Alexander Köplinger 216d2b2724 [sdks] Make sure python3-pip is installed for Linux Android builds
(cherry picked from commit ee636ee8d9)
2019-07-30 20:46:29 +02:00
Bernhard Urban 17ac7dcfc7 [android] switch to python offset tool (#15848)
Backport of #15834

Warning: Disables WebAssembly build on `2019-06`.
2019-07-29 14:55:21 +02:00
Ankit Jain cec37277df Update nuget.exe to 5.2.0, matching msbuild (#15851)
This is a bit delayed as the 5.2.0 `nuget.exe` wasn't available earlier.
2019-07-27 02:18:49 +02:00
Ankit Jain 1b250ede0a [msbuild] Bump to track HEAD of mono-2019-06
.. to pick up update to track upstream `vs16.3`.
https://github.com/mono/msbuild/pull/118
2019-07-26 18:32:55 -05:00
Ankit Jain 844d5d7c91 Bump msbuild to track mono-2019-06 2019-07-26 18:32:55 -05:00
monojenkins c16ec8c31a Fix http framework module that errors using SDK 2.1.801 (#15854)
- 2.1.801 seems to have a problem with:

```
						using (var uint8Buffer = Uint8Array.From(await request.Content.ReadAsByteArrayAsync ()))
```
2019-07-26 23:39:18 +02:00
Alexander Köplinger 761220d914 [sdks] Report xcode11 builds as a separate GitHub status 2019-07-25 23:45:15 +02:00
Alexander Köplinger f06da77a36 [sdks] Add special build job for building with Xcode 11 2019-07-25 17:13:23 +02:00
Alexander Köplinger c6949d7e3d Bump bockbuild to get pkg-config fix for https://github.com/mono/mono/issues/13804 2019-07-25 00:21:19 +02:00
Vlad Brezae 0cd93e984c [sre] Make creation of dynamic method synchronized (#15814)
[2019-06][sre] Make creation of dynamic method synchronized

Before, reflection_create_dynamic_method could end up being called by two threads simultaneously. At the end of the method the ilgen field of the MonoReflectionDynamicMethod object is cleared, to enable its collection since it is no longer needed. The problem with this is that it can confuse the other thread compiling the dynamic method, making it see that there is no body.

We could either bother making this clearing work with multiple threads (and I'm not 100% convinced that the rest of the code is thread safe), disable the clearing of ilgen (which would make us use more memory) or go with the simple approach of locking on this DynamicMethod so only one thread is compiling it at one time.

Backport of https://github.com/mono/mono/pull/15803
2019-07-24 13:35:56 -04:00
monojenkins d8286ce40b Bump mono/corefx@eb5ba09 (#15813)
Backport of #15799.
2019-07-24 17:30:52 +02:00
Zoltan Varga 4b7a92e09a [runtime] Fix the error message for exceptions created from MonoError.
Fixes https://github.com/mono/mono/issues/15692.
2019-07-23 19:08:44 +02:00
monojenkins 31bf75ed23 [2019-06] Fix ambiguous time at the end of DST period on Windo… (#15785)
As an addition to #15401 it should resolve XA-related issue.

Fixes #9664

Backport of #15548.
2019-07-23 11:15:04 +02:00
Alexander Köplinger 4f29e313aa Add more tests to tarball with EXTRA_DIST
(cherry picked from commit f497893a84)
2019-07-22 15:23:26 +02:00
Bernhard Urban afcf28a366 [2019-06][ios] Switch to python offset tool (#15756)
The C# version doesn't work on 64bit. It would require to update its dependencies, but Zoltan already added a Python version for WebAssembly which works fine.

Some notes:
* changed the semantics of `--xcode-path` in order to match the output of `xcode-select -p`
* switched to python3/pip3
* removed hack for arm64_32, it works properly now because the Python version uses stock `clang`
* removed some leftover usages of `XCODE32_DIR`.

Manually verified by comparing output with the C# version.

This will unblock the iOS team when building on Catalina/Xcode11, as they can't consume SDK archives there. See xamarin/xamarin-macios#6603 (comment)

Contributes to #9621
 master (#15744)
2019-07-19 23:14:17 +02:00
Alexander Köplinger 5bd33de631 [ci] Clean working directory after build in package and sdks build
(cherry picked from commit d159a38c47)
2019-07-19 23:02:19 +02:00
Alexander Köplinger a375e6a090 [packaging] Don't use dl.xamarin.com for gtksharp package anymore
(cherry picked from commit 6407d27000)
2019-07-19 23:02:09 +02:00
Alexander Köplinger 4db3b960c0 [sdks] Calculate sha1 of sdks archive and upload it
This was requested by the XA team.

(cherry picked from commit 49ed255707)
2019-07-19 20:06:31 +02:00
Alexander Köplinger ad226f5d37 [ci] Don't require master node for reportGitHubStatus method
In some jobs we use throttle() logic which would mean only one job can execute on master, blocking other jobs in the meantime.

We don't really care which node executes this so let Jenkins pick one.

(cherry picked from commit 7784f20a26)
2019-07-19 14:50:37 +02:00
Alexander Köplinger aef3c4c16b [sdks] Add linker sources to Android linux archive
(cherry picked from commit a02f3aaf4b)
2019-07-19 11:38:01 +02:00
monojenkins a2bcc639e1 [2019-06] [sdks] Add third party notices to sdks archives (#15755)
It takes all THIRD-PARTY-NOTICES.TXT and LICENSE files from the repo.
2019-07-19 11:09:57 +02:00
Bijan Tabatabai c636636084 Align mcs's tuple deconstruction behavior with csc
Before the code
(a, b) = (b, a);
was the equivalent of
a = b;
b = a;

now, it evaluates it to so something like
tmp1 = a;
tmp2 = b;
a = tmp2;
b = tmp1;

Signed-off-by: Bijan Tabatabai <[email protected]>
2019-07-19 05:01:11 -04:00
monojenkins 1c919ea11c [2019-06] Print two bytes at a time in mono_pe_file_map conversion error (#15726)
* Print two bytes at a time

* Iterate over gunichar2
2019-07-18 14:48:10 -07:00
monojenkins 4b15f9b3ee [mini] Runtime check if boxing is needed for DIM constrained calls (#15730)
This is to handle the following example.  The issue is that in Check() we have
a constrained call where the called method is IAdder`1<!!U>::PlusPlus() which
is a default interface method, and we need to determine at runtime whether to
box the this argument.

```
using System;

public interface IAdder<T> {
	int Add (int x);

	int PlusPlus () {
		return Add (1);
	}
}

interface IGen3<T> { }

struct Adder<T> : IAdder<IGen3<T>> {
	int _field;
	public int Add (int x) {
		_field = x + _field;
		return _field;
	}
}

public class P {
	public static int Check<T, U>(T t) where T : IAdder<U> {
		return t.PlusPlus () + t.PlusPlus ();
	}

	public static void Main () {
		var x = new Adder<object> ();
		int y = Check<Adder<object>, IGen3<object>> (x);
		Console.WriteLine ("expect 2, got {0}", y);
	}
}
```
2019-07-18 16:31:49 -04:00
Larry Ewing 17520a8622 Bump msbuild to msbuild:mono-2019-06 to stop disabling compiler server 2019-07-17 16:47:23 -04:00
monojenkins 048d95d86d Add com.apple.security.cs.allow-unsigned-executable-memory exception to hardened runtime entitlements (#15725)
This avoids the issue from https://github.com/mono/mono/issues/13445 when hardened runtime is enabled.
2019-07-17 20:50:23 +02:00
monojenkins c63edc338b [2019-06] [bcl] Sync recent NS2.1 changes (#15719)
1) Copied changes from corefx for System.Data (also mono/corefx#310)
2) New Tracing api is left empty (because the current API is empty too)
3) New AES types throw PNSE

Bump netstandard.dll AssemblyVersion to 2.1 (fixes #13412)

Backport of #15696.
2019-07-17 14:31:43 +02:00
Alexander Köplinger 74e1f4cbf9 Revert "[ci] Remove separate utils.groovy file" and run step on master instead
This reverts commit d732cef5fc.

The github commit status setter step requires a node context so we now just use on on master instead.

(cherry picked from commit 0509ceccb6)
2019-07-17 14:27:05 +02:00
monojenkins a79115dd7f [2019-06] [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools… (#15718)
* [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools profile

Requested by the XA team.

* [csproj] Update project files
2019-07-17 12:44:53 +02:00
Alexander Köplinger d410e07d8b [ci] Remove separate utils.groovy file
For some reason when using a separate file and loading the function
from there Jenkins still requires us to have a node() context when
calling the function which is not what we want.
2019-07-17 10:57:09 +02:00
Alexander Köplinger 058d207f58 [ci] Wait for signing outside of a node() block
We don't need to block the machine just for waiting on signing to finish.

(cherry picked from commit 39472e06a3)
2019-07-17 01:02:55 +02:00
monojenkins a5b77554e4 [2019-06] [System] Keep events from firing from a disposed 'De… (#15706)
Fixes https://github.com/mono/mono/issues/8747

Also try to stop other implementations from dispatching events past Dispose(), without guaranteeing it
2019-07-16 21:45:40 +02:00
Alexander Köplinger ce68c20971 [ci] Bump timeout for .pkg signing job
In preparation for notarizing our .pkg with Apple we need to bump the timeout
since it can take up to an hour for Apple's server to return a result.

(cherry picked from commit 7480830b36)
2019-07-16 21:32:31 +02:00
Alexander Köplinger edc9e70e04 [sdks] Use Azul Zulu OpenJDK for Android Linux build, since Debian 10 has no Java 8 2019-07-15 23:24:20 +02:00
Alexander Köplinger ea8da188ef [sdks] Update android linux builds to Debian 10
This was suggested by XA folks to fix the weird mingw llvm issues we've been seeing.

(cherry picked from commit 4b6f49bb06)
2019-07-15 21:36:27 +02:00
monojenkins fe2a0a8810 [2019-06] [debugger] Don't list finalizer gc thread if it's not executing managed code (#15683)
* The example that reproduces the bug does this: var thread = vm.GetThreads()[0]; greetingValue = program.InvokeMethod(thread, greeting, new Value[0]) as StringMirror;

But sometimes the Thread[0] was the MainThread of the program and it works, and other times was the GC Finalizer, but when it tries to run something on GC Finalizer thread, the thread is not suspended and throws an exception.
In the fix I removed the Finalizer thread of the list that is returned when CMD_VM_ALL_THREADS is called.

Fix #13311

* The example that reproduces the bug does this: var thread = vm.GetThreads()[0]; greetingValue = program.InvokeMethod(thread, greeting, new Value[0]) as StringMirror;

But sometimes the Thread[0] was the MainThread of the program and it works, and other times was the GC Finalizer, but when it tries to run something on GC Finalizer thread, the thread is not suspended and throws an exception.
In the fix I removed the Finalizer thread if ti's not executing a managed code of the list that is returned when CMD_VM_ALL_THREADS is called.

Fix #13311

* Changing what was suggested by Aleksey.

* Removing unused attribute.
2019-07-15 17:14:15 +02:00
monojenkins 7fd5a4da49 [2019-06] Fix marshalling bugs for StringBuilder parameters (#15685)
Fixes #15366 and #15306.

- Fixes NULL termination of `StringBuilder` to UTF-16 marshalling. Additionally an extra NULL is added as defense-in-depth protection for incorrect handling in native calls. This extra NULL adds protection when trying to marshal the NULL-terminated string back and the native function doesn't null-terminate at the size of the buffer.
- Fixes marshalling back from UTF-16/UTF-8 buffers into chunked `StringBuilder`.
- Fixes missing `mono_marshal_free` call causing a memory leak when marshalling `StringBuilder` to UTF-16 and back.
- Skips marshalling back to `StringBuilder` when the parameter doesn't have `[Out]` attribute.

Unit tests are still TBD. Not sure where to put them and how to structure them. Any help would be appreciated. /cc @vargaz @EgorBo 

Backport of #15392.
2019-07-15 16:45:55 +02:00
monojenkins f688049114 [2019-06] [ios] Nil Check to Ensure Local TimeZone Id is Set F… (#15690) 2019-07-15 16:06:45 +02:00
Alexander Köplinger a56786e89e [System] Remove FileSystemWatcher tests from monotouch/xammac
They were already excluded from watch and tv but they don't work on any mobile Apple profile.
2019-07-15 15:16:09 +02:00
monojenkins 675c744a7e [2019-06] [ios] Ensure Local TimeZone Id is Set For TimeZoneIn… (#15670)
* Fixes https://github.com/mono/mono/issues/14839

The local timezone name was not known and defaulted to "Local".
This change adds an sdk function that returns the local name.

Backport of #15667.
2019-07-15 15:11:19 +02:00
Alexander Köplinger 5098258cce [ci] Upload mac-entitlements.plist to azure when uploading .pkg
So it can be used by the codesigning job.

(cherry picked from commit e3d5b23ff9)
2019-07-12 19:48:08 +02:00
Jo Shields 4ec4736af4 Add hardened runtime testing to Catalina lane (#15589)
* Permit DYLD mangling, so p/invokes work from non-system paths

* Add com.apple.security.cs.disable-library-validation

This is required for the .dylibs used in p/invokes to work.

* Use adhoc signing

(cherry picked from commit 202dca075b)
2019-07-12 19:48:08 +02:00
monojenkins 5debe32373 [2019-06] Fix offsets tool for Android NDK r20 (#15650)
Android NDK r20 contains a header file (`ioctl.h`) with this code:

    int ioctl(int __fd, int __request, ...);

    #if !defined(BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD)
    /* enable_if(1) just exists to break overloading ties. */
    int ioctl(int __fd, unsigned __request, ...) __overloadable __enable_if(1, "") __RENAME(ioctl);
    #endif

The `_RENAME` macro is defined as `__asm__(x)` and serves the purpose of
replacing an existing symbol with the new one, decorated with the keyword.
Unfortunately, Mono's `offsets-tool` doesn't parse the header properly as
`CppSharp` doesn't understand `__asm__`:

    Processing triple: armv7-none-linux-androideabi
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/metadata/metadata-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/mini/mini-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    make: *** [/Users/rodo/git/xa2/external/mono/sdks/builds/android-cross-arm-release/armv7-linux-android.h] Error 1

The workaround, implemented in this commit, is to define the
`BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD` macro thus removing the "conflicting"
declaration from the picture, which changes nothing from the offset tool's point
of view and lets us process the headers in NDK r20 properly.
2019-07-11 20:39:42 +02:00
Alexander Köplinger 5898c9fd07 Bump bockbuild to fix gtk build break 2019-07-11 17:30:51 +02:00
Alexander Köplinger aed9c000ec Bump bockbuild to get https://github.com/mono/bockbuild/pull/105 2019-07-11 16:09:22 +02:00
monojenkins ba719acaa2 [2019-06] Add TypeForwards for FileSystemName & CryptographicOperations (#15657)
* Add TypeForwards for:

* System.IO.Enumeration.FileSystemName
* System.Security.Cryptography.CryptographicOperations

Since these were removed in NS 2.1

* Bump API snapshot submodule
2019-07-11 01:10:10 +02:00
Steve Pfister dfd02351f4 [2019-06] Bump mono/corefx@17dd540 (#15653) 2019-07-10 23:57:36 +02:00
monojenkins 4c710e041c [2019-06] [System]: Allow reading fully buffered `HttpWebRespo… (#15638)
[System]: Allow reading fully buffered `HttpWebResponse` body after close.

If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.

* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
   call down after argument checks and call a new protected abstract method
   called `TryReadFromBufferedContent ()` prior to it.

* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
  If the entire response body has been buffered, then `bufferedEntireContent`
  is true and `innerStream is BufferedReadStream` and we can call the new
  `BufferedReadStream.TryReadFromBuffer ()`.

* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
   after reading the entire response body.

* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.

Fixes #10645.

Backport of #15220.
2019-07-10 10:45:12 +02:00
monojenkins d99e13bdea [2019-06] [llvm] avoid FP elimination on iOS/armv7 (#15621)
[2019-06] [llvm] avoid FP elimination on iOS/armv7

Instead of
```
push    {r4, r5, r6, r7, r10, r11, lr}
```

LLVM will generate
```
push    {r4, r5, r6, r7, lr}
add     r7, sp, #12
push    {r9, r10}
```

Seems like this https://github.com/mono/llvm/commit/a04e9e4a0af16f15ace258e81448b7eeca5ff599 assumes that `-disable-fp-elim` is passed for iOS targets. I wondered why this wouldn't break things for them, but when you run `clang` from Xcode with `-v`, you will discover that its driver passes `-mdisable-fp-elim`.

Together with https://github.com/mono/llvm/pull/48 this fixes crashes with FullAOT+LLVM on iOS 32bit.

Fixes https://github.com/mono/mono/issues/15058

Contributes to https://github.com/mono/mono/issues/9621



Backport of #15617.

/cc @lewurm
2019-07-09 17:25:44 -04:00
Alexander Köplinger c608cf3eaf Make RemoteExecutor a no-op on all mobile profiles (#15611)
Starting another Mono process for the remote executor doesn't work on any of them.

Helps with https://github.com/mono/mono/issues/15328

(cherry picked from commit b2ba8e62ca)
2019-07-09 11:45:35 +02:00
monojenkins 90d32a9d62 Crashes on running the boring.il test from coreclr, the problem was that using [System.Runtime]System.String the type was not understood as the primitive type string. Using ilasm from coreclr and running on mono it works, because of that I decided to fix on ilasm and not on runtime. (#15609)
Fixes #13923
2019-07-08 17:48:55 -03:00
Alexander Köplinger 329145ee73 [sdks] Fix android sdks job, take 2
It needs to work for both debug and release configs.

(cherry picked from commit 784449a077)
2019-07-08 19:35:09 +02:00
Alexander Köplinger a48723b842 [sdks] Fix android sdks job
It would try to execute the .stamp-android-loader-path target in unrelated configurations.

(cherry picked from commit 305b2e76e6)
2019-07-08 18:06:22 +02:00
Nikita Voronchev 08dd9882ce [2019-06] [System.Windows.Forms] Fix crash of DataGridView in in case of assigning a custom Column Header. (#15461)
This PR fixes the crash of DataGridView in case of assigning a custom Column Header.

Backport of #15135
2019-07-05 17:42:07 +02:00
monojenkins 16bf40461d [2019-06] [android] switch cross compilers host from 32bit to… (#15581)
Untested, but we need to do it anyway soon. This will help us to understand the situation on Xamarin.Android while finishing the work for Xamarin.iOS.

Contributes to https://github.com/mono/mono/issues/9621

Backport of https://github.com/mono/mono/issues/15553.
2019-07-05 17:41:08 +02:00
monojenkins 91f98eb2c8 [2019-06] [sdks] Use @loader_path for mono and libmonosgen (#15587)
* [sdks] Use @loader_path for mono and libmonosgen

This way we are able to dynamically load libmonosgen and also use host
mono binary.

The library dependencies look like this after the change:
```
otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...

otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...
```
2019-07-05 17:40:30 +02:00
Steve Pfister 2629c6d8f8 [2019-06] Bump mono/corefx@9354dfd (#15547) 2019-07-05 14:34:55 +02:00
Ankit Jain 972d548394 [2019-06] Bump msbuild to track mono-2019-06 (#15525)
* Bump msbuild to track mono-2019-06

* Bump nuget.exe to 5.1.0 to match what msbuild has.
2019-07-03 10:57:13 +02:00
Alexander Köplinger 39e8153c00 [2019-06] Bump corefx (#15519)
* [2019-06] Bump corefx

* Bump API snapshot submodule
2019-07-03 01:50:47 +02:00
monojenkins 0f75b36c0f [2019-06] [System.Net.Http]: Delete broken `Send_Transfer_Enco… (#15496)
This test was checking a broken Mono-specific behavior, which has been
removed after we switched to the new corefx-based handler. It also does
not make much sense to use a `Transfer-Encoding` header with a GET request.
2019-07-02 01:29:29 +02:00
Alexander Köplinger ffbba36f45 [2019-06] Bump bockbuild to bring in fix for mono/mono#14957 2019-07-02 00:32:12 +02:00
Thays Grazia 567964aff5 Fixes #13925. (#15489)
Backport https://github.com/mono/mono/pull/15433 to 2019-06.
2019-07-01 19:01:09 -03:00
Aleksey Kliger f55e0a671c [loader] remap only System.Threading.Tasks.Dataflow older than 4.0.0.0
Allow through requests for newer versions unchanged.
2019-07-01 17:20:56 +02:00
monojenkins 3702e62c7d [jit] Avoid calling mini_jit_info_table_find () in mini_add_method_trampoline () unless needed. (#15414) 2019-06-30 19:07:07 -04:00
monojenkins 41aad78a9d CreateNLSocket and CloseNLSocket should return gpointer (#15431)
In managed, these functions are used as `static external IntPtr`. This means that previously on arm64 the top bits of the return value were garbage. A comparison with 64-bit -1 like in LinuxNetworkChange.EnsureSocket would never be true, which was causing us to hit other assertions in the runtime.
Co-authored-by: Aleksey Kliger <[email protected]>
2019-06-30 15:06:13 -04:00
Aleksey Kliger fc63d85a38 [sdks] dynamically link libmonosgen to mono-sgen for monodroid host builds 2019-06-28 21:46:35 -05:00
Aleksey Kliger 605f7f04fc [runtime] Mark mono_file_map_error as MONO_API, always
Fixed the `--with-static_mono=no` build on non-Win32
2019-06-28 21:46:35 -05:00
Aleksey Kliger d8e2a049fc [configure] Cosmetic AC_MSG_WARN fix 2019-06-28 21:46:35 -05:00
monojenkins 9fd0d07cfb Correct env vars used for Android directory override (#15471)
This fixes https://github.com/mono/mono/pull/15453

The issue is the env var names differ vs. Xamarin.Android
2019-06-28 15:33:20 -04:00
monojenkins 8c50c1a047 [2019-06] Use /mnt/scratch for Android SDK, to match https://g… (#15468) 2019-06-28 19:23:35 +02:00
monojenkins 89867fd4ea [2019-06] Fix year+ old MIPS compile break. (#15460)
https://github.com/mono/mono/issues/15441
2019-06-28 11:29:50 +02:00
monojenkins 25a7d8c97c [2019-06] Fix time zone transition out of DST (#15445)
* Time zone DST out fix

* Add unit tests for Europe/Vatican, Iran and Europe/Guernsey time zones

* Disable the tests on WINAOT; fixed Iran time zone id
2019-06-27 20:48:40 +02:00
monojenkins dec365d6fd [sgen] Fix crashes in sgen due to xdomain refs (#15426)
CADMessages are passed directly between appdomains. When using contained data, we need to make sure we don't create any cross domain references.

When domains are unloaded, all threads executing code in that appdomain are aborted and the entire heap is traversed and objects in that appdomain are cleared. If any objects in other domains are directly referencing objects in the unloaded domain, they will reference junk. If any, cross domain refs should only be allowed when the referenced object is in the root domain which is never unloaded.
2019-06-27 12:37:19 +02:00
lateralusX 68d1fceba9 Fix race related to final event signaling in RegisteredWaitHandle.
We have seen sporadic failures in thread pool tests on CI related to
test case:

System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
          at System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest () [0x00143] in D:\j\workspace\z3\label\w64\external\corefx\src\System.Threading.ThreadPool\tests\ThreadPoolTests.cs:337
          at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
          at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in D:\j\workspace\z3\label\w64\mcs\class\corlib\System.Reflection\RuntimeMethodInfo.cs:395

Finally managed to track down the issue to a race when using the same final
event in sequential calls to RegisterWaitForSingleObject. The event could, depending on
timing, get signaled multiple times for the same RegisterWaitHandle instance
causing a thread waiting for completion on another RegisterWaitHandle using same event
to be signaled before that callback was completed. That will cause the race seen in the
failing test above.

Fix make sure we only signal the final event one time per RegisterWaitHandle
instance.
2019-06-26 07:12:16 -05:00
monojenkins 9bfdc5c38b [2019-06] [jit] Fix interface cast (#15375)
* [jit] Throw exception if casting fails for special array interface

After the standard interface check (mini_emit_iface_cast), for special interfaces we need to do additional checks. We were skipping these checks if the object was not an array, without throwing a cast exception. This code path is reached for configurations where remoting is disabled.

Fixes https://github.com/mono/mono/issues/14729

* [tests] Add test for interface casting case

This test would have failed on fullaot lane for example.
2019-06-26 11:38:39 +02:00
Alexander Köplinger ce79aff5be [System] Fix monotouch_watch xunit tests build
Forgot to exclude one more file.

(cherry picked from commit 41767754d5)
2019-06-25 22:22:42 +02:00
monojenkins f481f794ce [2019-06] Exclude a few tests on monotouch_watch/xammac (#15409)
* [System.Core] Exclude AnonymousPipe tests on monotouch/xammac profiles

They should only be run on xammac_net_4_5.

Fixes https://github.com/mono/mono/issues/15329

* [System] Exclude WebClient and WebSocket tests on monotouch_watch

Fixes https://github.com/mono/mono/issues/15265.
2019-06-25 20:03:44 +02:00
Martin Baulig ed3321827e [test-helpers]Don't use named Mutexes on mobile. 2019-06-25 08:46:26 +02:00
Alexander Köplinger c5d9285ad3 [sdks] Do the WATCHOS5_VERSION -> WATCHOS64_32_VERSION rename in run-jenkins.sh too
It was missed in https://github.com/mono/mono/pull/15202
2019-06-24 19:14:14 +02:00
Bernhard Urban d707d745a6 [arm] set exception handling model to dwarf for iOS
We recently switched the AOT compiler for iOS armv7 from 32bit host to 64bit host. At the same time we _also_ switched from LLVM 3.6 to the newer LLVM 6.0 fork we support.

In a Xamarin.iOS Release build for iOS armv7 exception handling would crash. Specifically the problem was that the exception object wasn't properly injected into the handler block. The way it works is, that the exception handler machinery passes the pointer to the exception object via `r0`.

With the newer LLVM the generated code of the exception handling block
looks like this:
```
LBB1_15:                                @ %EH_CLAUSE0_BB3
Ltmp8:
	ldr	r0, [sp, #24]
	str	r0, [sp, #4]
	ldr	r0, [sp, #28]
	ldr	r0, [sp, #12]
	ldr	r0, [r0]
	cmp	r0, #0
	beq	LBB1_17
```
thus overwriting `r0`, so that the pointer to the exception object is lost.  With the older LLVM 3.6 the generated code looks like this:
```
LBB1_8:                                 @ %EH_CLAUSE0_BB3
Ltmp8:
	str	r0, [sp, #4]
	ldr	r0, [r6]
	cmp	r0, #0
	beq	LBB1_10
```
correctly storing the exception object into a stack slot for later usage.

After some time I figured out that there are _different_ exception handling models. Depending on the model, the exception pointer is passed in `r0` or not:
https://github.com/mono/llvm/blob/2bd2f1db1803f7b36687e5abf912c69baa848305/lib/Target/ARM/ARMISelLowering.cpp#L14416-L14428

`SjLj` stands for "SetJump / LongJump" and  means that exception handling is implemented with this. On iOS this is the default model that is used. We want `dwarf` instead, so we tell this `llc` now and the generated code handles it correctly.

So why was it working before? In our older LLVM 3.6 fork we hardcoded it:
https://github.com/mono/llvm/blob/f80899cb3eb75f7f5640b4519e83bd96991bffb8/lib/Target/ARM/ARMISelLowering.cpp#L756-L762

The `-exception-model=` option is also not available in LLVM 3.6.

Contributes to https://github.com/mono/mono/issues/15058 and https://github.com/mono/mono/issues/9621
2019-06-24 09:48:54 +02:00
Zoltan Varga d0294cf582 [aot] Fix a deadlock in init_plt () by calling create_trampoline () outside the lock.
Fixes https://github.com/mono/mono/issues/15330.
2019-06-24 09:33:44 +02:00
Vlad Brezae 3b576e58c7 [arm64] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-24 09:22:39 +02:00
Vlad Brezae 2878c33dfe [arm] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-24 09:22:39 +02:00
Vlad Brezae 1398a3d33a [amd64] Fix location saving for *this*
With static methods we were saving location for irrelevant argument.
2019-06-24 09:22:39 +02:00
Aleksey Kliger (λgeek) 6bed4fc2ff [ci] One more workaround for Git 2.22.0 (#15325)
Homebrew Git 2.22.0 misparses the submodule command and passes arguments like --hard and -xdff to git-submodule instead of to git-reset or git-clean.  Passing the entire command as a single argument seems to help.

Git mailing list bug report: https://public-inbox.org/git/[email protected]/
2019-06-21 10:57:25 -04:00
monojenkins f74dc9a395 [2019-06] [debugger] Debugger doesn't break on unhandled exceptions (#15286)
* When the catch found is in a MONO_WRAPPER_RUNTIME_INVOKE the exception should be treated as an UNHANDLED by the debugger.

* Adding unit test.

* trying to fix side effect.

* I've tested the MonoTests.DebuggerTests.UnhandledException_2 test on VsWin using .net Framework and I think the test is wrong because the first break is in throw and the next break will be on breakpoint. In our implementation we were ignoring the unhandled exception and stopping only on breakpoint.
2019-06-21 11:24:21 +02:00
Larry Ewing b2987f714b Merge pull request #15292 from monojenkins/backport-pr-15255-to-2019-06
[2019-06] [jit] Make alias analysis properly handle alias cloberring
2019-06-20 22:42:53 -05:00
Vlad Brezae b869c961ce [jit] Make alias analysis properly handle alias cloberring
Alias analysis attempts to remove indirection when using OP_LDADDR (which fetches the address of a variable). For example :

	ldaddr r1 <- r0
	load_mem r2 <- r1

is equivalent to

	mov r2 <- r0

The remembered r1 alias needs to be forgotten in case r1 is changed. We were handling this for direct assignment with MOV, but not for the more subtle case where an alias to r1 would be passed to a call, which could lead to changing r1.

https://github.com/mono/mono/issues/14872
2019-06-20 18:42:58 +00:00
monojenkins 3379668399 [csproj] Update project files 2019-06-20 18:49:25 +02:00
Marek Safar a49a3ea553 Bump linker & cecil 2019-06-20 18:49:25 +02:00
Bernhard Urban 71e0854e2a [arm] fix some more host/target pointer size issues
As suggested by Jay in https://github.com/mono/mono/issues/15058#issuecomment-501995400

I haven't seen fixing it any specific issue, but we should still backport it to `2019-06` in order to avoid hard-to-debug regressions.

Contributes to https://github.com/mono/mono/issues/9621
2019-06-19 12:58:52 +02:00
Alexis Christoforides 9d1153af22 Improve robustness of post-install script
We have several issues (e.g. https://github.com/mono/mono/issues/14181) where the installer occasionally fails in the postinstall script for unknown causes, on some hosts.

This seems to be sometimes the culpable section and this patch should help trace the issue or alleviate the common symptoms of it (see https://github.com/mono/mono/issues/14181)
2019-06-19 08:48:57 +02:00
Aleksey Kliger 6091b8f209 [coop] Mark a few functions with MONO_PROFILER_API 2019-06-18 19:44:16 +02:00
Aleksey Kliger 7fcdd9f318 [runtime] Don't do GC Unsafe transition in mono_gchandle_free
Xamarin.Mac can call back here from a native thread that isn't attached to the
runtime.  SGen gchandle code is lockfree, and the profiler event can fire
without involving Mono.
2019-06-18 19:44:16 +02:00
Aleksey Kliger 5c36c00915 [profiler] Don't switch to GC Safe when locking if thread isn't attached to the runtime.
Some profiler events (for example gc_handle_deleted) could be raised on threads
that aren't attached to the runtime.  Since the thread won't participate in
Mono's stop the world (since the runtime can't see it), there's no need to do
the transition to GC Safe while the thread is trying to grab the profiler
buffer lock.
2019-06-18 19:44:16 +02:00
monojenkins a099adcbce [2019-06] Implementing reabstraction of default interface methods. (#15229)
* Implementing reabstraction of default interface methods.

* Fixing side effect on unit tests. Tested with another example and changed the implementation: using System; public interface I1 { void M1(); /*void M2(){ System.Console.WriteLine("I1.M2"); }*/ }

public interface I2 : I1
{
    abstract void I1.M1();
    /* void M2(){
        System.Console.WriteLine("I2.M2");
    }*/
}

class Test1 : I2
{
    static void Main()
    {
        I1 i1 = new Test1();
        i1.M1();
        //i1.M2();
    }

    void I1.M1()
    {
        System.Console.WriteLine("Test1.M1");
    }
}

* Fixing side effect on unit tests.

* Fixing side effect on unit tests.

* Implementing unit test. Fixing side effect.

* Fix unit test.

* Fixing reabstraction.il coreclr test.

* Changing comments.

* Changing things pointed by Aleksey.

* Changing things pointed by Aleksey. Implementing generics test.

* Changing things pointed by Aleksey.
2019-06-18 19:43:25 +02:00
Bernhard Urban 70732b33a8 [watch] consistent naming for watchOS min version
See https://github.com/xamarin/xamarin-macios/pull/6307
2019-06-18 16:38:09 +02:00
Bernhard Urban d68b18c9c2 [arm] use target_mgreg_t instead of plain pointer type in sdb breakpoint
Fixes Debug mode for iOS 32bit.

Contributes to https://github.com/mono/mono/issues/15058 & https://github.com/mono/mono/issues/9621
2019-06-17 16:52:42 +02:00
Aleksey Kliger 02311a91c1 [debugger-agent] Ignore thread_end event if thread is detached
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set.  In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.

Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace

```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
  * frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
    frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
    frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
    frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
    frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
    frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
    frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
    frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
    frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
    frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
    frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
    frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
    frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
    frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
    frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
2019-06-15 09:31:20 +02:00
Aleksey Kliger 0d00315b97 don't use 'gpointer' in a public header 2019-06-15 09:29:36 +02:00
Aleksey Kliger f73b774483 [mini] Move some MONO_API functions to public header
And mark them all MONO_RT_EXTERNAL_ONLY.

mono_set_use_llvm is new - Xamarin.Android previously assigned to mono_use_llvm
directly.
2019-06-15 09:29:36 +02:00
Jay Krell 507c4d8c99 Add missing read barrier to mono_lazy_initialize().
This is unfortunate.

The general pattern of:

if initialized
  use the data
else:
  initialize the data, possibly with lots of locks and barriers
  mark initialized

is racy, because "use the data" can be scheduled ahead of "if initialized".

"Barriers come in pairs" generally, and this was missing one.

It depends somewhat. If the data is all pointers, initialized in the else path,
from null to non-null, and use includes dereferencing, which is a common
but not universal case, then it is ok on all non-Alpha processors, due
to "data dependency".

But if the data includes reading globals, then there is a race.

This is a bit of a slow down on fast paths on arm, and possibly
other architectures.

There are barrier-free ways to solve this, involving a thread local,
that should seriously be considered, and applied throughout the runtime.

The runtime has a lot of on-demand initialization and a lot of looks racy.

See www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2660.htm.
Which would have to be adopted to be coop-friendly which should not be difficult.

Notice that the existing "lazy" mechanism is also not coop-friendly.
That is not changed by this PR.
2019-06-14 23:53:49 +02:00
Jay Krell b60c60f2ab Fix bug and race in mono_lazy_initialize. 2019-06-14 23:53:49 +02:00
monojenkins c15b300298 [2019-06] [System.Net.Http]: Add hack to the legacy HttpClient to pass down Timeout property. #12577. (#15014)
* [System.Net.Http]Add hack to the legacy HttpClient to pass down Timeout property. #12577.

* `HttpClientHandler`, `IMonoHttpClientHandler`: add new internal `MonoSetTimeout (TimeSpan)` function.

* `HttpClient.SendAsyncWorker()`: if `handler` is `HttpClientHandler`, call the new internal
  `MonoSetTimeout()` function to pass down the timeout.

* `MonoWebRequestHandler.SendAsync()`: this is only used here to pass the timeout to the `HttpWebRequest`.

* Address feedback.

* Update HttpMessageInvoker.cs

* Make it build.

* Really make it build (forgot monotouch_watch).
2019-06-14 10:51:02 +02:00
monojenkins 6752005e64 [2019-06] Fix Japan's new era fallback isn't reflected in Mono (#14984)
* Fix Japan's new era fallback isn't reflected in Mono

refs: <https://github.com/dotnet/coreclr/pull/23614>

* Fix array length

* Fix MonoTests.System.Globalization.CalendarTest

* Fix MonoTests.System.Globalization.EastAsianLunisolarCalendarTest

* Fix era index

refs: 1715e1d0e738ce95382ba372c24502967002a2f0

* Amend fixing

refs: 05af6e6f75fcea76e3d9e9f2f95f0caedb753aa7
2019-06-14 10:50:09 +02:00
Ryan Lucia a2424f031e Actually print out *filename correctly 2019-06-14 10:49:46 +02:00
Ryan Lucia 1517a02c09 Naming: err -> error 2019-06-14 10:49:46 +02:00
Ryan Lucia c7a57444dc Switch everything to MonoError 2019-06-14 10:49:46 +02:00
Ryan Lucia 7cf9bf95d4 Remove todos 2019-06-14 10:49:46 +02:00
Ryan Lucia 1477c73eba Print input string when conversion fails 2019-06-14 10:49:46 +02:00
Ryan Lucia 897621178b Remove redundant goto 2019-06-14 10:49:46 +02:00
Ryan Lucia f85685911c Migrate things to _error version where sensible 2019-06-14 10:49:46 +02:00
Ryan Lucia 7490bdd567 Add internal version of mono_unicode_to_external to propagate error 2019-06-14 10:49:46 +02:00
Imran Hameed 23bac8c6e6 Move ternary SSA cfold test from basic.cs to objects.cs. 2019-06-13 23:48:21 -05:00
Imran Hameed e1ebf711eb Treat ternary operations as non-constant... (#13873)
...during constant evaluation and propagation.
2019-06-13 23:48:21 -05:00
monojenkins 46f7a64e38 Bump API snapshot submodule 2019-06-13 08:42:09 +02:00
Zoltan Varga a020195ec7 [corlib] Add missing Unsafe.Unbox () method.
Fixes https://github.com/mono/mono/issues/14871.
2019-06-13 08:42:09 +02:00
Aleksey Kliger 43f00b12ad [llvm] bump to mono/llvm@2bd2f1db18
Emit .fnend directives on arm even for functions which don't have a corresponding mono method.

Fixes https://github.com/mono/mono/issues/15048
2019-06-13 08:38:52 +02:00
Aleksey Kliger e126d67a6e [runtime] Switch to GC Unsafe for all GC external API functions
Fixes https://github.com/mono/mono/issues/14975
2019-06-12 12:55:27 -05:00
monojenkins 0ef4a3b0d9 Trying to unrevert https://github.com/mono/mono/pull/7390. Couldn't reproduce the same side effect as that caused before. (#14969)
This PR fixes #7377
2019-06-12 14:44:28 -03:00
Zoltan Varga 9a5bddb57d [runtime] Fix gshared support for BeginInvoke ()/EndInvoke () wrappers.
Fixes https://github.com/mono/mono/issues/14972.
2019-06-12 18:56:09 +02:00
Zoltan Varga 05e324bce7 [aot] Open AOT profile files using "rb" so it works on windows.
Fixes https://github.com/xamarin/xamarin-android/issues/3168.
2019-06-12 08:31:14 +02:00
monojenkins b96e503690 [2019-06] [llvm] Fix disabling direct calls and inlining (#14886)
* [llvm] Enable disabling inlining on llvm

* [llvm] Fix disabling direct calls on llvm

* [llvm] Rename mono_inited even when doing self_init (generated init functions refer to it)
2019-06-11 18:44:39 +02:00
monojenkins c2c9c2e012 [install] Include .exe extension when symlinking the mono binary (#14965)
Prevent the SDKs from making dangling symlinks when packaging the MXE
cross-compilers: should hav

    mono.exe -> mono-sgen.exe

rather than

    mono -> mono-sgen (dangling).
2019-06-11 07:31:29 -04:00
Martin Baulig 547b34f0ed Do the same thing in HttpListener. 2019-06-08 11:24:54 +02:00
Martin Baulig 63a2ab73ed Implement RSACertificateExtensions.CopyWithPrivateKey(). #14152.
* RSACertificateExtensions.CopyWithPrivateKey(): implement.

Since `RSACertificateExtensions` lives in `System.Core` and only `mscorlib`
internals (but not `System` internals) are visible to it, we need to use a
little trick here:

* System.Security.Cryptography.X509Certificates.X509CertificateImpl:
  add new abstract methods CopyWithPrivateKey() and CreateCertificate().

* System.Security.Cryptography.X509Certificates.X509Certificate2Impl:
  sealed implement them here.

The `httpcfg` tool, only checks whether the key is valid, so we don't really
need to call `CopyWithPrivateKey()` (which requires us to take a dependency
on `System.Core`).
2019-06-08 11:24:54 +02:00
monojenkins ddef564e34 [2019-06] [runtime] Make cross-appdomain wrappers throw NIE for methods with ref struct args (#14867)
* [AppDomain] Cross-appdomain wrapper throws NIE for methods with ref struct arguments

If we have a ref struct and a method that takes it as an argument:

```
[Serializable]
public ref struct R {
  public int i;
}

public interface I {
  public void M1 (R r, SomeClass o);
}

public class M : MarshalByRefObject, I {
  public void M1 (R r, SomeClass o) { ... }
}
```

If we create an instance of M in another domain and try to call it:
```
  I obj = (I)otherDomain.CreateInstanceAndUnwrap (typeof(M).Assembly.FullName,
  nameof(M));
  R r;
  r.i = 42;
  SomeClass o = ...;
  obj.M1 (r, o);
```

Mono will need to create a wrapper that can invoke M.M1 in the other domain.
The way the wrapper works in `mono_marshal_get_xappdomain_invoke` is by
creating an array of all the arguments that need to be serialized (see
`mono_get_xdomain_marshal_type`) and then serializing the whole array (by
calling out to `System.Runtime.Remoting.RemotingServices.SerializeCallData`).
In the example it would make a two element array and try to serialize `r` and
`o` into it. For valuetypes it normally does this by boxing the argument.
However since `R` is a ref-struct, boxing it is not allowed.

This works on .NET Framework.

However for Mono we would need to serilize `R` without boxing it, which seems
challenging with our current setup.

So for now we generate a wrapper that just throws a NotImplementedException if
it is ever called.

This fixes https://github.com/mono/mono/issues/14809 (in that marshalling a
TextWriter across domains works again) but in an unsatisfactory way (because
you can't call any of the ReadOnlySpan<char> overloads on the transparent proxy
object).

* [corlib] Add back Serializable attribute on some reflection classes

Tests that use https://www.nuget.org/packages/NUnit.ApplicationDomain/
need to serialize MethodInfo (and hence MethodBase).

* [corlib] Add test for transparent proxies for classes with methods that take ref struct args

Regression test for https://github.com/mono/mono/issues/14809

* Bump API snapshot submodule
2019-06-07 13:07:13 +02:00
monojenkins fed776b886 [ios] switch cross32/crosswatch to be executed on x86_64 and use newer LLVM (#14856)
Contributes to https://github.com/mono/mono/issues/9621
Fixes https://github.com/mono/mono/issues/14841

Note that it breaks iOS 32bit on devices. The Apple Watch _might_ still works (only tested on `2019-02`).  It unblocks some build related issues on the Xamarin.iOS `2019-06` integration, and we have to do it anyway. I'll come back to it to fix device related issues.
2019-06-07 00:01:18 +02:00
Bernhard Urban 23b831032a [mach] cast objc_msgSend{,Super} usages (#14836)
Fixes #14792

(cherry picked from commit 812357ec51)
2019-06-06 20:16:03 +02:00
monojenkins 2efb7058f5 [2019-06] [llvm] emit call to mini_llvmonly_throw_nullref_exception without exceptions in mind (#14846)
Fixes https://github.com/mono/mono/issues/14243
2019-06-06 19:24:26 +02:00
monojenkins efddb9595c [2019-06] [arm64] Set a llvm target layout. Assert in llvm if the target layout is not set, the defaults used by 'opt' are not correct. (#14844)
It is important to provide this, otherwise the `opt` command will make modifications of every GC safepoint load. For the SmartHotel360 Android app, this can result in `opt` taking 19 minutes and 20 seconds to finish the optimization of the Mono.Android.dll AOT modules.

### IR Example: 

Before Opt:
```
   >   %GC_SAFE_POINT_FLAG_7 = load i64*, i64** getelementptr inbounds ([166068 x i64*], [166068 x i64*]* @mono_aot_Mono_Android_llvm_got, i32 0, i32 7), !invariant.load !139597
    >   %4 = load volatile i64, i64* %GC_SAFE_POINT_FLAG_7
    >   %5 = icmp eq i64 %4, 0
    >   br i1 %5, label %CONT_BB2, label %POLL_BB1
```

After Opt:
```
    <   %GC_SAFE_POINT_FLAG_7 = load i64*, i64** getelementptr inbounds ([166068 x i64*], [166068 x i64*]* @mono_aot_Mono_Android_llvm_got, i64 0, i64 7), align 8, !invariant.load !139597
    <   %3 = load volatile i64, i64* %GC_SAFE_POINT_FLAG_7, align 4
    <   %4 = icmp eq i64 %3, 0
    <   br i1 %4, label %BB2, label %POLL_BB1
```

This has a huge impact on build time, giving InstCombine a huge slowdown. This change removes the need for opt to rewrite all of these basic blocks. 

Backport of #14714.
2019-06-06 18:48:49 +02:00
monojenkins c84d913f41 [2019-06] [sdks] Use 7z format for the android/ios/mac archives (#14838)
It has way better compression performance.
Kept WebAssembly on .zip for now since external users took dependencies on it [1].

Fixes https://github.com/mono/mono/issues/14215

[1] https://github.com/nventive/Uno.Wasm.Bootstrap/blob/e2bf8946e162642ca1bdf4f003f8de4a94954f73/Readme.md#updating-the-mono-sdk-build
2019-06-06 12:26:52 +02:00
Martin Baulig a59e941662 Bump CoreFX to pickup the fix for #14214.
HttpClient.SendAsync() should not attempt to read response body on a HEAD request.

* CoreFX PR https://github.com/dotnet/corefx/pull/38129.

* Backported in https://github.com/mono/corefx/pull/297.

* Fixes https://github.com/mono/mono/issues/14214.
2019-06-06 09:33:30 +02:00
Ankit Jain b33f230a80 Bump msbuild to track xplat-master (#14816) 2019-06-06 00:51:10 +02:00
monojenkins a2ffc3d3d5 [2019-06] Fix build on macOS 10.15 Catalina + xcode 11 (#14824)
1) CC_MD5_Init, CC_MD5_Update and CC_MD5_Final are deprecated.

```c
mono-md5.c:45:2: error: 'CC_MD5_Init' is deprecated: first deprecated in macOS 10.15 - This function is
      cryptographically broken and should not be used in security contexts. Clients should migrate to SHA256
      (or stronger). [-Werror,-Wdeprecated-declarations]
        CC_MD5_Init (ctx);
        ^
```

2) objc_msgSend is deprecated
```
mono-threads-mach-helper.c:95:17: error: too many arguments to function call, expected 0, have 2
                objc_msgSend (value, release);
                ~~~~~~~~~~~~  ^~~~~~~~~~~~~~
```
XCode has a special switch `Enable Strict Checking of objc_msgSend Calls` which is now `true` by default. It's fixed by `-DOBJC_OLD_DISPATCH_PROTOTYPES=1` flag to clang or the corresponding define in the source file.

3) Fixes https://github.com/mono/mono/issues/14793

It worked on pre 10.15 without `-framework GSS`, because it got pulled in through the dependencies. You can see it by running DYLD_PRINT_LIBRARIES=1 csharp and then trace it back. On Mojave /System/Library/Frameworks/GSS.framework/Versions/A/GSS got loaded as dependency of /usr/lib/libcups.2.dylib and so on. Unlike Windows, macOS will resolve the symbols to whatever is already available so it was just happy that someone loaded GSS before libmono-native[-compat/-unified].
2019-06-06 00:47:14 +02:00
monojenkins 8cfcd89a7a [2019-06] Reflect recent NS2.1 changes (#14822)
Due to few recent updates to dotnet/standard (NETStandard 2.1) we had 40 apicompat issues.
Mostly copy-pasted from dotnet/corefx.
Didn't implement Cryptography stuff yet.
Checked all profiles.

Implements #14609

Backport of #14679.
2019-06-05 21:47:53 +02:00
Alexander Köplinger f1fda814c2 [tests] Exclude more nunit categories on ios/mac (#14807)
A few more that we missed adding to the exclude file in the sdks archive.

(cherry picked from commit 3d9513efd4)
2019-06-05 16:01:44 +02:00
Aleksey Kliger 8ec87bbaec [runtime] Autoconf check for pthread_mutexattr_setprotocol
Android doesn't always declare that function.
2019-06-05 09:55:46 +02:00
monojenkins 1e6cd29e60 [2019-06] [Mono.Profiler.Log] Add profiled data reader APIs (#14763)
* [Mono.Profiler.Log] Add profiled data reader APIs

* Update API

* [csproj] Update project files
2019-06-05 01:24:46 +02:00
monojenkins 829949df54 [2019-06] [arm64] Correct exception filter stack frame size (#14770)
[2019-06] [arm64] Correct exception filter stack frame size

Fixes: https://github.com/mono/mono/issues/14170
Fixes: https://github.com/xamarin/xamarin-android/issues/3112

The `call_filter()` function generated by `mono_arch_get_call_filter()`
was overwriting a part of the previous stack frame because it was not
creating a large enough new stack frame for itself.  This had been
working by luck before the switch to building the runtime with Android
NDK r19.

I used a local build of [xamarin/xamarin-android/d16-1@87a80b][0] to
verify that this change resolved both of the issues linked above.  I
also confirmed that I was able to reintroduce the issues in my local
environment by removing the change and rebuilding.

After this change, the generated `call_filter()` function now starts by
reserving sufficient space on the stack to hold a 64-bit value at the
`ctx_offset` location.  The 64-bit `ctx` value is saved to the
`ctx_offset` location (offset 344 in the new stack frame) shortly
afterwards:

    stp	x29, x30, [sp,#-352]!
    mov	x29, sp
    str	x0, [x29,#344]

As expected, the invocation of `call_filter()` now no longer modifies
the top of the previous stack frame.

Top of the stack before `call_filter()`:

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

Top of the stack after `call_filter()` (unchanged):

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

<details>
<summary>Additional background information</summary>

### The original lucky, "good" behavior for [mono/mono/2018-08@725ba2a built with Android NDK r14][1]

 1. The `resume` parameter for `mono_handle_exception_internal()` is
    held in register `w23`.

 2. That register is saved into the current stack frame at offset 20 by
    a `str` instruction:

           0x00000000000bc1bc <+3012>:	str	w23, [sp,#20]

 3. `handle_exception_first_pass()` invokes `call_filter()` via a `blr`
    instruction:

        2279							filtered = call_filter (ctx, ei->data.filter);
           0x00000000000bc60c <+4116>:	add	x9, x22, x24, lsl #6
           0x00000000000bc610 <+4120>:	ldr	x8, [x8,#3120]
           0x00000000000bc614 <+4124>:	ldr	x1, [x9,#112]
           0x00000000000bc618 <+4128>:	add	x0, sp, #0x110
           0x00000000000bc61c <+4132>:	blr	x8

    Before the `blr` instruction, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x0a8b31d8	0x00000071
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

 4. `call_filter()` runs.  This function is generated by
    `mono_arch_get_call_filter()` and starts with:

           stp	x29, x30, [sp,#-336]!
           mov	x29, sp
           str	x0, [x29,#344]

    Note in particular how the first line subtracts 336 from `sp` to
    start a new stack frame, but the third line writes to a position
    that is 344 bytes beyond that, back within the *previous* stack
    frame.

 5. After the invocations of `call_filter()` and
    `handle_exception_first_pass()`, `w23` is restored from the stack:

           0x00000000000bc820 <+4648>:	ldr	w23, [sp,#20]

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

    Notice how `call_filter()` has overwritten bytes 8 through 15 of the
    stack frame.  In this original lucky scenario, this does not affect
    the value restored into register `w23` because that value starts at
    byte 20.

 6. `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000bb960 <+872>:	cbz	w23, 0xbb9c4 <mono_handle_exception_internal+972>

    Since `w23` is `0`, `mono_handle_exception_internal()` correctly
    continues into the `else` branch.

### The bad behavior for [mono/mono/2018-08@725ba2a built with Android NDK r19][2] works slightly differently

 1. As before, the local `resume` parameter starts in register `w23`.
 2. This time, the register is saved into the stack frame at offset 12
    (instead of 20):

           0x00000000000bed7c <+3200>:	str	w23, [sp,#12]

 3. As before, `handle_exception_first_pass()` invokes `call_filter()`
    via a `blr` instruction.

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

 4. `call_filter()` runs as before.  And the first few instructions of
    that function are the same as before:

        stp	x29, x30, [sp,#-336]!
        mov	x29, sp
        str	x0, [x29,#344]

 5. `w23` is again restored from the stack, but this time from offset 12:

           0x00000000000bf2c0 <+4548>:	ldr	w23, [sp,#12]

    The problem is that `call_filter()` has again overwritten bytes 8
    through 15 of the stack frame:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

    So after the `ldr` instruction, `w23` has an incorrect value of
    `0x7f` rather than the correct value `0`.

 6. As before, `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000be430 <+820>:	cbz	w23, 0xbe834 <mono_handle_exception_internal+1848>

    But this time because `w23` is *not* zero, execution continues
    *incorrectly* into the `if` branch rather than the `else` branch.
    The result is that `ji` is set to `0` rather than a valid
    `(MonoJitInfo *)` value.  This incorrect value for `ji` leads to a
    null pointer dereference, as seen in the bug reports.

[0]: https://github.com/xamarin/xamarin-android/tree/xamarin-android-9.3.0.22
[1]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1432/Azure/
[2]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1436/Azure/

</details>

Backport of #14757.

/cc @lambdageek @brendanzagaeski
2019-06-03 20:05:59 -04:00
Katelyn Gadd c28e96bda1 Bump roslyn to release 3.1.0 (#14775) 2019-06-03 22:37:04 +02:00
monojenkins 989433530a [sdks] Bump Android NDK, build tools and platform tools versions (#14758)
Try to match Xamarin.Android master

Build tools and Platform tools: https://github.com/xamarin/xamarin-android/blob/f9f5b6cd120484b13df74e94638e07411d8693fa/Configuration.props#L100-L102

NDK: https://github.com/xamarin/xamarin-android/blob/f9f5b6cd120484b13df74e94638e07411d8693fa/build-tools/scripts/Ndk.targets#L4
2019-06-03 12:50:33 +02:00
Alexis Christoforides 03bdf62922 [MacSDK] Properly handle PATHs with spaces during package post-install 2019-06-03 08:58:36 +02:00
Jo Shields da1fa28109 Bump version to 6.4.0 (#14727) 2019-05-31 12:53:23 -04:00
3566 changed files with 171471 additions and 194532 deletions

No files matched your search

+1 -7
View File
@@ -1,6 +1,3 @@
# diff settings
*.cs diff=csharp
# ensure LF endings on all checkouts
configure.ac crlf=input
config.rpath crlf=input
@@ -10,7 +7,6 @@ mkinstalldirs crlf=input
*.sources crlf=input
.gitattributes crlf=input
*akefile* crlf=input
m4/mono-output.m4 crlf=input
# ensure native line endings on checkout
*.c crlf
@@ -20,10 +16,8 @@ m4/mono-output.m4 crlf=input
*.sln crlf
*.*proj* crlf
*.bat text eol=crlf
*.cmd text eol=crlf
# don't do anything to line-endings. Let CRLFs/LFs go into the repo, and CRLF/LFs on checkout
*.bat -crlf
*.xml -crlf
# CRLF Handling
+50 -60
View File
@@ -15,15 +15,15 @@
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger @kg
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/acceptance-tests/profiler-stress* @alexrp @akoeplinger
/data @directhex @akoeplinger
/data/gdb @vargaz
/data/lldb @vargaz
/data/gdb @lewurm @vargaz
/data/lldb @lewurm @vargaz
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp
/docs/sources/mono-api-profiler.html @BrzVlad
/docs/riscv.md @alexrp @lewurm
/docs/sources/mono-api-profiler.html @alexrp
/external @akoeplinger @marek-safar
/external/api-snapshot @marek-safar @akoeplinger
@@ -31,120 +31,109 @@
/external/bockbuild @akoeplinger @alexischr
/external/boringssl @baulig
/external/cecil @marek-safar @akoeplinger
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/corefx @akoeplinger @marek-safar @egorbo @alexischr
/external/corert @akoeplinger @marek-safar @egorbo @alexischr
/external/linker @marek-safar
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/llvm @alexanderkyte @vargaz
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
/external/roslyn-binaries @kg @marek-safar @akoeplinger
/libgc @BrzVlad @vargaz @naricc
/libgc @BrzVlad @vargaz
/llvm @vargaz @SamMonoRT @imhameed
/llvm @vargaz @alexanderkyte
/man @marek-safar @miguel
/man/*prof* @BrzVlad
/man/*prof* @alexrp
/mcs @marek-safar @akoeplinger @egorbo
/mcs/build @marek-safar @akoeplinger
/mcs/class @steveisok @akoeplinger
/mcs/class @marek-safar @akoeplinger
/mcs/class/corlib/System.Reflection*/ @vargaz @lambdageek
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/Mono.Btls.Interface @baulig @steveisok
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/Mono.Btls.Interface @bauligg @marek-safar
/mcs/class/Mono.Data.Tds @egorbo
/mcs/class/Mono.Debugger.Soft @vargaz @thaystg
/mcs/class/Mono.Options @jonpryor @marek-safar
/mcs/class/Mono.Profiler.Log @lambdageek
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @steveisok
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @marek-safar
/mcs/class/System/Mono.AppleTls @baulig @rolf
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @steveisok
/mcs/class/System/Mono.Security.Interface @baulig @steveisok
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/System.Data* @egorbo @steveisok
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @marek-safar
/mcs/class/System/Mono.Security.Interface @baulig @marek-safar
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/System.Data* @egorbo @marek-safar
/mcs/errors @marek-safar
/mcs/ilasm @marek-safar
/mcs/jay @marek-safar
/mcs/mcs @marek-safar
/mcs/tests @marek-safar
/mcs/errors @marek-safar @lewurm
/mcs/ilasm @marek-safar @lewurm
/mcs/jay @marek-safar @lewurm
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar @lewurm
/mcs/tools @marek-safar @akoeplinger @steveisok
/mcs/tools @marek-safar @akoeplinger
/mcs/tools/gensources @kg @akoeplinger
/mcs/tools/resx2sr @baulig
/mcs/tools/upload-to-sentry @directhex @akoeplinger
/mcs/tools/upload-to-sentry @alexanderkyte @akoeplinger
/mono/arch @vargaz
/mono/arch @vargaz @lewurm
/mono/btls @baulig @marek-safar
/mono/dis @lambdageek @vargaz
/mono/eglib @vargaz @lambdageek
/mono/benchmark @SamMonoRT
/mono/benchmark @alexanderkyte @lewurm
/mono/metadata @vargaz @lambdageek @thaystg
/mono/metadata/*profiler* @lambdageek
/mono/metadata/handle* @lambdageek @vargaz
/mono/metadata/monitor* @brzvlad @vargaz
/mono/metadata/sgen* @brzvlad @vargaz @naricc
/mono/metadata/sgen* @brzvlad @vargaz
/mono/metadata/thread* @lateralusX @lambdageek
/mono/metadata/w32* @lateralusX @lambdageek
/mono/metadata/*-win* @lateralusX @lambdageek
/mono/metadata/appdomain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/assembly* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/domain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/loader* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @vargaz
/mono/mini/*exceptions* @vargaz @BrzVlad
/mono/mini @vargaz @lewurm @lambdageek
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @BrzVlad
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @alexrp @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas
/mono/native @baulig @egorbo @alexischr @marek-safar
/mono/profiler @BrzVlad @lambdageek
/mono/profiler @alexrp @lambdageek
/mono/sgen @BrzVlad @lambdageek
/mono/tools/offsets-tool @rolfbjarne
/mono/utils @vargaz @lambdageek
/mono/utils/atomic* @vargaz
/mono/utils/mono-hwcap* @vargaz
/mono/utils/mono-mem* @vargaz
/mono/utils/mono-merp* @alexischr @lambdageek
/mono/utils/mono-state* @alexischr @lambdageek
/mono/utils @vargaz @lewurm @lambdageek
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
/mono/utils/mono-mem* @lewurm @vargaz
/mono/utils/mono-threads* @lambdageek @vargaz
/mono/utils/*-win* @lateralusX @lambdageek
/msvc @lateralusX @kg @akoeplinger
/msvc/scripts @akoeplinger @kg
/netcore @marek-safar @akoeplinger @egorbo @vargaz @steveisok
/netcore @marek-safar @akoeplinger @egorbo
/packaging @alexischr @akoeplinger @directhex
/packaging/MacSDK/msbuild.py @radical @akoeplinger
/runtime @akoeplinger @marek-safar
/samples @lambdageek
/samples/profiler @BrzVlad @lambdageek
/samples @lambdageek @lewurm
/samples/profiler @alexrp @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
/scripts/ci @akoeplinger @directhex
/sdks @marek-safar @vargaz @lambdageek
/sdks @marek-safar @lewurm @vargaz
/sdks/builds/android* @alexischr @jonpryor
/sdks/builds/mxe* @alexischr @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
@@ -158,5 +147,6 @@
/tools @lambdageek @vargaz
/tools/locale-builder @marek-safar @egorbo
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/pedump @lambdageek
/tools/sgen @brzvlad @vargaz @naricc
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/sgen @brzvlad @vargaz
+1 -1
View File
@@ -43,5 +43,5 @@ Please paste the stack trace here if available.
```
<!--
You can join us on Discord (https://aka.ms/dotnet-discord) in the #monovm channel to discuss your reported issue
You can join us at https://gitter.im/mono/mono to discuss your reported issue
-->
+2 -3
View File
@@ -18,7 +18,6 @@
*.vssscc
*_i.c
*_p.c
*.mdb
*.ncb
*.suo
*.tlb
@@ -139,7 +138,7 @@ llvm/build/
llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
##############################################################################
# Arcade output directories
@@ -147,4 +146,4 @@ mcs/build/compiler-server.log
.packages
artifacts
.dotnet
.dotnet
+4 -7
View File
@@ -58,16 +58,13 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/llvm"]
path = external/llvm
url = git://github.com/mono/llvm.git
branch = release_60
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+1 -1
View File
@@ -76,7 +76,7 @@ The Licenses
These are the licenses used in Mono, the files are located:
### MIT License
### MIT X11 License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
+50 -15
View File
@@ -2,11 +2,32 @@ ACLOCAL_AMFLAGS = -I m4
AM_CFLAGS = $(WERROR_CFLAGS)
SUBDIRS = @MONO_SUBDIRS@
noinst_SUBDIRS = @MONO_NOINST_SUBDIRS@
DIST_SUBDIRS = $(SUBDIRS) m4 netcore
MONOTOUCH_SUBDIRS = $(libgc_dir) mono
if !ENABLE_NETCORE
# Some tools might not build when cross-compiling
if CROSS_COMPILING
tools_dir =
else
tools_dir = tools
endif
if ENABLE_MSVC
build_with_msvc = msvc
build_without_msvc =
else
build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
else
update_submodules = update_submodules
endif
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore
all: $(update_submodules)
@@ -24,10 +45,9 @@ EXTRA_DIST= \
winconfig.h \
code_of_conduct.md \
external \
netcore/Makefile \
mcs/class/referencesource
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_RUNTIME=false
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_MCS=false EXTERNAL_RUNTIME=false
# Distribute the 'mcs' tree too
GIT_DIR ?= $(srcdir)/.git
@@ -45,9 +65,6 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
@@ -94,10 +111,30 @@ mcs-do-compiler-tests:
bootstrap-world: compiler-tests
$(MAKE) install
install:
for mydir in $(filter-out $(noinst_SUBDIRS),$(SUBDIRS)); do \
(cd $${mydir} && ${MAKE} install) \
done
if INSTALL_MONOTOUCH
monotouch-do-build: config.h
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-build";; \
* ) target="all";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-build)
monotouch-do-clean:
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-clean";; \
* ) target="clean";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-clean)
endif
update-csproj:
-rm msvc/scripts/order
@@ -141,5 +178,3 @@ update-solution-files:
update-solution-files-with-tests:
$(MAKE) "GENPROJ_ARGS=2012 true true" update-solution-files
endif
+1 -1
View File
@@ -2,7 +2,7 @@
<configuration>
<packageSources>
<clear />
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
+7 -11
View File
@@ -5,13 +5,7 @@ standards for C# and the Common Language Runtime.
The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.org/)
Join us on [Discord](https://aka.ms/dotnet-discord) in the `#monovm` channel:
<a href="https://aka.ms/dotnet-discord">
<img src="https://img.shields.io/discord/732297728826277939?style=flat-square&label=Discord&logo=discord&logoColor=white&color=7289DA">
</a>
### Contents
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/mono/mono?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
1. [Compilation and Installation](#compilation-and-installation)
2. [Using Mono](#using-mono)
@@ -36,8 +30,7 @@ Join us on [Discord](https://aka.ms/dotnet-discord) in the `#monovm` channel:
| Windows | i386 | [![windows-i386][17]][18] |
| CentOS | s390x (cs) | [![centos-s390x][19]][20] |
| Debian 9 | ppc64el (cs) | [![debian-9-ppc64el][21]][22]|
| AIX 6.1 | ppc64 (cs) | [![aix-ppc64][23]][24] |
| FreeBSD 12 | amd64 (cs) | [![freebsd-amd64][25]][26] |
| AIX | ppc64 (cs) | [![aix-ppc64][23]][24] |
_(cs) = community supported architecture_
@@ -65,8 +58,6 @@ _(cs) = community supported architecture_
[22]: https://jenkins.mono-project.com/job/test-mono-mainline-community-chroot/label=debian-9-ppc64el
[23]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64/badge/icon
[24]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64
[25]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64/badge/icon
[26]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64
Compilation and Installation
============================
@@ -440,6 +431,7 @@ Disables compilation for the SSA optimization
framework, and the various SSA-based optimizations.
* `--enable-llvm`
* `--enable-loadedllvm`
* This enables the use of LLVM as a code generation engine
for Mono. The LLVM code generator and optimizer will be
@@ -452,6 +444,10 @@ full details and up-to-date information on this feature.
* You will need to have an LLVM built that Mono can link
against.
* The `--enable-loadedllvm` variant will make the LLVM backend
into a runtime-loadable module instead of linking it directly
into the main mono binary.
* `--enable-big-arrays` - Enable use of arrays with indexes larger
than Int32.MaxValue.
+5 -172
View File
@@ -1,185 +1,18 @@
Mono uses third-party libraries or other resources that may be
distributed under licenses different than the Mono software.
distributed under licenses different than the .NET Core software.
Attributions and license notices for test cases originally authored by
Attributions and license notices for test cases originally authored by
third parties can be found in the respective test directories.
In the event that we accidentally failed to list a required notice, please
In the event that we accidentally failed to list a required notice, please
bring it to our attention. Post an issue or email us:
[email protected]
The attached notices are provided for information only.
License notice for Mono
-----------------------
Note: multiple license notices are provided that occure in source code.
They are equivalent forms of the the MIT license from various copyright holders.
// Dual licensed under the terms of the MIT X11 or GNU GPL
// Copyright 2003-2011 Novell, Inc.
// Copyright 2011 Xamarin Inc
/**
*
* Copyright 2001-2003 Ximian, Inc
* Copyright 2003-2010 Novell, Inc.
* Copyright (C) 2012 Xamarin Inc
*
* Licensed under the MIT license. See LICENSE file in the project root for full license information.
*/
/******************************************************************************
* The MIT License
* Copyright (c) 2003 Novell Inc. www.novell.com
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the Software), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*******************************************************************************/
License notice for Apple
License notice for Apple
-------------------------
Content: https://github.com/mono/mono/blob/435f968102d0820ca65d63e1e741e4130b4d1303/support/ios/net/route.h
License: https://opensource.apple.com/apsl/
License for GCC
---------------
License: http://gcc.gnu.org/onlinedocs/gcc-4.6.4/libstdc++/manual/manual/license.html
Content: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini
Example: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini/decompose.c#L967
GCC RUNTIME LIBRARY EXCEPTION
Version 3.1, 31 March 2009
Copyright (C) 2009 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
This GCC Runtime Library Exception ("Exception") is an additional
permission under section 7 of the GNU General Public License, version
3 ("GPLv3"). It applies to a given file (the "Runtime Library") that
bears a notice placed by the copyright holder of the file stating that
the file is governed by GPLv3 along with this Exception.
When you use GCC to compile a program, GCC may combine portions of
certain GCC header files and runtime libraries with the compiled
program. The purpose of this Exception is to allow compilation of
non-GPL (including proprietary) programs to use, in this way, the
header files and runtime libraries covered by this Exception.
0. Definitions.
A file is an "Independent Module" if it either requires the Runtime
Library for execution after a Compilation Process, or makes use of an
interface provided by the Runtime Library, but is not otherwise based
on the Runtime Library.
"GCC" means a version of the GNU Compiler Collection, with or without
modifications, governed by version 3 (or a specified later version) of
the GNU General Public License (GPL) with the option of using any
subsequent versions published by the FSF.
"GPL-compatible Software" is software whose conditions of propagation,
modification and use would permit combination with GCC in accord with
the license of GCC.
"Target Code" refers to output from any compiler for a real or virtual
target processor architecture, in executable form or suitable for
input to an assembler, loader, linker and/or execution
phase. Notwithstanding that, Target Code does not include data in any
format that is used as a compiler intermediate representation, or used
for producing a compiler intermediate representation.
The "Compilation Process" transforms code entirely represented in
non-intermediate languages designed for human-written code, and/or in
Java Virtual Machine byte code, into Target Code. Thus, for example,
use of source code generators and preprocessors need not be considered
part of the Compilation Process, since the Compilation Process can be
understood as starting with the output of the generators or
preprocessors.
A Compilation Process is "Eligible" if it is done using GCC, alone or
with other GPL-compatible software, or if it is done without using any
work based on GCC. For example, using non-GPL-compatible Software to
optimize any GCC intermediate representations would not qualify as an
Eligible Compilation Process.
1. Grant of Additional Permission.
You have permission to propagate a work of Target Code formed by
combining the Runtime Library with Independent Modules, even if such
propagation would otherwise violate the terms of GPLv3, provided that
all Target Code was generated by Eligible Compilation Processes. You
may then convey such a combination under terms of your choice,
consistent with the licensing of the Independent Modules.
2. No Weakening of GCC Copyleft.
The availability of this Exception does not imply any general
presumption that third-party software is unaffected by the copyleft
requirements of the license of GCC.
Hopefully that text is self-explanatory. If it isn't, you need to speak to your lawyer, or the Free Software Foundation.
License notice for Silicon Graphics and HP
------------------------------------------
/*
* Copyright (c) 1996-1997
* Silicon Graphics Computer Systems, Inc.
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Silicon Graphics makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*
* Copyright (c) 2002
* Hewlett-Packard Company
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Hewlett-Packard Company makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*/
/*
* This implements standard-conforming allocators that interact with
* the garbage collector. Gc_alloctor<T> allocates garbage-collectable
* objects of type T. Traceable_allocator<T> allocates objects that
* are not temselves garbage collected, but are scanned by the
* collector for pointers to collectable objects. Traceable_alloc
* should be used for explicitly managed STL containers that may
* point to collectable objects.
*
* This code was derived from an earlier version of the GNU C++ standard
* library, which itself was derived from the SGI STL implementation.
*/
Content: https://github.com/mono/mono/blob/d58dfa185af1d48fb93efdef1d046762f214f0a7/libgc/include/gc_allocator.h
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"rev": "13a40eedacc799a78710aba5c8e4b2dcc80a7e61",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+9 -30
View File
@@ -104,10 +104,12 @@ if grep "^AM_PROG_LIBTOOL" configure.ac >/dev/null; then
fi
fi
# Parse parameters
#
# Plug in the extension module
#
has_ext_mod=false
ext_mod_args=''
has_disable_boehm=false
for PARAM; do
if [[ $PARAM =~ "--enable-extension-module" ]] ; then
has_ext_mod=true
@@ -115,14 +117,8 @@ for PARAM; do
ext_mod_args=`echo $PARAM | cut -d= -f2`
fi
fi
if [[ $PARAM =~ "--disable-boehm" ]] ; then
has_disable_boehm=true
fi
done
#
# Plug in the extension module
#
if test x$has_ext_mod = xtrue; then
pushd $top_srcdir../mono-extensions/scripts
sh ./prepare-repo.sh $ext_mod_args || exit 1
@@ -154,34 +150,17 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
# Update all submodules recursively to ensure everything is checked out
if test -e $srcdir/scripts/update_submodules.sh; then
(cd $srcdir && scripts/update_submodules.sh)
fi
if test x$has_disable_boehm = xfalse -a -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
echo "MONO_EXTRA_CONFIGURE_FLAGS is $MONO_EXTRA_CONFIGURE_FLAGS"
fi
host_conf_flag=
build_uname_all=`(uname -a) 2>/dev/null`
case "$build_uname_all" in
CYGWIN*)
if [[ "$@" != *"--host="* ]]; then
echo "Missing --host parameter, configure using ./configure --host=i686-w64-mingw32 or --host=x86_64-w64-mingw32"
echo "Falling back using --host=x86_64-w64-mingw32 as default."
host_conf_flag="--host=x86_64-w64-mingw32"
fi
;;
esac
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings $host_conf_flag" #--enable-iso-c
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings" #--enable-iso-c
if test x$NOCONFIGURE = x; then
echo Running $srcdir/configure $conf_flags "$@" ...
+86 -94
View File
@@ -574,8 +574,6 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ilasm", "mcs/ilasm/ilasm.cs
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "al", "mcs/tools/al/al.csproj", "{0A20671F-CCA9-457D-886C-0828C7D64128}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "aprofutil", "mcs/tools/aprofutil/aprofutil.csproj", "{658B566B-CED1-4C42-9926-C1AA930BA368}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "monolinker", "mcs/tools/linker/monolinker.csproj", "{D60432F1-3C12-4235-A1CC-4CA5575403BD}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "culevel", "mcs/tools/culevel/culevel.csproj", "{E8E246BD-CD0C-4734-A3C2-7F44796EC47B}"
@@ -1536,8 +1534,10 @@ Global
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|unreal.Build.0 = Debug|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.ActiveCfg = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.Build.0 = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.Build.0 = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.Build.0 = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.ActiveCfg = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.Build.0 = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|winaot.ActiveCfg = Release|winaot
@@ -1662,8 +1662,10 @@ Global
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|unreal.Build.0 = Debug|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.ActiveCfg = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.Build.0 = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.Build.0 = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.Build.0 = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.ActiveCfg = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.Build.0 = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|winaot.ActiveCfg = Release|winaot
@@ -2210,8 +2212,10 @@ Global
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|unreal.Build.0 = Debug|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.ActiveCfg = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.Build.0 = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.Build.0 = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.Build.0 = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.ActiveCfg = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.Build.0 = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|winaot.ActiveCfg = Release|winaot
@@ -2270,8 +2274,10 @@ Global
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|unreal.Build.0 = Debug|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.ActiveCfg = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.Build.0 = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.Build.0 = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.Build.0 = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.ActiveCfg = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.Build.0 = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|winaot.ActiveCfg = Release|winaot
@@ -2374,8 +2380,10 @@ Global
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|unreal.Build.0 = Debug|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.ActiveCfg = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.Build.0 = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.Build.0 = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.Build.0 = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.ActiveCfg = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.Build.0 = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|winaot.ActiveCfg = Release|winaot
@@ -2566,8 +2574,10 @@ Global
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|unreal.Build.0 = Debug|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.ActiveCfg = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.Build.0 = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.Build.0 = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.Build.0 = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.ActiveCfg = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.Build.0 = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|winaot.ActiveCfg = Release|winaot
@@ -3632,8 +3642,10 @@ Global
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|unreal.Build.0 = Debug|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.ActiveCfg = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.Build.0 = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.Build.0 = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.Build.0 = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.ActiveCfg = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.Build.0 = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|winaot.ActiveCfg = Release|winaot
@@ -3956,8 +3968,10 @@ Global
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|unreal.Build.0 = Debug|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.ActiveCfg = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.Build.0 = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.Build.0 = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.Build.0 = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.ActiveCfg = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.Build.0 = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|winaot.ActiveCfg = Release|winaot
@@ -4014,42 +4028,24 @@ Global
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|net_4_x.Build.0 = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.Build.0 = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.Build.0 = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.Build.0 = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.Build.0 = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.Build.0 = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.Build.0 = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.Build.0 = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.Build.0 = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.Build.0 = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.Build.0 = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.Build.0 = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.Build.0 = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.Build.0 = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.Build.0 = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.Build.0 = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.Build.0 = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.Build.0 = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.Build.0 = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.Build.0 = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.Build.0 = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|net_4_x.ActiveCfg = Release|net_4_x
@@ -4078,8 +4074,10 @@ Global
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|unreal.Build.0 = Debug|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.ActiveCfg = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.Build.0 = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.Build.0 = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.Build.0 = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.ActiveCfg = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.Build.0 = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|winaot.ActiveCfg = Release|winaot
@@ -4200,8 +4198,10 @@ Global
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|unreal.Build.0 = Debug|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.ActiveCfg = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.Build.0 = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.Build.0 = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.Build.0 = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.ActiveCfg = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.Build.0 = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4238,8 +4238,10 @@ Global
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|unreal.Build.0 = Debug|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.ActiveCfg = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.Build.0 = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.Build.0 = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.Build.0 = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.ActiveCfg = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.Build.0 = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4496,8 +4498,10 @@ Global
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|unreal.Build.0 = Debug|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.ActiveCfg = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.Build.0 = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.Build.0 = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.Build.0 = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.ActiveCfg = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.Build.0 = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|winaot.ActiveCfg = Release|winaot
@@ -6012,8 +6016,10 @@ Global
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|unreal.Build.0 = Debug|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.ActiveCfg = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.Build.0 = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.Build.0 = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.Build.0 = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.ActiveCfg = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.Build.0 = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|winaot.ActiveCfg = Release|winaot
@@ -6050,8 +6056,10 @@ Global
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|unreal.Build.0 = Debug|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.ActiveCfg = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.Build.0 = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.Build.0 = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.Build.0 = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.ActiveCfg = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.Build.0 = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|winaot.ActiveCfg = Release|winaot
@@ -6528,8 +6536,10 @@ Global
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|unreal.Build.0 = Debug|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.ActiveCfg = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.Build.0 = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.Build.0 = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.Build.0 = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.ActiveCfg = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.Build.0 = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|winaot.ActiveCfg = Release|winaot
@@ -6566,8 +6576,10 @@ Global
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|unreal.Build.0 = Debug|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.ActiveCfg = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.Build.0 = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.Build.0 = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.Build.0 = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.ActiveCfg = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.Build.0 = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|winaot.ActiveCfg = Release|winaot
@@ -9844,8 +9856,10 @@ Global
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|unreal.Build.0 = Debug|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.ActiveCfg = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.Build.0 = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.Build.0 = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.Build.0 = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.ActiveCfg = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.Build.0 = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|winaot.ActiveCfg = Release|winaot
@@ -10120,28 +10134,6 @@ Global
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|winaot.ActiveCfg = Release|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Debug|xammac.ActiveCfg = Debug|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|xammac.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.Build.0 = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.Build.0 = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monodroid.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|orbis.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|orbis.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|unreal.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|unreal.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|wasm.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|wasm.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|winaot.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|winaot.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|xammac.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|xammac.ActiveCfg = Release|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.Build.0 = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Release|net_4_x.ActiveCfg = Release|net_4_x
+48 -4
View File
@@ -1,6 +1,50 @@
# Code of Conduct
# Contributor Code of Conduct
This project has adopted the code of conduct defined by the Contributor Covenant
to clarify expected behavior in our community.
As contributors and maintainers of this project, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating
documentation, submitting pull requests or patches, and other activities.
For more information, see the [.NET Foundation Code of Conduct](https://dotnetfoundation.org/code-of-conduct).
We are committed to making participation in this project a harassment-free
experience for everyone, regardless of level of experience, gender, gender
identity and expression, sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, or nationality.
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing other's private information, such as physical or electronic
addresses, without explicit permission
* Other unethical or unprofessional conduct
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
By adopting this Code of Conduct, project maintainers commit themselves to
fairly and consistently applying these principles to every aspect of managing
this project. Project maintainers who do not follow or enforce the Code of
Conduct may be permanently removed from the project team.
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting a project maintainer at miguel@xamarin.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. Maintainers are
obligated to maintain confidentiality with regard to the reporter of an
incident.
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 1.3.0, available at
[http://contributor-covenant.org/version/1/3/0/][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/3/0/
+479 -942
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -6,7 +6,7 @@
<dllmap dll="libintl" name="bind_textdomain_codeset" target="@LIBC@" os="solaris"/>
<dllmap dll="libintl" target="@INTL@" os="!windows"/>
<dllmap dll="i:libxslt.dll" target="libxslt@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="@ODBC@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libodbc@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libiodbc.dylib" os="osx"/>
<dllmap dll="oci" target="libclntsh@libsuffix@" os="!windows"/>
<dllmap dll="db2cli" target="libdb2_36@libsuffix@" os="!windows"/>
+13 -13
View File
@@ -1,9 +1,8 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
print('%c thread #%i' % ('*' if current_thread else ' ', thread.idx))
print '%c thread #%i' % ('*' if current_thread else ' ', thread.idx)
if current_thread:
selected_frame = thread.GetSelectedFrame()
@@ -15,16 +14,17 @@ def print_frames(thread, num_frames, current_thread):
if function_name == "interp_exec_method_full":
try:
s = 'frame->imethod->method'
methoddesc = frame.EvaluateExpression('(char*) mono_method_full_name (' + s + ', 1)').summary[1:-1]
klassname = frame.EvaluateExpression('(char*) ' + s + '->klass->name').summary[1:-1]
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
ipoffset -= frame.EvaluateExpression('imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
var = '%s::%s @ %d %s || %s' % (klassname, methodname, ipoffset, insn, frame)
except Exception as e:
traceback.print_exc()
print "DBG: execfail:" + str(e)
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -32,7 +32,7 @@ def print_frames(thread, num_frames, current_thread):
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
var = 'transforming %s::%s || %s' % (klassname, methodname, frame)
except Exception as e:
print("DBG: transformfail:" + str(e))
print "DBG: transformfail:" + str(e)
elif pc[0] == '0':
try:
framestr = frame.EvaluateExpression('(char*)mono_pmip((void*)%s)' % pc).summary[1:-1]
@@ -40,7 +40,7 @@ def print_frames(thread, num_frames, current_thread):
except:
pass
print(' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var))
print ' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var)
def monobt(debugger, command, result, dict):
opts = {'all_bt': False, 'num_frames': None}
@@ -50,20 +50,20 @@ def monobt(debugger, command, result, dict):
elif command.isdigit():
opts['num_frames'] = int(command)
elif command != '':
print('error: monobt [<number>|all]')
print 'error: monobt [<number>|all]'
return
target = debugger.GetSelectedTarget()
process = target.process
if not process.IsValid():
print('error: invalid process')
print 'error: invalid process'
return
if opts['all_bt']:
for thread in process.threads:
print_frames(thread, len(thread), process.selected_thread == thread)
print('')
print ''
else:
thread = process.selected_thread
num_frames = len(thread) if opts['num_frames'] is None else opts['num_frames']
@@ -76,4 +76,4 @@ def __lldb_init_module (debugger, dict):
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f monobt.monobt monobt')
debugger.HandleCommand('command alias mbt monobt')
print('"monobt" command installed')
print '"monobt" command installed'
@@ -242,7 +242,7 @@ namespace HtmlAgilityPack
}
/// <summary>
/// Checks for existence of attribute with given name
/// Checks for existance of attribute with given name
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
+1 -1
View File
@@ -22,7 +22,7 @@ Layout
as well as the upcoming Monodoc tools and index creation
tools).
The files can contain any amount of extra information.
The files can contain any ammount of extra information.
The documentation from the source code is then merged with
these files and the header/footer information to produce
-23
View File
@@ -18,27 +18,6 @@ convert.exe: $(srcdir)/convert.cs AgilityPack.dll
AgilityPack.dll:
$(CSCOMPILE) -r:$(topdir)/class/lib/$(PROFILE)/mscorlib.dll -r:$(topdir)/class/lib/$(PROFILE)/System.dll -r:$(topdir)/class/lib/$(PROFILE)/System.Xml.dll -target:library -out:$@ $(srcdir)/HtmlAgilityPack/*.cs
ifdef MCS_MODE
monoapi.zip:
touch $@
monoapi.tree: monoapi.zip
touch $@
mono-tools.zip:
touch $@
mono-tools.tree: mono-tools.zip
touch $@
mono-file-formats.zip:
touch $@
mono-file-formats.tree: mono-file-formats.zip
touch $@
else
monoapi.zip: monoapi.tree
@test -f $@ || { rm -f $< && $(MAKE) $<; }
@@ -57,7 +36,5 @@ mono-file-formats.zip: mono-file-formats.tree
mono-file-formats.tree: $(srcdir)/mono-file-formats.config $(srcdir)/docs.make
$(MDOC) assemble -o mono-file-formats -f man $<
endif
.doc-stamp:
+39 -61
View File
@@ -2,58 +2,36 @@
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documentation
## Useful RISC-V documents
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications/)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa/)
* [RISC-V Debug Specification Standard](https://github.com/riscv/riscv-debug-spec/blob/master/riscv-debug-spec.pdf)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual/blob/master/riscv-asm.md)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Linux](https://github.com/riscv/riscv-linux)
* [RISC-V LLD](https://github.com/riscv/riscv-lld)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V QEMU](https://github.com/riscv/riscv-qemu)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
* [lowRISC Organization](https://github.com/lowrisc)
* [RISC-V LLVM](https://github.com/lowrisc/riscv-llvm)
## Setting up a cross environment
### Debian/Ubuntu packages
Setting up a cross environment with a Linux toolchain and QEMU is quite easy.
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
First, add these to your `.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
@@ -61,42 +39,47 @@ export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Next, install some dependencies needed to build the toolchain:
Install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev libusb-1.0-0-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev device-tree-compiler pkg-config libexpat-dev libglib2.0-dev libpixman-1-dev
```
Finally, build the toolchain:
Now you can build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ git clone --recursive [email protected]:riscv/riscv-tools.git
$ cd riscv-tools
$ ./build.sh
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ ./configure --prefix=$RISCV --enable-multilib
$ make linux
$ cd ../..
$ git clone --recursive [email protected]:riscv/riscv-qemu.git
$ cd riscv-qemu
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --disable-werror
$ make
$ make install
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
@@ -106,24 +89,19 @@ Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 ./mono/mini/mono hello.exe
$ qemu-riscv64 $RISCV/sysroot/bin/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
$ qemu-riscv64 -g 12345 ./mono/mini/mono-sgen --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono-sgen
```
## Things to be done
Things that need to be done beyond the basic 64-bit port, in no particular
order:
In no particular order:
* Complete the soft float port.
* Complete the 32-bit port.
-6
View File
@@ -109,12 +109,6 @@ typedef struct {
<h3>Write Barriers</h3>
<p>SGen is a concurrent and generational GC, features which require
tracking changes to the state of the heap. This is achieved through
write barriers. Whenever native code is changing the state of the
heap by storing references into another managed object, it needs to
do it using this write barrier API.
<h4><a name="api:mono_gc_wbarrier_arrayref_copy">mono_gc_wbarrier_arrayref_copy</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_nostore">mono_gc_wbarrier_generic_nostore</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_store">mono_gc_wbarrier_generic_store</a></h4>
+4 -16
View File
@@ -1,27 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>39d545dc14bc4d5a546e5e2f58d1ebaca72fcd61</Sha>
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.20201.2">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>bce0a98620c1c5a110b2bba9912f3d5929069c6b</Sha>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.20201.2">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>bce0a98620c1c5a110b2bba9912f3d5929069c6b</Sha>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
+1 -8
View File
@@ -1,14 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>5.0.0</VersionPrefix>
<VersionPrefix>1.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
</Project>
@@ -1,9 +1,11 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $DotnetSymbolVersion # Version of dotnet symbol to use
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SymbolToolPath # Full path to directory where dotnet symbol-tool was installed
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
@@ -19,39 +21,36 @@ function FirstMatchingSymbolDescriptionOrDefault {
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + '\' + $FileName
$SymbolPath = $SymbolsPath + "\" + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, '.pdb')
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, '.ni.pdb')
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, '.so.dbg')
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, '.dylib.dwarf')
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$dotnetSymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetSymbolExe = Resolve-Path "$dotnetSymbolExe\dotnet-symbol.exe"
& $dotnetSymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return 'PDB'
return "PDB"
}
elseif (Test-Path $NGenPdb) {
return 'NGen PDB'
return "NGen PDB"
}
elseif (Test-Path $SODbg) {
return 'DBG for SO'
return "DBG for SO"
}
elseif (Test-Path $DylibDwarf) {
return 'Dwarf for Dylib'
return "Dwarf for Dylib"
}
elseif (Test-Path $SymbolPath) {
return 'Module'
return "Module"
}
else {
return $null
@@ -65,12 +64,11 @@ function CountMissingSymbols {
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @('.dll', '.exe', '.so', '.dylib')
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
# How many files are missing symbol information
$MissingSymbols = 0
@@ -78,38 +76,41 @@ function CountMissingSymbols {
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath 'Symbols'
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
# Makes easier to reference `symbol tool`
Push-Location $SymbolToolPath
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match '\\ref\\') {
Write-Host "`t Ignoring reference assembly file " $_.FullName
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--microsoft-symbol-server' $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--internal-server' $SymbolsPath
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
Write-Host -NoNewLine "`t Checking file " $_.FullName "... "
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host 'No symbols found on MSDL or SymWeb!'
Write-Host "No symbols found on MSDL or SymWeb!"
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host 'No symbols found on MSDL!'
Write-Host "No symbols found on MSDL!"
}
else {
Write-Host 'No symbols found on SymWeb!'
Write-Host "No symbols found on SymWeb!"
}
}
}
@@ -128,61 +129,30 @@ function CheckSymbolsAvailable {
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match 'Microsoft\.DotNet\.Darc\.') {
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match 'Microsoft\.DotNet\.Maestro\.Tasks\.') {
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
Write-Error "Missing symbols for $Status modules in the package $FileName"
}
Write-Host
}
}
function InstallDotnetSymbol {
$dotnetSymbolPackageName = 'dotnet-symbol'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$dotnetSymbolPackageName*") -and ($toolList -like "*$dotnetSymbolVersion*")) {
Write-Host "dotnet-symbol version $dotnetSymbolVersion is already installed."
}
else {
Write-Host "Installing dotnet-symbol version $dotnetSymbolVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $dotnetSymbolPackageName --version $dotnetSymbolVersion --verbosity "minimal" --global
}
}
try {
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
InstallDotnetSymbol
CheckSymbolsAvailable
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message $_
ExitWithExitCode 1
}
CheckSymbolsAvailable
-11
View File
@@ -1,11 +0,0 @@
@{
IncludeRules=@('PSAvoidUsingCmdletAliases',
'PSAvoidUsingWMICmdlet',
'PSAvoidUsingPositionalParameters',
'PSAvoidUsingInvokeExpression',
'PSUseDeclaredVarsMoreThanAssignments',
'PSUseCmdletCorrectly',
'PSStandardDSCFunctionsInResource',
'PSUseIdenticalMandatoryParametersForDSC',
'PSUseIdenticalParametersForDSC')
}
+83
View File
@@ -0,0 +1,83 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PushToStaticFeed task that will read the build asset
manifest and publish the assets described in the manifest to
informed target feeds.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(MSBuildThisFileDirectory)DefaultVersions.props" Condition="Exists('$(MSBuildThisFileDirectory)DefaultVersions.props')" />
<!--
This won't be necessary once we solve this issue:
https://github.com/dotnet/arcade/issues/2266
-->
<Import Project="$(MSBuildThisFileDirectory)ArtifactsCategory.props" Condition="Exists('$(MSBuildThisFileDirectory)ArtifactsCategory.props')" />
<Import Project="$(NuGetPackageRoot)microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<Target Name="PublishToFeed">
<Error Condition="'$(ArtifactsCategory)' == ''" Text="ArtifactsCategory: The artifacts' category produced by the build wasn't provided." />
<Error Condition="'$(AccountKeyToStaticFeed)' == ''" Text="AccountKeyToStaticFeed: Account key for target feed wasn't provided." />
<Error Condition="'$(ManifestsBasePath)' == ''" Text="Full path to asset manifests directory wasn't provided." />
<Error Condition="'$(BlobBasePath)' == '' AND '$(PackageBasePath)' == ''" Text="A valid full path to BlobBasePath of PackageBasePath is required." />
<ItemGroup>
<!-- Include all manifests found in the manifest folder. -->
<ManifestFiles Include="$(ManifestsBasePath)*.xml" />
</ItemGroup>
<Error Condition="'@(ManifestFiles)' == ''" Text="No manifest file was found in the provided path: $(ManifestsBasePath)" />
<!--
For now the type of packages being published will be informed for the whole build.
Eventually this will be specified on a per package basis:
TODO: https://github.com/dotnet/arcade/issues/2266
-->
<PropertyGroup>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCORE'">https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCOREVALIDATION'">https://dotnetfeed.blob.core.windows.net/arcade-validation/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORETOOLING'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ENTITYFRAMEWORKCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETEXTENSIONS'">https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORECLR'">https://dotnetfeed.blob.core.windows.net/dotnet-coreclr/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORESDK'">https://dotnetfeed.blob.core.windows.net/dotnet-sdk/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSINTERNAL'">https://dotnetfeed.blob.core.windows.net/dotnet-tools-internal/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSET'">https://dotnetfeed.blob.core.windows.net/dotnet-toolset/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'WINDOWSDESKTOP'">https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'NUGETCLIENT'">https://dotnetfeed.blob.core.windows.net/nuget-nugetclient/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETENTITYFRAMEWORK6'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframework6/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETBLAZOR'">https://dotnetfeed.blob.core.windows.net/aspnet-blazor/index.json</TargetStaticFeed>
</PropertyGroup>
<Error
Condition="'$(TargetStaticFeed)' == ''"
Text="'$(ArtifactsCategory)' wasn't recognized as a valid artifact category. Valid categories are: '.NetCore' and '.NetCoreValidation'" />
<!-- Iterate publishing assets from each manifest file. -->
<PushArtifactsInManifestToFeed
ExpectedFeedUrl="$(TargetStaticFeed)"
AccountKey="$(AccountKeyToStaticFeed)"
BARBuildId="$(BARBuildId)"
MaestroApiEndpoint="$(MaestroApiEndpoint)"
BuildAssetRegistryToken="$(BuildAssetRegistryToken)"
Overwrite="$(OverrideAssetsWithSameName)"
PassIfExistingItemIdentical="$(PassIfExistingItemIdentical)"
MaxClients="$(MaxParallelUploads)"
UploadTimeoutInMinutes="$(MaxUploadTimeoutInMinutes)"
AssetManifestPath="%(ManifestFiles.Identity)"
BlobAssetsBasePath="$(BlobBasePath)"
PackageAssetsBasePath="$(PackageBasePath)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
</ItemGroup>
</Project>
+82
View File
@@ -0,0 +1,82 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PublishSymbols tasks to publish symbols to MSDL and SymWeb.
Parameters:
- PDBArtifactsDirectory : Full path to directory containing PDB files to be published.
- BlobBasePath : Full path containing *.symbols.nupkg packages to be published.
- DotNetSymbolServerTokenMsdl : PAT to access MSDL.
- DotNetSymbolServerTokenSymWeb : PAT to access SymWeb.
- DotNetSymbolExpirationInDays : Expiration days for published packages. Default is 3650.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(NuGetPackageRoot)microsoft.symboluploader.build.task\$(SymbolUploaderVersion)\build\PublishSymbols.targets" />
<Target Name="PublishSymbols">
<ItemGroup>
<FilesToPublishToSymbolServer Include="$(PDBArtifactsDirectory)\*.pdb"/>
<PackagesToPublishToSymbolServer Include="$(BlobBasePath)\*.symbols.nupkg"/>
<!--
These packages from Arcade-Services include some native libraries that
our current symbol uploader can't handle. Below is a workaround until
we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
-->
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Darc.*" />
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Maestro.Tasks.*" />
</ItemGroup>
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
<Message
Importance="High"
Text="No symbol package(s) were found to publish."
Condition="$(PublishToSymbolServer) == false" />
<!-- Symbol Uploader: MSDL -->
<Message Importance="High" Text="Publishing symbol packages to MSDL ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenMsdl)"
SymbolServerPath="https://microsoftpublicsymbols.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
<!--
Symbol Uploader: SymWeb
Watson, VS insertion testings and the typical internal dev usage require SymWeb.
Currently we need to call the task twice (https://github.com/dotnet/core-eng/issues/3489).
-->
<Message Importance="High" Text="Publishing symbol packages to SymWeb ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenSymWeb)"
SymbolServerPath="https://microsoft.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.SymbolUploader.Build.Task" Version="$(SymbolUploaderVersion)" />
</ItemGroup>
</Project>
-141
View File
@@ -1,141 +0,0 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile $(Build.SourcesDirectory)/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
if (!$Password) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Please supply a valid PAT'
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
$dotnet3Source = $sources.SelectSingleNode("add[@key='dotnet3']")
if ($dotnet3Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$dotnet31Source = $sources.SelectSingleNode("add[@key='dotnet3.1']")
if ($dotnet31Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$doc.Save($filename)
-149
View File
@@ -1,149 +0,0 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $(Build.SourcesDirectory)/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
Write-PipelineTelemetryError -Category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [ -z "$CredToken" ]; then
Write-PipelineTelemetryError -category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Please supply a valid PAT"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
PackageSources=()
# Ensure dotnet3-internal and dotnet3-internal-transport are in the packageSources if the public dotnet3 feeds are present
grep -i "<add key=\"dotnet3\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal')
grep -i "<add key=\"dotnet3-internal-transport\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal-transport')
fi
# Ensure dotnet3.1-internal and dotnet3.1-internal-transport are in the packageSources if the public dotnet3.1 feeds are present
grep -i "<add key=\"dotnet3.1\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3.1-internal\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal')
grep -i "<add key=\"dotnet3.1-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal-transport')
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources+="$IFS"
PackageSources+=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" $ConfigFile
fi
done
+83
View File
@@ -0,0 +1,83 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
- PackageBasePath : Directory containing all files that need to be validated.
- SignCheckVersion : Version of SignCheck package to be used.
- SignValidationExclusionList : ItemGroup containing exclusion list to be forwarded to SignCheck.
- EnableJarSigningCheck : Whether .jar files should be validated.
- EnableStrongNameCheck : Whether strong name check should be performed.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<!--
From 'Signing.props' we import $(SignValidationExclusionList)
-->
<Import Project="$(MSBuildThisFileDirectory)Signing.props" Condition="Exists('$(MSBuildThisFileDirectory)Signing.props')" />
<Target Name="ValidateSigning">
<PropertyGroup>
<SignCheckToolPath>$(NuGetPackageRoot)Microsoft.DotNet.SignCheck\$(SignCheckVersion)\tools\Microsoft.DotNet.SignCheck.exe</SignCheckToolPath>
<SignCheckInputDir>$(PackageBasePath)</SignCheckInputDir>
<SignCheckLog>signcheck.log</SignCheckLog>
<SignCheckErrorLog>signcheck.errors.log</SignCheckErrorLog>
<SignCheckExclusionsFile>signcheck.exclusions.txt</SignCheckExclusionsFile>
</PropertyGroup>
<ItemGroup>
<!--
Documentation for these arguments is available here:
https://github.com/dotnet/arcade/tree/master/src/SignCheck
-->
<SignCheckArgs Include="--recursive" />
<SignCheckArgs Include="--traverse-subfolders" />
<SignCheckArgs Include="--file-status AllFiles" />
<SignCheckArgs Include="--log-file $(SignCheckLog)" />
<SignCheckArgs Include="--error-log-file $(SignCheckErrorLog)" />
<SignCheckArgs Include="--input-files $(SignCheckInputDir)" />
<SignCheckArgs Include="--exclusions-file $(SignCheckExclusionsFile)" Condition="'@(SignValidationExclusionList)' != ''" />
<SignCheckArgs Include="--verify-jar" Condition="'$(EnableJarSigningCheck)' == 'true'" />
<SignCheckArgs Include="--verify-strongname" Condition="'$(EnableStrongNameCheck)' == 'true'" />
</ItemGroup>
<WriteLinesToFile
File="$(SignCheckExclusionsFile)"
Lines="@(SignValidationExclusionList)"
Condition="'@(SignValidationExclusionList)' != ''"
Overwrite="true"
Encoding="Unicode"/>
<!--
IgnoreExitCode='true' because the tool doesn't return '0' on success.
-->
<Exec
Command="&quot;$(SignCheckToolPath)&quot; @(SignCheckArgs, ' ')"
IgnoreExitCode='true'
ConsoleToMsBuild="false"
StandardErrorImportance="high" />
<Error
Text="Signing validation failed. Check $(SignCheckErrorLog) for more information."
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
<Message
Text="##vso[artifact.upload containerfolder=LogFiles;artifactname=LogFiles]{SignCheckErrorLog}"
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.SignCheck" Version="$(SignCheckVersion)" />
</ItemGroup>
</Project>
+184
View File
@@ -0,0 +1,184 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SourceLinkToolPath, # Full path to directory where dotnet SourceLink CLI was installed
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
$RepoTreeURL = -Join("https://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
Measure-Command { ValidateSourceLinkLinks }
+49 -60
View File
@@ -18,7 +18,6 @@ Param(
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
@@ -26,55 +25,49 @@ Param(
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
# Unset 'Platform' environment variable to avoid unwanted collision in InstallDotNetCore.targets file
# some computer has this env var defined (e.g. Some HP)
if($env:Platform) {
$env:Platform=""
}
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot 'restore-toolset.ps1'
$script = Join-Path $EngRoot "restore-toolset.ps1"
if (Test-Path $script) {
. $script
@@ -85,17 +78,13 @@ function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'Build.binlog') } else { '' }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { '' }
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
@@ -120,15 +109,7 @@ function Build {
}
try {
if ($clean) {
if (Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host 'Artifacts directory deleted.'
}
exit 0
}
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
exit 0
}
@@ -138,15 +119,23 @@ try {
$nodeReuse = $false
}
if ($restore) {
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
InitializeNativeTools
}
Build
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+12 -12
View File
@@ -26,7 +26,6 @@ usage()
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
@@ -63,7 +62,6 @@ publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
@@ -84,9 +82,6 @@ while [[ $# > 0 ]]; do
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
@@ -201,15 +196,20 @@ function Build {
ExitWithExitCode 0
}
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
if [[ "$restore" == true ]]; then
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
InitializeNativeTools
fi
@@ -0,0 +1,41 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)
## Specify the toolchain
set(TOOLCHAIN "arm-linux-androideabi")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
add_compile_options(-mfloat-abi=soft)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/arm-linux-androideabi/)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
@@ -0,0 +1,42 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
## Specify the toolchain
set(TOOLCHAIN "aarch64-linux-android")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
## Needed for Android or bionic specific conditionals
add_compile_options(-D__ANDROID__)
add_compile_options(-D__BIONIC__)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+3 -3
View File
@@ -157,15 +157,15 @@ fetch_tizen_pkgs()
Inform "Initialize arm base"
fetch_tizen_pkgs_init standard base
Inform "fetch common packages"
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel libatomic
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel
fetch_tizen_pkgs noarch linux-glibc-devel
Inform "fetch coreclr packages"
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel tizen-release lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l libcom_err libcom_err-devel zlib zlib-devel libopenssl libopenssl-devel krb5 krb5-devel libcurl libcurl-devel
Inform "Initialize standard unified"
fetch_tizen_pkgs_init standard unified
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l gssdp gssdp-devel tizen-release
fetch_tizen_pkgs armv7l gssdp gssdp-devel
+51 -45
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
set -e
__NDK_Version=r21
__NDK_Version=r14
usage()
{
@@ -16,11 +16,11 @@ usage()
echo.
echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"
echo "you can set the NDK_DIR environment variable to have this script use that installation of the NDK."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.28-arm64. This file is to replace '/etc/os-release', which is not available for Android."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.21-arm64. This file is to replace '/etc/os-release', which is not available for Android."
exit 1
}
__ApiLevel=28 # The minimum platform for arm64 is API level 21 but the minimum version that support glob(3) is 28. See $ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/glob.h
__ApiLevel=21 # The minimum platform for arm64 is API level 21
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
@@ -53,20 +53,13 @@ for i in "$@"
done
# Obtain the location of the bash script to figure out where the root of the repo is.
__ScriptBaseDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__CrossDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"
if [[ ! -f "$__CrossDir" ]]; then
mkdir -p "$__CrossDir"
fi
# Resolve absolute path to avoid `../` in build logs
__CrossDir="$( cd "$__CrossDir" && pwd )"
__NDK_Dir="$__CrossDir/android-ndk-$__NDK_Version"
__lldb_Dir="$__CrossDir/lldb"
__ToolchainDir="$__CrossDir/android-ndk-$__NDK_Version"
__Android_Cross_Dir="$__CrossDir/android-rootfs"
__NDK_Dir="$__Android_Cross_Dir/android-ndk-$__NDK_Version"
__libunwind_Dir="$__Android_Cross_Dir/libunwind"
__lldb_Dir="$__Android_Cross_Dir/lldb"
__ToolchainDir="$__Android_Cross_Dir/toolchain/$__BuildArch"
if [[ -n "$TOOLCHAIN_DIR" ]]; then
__ToolchainDir=$TOOLCHAIN_DIR
@@ -85,47 +78,60 @@ echo "Target Toolchain location: $__ToolchainDir"
if [ ! -d $__NDK_Dir ]; then
echo Downloading the NDK into $__NDK_Dir
mkdir -p $__NDK_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__CrossDir
wget -nv -nc --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__Android_Cross_Dir
fi
if [ ! -d $__lldb_Dir ]; then
mkdir -p $__lldb_Dir
echo Downloading LLDB into $__lldb_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
fi
echo "Download dependencies..."
__TmpDir=$__CrossDir/tmp/$__BuildArch/
mkdir -p "$__TmpDir"
# Create the RootFS for both arm64 as well as aarch
rm -rf $__Android_Cross_Dir/toolchain
# combined dependencies for coreclr, installer and libraries
__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"
echo Generating the $__BuildArch toolchain
$__NDK_Dir/build/tools/make_standalone_toolchain.py --arch $__BuildArch --api $__ApiLevel --install-dir $__ToolchainDir
for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\
grep -A15 "Package: \(${__AndroidPackages// /\\|}\)" | grep -v "static\|tool" | grep Filename); do
# Install the required packages into the toolchain
# TODO: Add logic to get latest pkg version instead of specific version number
rm -rf $__Android_Cross_Dir/deb/
rm -rf $__Android_Cross_Dir/tmp
if [[ "$path" != "Filename:" ]]; then
echo "Working on: $path"
wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"
fi
done
mkdir -p $__Android_Cross_Dir/deb/
mkdir -p $__Android_Cross_Dir/tmp/$arch/
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu-dev_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb
cp -R "$__TmpDir/data/data/com.termux/files/usr/"* "$__ToolchainDir/sysroot/usr/"
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob-dev_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support-dev_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma-dev_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind-dev_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb
echo Unpacking Termux packages
dpkg -x $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
cp -R $__Android_Cross_Dir/tmp/$__AndroidArch/data/data/com.termux/files/usr/* $__ToolchainDir/sysroot/usr/
# Generate platform file for build.sh script to assign to __DistroRid
echo "Generating platform file..."
echo "RID=android.${__ApiLevel}-${__BuildArch}" > $__ToolchainDir/sysroot/android_platform
echo "Now to build coreclr, libraries and installers; run:"
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory coreclr
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory libraries
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory installer
echo "RID=android.21-arm64" > $__ToolchainDir/sysroot/android_platform
echo Now run:
echo CONFIG_DIR=\`realpath cross/android/$__BuildArch\` ROOTFS_DIR=\`realpath $__ToolchainDir/sysroot\` ./build.sh cross $__BuildArch skipgenerateversion skipnuget cmakeargs -DENABLE_LLDBPLUGIN=0
+28 -73
View File
@@ -2,16 +2,15 @@
usage()
{
echo "Usage: $0 [BuildArch] [CodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "Usage: $0 [BuildArch] [LinuxCodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "BuildArch can be: arm(default), armel, arm64, x86"
echo "CodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo " for FreeBSD can be: freebsd11 or freebsd12."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine and FReeBSD"
echo "LinuxCodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine"
echo "--skipunmount - optional, will skip the unmount of rootfs folder."
exit 1
}
__CodeName=xenial
__LinuxCodeName=xenial
__CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__InitialDir=$PWD
__BuildArch=arm
@@ -26,9 +25,8 @@ __UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackagesEdgeTesting=" lldb-dev"
__AlpinePackagesEdgeMain=" llvm9-libs"
__AlpinePackagesEdgeMain+=" python3"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
# symlinks fixer
__UbuntuPackages+=" symlinks"
@@ -54,15 +52,6 @@ __AlpinePackages+=" krb5-dev"
__AlpinePackages+=" openssl-dev"
__AlpinePackages+=" zlib-dev"
__FreeBSDBase="12.1-RELEASE"
__FreeBSDPkg="1.10.5"
__FreeBSDPackages="libunwind"
__FreeBSDPackages+=" icu"
__FreeBSDPackages+=" libinotify"
__FreeBSDPackages+=" lttng-ust"
__FreeBSDPackages+=" llvm-90"
__FreeBSDPackages+=" krb5"
__UnprocessedBuildArgs=
while :; do
if [ $# -le 0 ]; then
@@ -91,7 +80,7 @@ while :; do
__BuildArch=armel
__UbuntuArch=armel
__UbuntuRepo="http://ftp.debian.org/debian/"
__CodeName=jessie
__LinuxCodeName=jessie
;;
x86)
__BuildArch=x86
@@ -120,36 +109,36 @@ while :; do
unset __LLDB_Package
;;
trusty) # Ubuntu 14.04
if [ "$__CodeName" != "jessie" ]; then
__CodeName=trusty
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=trusty
fi
;;
xenial) # Ubuntu 16.04
if [ "$__CodeName" != "jessie" ]; then
__CodeName=xenial
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=xenial
fi
;;
zesty) # Ubuntu 17.04
if [ "$__CodeName" != "jessie" ]; then
__CodeName=zesty
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=zesty
fi
;;
bionic) # Ubuntu 18.04
if [ "$__CodeName" != "jessie" ]; then
__CodeName=bionic
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=bionic
fi
;;
jessie) # Debian 8
__CodeName=jessie
__LinuxCodeName=jessie
__UbuntuRepo="http://ftp.debian.org/debian/"
;;
stretch) # Debian 9
__CodeName=stretch
__LinuxCodeName=stretch
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
buster) # Debian 10
__CodeName=buster
__LinuxCodeName=buster
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
@@ -159,22 +148,14 @@ while :; do
usage;
exit 1;
fi
__CodeName=
__LinuxCodeName=
__UbuntuRepo=
__Tizen=tizen
;;
alpine)
__CodeName=alpine
__LinuxCodeName=alpine
__UbuntuRepo=
;;
freebsd11)
__FreeBSDBase="11.3-RELEASE"
;&
freebsd12)
__CodeName=freebsd
__BuildArch=x64
__SkipUnmount=1
;;
--skipunmount)
__SkipUnmount=1
;;
@@ -200,7 +181,7 @@ if [ -z "$__RootfsDir" ] && [ ! -z "$ROOTFS_DIR" ]; then
fi
if [ -z "$__RootfsDir" ]; then
__RootfsDir="$__CrossDir/../../../.tools/rootfs/$__BuildArch"
__RootfsDir="$__CrossDir/rootfs/$__BuildArch"
fi
if [ -d "$__RootfsDir" ]; then
@@ -210,50 +191,24 @@ if [ -d "$__RootfsDir" ]; then
rm -rf $__RootfsDir
fi
if [[ "$__CodeName" == "alpine" ]]; then
if [[ "$__LinuxCodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.9
__AlpineVersion=3.7
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeMain
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeTesting
add $__AlpinePackages
rm -r $__ApkToolsDir
elif [[ "$__CodeName" == "freebsd" ]]; then
mkdir -p $__RootfsDir/usr/local/etc
wget -O - https://download.freebsd.org/ftp/releases/amd64/${__FreeBSDBase}/base.txz | tar -C $__RootfsDir -Jxf - ./lib ./usr/lib ./usr/libdata ./usr/include ./usr/share/keys ./etc ./bin/freebsd-version
# For now, ask for 11 ABI even on 12. This can be revisited later.
echo "ABI = \"FreeBSD:11:amd64\"; FINGERPRINTS = \"${__RootfsDir}/usr/share/keys\"; REPOS_DIR = [\"${__RootfsDir}/etc/pkg\"]; REPO_AUTOUPDATE = NO; RUN_SCRIPTS = NO;" > ${__RootfsDir}/usr/local/etc/pkg.conf
echo "FreeBSD: { url: "pkg+http://pkg.FreeBSD.org/\${ABI}/quarterly", mirror_type: \"srv\", signature_type: \"fingerprints\", fingerprints: \"${__RootfsDir}/usr/share/keys/pkg\", enabled: yes }" > ${__RootfsDir}/etc/pkg/FreeBSD.conf
mkdir -p $__RootfsDir/tmp
# get and build package manager
wget -O - https://github.com/freebsd/pkg/archive/${__FreeBSDPkg}.tar.gz | tar -C $__RootfsDir/tmp -zxf -
cd $__RootfsDir/tmp/pkg-${__FreeBSDPkg}
./autogen.sh && ./configure --prefix=$__RootfsDir/host && make install
rm -rf $__RootfsDir/tmp/pkg-${__FreeBSDPkg}
# install packages we need.
$__RootfsDir/host/sbin/pkg -r $__RootfsDir -C $__RootfsDir/usr/local/etc/pkg.conf update
$__RootfsDir/host/sbin/pkg -r $__RootfsDir -C $__RootfsDir/usr/local/etc/pkg.conf install --yes $__FreeBSDPackages
elif [[ -n $__CodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__CodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__CodeName $__RootfsDir/etc/apt/sources.list
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__LinuxCodeName $__RootfsDir/etc/apt/sources.list
chroot $__RootfsDir apt-get update
chroot $__RootfsDir apt-get -f -y install
chroot $__RootfsDir apt-get -y install $__UbuntuPackages
@@ -263,7 +218,7 @@ elif [[ -n $__CodeName ]]; then
umount $__RootfsDir/*
fi
if [[ "$__BuildArch" == "arm" && "$__CodeName" == "trusty" ]]; then
if [[ "$__BuildArch" == "arm" && "$__LinuxCodeName" == "trusty" ]]; then
pushd $__RootfsDir
patch -p1 < $__CrossDir/$__BuildArch/trusty.patch
patch -p1 < $__CrossDir/$__BuildArch/trusty-lttng-2.4.patch
+37 -61
View File
@@ -1,11 +1,7 @@
set(CROSS_ROOTFS $ENV{ROOTFS_DIR})
set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
if(EXISTS ${CROSS_ROOTFS}/bin/freebsd-version)
set(CMAKE_SYSTEM_NAME FreeBSD)
else()
set(CMAKE_SYSTEM_NAME Linux)
endif()
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
if(TARGET_ARCH_NAME STREQUAL "armel")
@@ -31,17 +27,10 @@ elseif(TARGET_ARCH_NAME STREQUAL "arm64")
elseif(TARGET_ARCH_NAME STREQUAL "x86")
set(CMAKE_SYSTEM_PROCESSOR i686)
set(TOOLCHAIN "i686-linux-gnu")
elseif (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
set(CMAKE_SYSTEM_PROCESSOR "x86_64")
set(triple "x86_64-unknown-freebsd11")
else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
@@ -50,53 +39,50 @@ if(TARGET_ARCH_NAME STREQUAL "armel")
endif()
endif()
if("$ENV{__DistroRid}" MATCHES "android.*")
if(TARGET_ARCH_NAME STREQUAL "arm")
set(ANDROID_ABI armeabi-v7a)
elseif(TARGET_ARCH_NAME STREQUAL "arm64")
set(ANDROID_ABI arm64-v8a)
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
endif()
# extract platform number required by the NDK's toolchain
string(REGEX REPLACE ".*\\.([0-9]+)-.*" "\\1" ANDROID_PLATFORM "$ENV{__DistroRid}")
set(ANDROID_TOOLCHAIN clang)
set(FEATURE_EVENT_TRACE 0) # disable event trace as there is no lttng-ust package in termux repository
set(CMAKE_SYSTEM_LIBRARY_PATH "${CROSS_ROOTFS}/usr/lib")
set(CMAKE_SYSTEM_INCLUDE_PATH "${CROSS_ROOTFS}/usr/include")
# include official NDK toolchain script
include(${CROSS_ROOTFS}/../build/cmake/android.toolchain.cmake)
elseif (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
# we cross-compile by instructing clang
set(CMAKE_C_COMPILER_TARGET ${triple})
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
else()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
endif()
endforeach()
endmacro()
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_link_options(-m32)
add_compile_param(CROSS_LINK_FLAGS "-m32")
endif()
# Specify compile options
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$" AND NOT "$ENV{__DistroRid}" MATCHES "android.*")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
@@ -104,17 +90,7 @@ endif()
if(TARGET_ARCH_NAME MATCHES "^(arm|armel)$")
add_compile_options(-mthumb)
if (NOT DEFINED CLR_ARM_FPU_TYPE)
set (CLR_ARM_FPU_TYPE vfpv3)
endif (NOT DEFINED CLR_ARM_FPU_TYPE)
add_compile_options (-mfpu=${CLR_ARM_FPU_TYPE})
if (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
set (CLR_ARM_FPU_CAPABILITY 0x7)
endif (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
add_definitions (-DCLR_ARM_FPU_CAPABILITY=${CLR_ARM_FPU_CAPABILITY})
add_compile_options(-mfpu=vfpv3)
if(TARGET_ARCH_NAME STREQUAL "armel")
add_compile_options(-mfloat-abi=softfp)
if(DEFINED TIZEN_TOOLCHAIN)
@@ -127,7 +103,7 @@ elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths for builds that need lldb.
# Set LLDB include and library paths
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
@@ -155,7 +131,7 @@ if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+11 -25
View File
@@ -1,21 +1,20 @@
param (
$darcVersion = $null,
$versionEndpoint = 'https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16',
$verbosity = 'minimal',
$toolpath = $null
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
)
$verbosity = "m"
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion, $toolpath) {
$darcCliPackageName = 'microsoft.dotnet.darc'
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list -g
$toolList = Invoke-Expression "& `"$dotnet`" tool list -g"
if ($toolList -like "*$darcCliPackageName*") {
& "$dotnet" tool uninstall $darcCliPackageName -g
Invoke-Expression "& `"$dotnet`" tool uninstall $darcCliPackageName -g"
}
# If the user didn't explicitly specify the darc version,
@@ -23,25 +22,12 @@ function InstallDarcCli ($darcVersion, $toolpath) {
if (-not $darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
$arcadeServicesSource = 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
if (-not $toolpath) {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity --tool-path '$toolpath'"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Invoke-Expression "& `"$dotnet`" tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
}
try {
InstallDarcCli $darcVersion $toolpath
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Darc' -Message $_
ExitWithExitCode 1
}
InstallDarcCli $darcVersion
+8 -26
View File
@@ -2,8 +2,7 @@
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint='https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16'
verbosity='minimal'
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
@@ -16,14 +15,6 @@ while [[ $# > 0 ]]; do
versionEndpoint=$2
shift
;;
--verbosity)
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
@@ -43,6 +34,7 @@ while [[ -h "$source" ]]; do
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity=m
. "$scriptroot/tools.sh"
@@ -56,27 +48,17 @@ function InstallDarcCli {
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
local arcadeServicesSource="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
}
InstallDarcCli
+11 -12
View File
@@ -1,27 +1,26 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = 'minimal',
[string] $architecture = '',
[string] $version = 'Latest',
[string] $runtime = 'dotnet',
[string] $RuntimeSourceFeed = '',
[string] $RuntimeSourceFeedKey = ''
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet"
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq 'x86') {
$installdir = Join-Path $installdir 'x86'
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
}
InstallDotNet $installdir $version $architecture $runtime $true
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+5 -45
View File
@@ -11,13 +11,9 @@ while [[ -h "$source" ]]; do
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
@@ -33,56 +29,20 @@ while [[ $# > 0 ]]; do
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
Write-PipelineTelemetryError -Category 'Build' -Message "Invalid argument: $1"
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv*l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
. "$scriptroot/tools.sh"
dotnetRoot="$repo_root/.dotnet"
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
local exit_code=$?
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "dotnet-install.sh failed (exit code '$exit_code')." >&2
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
@@ -1,13 +0,0 @@
param(
[string] $token
)
. $PSScriptRoot\pipeline-logging-functions.ps1
# Write-PipelineSetVariable will no-op if a variable named $ci is not defined
# Since this script is only ever called in AzDO builds, just universally set it
$ci = $true
Write-PipelineSetVariable -Name 'VSS_NUGET_ACCESSTOKEN' -Value $token -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'VSS_NUGET_URI_PREFIXES' -Value 'https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/' -IsMultiJobVariable $false
+34 -33
View File
@@ -3,46 +3,46 @@ Param(
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion, # darc's version
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-PipelineTelemetryError -Category 'Arcade' -Message "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
$ErrorActionPreference = 'Stop'
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
Push-Location $PSScriptRoot
Write-Host 'Installing darc...'
Write-Host "Installing darc..."
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode 'Running darc-init'
CheckExitCode "Running darc-init"
$engCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
$installBin = Join-Path $graphvizInstallDir 'bin'
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
Write-Host 'Installing dot...'
Write-Host "Installing dot..."
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
$darcExe = Resolve-Path "$darcExe\darc.exe"
Create-Directory $outputFolder
# Generate 3 graph descriptions:
# 1. Flat with coherency information
# 2. Graphviz (dot) file
@@ -51,36 +51,37 @@ try {
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = @( '--github-pat', "$gitHubPat", '--azdev-pat', "$azdoPat", '--password', "$barToken" )
$baseOptions = "get-dependency-graph --github-pat $gitHubPat --azdev-pat $azdoPat --password $barToken"
if ($includeToolset) {
Write-Host 'Toolsets will be included in the graph...'
$baseOptions += @( '--include-toolset' )
Write-Host "Toolsets will be included in the graph..."
$baseOptions += " --include-toolset"
}
Write-Host 'Generating standard dependency graph...'
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
CheckExitCode 'Generating normal dependency graph'
Write-Host "Generating standard dependency graph..."
Invoke-Expression "& `"$darcExe`" $baseOptions --output-file $normalGraphFilePath"
CheckExitCode "Generating normal dependency graph"
Write-Host 'Generating flat dependency graph and graphviz file...'
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
CheckExitCode 'Generating flat and graphviz dependency graph'
Write-Host "Generating flat dependency graph and graphviz file..."
Invoke-Expression "& `"$darcExe`" $baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath"
CheckExitCode "Generating flat and graphviz dependency graph"
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
CheckExitCode 'Generating graphviz image'
Invoke-Expression "& `"$dotFilePath`" -Tpng -o'$graphVizImageFilePath' `"$graphVizFilePath`""
CheckExitCode "Generating graphviz image"
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host 'This might be a toolset repo which includes only toolset dependencies. ' -NoNewline -ForegroundColor Yellow
Write-Host 'Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again...' -ForegroundColor Yellow
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
} finally {
Pop-Location
Pop-Location
}
+28 -38
View File
@@ -35,7 +35,7 @@ File path to global.json file
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external',
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
@@ -45,27 +45,26 @@ Param (
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName 'global.json'
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
}
Set-StrictMode -version 2.0
$ErrorActionPreference='Stop'
$ErrorActionPreference="Stop"
. $PSScriptRoot\pipeline-logging-functions.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq 'Continue'
$Verbose = $VerbosePreference -Eq "Continue"
$EngCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir 'bin'
$InstallerPath = Join-Path $EngCommonBaseDir 'install-tool.ps1'
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
# Process tools list
Write-Host "Processing $GlobalJsonFile"
@@ -75,59 +74,51 @@ try {
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand 'native-tools' -ErrorAction SilentlyContinue
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
$ToolVersion = $_.Value
$LocalInstallerArguments = @{ ToolName = "$ToolName" }
$LocalInstallerArguments += @{ InstallPath = "$InstallBin" }
$LocalInstallerArguments += @{ BaseUri = "$BaseUri" }
$LocalInstallerArguments += @{ CommonLibraryDirectory = "$EngCommonBaseDir" }
$LocalInstallerArguments += @{ Version = "$ToolVersion" }
$LocalInstallerCommand = $InstallerPath
$LocalInstallerCommand += " -ToolName $ToolName"
$LocalInstallerCommand += " -InstallPath $InstallBin"
$LocalInstallerCommand += " -BaseUri $BaseUri"
$LocalInstallerCommand += " -CommonLibraryDirectory $EngCommonBaseDir"
$LocalInstallerCommand += " -Version $ToolVersion"
if ($Verbose) {
$LocalInstallerArguments += @{ Verbose = $True }
$LocalInstallerCommand += " -Verbose"
}
if (Get-Variable 'Force' -ErrorAction 'SilentlyContinue') {
if($Force) {
$LocalInstallerArguments += @{ Force = $True }
$LocalInstallerCommand += " -Force"
}
}
if ($Clean) {
$LocalInstallerArguments += @{ Clean = $True }
$LocalInstallerCommand += " -Clean"
}
Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$InstallerPath $($LocalInstallerArguments.Keys.ForEach({"-$_ '$($LocalInstallerArguments.$_)'"}) -join ' ')'"
& $InstallerPath @LocalInstallerArguments
Write-Verbose "Executing '$LocalInstallerCommand'"
Invoke-Expression "$LocalInstallerCommand"
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
$showNativeToolsWarning = $true
if ((Get-Variable 'DoNotDisplayNativeToolsInstallationWarnings' -ErrorAction 'SilentlyContinue') -and $DoNotDisplayNativeToolsInstallationWarnings) {
$showNativeToolsWarning = $false
}
if ($showNativeToolsWarning) {
Write-Warning $errMsg
}
Write-Warning $errMsg
$toolInstallationFailure = $true
} else {
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host $errMsg
Write-Error $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host 'Native tools bootstrap failed'
exit 1
}
}
else {
Write-Host 'No native tools defined in global.json'
Write-Host "No native tools defined in global.json"
exit 0
}
@@ -135,18 +126,17 @@ try {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host 'Native tools are available from ' (Convert-Path -Path $InstallBin)
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message 'Native tools install directory does not exist, installation failed'
Write-Error "Native tools install directory does not exist, installation failed"
exit 1
}
exit 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message $_
ExitWithExitCode 1
Write-Host $_
Write-Host $_.Exception
exit 1
}
+11 -43
View File
@@ -12,7 +12,6 @@ retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/pipeline-logging-functions.sh
. $scriptroot/native/common-library.sh
while (($# > 0)); do
@@ -34,14 +33,6 @@ while (($# > 0)); do
force=true
shift 1
;;
--donotabortonfailure)
donotabortonfailure=true
shift 1
;;
--donotdisplaywarnings)
donotdisplaywarnings=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
@@ -60,8 +51,6 @@ while (($# > 0)); do
echo " - (default) %USERPROFILE%/.netcoreeng/native"
echo ""
echo " --clean Switch specifying not to install anything, but cleanup native asset folders"
echo " --donotabortonfailure Switch specifiying whether to abort native tools installation on failure"
echo " --donotdisplaywarnings Switch specifiying whether to display warnings during native tools installation on failure"
echo " --force Clean and then install tools"
echo " --help Print help and exit"
echo ""
@@ -81,7 +70,8 @@ function ReadGlobalJsonNativeTools {
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=$( echo "$native_tools_list" | sed 's/\s//g' | sed 's/,/\n/g' )
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[:space:]')"
local old_IFS=$IFS
while read -r line; do
@@ -102,7 +92,6 @@ if [[ -z $install_directory ]]; then
fi
install_bin="${native_base_dir}/bin"
installed_any=false
ReadGlobalJsonNativeTools
@@ -114,12 +103,11 @@ else
for tool in "${!native_assets[@]}"
do
tool_version=${native_assets[$tool]}
installer_path="$native_installer_dir/install-$tool.sh"
installer_command="$installer_path"
installer_name="install-$tool.sh"
installer_command="$native_installer_dir/$installer_name"
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
echo $installer_command
if [[ $force = true ]]; then
installer_command+=" --force"
@@ -129,29 +117,11 @@ else
installer_command+=" --clean"
fi
if [[ -a $installer_path ]]; then
$installer_command
if [[ $? != 0 ]]; then
if [[ $donotabortonfailure = true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
exit 1
fi
else
$installed_any = true
fi
else
if [[ $donotabortonfailure == true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
exit 1
fi
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
exit 1
fi
done
fi
@@ -164,10 +134,8 @@ if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
if [[ $installed_any = true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Native tools install directory does not exist, installation failed"
exit 1
fi
echo "Native tools install directory does not exist, installation failed" >&2
exit 1
fi
exit 0
-133
View File
@@ -1,133 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $Operation,
[string] $AuthToken,
[string] $CommitSha,
[string] $RepoName,
[switch] $IsFeedPrivate
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables. This should ONLY be called from identified
# internal builds
function SetupCredProvider {
param(
[string] $AuthToken
)
# Install the Cred Provider NuGet plugin
Write-Host 'Setting up Cred Provider NuGet plugin in the agent...'
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host 'Installing plugin...'
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-PipelineTelemetryError -Category 'Arcade' -Message 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
}
else {
Write-Host 'CredProvider plugin was installed correctly!'
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'NuGet.config file not found in repo root!'
ExitWithExitCode 1
}
$endpoints = New-Object System.Collections.ArrayList
$nugetConfigPackageSources = Select-Xml -Path $nugetConfigPath -XPath "//packageSources/add[contains(@key, 'darc-int-')]/@value" | foreach{$_.Node.Value}
if (($nugetConfigPackageSources | Measure-Object).Count -gt 0 ) {
foreach ($stableRestoreResource in $nugetConfigPackageSources) {
$trimmedResource = ([string]$stableRestoreResource).Trim()
[void]$endpoints.Add(@{endpoint="$trimmedResource"; password="$AuthToken"})
}
}
if (($endpoints | Measure-Object).Count -gt 0) {
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
$endpointCredentials = @{endpointCredentials=$endpoints} | ConvertTo-Json -Compress
# Create the environment variables the AzDo way
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $endpointCredentials -Properties @{
'variable' = 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS'
'issecret' = 'false'
}
# We don't want sessions cached since we will be updating the endpoints quite frequently
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data 'False' -Properties @{
'variable' = 'NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED'
'issecret' = 'false'
}
}
else
{
Write-Host 'No internal endpoints found in NuGet.config'
}
}
#Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
$dotnetTempDir = "$RepoRoot\dotnet"
$dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
$dotnet = "$dotnetTempDir\dotnet.exe"
$restoreProjPath = "$PSScriptRoot\restore.proj"
Write-Host "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Out-File "$restoreProjPath"
& $dotnet restore $restoreProjPath
Write-Host 'Arcade SDK restored!'
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
}
if (Test-Path -Path $dotnetTempDir) {
Remove-Item $dotnetTempDir -Recurse
}
}
try {
Push-Location $PSScriptRoot
if ($Operation -like 'setup') {
SetupCredProvider $AuthToken
}
elseif ($Operation -like 'install-restore') {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-PipelineTelemetryError -Category 'Arcade' -Message "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
}
finally {
Pop-Location
}
-142
View File
@@ -1,142 +0,0 @@
#!/usr/bin/env bash
set -e
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables.
# This should ONLY be called from identified internal builds
function SetupCredProvider {
local authToken=$1
# Install the Cred Provider NuGet plugin
echo "Setting up Cred Provider NuGet plugin in the agent..."...
echo "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
local url="https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh"
echo "Writing the contents of 'installcredprovider.ps1' locally..."
local installcredproviderPath="installcredprovider.sh"
if command -v curl > /dev/null; then
curl $url > "$installcredproviderPath"
else
wget -q -O "$installcredproviderPath" "$url"
fi
echo "Installing plugin..."
. "$installcredproviderPath"
echo "Deleting local copy of 'installcredprovider.sh'..."
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
Write-PipelineTelemetryError -category 'Build' 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
fi
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
Write-PipelineTelemetryError -category 'Build' "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
local endpoints='['
local nugetConfigPackageValues=`cat "$nugetConfigPath" | grep "key=\"darc-int-"`
local pattern="value=\"(.*)\""
for value in $nugetConfigPackageValues
do
if [[ $value =~ $pattern ]]; then
local endpoint="${BASH_REMATCH[1]}"
endpoints+="{\"endpoint\": \"$endpoint\", \"password\": \"$authToken\"},"
fi
done
endpoints=${endpoints%?}
endpoints+=']'
if [ ${#endpoints} -gt 2 ]; then
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
local endpointCredentials="{\"endpointCredentials\": "$endpoints"}"
echo "##vso[task.setvariable variable=VSS_NUGET_EXTERNAL_FEED_ENDPOINTS]$endpointCredentials"
echo "##vso[task.setvariable variable=NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED]False"
else
echo "No internal endpoints found in NuGet.config"
fi
}
# Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
local dotnetTempDir="$repo_root/dotnet"
local dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
local restoreProjPath="$repo_root/eng/common/restore.proj"
echo "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
echo "<Project Sdk=\"Microsoft.DotNet.Arcade.Sdk\"/>" > "$restoreProjPath"
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
local res=`$dotnetTempDir/dotnet restore $restoreProjPath`
echo "Arcade SDK restored!"
# Cleanup
if [ "$restoreProjPath" ]; then
rm "$restoreProjPath"
fi
if [ "$dotnetTempDir" ]; then
rm -r $dotnetTempDir
fi
}
source="${BASH_SOURCE[0]}"
operation=''
authToken=''
repoName=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--operation)
operation=$2
shift
;;
--authtoken)
authToken=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ "$operation" = "setup" ]; then
SetupCredProvider $authToken
elif [ "$operation" = "install-restore" ]; then
InstallDotNetSdkAndRestoreArcade
else
echo "Unknown operation '$operation'!"
fi
+3 -2
View File
@@ -1,6 +1,6 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = 'minimal',
[string] $verbosity = "minimal",
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
@@ -18,8 +18,9 @@ try {
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+3 -34
View File
@@ -59,38 +59,9 @@ function DownloadAndExtract {
-Verbose:$Verbose
if ($UnzipStatus -Eq $False) {
# Retry Download one more time with Force=true
$DownloadRetryStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries 1 `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$True `
-Verbose:$Verbose
if ($DownloadRetryStatus -Eq $False) {
Write-Error "Last attempt of download failed as well"
return $False
}
# Retry unzip again one more time with Force=true
$UnzipRetryStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$True `
-Verbose:$Verbose
if ($UnzipRetryStatus -Eq $False)
{
Write-Error "Last attempt of unzip failed as well"
# Clean up partial zips and extracts
if (Test-Path $TempToolPath) {
Remove-Item $TempToolPath -Force
}
if (Test-Path $InstallDirectory) {
Remove-Item $InstallDirectory -Force -Recurse
}
return $False
}
Write-Error "Unzip failed"
return $False
}
return $True
}
@@ -152,8 +123,6 @@ function Get-File {
}
else {
Write-Verbose "Downloading $Uri"
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
while($Attempt -Lt $DownloadRetries)
{
try {
@@ -240,7 +209,7 @@ function New-ScriptShim {
Remove-Item (Join-Path $ShimDirectory "$ShimName.exe")
}
& "$ShimDirectory\WinShimmer\winshimmer.exe" $ShimName $ToolFilePath $ShimDirectory
Invoke-Expression "$ShimDirectory\WinShimmer\winshimmer.exe $ShimName $ToolFilePath $ShimDirectory"
return $True
}
catch {
+5 -5
View File
@@ -34,7 +34,7 @@ function ExpandZip {
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to remove '$output_directory'"
echo Unable to remove '$output_directory'>&2
return 1
fi
fi
@@ -45,7 +45,7 @@ function ExpandZip {
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to extract '$zip_path'"
echo "Unable to extract '$zip_path'" >&2
return 1
fi
@@ -117,7 +117,7 @@ function DownloadAndExtract {
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to download '$uri' to '$temp_tool_path'."
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
return 1
fi
@@ -125,7 +125,7 @@ function DownloadAndExtract {
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to extract '$temp_tool_path' to '$installDir'."
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
return 1
fi
@@ -148,7 +148,7 @@ function NewScriptShim {
fi
if [[ ! -f $tool_file_path ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Specified tool file path:'$tool_file_path' does not exist"
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
return 1
fi
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake-test"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
exit 0
+3 -3
View File
@@ -69,7 +69,7 @@ tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
uri="${base_uri}/$tool_folder/cmake/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
@@ -101,7 +101,7 @@ fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
echo "Installation failed" >&2
exit 1
fi
@@ -110,7 +110,7 @@ fi
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
echo "Shim generation failed" >&2
exit 1
fi
+5 -7
View File
@@ -46,8 +46,6 @@ Param (
[int] $RetryWaitTimeInSeconds = 30
)
. $PSScriptRoot\..\pipeline-logging-functions.ps1
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
@@ -95,7 +93,7 @@ try {
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-PipelineTelemetryError "Installation failed" -Category "NativeToolsetBootstrapping"
Write-Error "Installation failed"
exit 1
}
}
@@ -105,7 +103,7 @@ try {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Host "$ToolName was not found in $ToolFilePath."
Write-Error "$ToolName was not found in $ToolFilePath."
exit 1
}
@@ -119,14 +117,14 @@ try {
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-PipelineTelemetryError "Generate shim failed" -Category "NativeToolsetBootstrapping"
Write-Error "Generate shim failed"
return 1
}
exit 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "NativeToolsetBootstrapping" -Message $_
Write-Host $_
Write-Host $_.Exception
exit 1
}
@@ -1,121 +0,0 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<WorkItemCommand>%HELIX_CORRELATION_PAYLOAD%\performance\scripts\benchmarks_ci.py --csproj %HELIX_CORRELATION_PAYLOAD%\performance\$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd;set PYTHONPATH=%HELIX_WORKITEM_PAYLOAD%\scripts%3B%HELIX_WORKITEM_PAYLOAD%</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
<DotnetExe>%HELIX_CORRELATION_PAYLOAD%\performance\tools\dotnet\$(Architecture)\dotnet.exe</DotnetExe>
<Percent>%25%25</Percent>
<XMLResults>%HELIX_WORKITEM_ROOT%\testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
<BaseDirectory>$HELIX_CORRELATION_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)/performance</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'true'">
<BaseDirectory>$HELIX_WORKITEM_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT'">
<WorkItemCommand>$(PerformanceDirectory)/scripts/benchmarks_ci.py --csproj $(PerformanceDirectory)/$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<BaselineCoreRun>$(BaseDirectory)/Baseline_Core_Root/corerun</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>$(PerformanceDirectory)/src/tools/ResultsComparer/ResultsComparer.csproj</ResultsComparer>
<DotnetExe>$(PerformanceDirectory)/tools/dotnet/$(Architecture)/dotnet</DotnetExe>
<Percent>%25</Percent>
<XMLResults>$HELIX_WORKITEM_ROOT/testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(UseBaselineCoreRun)' == 'true'">
<BaselineCoreRunArgument>--corerun $(BaselineCoreRun)</BaselineCoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>
<PropertyGroup Condition="'$(_Framework)' != 'net461'">
<WorkItemCommand>$(WorkItemCommand) $(CliArguments)</WorkItemCommand>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<PropertyGroup>
<PartitionCount>5</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="$(BuildConfig).Partition0" Index="0" />
<Partition Include="$(BuildConfig).Partition1" Index="1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>
<PropertyGroup Condition="'$(Compare)' == 'true'">
<FailOnTestFailure>false</FailOnTestFailure>
</PropertyGroup>
<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults);$(FinalCommand)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<HelixWorkItem Include="Crossgen System.Private.Xml.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Private.Xml.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen System.Linq.Expressions.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Linq.Expressions.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.VisualBasic.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.VisualBasic.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.CSharp.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.CSharp.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
</ItemGroup>
</Project>
@@ -1,114 +0,0 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $BaselineCoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp5.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="Libraries Runtime",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode RunKind=$Kind"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)
$UseBaselineCoreRun = ($BaselineCoreRootDirectory -ne [string]::Empty)
$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
$WorkItemDirectory = (Join-Path $SourceDirectory "workitem")
$ExtraBenchmarkDotNetArguments = "--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
$Creator = $env:BUILD_DEFINITIONNAME
$PerfLabArguments = ""
$HelixSourcePrefix = "pr"
$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"
if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS5.Open"
}
if ($Compare) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf.Open"
$PerfLabArguments = ""
$ExtraBenchmarkDotNetArguments = ""
}
if ($Internal) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}
# FIX ME: This is a workaround until we get this from the actual pipeline
$CommonSetupArguments="--channel master --queue $Queue --build-number $BuildNumber --build-configs $Configurations --architecture $Architecture"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
#This grabs the LKG version number of dotnet and passes it to our scripts
$VersionJSON = Get-Content global.json | ConvertFrom-Json
$DotNetVersion = $VersionJSON.tools.dotnet
$SetupArguments = "--dotnet-versions $DotNetVersion $SetupArguments"
if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"
robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}
if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}
if ($UseBaselineCoreRun) {
$NewBaselineCoreRoot = (Join-Path $PayloadDirectory "Baseline_Core_Root")
Move-Item -Path $BaselineCoreRootDirectory -Destination $NewBaselineCoreRoot
}
$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory
# Set variables that we will need to have in future steps
$ci = $true
. "$PSScriptRoot\..\pipeline-logging-functions.ps1"
# Directories
Write-PipelineSetVariable -Name 'PayloadDirectory' -Value "$PayloadDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerformanceDirectory' -Value "$PerformanceDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'WorkItemDirectory' -Value "$WorkItemDirectory" -IsMultiJobVariable $false
# Script Arguments
Write-PipelineSetVariable -Name 'Python' -Value "py -3" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'ExtraBenchmarkDotNetArguments' -Value "$ExtraBenchmarkDotNetArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'SetupArguments' -Value "$SetupArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerfLabArguments' -Value "$PerfLabArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'BDNCategories' -Value "$RunCategories" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseBaselineCoreRun' -Value "$UseBaselineCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Compare' -Value "$Compare" -IsMultiJobVariable $false
# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Queue' -Value "$Queue" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'HelixSourcePrefix' -Value "$HelixSourcePrefix" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name '_BuildConfig' -Value "$Architecture.$Kind.$Framework" -IsMultiJobVariable $false
exit 0
-223
View File
@@ -1,223 +0,0 @@
#!/usr/bin/env bash
source_directory=$BUILD_SOURCESDIRECTORY
core_root_directory=
baseline_core_root_directory=
architecture=x64
framework=netcoreapp5.0
compilation_mode=tiered
repository=$BUILD_REPOSITORY_NAME
branch=$BUILD_SOURCEBRANCH
commit_sha=$BUILD_SOURCEVERSION
build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="Libraries Runtime"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations="CompliationMode=$compilation_mode RunKind=$kind"
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--sourcedirectory)
source_directory=$2
shift 2
;;
--corerootdirectory)
core_root_directory=$2
shift 2
;;
--baselinecorerootdirectory)
baseline_core_root_directory=$2
shift 2
;;
--architecture)
architecture=$2
shift 2
;;
--framework)
framework=$2
shift 2
;;
--compilationmode)
compilation_mode=$2
shift 2
;;
--repository)
repository=$2
shift 2
;;
--branch)
branch=$2
shift 2
;;
--commitsha)
commit_sha=$2
shift 2
;;
--buildnumber)
build_number=$2
shift 2
;;
--kind)
kind=$2
shift 2
;;
--runcategories)
run_categories=$2
shift 2
;;
--csproj)
csproj=$2
shift 2
;;
--internal)
internal=true
shift 1
;;
--compare)
compare=true
shift 1
;;
--configurations)
configurations=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --corerootdirectory <value> Directory where Core_Root exists, if running perf testing with --corerun"
echo " --architecture <value> Architecture of the testing being run"
echo " --configurations <value> List of key=value pairs that will be passed to perf testing infrastructure."
echo " ex: --configurations \"CompilationMode=Tiered OptimzationLevel=PGO\""
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --framework <value> The framework to run, if not running in master"
echo " --compliationmode <value> The compilation mode if not passing --configurations"
echo " --sourcedirectory <value> The directory of the sources. Defaults to env:BUILD_SOURCESDIRECTORY"
echo " --repository <value> The name of the repository in the <owner>/<repository name> format. Defaults to env:BUILD_REPOSITORY_NAME"
echo " --branch <value> The name of the branch. Defaults to env:BUILD_SOURCEBRANCH"
echo " --commitsha <value> The commit sha1 to run against. Defaults to env:BUILD_SOURCEVERSION"
echo " --buildnumber <value> The build number currently running. Defaults to env:BUILD_BUILDNUMBER"
echo " --csproj The relative path to the benchmark csproj whose tests should be run. Defaults to src\benchmarks\micro\MicroBenchmarks.csproj"
echo " --kind <value> Related to csproj. The kind of benchmarks that should be run. Defaults to micro"
echo " --runcategories <value> Related to csproj. Categories of benchmarks to run. Defaults to \"coreclr corefx\""
echo " --internal If the benchmarks are running as an official job."
echo ""
exit 0
;;
esac
done
if [ "$repository" == "dotnet/performance" ] || [ "$repository" == "dotnet-performance" ]; then
run_from_perf_repo=true
fi
if [ -z "$configurations" ]; then
configurations="CompliationMode=$compilation_mode"
fi
if [ -z "$core_root_directory" ]; then
use_core_run=false
fi
if [ -z "$baseline_core_root_directory" ]; then
use_baseline_core_run=false
fi
payload_directory=$source_directory/Payload
performance_directory=$payload_directory/performance
workitem_directory=$source_directory/workitem
extra_benchmark_dotnet_arguments="--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
perflab_arguments=
queue=Ubuntu.1804.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"
if [[ "$compare" == true ]]; then
extra_benchmark_dotnet_arguments=
perflab_arguments=
# No open queues for arm64
if [[ "$architecture" = "arm64" ]]; then
echo "Compare not available for arm64"
exit 1
fi
queue=Ubuntu.1804.Amd64.Tiger.Perf.Open
fi
if [[ "$internal" == true ]]; then
perflab_arguments="--upload-to-perflab-container"
helix_source_prefix="official"
creator=
extra_benchmark_dotnet_arguments=
if [[ "$architecture" = "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
queue=Ubuntu.1804.Amd64.Tiger.Perf
fi
fi
common_setup_arguments="--channel master --queue $queue --build-number $build_number --build-configs $configurations --architecture $architecture"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
# Get the tools section from the global.json.
# This grabs the LKG version number of dotnet and passes it to our scripts
dotnet_version=`cat global.json | python3 -c 'import json,sys;obj=json.load(sys.stdin);print(obj["tools"]["dotnet"])'`
setup_arguments="--dotnet-versions $dotnet_version $setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
payload_directory=
workitem_directory=$source_directory
performance_directory=$workitem_directory
setup_arguments="--perf-hash $commit_sha $common_setup_arguments"
else
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $performance_directory
docs_directory=$performance_directory/docs
mv $docs_directory $workitem_directory
fi
if [[ "$use_core_run" = true ]]; then
new_core_root=$payload_directory/Core_Root
mv $core_root_directory $new_core_root
fi
if [[ "$use_baseline_core_run" = true ]]; then
new_baseline_core_root=$payload_directory/Baseline_Core_Root
mv $baseline_core_root_directory $new_baseline_core_root
fi
ci=true
_script_dir=$(pwd)/eng/common
. "$_script_dir/pipeline-logging-functions.sh"
# Make sure all of our variables are available for future steps
Write-PipelineSetVariable -name "UseCoreRun" -value "$use_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "UseBaselineCoreRun" -value "$use_baseline_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "Architecture" -value "$architecture" -is_multi_job_variable false
Write-PipelineSetVariable -name "PayloadDirectory" -value "$payload_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerformanceDirectory" -value "$performance_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "WorkItemDirectory" -value "$workitem_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "Queue" -value "$queue" -is_multi_job_variable false
Write-PipelineSetVariable -name "SetupArguments" -value "$setup_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "Python" -value "$python3" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerfLabArguments" -value "$perflab_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "ExtraBenchmarkDotNetArguments" -value "$extra_benchmark_dotnet_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "BDNCategories" -value "$run_categories" -is_multi_job_variable false
Write-PipelineSetVariable -name "TargetCsproj" -value "$csproj" -is_multi_job_variable false
Write-PipelineSetVariable -name "RunFromPerfRepo" -value "$run_from_perf_repo" -is_multi_job_variable false
Write-PipelineSetVariable -name "Creator" -value "$creator" -is_multi_job_variable false
Write-PipelineSetVariable -name "HelixSourcePrefix" -value "$helix_source_prefix" -is_multi_job_variable false
Write-PipelineSetVariable -name "Kind" -value "$kind" -is_multi_job_variable false
Write-PipelineSetVariable -name "_BuildConfig" -value "$architecture.$kind.$framework" -is_multi_job_variable false
Write-PipelineSetVariable -name "Compare" -value "$compare" -is_multi_job_variable false
-242
View File
@@ -1,242 +0,0 @@
# Source for this file was taken from https://github.com/microsoft/azure-pipelines-task-lib/blob/11c9439d4af17e6475d9fe058e6b2e03914d17e6/powershell/VstsTaskSdk/LoggingCommandFunctions.ps1 and modified.
# NOTE: You should not be calling these method directly as they are likely to change. Instead you should be calling the Write-Pipeline* functions defined in tools.ps1
$script:loggingCommandPrefix = '##vso['
$script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"?
New-Object psobject -Property @{ Token = ';' ; Replacement = '%3B' }
New-Object psobject -Property @{ Token = "`r" ; Replacement = '%0D' }
New-Object psobject -Property @{ Token = "`n" ; Replacement = '%0A' }
New-Object psobject -Property @{ Token = "]" ; Replacement = '%5D' }
)
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Category,
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force)
$PSBoundParameters.Remove('Category') | Out-Null
if($Force -Or ((Test-Path variable:ci) -And $ci)) {
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
}
$PSBoundParameters.Remove('Message') | Out-Null
$PSBoundParameters.Add('Message', $Message)
Write-PipelineTaskError @PSBoundParameters
}
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force
)
if(!$Force -And (-Not (Test-Path variable:ci) -Or !$ci)) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
$PSBoundParameters.Remove('Force') | Out-Null
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Name,
[string]$Value,
[switch]$Secret,
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
'isOutput' = $IsMultiJobVariable
} -AsOutput:$AsOutput
}
}
function Write-PipelinePrependPath {
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
<########################################
# Private functions.
########################################>
function Format-LoggingCommandData {
[CmdletBinding()]
param([string]$Value, [switch]$Reverse)
if (!$Value) {
return ''
}
if (!$Reverse) {
foreach ($mapping in $script:loggingCommandEscapeMappings) {
$Value = $Value.Replace($mapping.Token, $mapping.Replacement)
}
} else {
for ($i = $script:loggingCommandEscapeMappings.Length - 1 ; $i -ge 0 ; $i--) {
$mapping = $script:loggingCommandEscapeMappings[$i]
$Value = $Value.Replace($mapping.Replacement, $mapping.Token)
}
}
return $Value
}
function Format-LoggingCommand {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Area,
[Parameter(Mandatory = $true)]
[string]$Event,
[string]$Data,
[hashtable]$Properties)
# Append the preamble.
[System.Text.StringBuilder]$sb = New-Object -TypeName System.Text.StringBuilder
$null = $sb.Append($script:loggingCommandPrefix).Append($Area).Append('.').Append($Event)
# Append the properties.
if ($Properties) {
$first = $true
foreach ($key in $Properties.Keys) {
[string]$value = Format-LoggingCommandData $Properties[$key]
if ($value) {
if ($first) {
$null = $sb.Append(' ')
$first = $false
} else {
$null = $sb.Append(';')
}
$null = $sb.Append("$key=$value")
}
}
}
# Append the tail and output the value.
$Data = Format-LoggingCommandData $Data
$sb.Append(']').Append($Data).ToString()
}
function Write-LoggingCommand {
[CmdletBinding(DefaultParameterSetName = 'Parameters')]
param(
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Area,
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Event,
[Parameter(ParameterSetName = 'Parameters')]
[string]$Data,
[Parameter(ParameterSetName = 'Parameters')]
[hashtable]$Properties,
[Parameter(Mandatory = $true, ParameterSetName = 'Object')]
$Command,
[switch]$AsOutput)
if ($PSCmdlet.ParameterSetName -eq 'Object') {
Write-LoggingCommand -Area $Command.Area -Event $Command.Event -Data $Command.Data -Properties $Command.Properties -AsOutput:$AsOutput
return
}
$command = Format-LoggingCommand -Area $Area -Event $Event -Data $Data -Properties $Properties
if ($AsOutput) {
$command
} else {
Write-Host $command
}
}
function Write-LogIssue {
[CmdletBinding()]
param(
[ValidateSet('warning', 'error')]
[Parameter(Mandatory = $true)]
[string]$Type,
[string]$Message,
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
$command = Format-LoggingCommand -Area 'task' -Event 'logissue' -Data $Message -Properties @{
'type' = $Type
'code' = $ErrCode
'sourcepath' = $SourcePath
'linenumber' = $LineNumber
'columnnumber' = $ColumnNumber
}
if ($AsOutput) {
return $command
}
if ($Type -eq 'error') {
$foregroundColor = $host.PrivateData.ErrorForegroundColor
$backgroundColor = $host.PrivateData.ErrorBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Red
$backgroundColor = [System.ConsoleColor]::Black
}
} else {
$foregroundColor = $host.PrivateData.WarningForegroundColor
$backgroundColor = $host.PrivateData.WarningBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Yellow
$backgroundColor = [System.ConsoleColor]::Black
}
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
-179
View File
@@ -1,179 +0,0 @@
#!/usr/bin/env bash
function Write-PipelineTelemetryError {
local telemetry_category=''
local force=false
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-category|-c)
telemetry_category=$2
shift
;;
-force|-f)
force=true
;;
-*)
function_args+=("$1 $2")
shift
;;
*)
message=$*
;;
esac
shift
done
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
if [[ $force == true ]]; then
function_args+=("-force")
fi
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
local message_type="error"
local sourcepath=''
local linenumber=''
local columnnumber=''
local error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-ln)
linenumber=$2
shift
;;
-columnnumber|-cn)
columnnumber=$2
shift
;;
-errcode|-e)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
local message="##vso[task.logissue"
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
function Write-PipelineSetVariable {
if [[ "$ci" != true ]]; then
return
fi
local name=''
local value=''
local secret=false
local as_output=false
local is_multi_job_variable=true
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-name|-n)
name=$2
shift
;;
-value|-v)
value=$2
shift
;;
-secret|-s)
secret=true
;;
-as_output|-a)
as_output=true
;;
-is_multi_job_variable|-i)
is_multi_job_variable=$2
shift
;;
esac
shift
done
value=${value/;/%3B}
value=${value/\\r/%0D}
value=${value/\\n/%0A}
value=${value/]/%5D}
local message="##vso[task.setvariable variable=$name;isSecret=$secret;isOutput=$is_multi_job_variable]$value"
if [[ "$as_output" == true ]]; then
$message
else
echo "$message"
fi
}
function Write-PipelinePrependPath {
local prepend_path=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-path|-p)
prepend_path=$2
shift
;;
esac
shift
done
export PATH="$prepend_path:$PATH"
if [[ "$ci" == true ]]; then
echo "##vso[task.prependpath]$prepend_path"
fi
}
@@ -1,48 +0,0 @@
param(
[Parameter(Mandatory=$true)][int] $BuildId,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that the channel we are going to promote the build to exist
$channelInfo = Get-MaestroChannel -ChannelId $ChannelId
if (!$channelInfo) {
Write-PipelineTelemetryCategory -Category 'PromoteBuild' -Message "Channel with BAR ID $ChannelId was not found in BAR!"
ExitWithExitCode 1
}
# Get info about which channel(s) the build has already been promoted to
$buildInfo = Get-MaestroBuild -BuildId $BuildId
if (!$buildInfo) {
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "Build with BAR ID $BuildId was not found in BAR!"
ExitWithExitCode 1
}
# Find whether the build is already assigned to the channel or not
if ($buildInfo.channels) {
foreach ($channel in $buildInfo.channels) {
if ($channel.Id -eq $ChannelId) {
Write-Host "The build with BAR ID $BuildId is already on channel $ChannelId!"
ExitWithExitCode 0
}
}
}
Write-Host "Promoting build '$BuildId' to channel '$ChannelId'."
Assign-BuildToChannel -BuildId $BuildId -ChannelId $ChannelId
Write-Host 'done.'
}
catch {
Write-Host $_
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
ExitWithExitCode 1
}
@@ -1,25 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $PromoteToChannels, # List of channels that the build should be promoted to
[Parameter(Mandatory=$true)][array] $AvailableChannelIds # List of channel IDs available in the YAML implementation
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that every channel that Maestro told to promote the build to
# is available in YAML
$PromoteToChannelsIds = $PromoteToChannels -split "\D" | Where-Object { $_ }
foreach ($id in $PromoteToChannelsIds) {
if (($id -ne 0) -and ($id -notin $AvailableChannelIds)) {
Write-PipelineTaskError -Type 'warning' -Message "Channel $id is not present in the post-build YAML configuration!"
}
}
Write-Host 'done.'
}
catch {
Write-Host $_
Write-PipelineTelemetryError -Category 'CheckChannelConsistency' -Message "There was an error while trying to check consistency of Maestro default channels for the build and post-build YAML configuration."
ExitWithExitCode 1
}
@@ -1,24 +0,0 @@
# This script validates NuGet package metadata information using this
# tool: https://github.com/NuGet/NuGetGallery/tree/jver-verify/src/VerifyMicrosoftPackage
param(
[Parameter(Mandatory=$true)][string] $PackagesPath, # Path to where the packages to be validated are
[Parameter(Mandatory=$true)][string] $ToolDestinationPath # Where the validation tool should be downloaded to
)
try {
. $PSScriptRoot\post-build-utils.ps1
$url = 'https://raw.githubusercontent.com/NuGet/NuGetGallery/3e25ad135146676bcab0050a516939d9958bfa5d/src/VerifyMicrosoftPackage/verify.ps1'
New-Item -ItemType 'directory' -Path ${ToolDestinationPath} -Force
Invoke-WebRequest $url -OutFile ${ToolDestinationPath}\verify.ps1
& ${ToolDestinationPath}\verify.ps1 ${PackagesPath}\*.nupkg
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NuGetValidation' -Message $_
ExitWithExitCode 1
}
@@ -1,91 +0,0 @@
# Most of the functions in this file require the variables `MaestroApiEndPoint`,
# `MaestroApiVersion` and `MaestroApiAccessToken` to be globally available.
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
. $PSScriptRoot\..\tools.ps1
function Create-MaestroApiRequestHeaders([string]$ContentType = 'application/json') {
Validate-MaestroVars
$headers = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$headers.Add('Accept', $ContentType)
$headers.Add('Authorization',"Bearer $MaestroApiAccessToken")
return $headers
}
function Get-MaestroChannel([int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}?api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroBuild([int]$BuildId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/builds/${BuildId}?api-version=$MaestroApiVersion"
$result = try { return Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroSubscriptions([string]$SourceRepository, [int]$ChannelId) {
Validate-MaestroVars
$SourceRepository = [System.Web.HttpUtility]::UrlEncode($SourceRepository)
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions?sourceRepository=$SourceRepository&channelId=$ChannelId&api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}/builds/${BuildId}?api-version=$MaestroApiVersion"
Invoke-WebRequest -Method Post -Uri $apiEndpoint -Headers $apiHeaders | Out-Null
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Validate-MaestroVars {
try {
Get-Variable MaestroApiEndPoint -Scope Global | Out-Null
Get-Variable MaestroApiVersion -Scope Global | Out-Null
Get-Variable MaestroApiAccessToken -Scope Global | Out-Null
if (!($MaestroApiEndPoint -Match '^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
ExitWithExitCode 1
}
if (!($MaestroApiVersion -Match '^[0-9]{4}-[0-9]{2}-[0-9]{2}$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
ExitWithExitCode 1
}
}
catch {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message 'Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script.'
Write-Host $_
ExitWithExitCode 1
}
}
@@ -1,257 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$false)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$false)][string] $GHCommit, # GitHub commit SHA used to build the packages
[Parameter(Mandatory=$true)][string] $SourcelinkCliVersion # Version of SourceLink CLI to use
)
. $PSScriptRoot\post-build-utils.ps1
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
# Maximum number of jobs to run in parallel
$MaxParallelJobs = 6
# Wait time between check for system load
$SecondsBetweenLoadChecks = 10
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
. $using:PSScriptRoot\..\tools.ps1
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-Host "Input file does not exist: $PackagePath"
return 1
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Validating ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath) | Out-Null
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith('.resources.dll')) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
$sourcelinkExe = "$env:USERPROFILE\.dotnet\tools"
$sourcelinkExe = Resolve-Path "$sourcelinkExe\sourcelink.exe"
$SourceLinkInfos = & $sourcelinkExe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = "https://raw.githubusercontent.com/${using:GHRepoName}/${using:GHCommit}/"
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match 'github' -or $Uri.Host -match 'githubusercontent')) {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
write-host $_
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
}
catch {
}
finally {
$zip.Dispose()
}
if ($FailedFiles -eq 0) {
Write-Host 'Passed.'
return 0
}
else {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$PackagePath has broken SourceLink links."
return 1
}
}
function ValidateSourceLinkLinks {
if ($GHRepoName -ne '' -and !($GHRepoName -Match '^[^\s\/]+/[^\s\/]+$')) {
if (!($GHRepoName -Match '^[^\s-]+-[^\s]+$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
ExitWithExitCode 1
}
else {
$GHRepoName = $GHRepoName -replace '^([^\s-]+)-([^\s]+)$', '$1/$2';
}
}
if ($GHCommit -ne '' -and !($GHCommit -Match '^[0-9a-fA-F]{40}$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
ExitWithExitCode 1
}
if ($GHRepoName -ne '' -and $GHCommit -ne '') {
$RepoTreeURL = -Join('http://api.github.com/repos/', $GHRepoName, '/git/trees/', $GHCommit, '?recursive=1')
$CodeExtensions = @('.cs', '.vb', '.fs', '.fsi', '.fsx', '.fsscript')
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL -UseBasicParsing | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL . Execution will proceed without caching."
}
}
elseif ($GHRepoName -ne '' -or $GHCommit -ne '') {
Write-Host 'For using the http caching mechanism both GHRepoName and GHCommit should be informed.'
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName | Out-Null
$NumJobs = @(Get-Job -State 'Running').Count
while ($NumJobs -ge $MaxParallelJobs) {
Write-Host "There are $NumJobs validation jobs running right now. Waiting $SecondsBetweenLoadChecks seconds to check again."
sleep $SecondsBetweenLoadChecks
$NumJobs = @(Get-Job -State 'Running').Count
}
foreach ($Job in @(Get-Job -State 'Completed')) {
Receive-Job -Id $Job.Id
Remove-Job -Id $Job.Id
}
}
$ValidationFailures = 0
foreach ($Job in @(Get-Job)) {
$jobResult = Wait-Job -Id $Job.Id | Receive-Job
if ($jobResult -ne '0') {
$ValidationFailures++
}
}
if ($ValidationFailures -gt 0) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$ValidationFailures package(s) failed validation."
ExitWithExitCode 1
}
}
function InstallSourcelinkCli {
$sourcelinkCliPackageName = 'sourcelink'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$sourcelinkCliPackageName*") -and ($toolList -like "*$sourcelinkCliVersion*")) {
Write-Host "SourceLink CLI version $sourcelinkCliVersion is already installed."
}
else {
Write-Host "Installing SourceLink CLI version $sourcelinkCliVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $sourcelinkCliPackageName --version $sourcelinkCliVersion --verbosity "minimal" --global
}
}
try {
InstallSourcelinkCli
ValidateSourceLinkLinks
}
catch {
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'SourceLink' -Message $_
ExitWithExitCode 1
}
@@ -1,64 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $SourceRepo,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
if (!$subscriptions) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: ${$subscription.id}"
[void]$subscriptionsToTrigger.Add($subscription.id)
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host 'done.'
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message 'At least one subscription failed to be triggered...'
ExitWithExitCode 1
}
else {
Write-Host 'All subscriptions were triggered successfully!'
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message $_
ExitWithExitCode 1
}
+10 -9
View File
@@ -1,8 +1,8 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $configuration = 'Debug',
[string] $configuration = "Debug",
[string] $task,
[string] $verbosity = 'minimal',
[string] $verbosity = "minimal",
[string] $msbuildEngine = $null,
[switch] $restore,
[switch] $prepareMachine,
@@ -32,7 +32,7 @@ function Print-Usage() {
}
function Build([string]$target) {
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$logSuffix = if ($target -eq "Execute") { "" } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
@@ -46,32 +46,33 @@ function Build([string]$target) {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
exit 0
}
if ($task -eq "") {
Write-PipelineTelemetryError -Category 'Build' -Message "Missing required parameter '-task <value>'" -ForegroundColor Red
Write-Host "Missing required parameter '-task <value>'" -ForegroundColor Red
Print-Usage
ExitWithExitCode 1
}
$taskProject = GetSdkTaskProject $task
if (!(Test-Path $taskProject)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Unknown task: $task" -ForegroundColor Red
Write-Host "Unknown task: $task" -ForegroundColor Red
ExitWithExitCode 1
}
if ($restore) {
Build 'Restore'
Build "Restore"
}
Build 'Execute'
Build "Execute"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
-13
View File
@@ -1,13 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
<packageSources>
<clear />
<add key="guardian" value="https://securitytools.pkgs.visualstudio.com/_packaging/Guardian/nuget/v3/index.json" />
</packageSources>
<disabledPackageSources>
<clear />
</disabledPackageSources>
</configuration>
-114
View File
@@ -1,114 +0,0 @@
Param(
[string] $GuardianPackageName, # Required: the name of guardian CLI package (not needed if GuardianCliLocation is specified)
[string] $NugetPackageDirectory, # Required: directory where NuGet packages are installed (not needed if GuardianCliLocation is specified)
[string] $GuardianCliLocation, # Optional: Direct location of Guardian CLI executable if GuardianPackageName & NugetPackageDirectory are not specified
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ('artifacts')), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
[bool] $TsaPublish=$False, # Optional: true will publish results to TSA; only set to true after onboarding to TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaBranchName=$env:BUILD_SOURCEBRANCH, # Optional: required for TSA publish; defaults to $(Build.SourceBranchName); TSA is the automated framework used to upload test results as bugs.
[string] $TsaRepositoryName=$env:BUILD_REPOSITORY_NAME, # Optional: TSA repository name; will be generated automatically if not submitted; TSA is the automated framework used to upload test results as bugs.
[string] $BuildNumber=$env:BUILD_BUILDNUMBER, # Optional: required for TSA publish; defaults to $(Build.BuildNumber)
[bool] $UpdateBaseline=$False, # Optional: if true, will update the baseline in the repository; should only be run after fixing any issues which need to be fixed
[bool] $TsaOnboard=$False, # Optional: if true, will onboard the repository to TSA; should only be run once; TSA is the automated framework used to upload test results as bugs.
[string] $TsaInstanceUrl, # Optional: only needed if TsaOnboard or TsaPublish is true; the instance-url registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the codebase registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaProjectName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the project registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaNotificationEmail, # Optional: only needed if TsaOnboard is true; the email(s) which will receive notifications of TSA bug filings (e.g. [email protected]); TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel='Standard', # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
)
try {
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path 'tools' 'guardian.cmd'))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Invalid Guardian CLI Location.'
ExitWithExitCode 1
}
& $(Join-Path $PSScriptRoot 'init-sdl.ps1') -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory '.gdn'
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not onboard to TSA -- not all required values ($TsaCodebaseName, $TsaNotificationEmail, $TsaCodebaseAdmin, $TsaBugAreaPath) were specified.'
ExitWithExitCode 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Update baseline'
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-publish failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not publish to TSA -- not all required values ($TsaBranchName, $BuildNumber) were specified.'
ExitWithExitCode 1
}
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
exit 1
}
@@ -1,80 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where artifact packages are stored
[Parameter(Mandatory=$true)][string] $ExtractPath # Full path to directory where the packages will be extracted
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
function ExtractArtifacts {
if (!(Test-Path $InputPath)) {
Write-Host "Input Path does not exist: $InputPath"
ExitWithExitCode 0
}
$Jobs = @()
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ExtractPackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Extracting ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
finally {
$zip.Dispose()
}
}
Measure-Command { ExtractArtifacts }
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
-67
View File
@@ -1,67 +0,0 @@
Param(
[string] $GuardianCliLocation,
[string] $Repository,
[string] $BranchName='master',
[string] $WorkingDirectory,
[string] $AzureDevOpsAccessToken,
[string] $GuardianLoggerLevel='Standard'
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
# Construct basic auth from AzDO access token; construct URI to the repository's gdn folder stored in that repository; construct location of zip file
$encodedPat = [Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$AzureDevOpsAccessToken"))
$escapedRepository = [Uri]::EscapeDataString("/$Repository/$BranchName/.gdn")
$uri = "https://dev.azure.com/dnceng/internal/_apis/git/repositories/sdl-tool-cfg/Items?path=$escapedRepository&versionDescriptor[versionOptions]=0&`$format=zip&api-version=5.0-preview.1"
$zipFile = "$WorkingDirectory/gdn.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
$gdnFolder = (Join-Path $WorkingDirectory '.gdn')
try {
# We try to download the zip; if the request fails (e.g. the file doesn't exist), we catch it and init guardian instead
Write-Host 'Downloading gdn folder from internal config repostiory...'
Invoke-WebRequest -Headers @{ "Accept"="application/zip"; "Authorization"="Basic $encodedPat" } -Uri $uri -OutFile $zipFile
if (Test-Path $gdnFolder) {
# Remove the gdn folder if it exists (it shouldn't unless there's too much caching; this is just in case)
Remove-Item -Force -Recurse $gdnFolder
}
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipFile, $WorkingDirectory)
Write-Host $gdnFolder
ExitWithExitCode 0
} catch [System.Net.WebException] { } # Catch and ignore webexception
try {
# if the folder does not exist, we'll do a guardian init and push it to the remote repository
Write-Host 'Initializing Guardian...'
Write-Host "$GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel"
& $GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian init failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We create the mainbaseline so it can be edited later
Write-Host "$GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline"
& $GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian baseline failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
& $(Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Initialize gdn folder'
ExitWithExitCode 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
-4
View File
@@ -1,4 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Guardian.Cli" version="0.7.2"/>
</packages>
-69
View File
@@ -1,69 +0,0 @@
Param(
[string] $Repository,
[string] $BranchName='master',
[string] $GdnFolder,
[string] $AzureDevOpsAccessToken,
[string] $PushReason
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP 'sdl'
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git clone failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) '.gdn'
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host 'git add .'
git add .
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git add failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git commit failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host 'git push'
git push
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git push failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# Return to the original directory
Pop-Location
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
-73
View File
@@ -1,73 +0,0 @@
Param(
[string] $GuardianCliLocation,
[string] $WorkingDirectory,
[string] $TargetDirectory,
[string] $GdnFolder,
[string[]] $ToolsList,
[string] $UpdateBaseline,
[string] $GuardianLoggerLevel='Standard',
[string[]] $CrScanAdditionalRunConfigParams,
[string[]] $PoliCheckAdditionalRunConfigParams
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
. $PSScriptRoot\..\tools.ps1
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder 'r'
$ValidPath = Test-Path $GuardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Invalid Guardian CLI Location."
ExitWithExitCode 1
}
$configParam = @('--config')
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq 'credscan') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
if ($tool -eq 'policheck') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
$configParam+=$gdnConfigFile
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
-89
View File
@@ -1,89 +0,0 @@
parameters:
enable: 'false' # Whether the SDL validation job should execute or not
overrideParameters: '' # Optional: to override values for parameters.
additionalParameters: '' # Optional: parameters that need user specific values eg: '-SourceToolsList @("abc","def") -ArtifactToolsList @("ghi","jkl")'
# There is some sort of bug (has been reported) in Azure DevOps where if this parameter is named
# 'continueOnError', the parameter value is not correctly picked up.
# This can also be remedied by the caller (post-build.yml) if it does not use a nested parameter
sdlContinueOnError: false # optional: determines whether to continue the build if the step errors;
dependsOn: '' # Optional: dependencies of the job
artifactNames: '' # Optional: patterns supplied to DownloadBuildArtifacts
# Usage:
# artifactNames:
# - 'BlobArtifacts'
# - 'Artifacts_Windows_NT_Release'
jobs:
- job: Run_SDL
dependsOn: ${{ parameters.dependsOn }}
displayName: Run SDL tool
condition: eq( ${{ parameters.enable }}, 'true')
variables:
- group: DotNet-VSTS-Bot
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
name: Hosted VS2017
steps:
- checkout: self
clean: true
- ${{ if ne(parameters.artifactNames, '') }}:
- ${{ each artifactName in parameters.artifactNames }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: ${{ artifactName }}
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if eq(parameters.artifactNames, '') }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: specific files
itemPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
displayName: Extract Blob Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
displayName: Extract Package Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
- task: NuGetCommand@2
displayName: 'Install Guardian'
inputs:
restoreSolution: $(Build.SourcesDirectory)\eng\common\sdl\packages.config
feedsToUse: config
nugetConfigPath: $(Build.SourcesDirectory)\eng\common\sdl\NuGet.config
externalFeedCredentials: GuardianConnect
restoreDirectory: $(Build.SourcesDirectory)\.packages
- ${{ if ne(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1 ${{ parameters.overrideParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
- ${{ if eq(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1
-GuardianPackageName Microsoft.Guardian.Cli.0.7.2
-NugetPackageDirectory $(Build.SourcesDirectory)\.packages
-AzureDevOpsAccessToken $(dn-bot-dotnet-build-rw-code-rw)
${{ parameters.additionalParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
+64 -79
View File
@@ -1,35 +1,64 @@
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
parameters:
# Job schema parameters - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
cancelTimeoutInMinutes: ''
condition: ''
container: ''
continueOnError: false
container: ''
dependsOn: ''
displayName: ''
pool: ''
steps: []
pool: ''
strategy: ''
timeoutInMinutes: ''
variables: []
workspace: ''
# Job base template specific parameters
# See schema documentation - https://github.com/dotnet/arcade/blob/master/Documentation/AzureDevOps/TemplateSchema.md
artifacts: ''
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
enablePublishUsingPipelines: false
mergeTestResults: false
testRunTitle: $(AgentOsName)-$(BuildConfiguration)-xunit
# Optional: enable sending telemetry
enableTelemetry: false
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: define the helix type for telemetry (example: 'build/product/')
helixType: ''
# Required: name of the job
name: ''
preSteps: []
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- job: ${{ parameters.name }}
@@ -61,7 +90,7 @@ jobs:
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ if ne(parameters.enableTelemetry, 'false') }}:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- ${{ each variable in parameters.variables }}:
@@ -93,12 +122,21 @@ jobs:
workspace: ${{ parameters.workspace }}
steps:
- ${{ if ne(parameters.preSteps, '') }}:
- ${{ each preStep in parameters.preSteps }}:
- ${{ preStep }}
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendStartTelemetry@0
displayName: 'Send Helix Start Telemetry'
inputs:
helixRepo: ${{ parameters.helixRepo }}
${{ if ne(parameters.helixType, '') }}:
helixType: ${{ parameters.helixType }}
buildConfig: $(_BuildConfig)
runAsPublic: ${{ parameters.runAsPublic }}
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
@@ -110,16 +148,6 @@ jobs:
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- task: NuGetAuthenticate@0
- ${{ if or(eq(parameters.artifacts.download, 'true'), ne(parameters.artifacts.download, '')) }}:
- task: DownloadPipelineArtifact@2
inputs:
buildType: current
artifactName: ${{ coalesce(parameters.artifacts.download.name, 'Artifacts_$(Agent.OS)_$(_BuildConfig)') }}
targetPath: ${{ coalesce(parameters.artifacts.download.path, 'artifacts') }}
itemPattern: ${{ coalesce(parameters.artifacts.download.pattern, '**') }}
- ${{ each step in parameters.steps }}:
- ${{ step }}
@@ -132,60 +160,20 @@ jobs:
env:
TeamName: $(_TeamName)
- ${{ if ne(parameters.artifacts.publish, '') }}:
- ${{ if or(eq(parameters.artifacts.publish.artifacts, 'true'), ne(parameters.artifacts.publish.artifacts, '')) }}:
- task: CopyFiles@2
displayName: Gather binaries for publish to artifacts
inputs:
SourceFolder: 'artifacts/bin'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/bin'
- task: CopyFiles@2
displayName: Gather packages for publish to artifacts
inputs:
SourceFolder: 'artifacts/packages'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/packages'
- task: PublishBuildArtifacts@1
displayName: Publish pipeline artifacts
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.logs, 'true'), ne(parameters.artifacts.publish.logs, '')) }}:
- publish: artifacts/log
artifact: ${{ coalesce(parameters.artifacts.publish.logs.name, 'Logs_Build_$(Agent.Os)_$(_BuildConfig)') }}
displayName: Publish logs
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- ${{ if and(ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.ArtifactStagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendEndTelemetry@0
displayName: 'Send Helix End Telemetry'
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if ne(parameters.enablePublishBuildArtifacts, 'false') }}:
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
ArtifactName: $(Agent.Os)_$(Agent.JobName)
continueOnError: true
condition: always()
@@ -196,12 +184,10 @@ jobs:
testResultsFormat: 'xUnit'
testResultsFiles: '*.xml'
searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults/$(_BuildConfig)'
testRunTitle: ${{ parameters.testRunTitle }}
mergeTestResults: ${{ parameters.mergeTestResults }}
continueOnError: true
condition: always()
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(variables['_PublishUsingPipelines'], 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
@@ -209,7 +195,6 @@ jobs:
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
-95
View File
@@ -1,95 +0,0 @@
parameters:
steps: [] # optional -- any additional steps that need to happen before pulling down the performance repo and sending the performance benchmarks to helix (ie building your repo)
variables: [] # optional -- list of additional variables to send to the template
jobName: '' # required -- job name
displayName: '' # optional -- display name for the job. Will use jobName if not passed
pool: '' # required -- name of the Build pool
container: '' # required -- name of the container
osGroup: '' # required -- operating system for the job
extraSetupParameters: '' # optional -- extra arguments to pass to the setup script
frameworks: ['netcoreapp3.0'] # optional -- list of frameworks to run against
continueOnError: 'false' # optional -- determines whether to continue the build if the step errors
dependsOn: '' # optional -- dependencies of the job
timeoutInMinutes: 320 # optional -- timeout for the job
enableTelemetry: false # optional -- enable for telemetry
jobs:
- template: ../jobs/jobs.yml
parameters:
dependsOn: ${{ parameters.dependsOn }}
enableTelemetry: ${{ parameters.enableTelemetry }}
enablePublishBuildArtifacts: true
continueOnError: ${{ parameters.continueOnError }}
jobs:
- job: '${{ parameters.jobName }}'
${{ if ne(parameters.displayName, '') }}:
displayName: '${{ parameters.displayName }}'
${{ if eq(parameters.displayName, '') }}:
displayName: '${{ parameters.jobName }}'
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ each variable in parameters.variables }}:
- ${{ if ne(variable.name, '') }}:
- name: ${{ variable.name }}
value: ${{ variable.value }}
- ${{ if ne(variable.group, '') }}:
- group: ${{ variable.group }}
- IsInternal: ''
- HelixApiAccessToken: ''
- HelixPreCommand: ''
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq( parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'set "PERFLAB_UPLOAD_TOKEN=$(PerfCommandUploadToken)"'
- IsInternal: -Internal
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'export PERFLAB_UPLOAD_TOKEN="$(PerfCommandUploadTokenLinux)"'
- IsInternal: --internal
- group: DotNet-HelixApi-Access
- group: dotnet-benchview
workspace:
clean: all
pool:
${{ parameters.pool }}
container: ${{ parameters.container }}
strategy:
matrix:
${{ each framework in parameters.frameworks }}:
${{ framework }}:
_Framework: ${{ framework }}
steps:
- checkout: self
clean: true
# Run all of the steps to setup repo
- ${{ each step in parameters.steps }}:
- ${{ step }}
- powershell: $(Build.SourcesDirectory)\eng\common\performance\performance-setup.ps1 $(IsInternal) -Framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Windows)
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Build.SourcesDirectory)/eng/common/performance/performance-setup.sh $(IsInternal) --framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Unix)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments)
displayName: Run ci setup script
# Run perf testing in helix
- template: /eng/common/templates/steps/perf-send-to-helix.yml
parameters:
HelixSource: '$(HelixSourcePrefix)/$(Build.Repository.Name)/$(Build.SourceBranch)' # sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'test/performance/$(Kind)/$(_Framework)/$(Architecture)'
HelixAccessToken: $(HelixApiAccessToken)
HelixTargetQueues: $(Queue)
HelixPreCommands: $(HelixPreCommand)
Creator: $(Creator)
WorkItemTimeout: 4:00 # 4 hours
WorkItemDirectory: '$(WorkItemDirectory)' # WorkItemDirectory can not be empty, so we send it some docs to keep it happy
CorrelationPayloadDirectory: '$(PayloadDirectory)' # it gets checked out to a folder with shorter path than WorkItemDirectory so we can avoid file name too long exceptions
@@ -37,12 +37,6 @@ jobs:
- name: _BuildConfig
value: ${{ parameters.configuration }}
- group: Publish-Build-Assets
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
@@ -53,10 +47,6 @@ jobs:
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: NuGetAuthenticate@0
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
@@ -69,23 +59,6 @@ jobs:
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: powershell@2
displayName: Create ReleaseConfigs Artifact
inputs:
targetType: inline
script: |
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(BARBuildId)
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value "$(DefaultChannels)"
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(IsStableBuild)
- task: PublishBuildArtifacts@1
displayName: Publish ReleaseConfigs Artifact
inputs:
PathtoPublish: '$(Build.StagingDirectory)/ReleaseConfigs.txt'
PublishLocation: Container
ArtifactName: ReleaseConfigs
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
+46 -28
View File
@@ -1,10 +1,19 @@
parameters:
# See schema documentation in /Documentation/AzureDevOps/TemplateSchema.md
# Optional: 'true' if failures in job.yml job should not fail the job
continueOnError: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Enable publishing using release pipelines
enablePublishUsingPipelines: false
@@ -14,9 +23,19 @@ parameters:
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
# if enabled then the 'helixRepo' parameter should also be specified
enableTelemetry: false
# Required: A collection of jobs to run - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
jobs: []
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: Override automatically derived dependsOn value for "publish build assets" job
publishBuildAssetsDependsOn: ''
@@ -43,30 +62,29 @@ jobs:
name: ${{ job.job }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if eq(parameters.graphFileGeneration.enabled, true) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if and(eq(parameters.graphFileGeneration.enabled, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
@@ -1,172 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
/p:PublishToMSDL=false
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=true
/p:ChecksumsTargetStaticFeed=$(InternalChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(InternalChecksumsBlobFeedKey)
/p:InstallersTargetStaticFeed=$(InternalInstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(InternalInstallersBlobFeedKey)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:PublishToMSDL=false
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,178 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
# If the channel name is empty, no links will be generated
akaMSChannelName: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
- name: ArtifactsCategory
value: ${{ coalesce(variables._DotNetArtifactsCategory, '.NETCore') }}
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(ArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:LatestLinkShortUrlPrefix=dotnet/'${{ parameters.akaMSChannelName }}'
/p:AkaMSClientId=$(akams-client-id)
/p:AkaMSClientSecret=$(akams-client-secret)
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,99 +0,0 @@
variables:
- group: AzureDevOps-Artifact-Feeds-Pats
- group: DotNet-Blob-Feed
- group: DotNet-DotNetCli-Storage
- group: DotNet-MSRC-Storage
- group: Publish-Build-Assets
# .NET Core 3.1 Dev
- name: PublicDevRelease_31_Channel_Id
value: 128
# .NET 5 Dev
- name: Net_5_Dev_Channel_Id
value: 131
# .NET Eng - Validation
- name: Net_Eng_Validation_Channel_Id
value: 9
# .NET Eng - Latest
- name: Net_Eng_Latest_Channel_Id
value: 2
# .NET 3 Eng - Validation
- name: NET_3_Eng_Validation_Channel_Id
value: 390
# .NET 3 Eng
- name: NetCore_3_Tools_Channel_Id
value: 344
# .NET Core 3.0 Internal Servicing
- name: InternalServicing_30_Channel_Id
value: 184
# .NET Core 3.0 Release
- name: PublicRelease_30_Channel_Id
value: 19
# .NET Core 3.1 Release
- name: PublicRelease_31_Channel_Id
value: 129
# General Testing
- name: GeneralTesting_Channel_Id
value: 529
# .NET Core 3.1 Blazor Features
- name: NetCore_31_Blazor_Features_Channel_Id
value: 531
# .NET Core Experimental
- name: NetCore_Experimental_Channel_Id
value: 562
# Whether the build is internal or not
- name: IsInternalBuild
value: ${{ and(ne(variables['System.TeamProject'], 'public'), contains(variables['Build.SourceBranch'], 'internal')) }}
# Default Maestro++ API Endpoint and API Version
- name: MaestroApiEndPoint
value: "https://maestro-prod.westus2.cloudapp.azure.com"
- name: MaestroApiAccessToken
value: $(MaestroAccessToken)
- name: MaestroApiVersion
value: "2019-01-16"
- name: SourceLinkCLIVersion
value: 3.0.0
- name: SymbolToolVersion
value: 1.0.1
# Feed Configurations
# These should include the suffix "/index.json"
# Default locations for Installers and checksums
# Public Locations
- name: ChecksumsBlobFeedUrl
value: https://dotnetclichecksums.blob.core.windows.net/dotnet/index.json
- name: InstallersBlobFeedUrl
value: https://dotnetcli.blob.core.windows.net/dotnet/index.json
# Private Locations
- name: InternalChecksumsBlobFeedUrl
value: https://dotnetclichecksumsmsrc.blob.core.windows.net/dotnet/index.json
- name: InternalChecksumsBlobFeedKey
value: $(dotnetclichecksumsmsrc-storage-key)
- name: InternalInstallersBlobFeedUrl
value: https://dotnetclimsrc.blob.core.windows.net/dotnet/index.json
- name: InternalInstallersBlobFeedKey
value: $(dotnetclimsrc-access-key)
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
@@ -1,382 +0,0 @@
parameters:
enableSourceLinkValidation: false
enableSigningValidation: true
enableSymbolValidation: false
enableNugetValidation: true
publishInstallersAndChecksums: false
SDLValidationParameters:
enable: false
continueOnError: false
params: ''
artifactNames: ''
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
symbolPublishingAdditionalParameters: ''
artifactsPublishingAdditionalParameters: ''
signingValidationAdditionalParameters: ''
# Which stages should finish execution before post-build stages start
validateDependsOn:
- build
publishDependsOn:
- Validate
# Channel ID's instantiated in this file.
# When adding a new channel implementation the call to `check-channel-consistency.ps1`
# needs to be updated with the new channel ID
NetEngLatestChannelId: 2
NetEngValidationChannelId: 9
NetDev5ChannelId: 131
GeneralTestingChannelId: 529
NETCoreToolingDevChannelId: 548
NETCoreToolingReleaseChannelId: 549
NETInternalToolingChannelId: 551
NETCoreExperimentalChannelId: 562
NetEngServicesIntChannelId: 678
NetEngServicesProdChannelId: 679
Net5Preview2ChannelId: 738
Net5Preview3ChannelId: 739
NetCoreSDK313xxChannelId: 759
NetCoreSDK313xxInternalChannelId: 760
stages:
- stage: Validate
dependsOn: ${{ parameters.validateDependsOn }}
displayName: Validate
variables:
- template: common-variables.yml
jobs:
- template: setup-maestro-vars.yml
- job:
displayName: Post-build Checks
dependsOn: setupMaestroVars
variables:
- name: InitialChannels
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'] ]
- name: PromoteToMaestroChannelId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Maestro Channels Consistency
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/check-channel-consistency.ps1
arguments: -PromoteToChannels "$(InitialChannels)[$(PromoteToMaestroChannelId)]"
-AvailableChannelIds ${{parameters.NetEngLatestChannelId}},${{parameters.NetEngValidationChannelId}},${{parameters.NetDev5ChannelId}},${{parameters.GeneralTestingChannelId}},${{parameters.NETCoreToolingDevChannelId}},${{parameters.NETCoreToolingReleaseChannelId}},${{parameters.NETInternalToolingChannelId}},${{parameters.NETCoreExperimentalChannelId}},${{parameters.NetEngServicesIntChannelId}},${{parameters.NetEngServicesProdChannelId}},${{parameters.Net5Preview2ChannelId}},${{parameters.Net5Preview3ChannelId}},${{parameters.NetCoreSDK313xxChannelId}},${{parameters.NetCoreSDK313xxInternalChannelId}}
- job:
displayName: NuGet Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableNugetValidation }}, 'true')
pool:
vmImage: 'windows-2019'
variables:
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- job:
displayName: Signing Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSigningValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
${{ parameters.signingValidationAdditionalParameters }}
- template: ../steps/publish-logs.yml
parameters:
StageLabel: 'Validation'
JobLabel: 'Signing'
- job:
displayName: SourceLink Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSourceLinkValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
enable: ${{ parameters.SDLValidationParameters.enable }}
dependsOn: setupMaestroVars
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
artifactNames: ${{ parameters.SDLValidationParameters.artifactNames }}
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev5_Publish'
channelName: '.NET 5 Dev'
akaMSChannelName: 'net5/dev'
channelId: ${{ parameters.NetDev5ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview2_Publish'
channelName: '.NET 5 Preview 2'
akaMSChannelName: 'net5/preview2'
channelId: ${{ parameters.Net5Preview2ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview3_Publish'
channelName: '.NET 5 Preview 3'
akaMSChannelName: 'net5/preview3'
channelId: ${{ parameters.Net5Preview3ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Latest_Publish'
channelName: '.NET Eng - Latest'
akaMSChannelName: 'eng/daily'
channelId: ${{ parameters.NetEngLatestChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Validation_Publish'
channelName: '.NET Eng - Validation'
akaMSChannelName: 'eng/validation'
channelId: ${{ parameters.NetEngValidationChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'General_Testing_Publish'
channelName: 'General Testing'
akaMSChannelName: 'generaltesting'
channelId: ${{ parameters.GeneralTestingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Dev_Publishing'
channelName: '.NET Core Tooling Dev'
channelId: ${{ parameters.NETCoreToolingDevChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Release_Publishing'
channelName: '.NET Core Tooling Release'
channelId: ${{ parameters.NETCoreToolingReleaseChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NET_Internal_Tooling_Publishing'
channelName: '.NET Internal Tooling'
channelId: ${{ parameters.NETInternalToolingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Experimental_Publishing'
channelName: '.NET Core Experimental'
channelId: ${{ parameters.NETCoreExperimentalChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Int_Publish'
channelName: '.NET Eng Services - Int'
channelId: ${{ parameters.NetEngServicesIntChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Prod_Publish'
channelName: '.NET Eng Services - Prod'
channelId: ${{ parameters.NetEngServicesProdChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_SDK_313xx_Publishing'
channelName: '.NET Core SDK 3.1.3xx'
channelId: ${{ parameters.NetCoreSDK313xxChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_SDK_313xx_Internal_Publishing'
channelName: '.NET Core SDK 3.1.3xx Internal'
channelId: ${{ parameters.NetCoreSDK313xxInternalChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-symbols/nuget/v3/index.json'
@@ -1,81 +0,0 @@
jobs:
- job: setupMaestroVars
displayName: Setup Maestro Vars
variables:
- template: common-variables.yml
- name: BuildId
value: $[ coalesce(variables.BARBuildId, 0) ]
- name: PromoteToChannelId
value: $[ coalesce(variables.PromoteToMaestroChannelId, 0) ]
pool:
vmImage: 'windows-2019'
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
displayName: Download Release Configs
condition: eq(variables.PromoteToChannelId, 0)
inputs:
buildType: current
artifactName: ReleaseConfigs
- task: PowerShell@2
name: setReleaseVars
displayName: Set Release Configs Vars
inputs:
targetType: inline
script: |
try {
if ($Env:PromoteToChannelId -eq 0) {
$Content = Get-Content $(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
$AzureDevOpsProject = $Env:System_TeamProject
$AzureDevOpsBuildDefinitionId = $Env:System_DefinitionId
$AzureDevOpsBuildId = $Env:Build_BuildId
$PromoteToMaestroChannelId = 0
}
else {
$buildApiEndpoint = "${Env:MaestroApiEndPoint}/api/builds/${Env:BARBuildId}?api-version=${Env:MaestroApiVersion}"
$apiHeaders = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$apiHeaders.Add('Accept', 'application/json')
$apiHeaders.Add('Authorization',"Bearer ${Env:MAESTRO_API_TOKEN}")
$buildInfo = try { Invoke-WebRequest -Method Get -Uri $buildApiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
$BarId = $Env:BARBuildId
$Channels = 'None'
#TODO: Fix this once this issue is done: https://github.com/dotnet/arcade/issues/3834
$IsStableBuild = 'False'
$AzureDevOpsProject = $buildInfo.azureDevOpsProject
$AzureDevOpsBuildDefinitionId = $buildInfo.azureDevOpsBuildDefinitionId
$AzureDevOpsBuildId = $buildInfo.azureDevOpsBuildId
$PromoteToMaestroChannelId = $Env:PromoteToMaestroChannelId
}
Write-Host "##vso[task.setvariable variable=BARBuildId;isOutput=true]$BarId"
Write-Host "##vso[task.setvariable variable=InitialChannels;isOutput=true]$Channels"
Write-Host "##vso[task.setvariable variable=IsStableBuild;isOutput=true]$IsStableBuild"
Write-Host "##vso[task.setvariable variable=AzDOProjectName;isOutput=true]$AzureDevOpsProject"
Write-Host "##vso[task.setvariable variable=AzDOPipelineId;isOutput=true]$AzureDevOpsBuildDefinitionId"
Write-Host "##vso[task.setvariable variable=AzDOBuildId;isOutput=true]$AzureDevOpsBuildId"
Write-Host "##vso[task.setvariable variable=PromoteToMaestroChannelId;isOutput=true]$PromoteToMaestroChannelId"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
exit 1
}
env:
MAESTRO_API_TOKEN: $(MaestroApiAccessToken)
@@ -1,13 +0,0 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Triggering subscriptions
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/trigger-subscriptions.ps1
arguments: -SourceRepo $(Build.Repository.Uri)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,13 +0,0 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/add-build-to-channel.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,66 +0,0 @@
# Please remember to update the documentation if you make changes to these parameters!
parameters:
HelixSource: 'pr/default' # required -- sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'tests/default/' # required -- Helix telemetry which identifies what type of data this is; should include "test" for clarity and must end in '/'
HelixBuild: $(Build.BuildNumber) # required -- the build number Helix will use to identify this -- automatically set to the AzDO build number
HelixTargetQueues: '' # required -- semicolon delimited list of Helix queues to test on; see https://helix.dot.net/ for a list of queues
HelixAccessToken: '' # required -- access token to make Helix API requests; should be provided by the appropriate variable group
HelixPreCommands: '' # optional -- commands to run before Helix work item execution
HelixPostCommands: '' # optional -- commands to run after Helix work item execution
WorkItemDirectory: '' # optional -- a payload directory to zip up and send to Helix; requires WorkItemCommand; incompatible with XUnitProjects
CorrelationPayloadDirectory: '' # optional -- a directory to zip up and send to Helix as a correlation payload
IncludeDotNetCli: false # optional -- true will download a version of the .NET CLI onto the Helix machine as a correlation payload; requires DotNetCliPackageType and DotNetCliVersion
DotNetCliPackageType: '' # optional -- either 'sdk' or 'runtime'; determines whether the sdk or runtime will be sent to Helix; see https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
DotNetCliVersion: '' # optional -- version of the CLI to send to Helix; based on this: https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Send job to Helix' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
continueOnError: false # optional -- determines whether to continue the build if the step errors; defaults to false
steps:
- powershell: $(Build.SourcesDirectory)\eng\common\msbuild.ps1 $(Build.SourcesDirectory)\eng\common\performance\perfhelixpublish.proj /restore /t:Test /bl:$(Build.SourcesDirectory)\artifacts\log\$env:BuildConfig\SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Windows)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $BUILD_SOURCESDIRECTORY/eng/common/msbuild.sh $BUILD_SOURCESDIRECTORY/eng/common/performance/perfhelixpublish.proj /restore /t:Test /bl:$BUILD_SOURCESDIRECTORY/artifacts/log/$BuildConfig/SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Unix)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
@@ -1,23 +0,0 @@
parameters:
StageLabel: ''
JobLabel: ''
steps:
- task: Powershell@2
displayName: Prepare Binlogs to Upload
inputs:
targetType: inline
script: |
New-Item -ItemType Directory $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
Move-Item -Path $(Build.SourcesDirectory)/artifacts/log/Debug/* $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
continueOnError: true
condition: always()
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/PostBuildLogs'
PublishLocation: Container
ArtifactName: PostBuildLogs
continueOnError: true
condition: always()
@@ -23,7 +23,6 @@ parameters:
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
HelixBaseUri: 'https://helix.dot.net/' # optional -- sets the Helix API base URI (allows targeting int)
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
@@ -56,7 +55,6 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
@@ -87,7 +85,6 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
+106 -229
View File
@@ -5,7 +5,7 @@
[bool]$ci = if (Test-Path variable:ci) { $ci } else { $false }
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { 'Debug' }
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { "Debug" }
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
@@ -24,7 +24,7 @@
[bool]$restore = if (Test-Path variable:restore) { $restore } else { $true }
# Adjusts msbuild verbosity level.
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { 'minimal' }
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { "minimal" }
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
[bool]$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { !$ci }
@@ -35,29 +35,23 @@
# Specifies which msbuild engine to use for build: 'vs', 'dotnet' or unspecified (determined based on presence of tools.vs in global.json).
[string]$msbuildEngine = if (Test-Path variable:msbuildEngine) { $msbuildEngine } else { $null }
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.ps1
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { 'v1' }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @('msbuild', 'dotnet', 'vbcscompiler') }
$disableConfigureToolsetImport = if (Test-Path variable:disableConfigureToolsetImport) { $disableConfigureToolsetImport } else { $null }
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
set-strictmode -version 2.0
$ErrorActionPreference = 'Stop'
$ErrorActionPreference = "Stop"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
function Create-Directory([string[]] $path) {
if (!(Test-Path $path)) {
New-Item -path $path -force -itemType 'Directory' | Out-Null
New-Item -path $path -force -itemType "Directory" | Out-Null
}
}
@@ -82,7 +76,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
$finished = $false
try {
while (-not $process.WaitForExit(100)) {
while (-not $process.WaitForExit(100)) {
# Non-blocking loop done to allow ctr-c interrupts
}
@@ -90,7 +84,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
return $global:LASTEXITCODE = $process.ExitCode
}
finally {
# If we didn't finish then an error occurred or the user hit ctrl-c. Either
# If we didn't finish then an error occured or the user hit ctrl-c. Either
# way kill the process
if (-not $finished) {
$process.Kill()
@@ -98,10 +92,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
}
}
# createSdkLocationFile parameter enables a file being generated under the toolset directory
# which writes the sdk's location into. This is only necessary for cmd --> powershell invocations
# as dot sourcing isn't possible.
function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
function InitializeDotNetCli([bool]$install) {
if (Test-Path variable:global:_DotNetInstallDir) {
return $global:_DotNetInstallDir
}
@@ -124,7 +115,7 @@ function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
# Find the first path on %PATH% that contains the dotnet.exe
if ($useInstalledDotNetCli -and (-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -eq $null)) {
$dotnetCmd = Get-Command 'dotnet.exe' -ErrorAction SilentlyContinue
$dotnetCmd = Get-Command "dotnet.exe" -ErrorAction SilentlyContinue
if ($dotnetCmd -ne $null) {
$env:DOTNET_INSTALL_DIR = Split-Path $dotnetCmd.Path -Parent
}
@@ -137,13 +128,13 @@ function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
if ((-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -ne $null) -and (Test-Path(Join-Path $env:DOTNET_INSTALL_DIR "sdk\$dotnetSdkVersion"))) {
$dotnetRoot = $env:DOTNET_INSTALL_DIR
} else {
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
if (-not (Test-Path(Join-Path $dotnetRoot "sdk\$dotnetSdkVersion"))) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
Write-Host "Unable to find dotnet with SDK version '$dotnetSdkVersion'" -ForegroundColor Red
ExitWithExitCode 1
}
}
@@ -151,87 +142,36 @@ function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
$env:DOTNET_INSTALL_DIR = $dotnetRoot
}
# Creates a temporary file under the toolset dir.
# The following code block is protecting against concurrent access so that this function can
# be called in parallel.
if ($createSdkLocationFile) {
do {
$sdkCacheFileTemp = Join-Path $ToolsetDir $([System.IO.Path]::GetRandomFileName())
}
until (!(Test-Path $sdkCacheFileTemp))
Set-Content -Path $sdkCacheFileTemp -Value $dotnetRoot
try {
Rename-Item -Force -Path $sdkCacheFileTemp 'sdk.txt'
} catch {
# Somebody beat us
Remove-Item -Path $sdkCacheFileTemp
}
}
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
# It also ensures that VS msbuild will use the downloaded sdk targets.
$env:PATH = "$dotnetRoot;$env:PATH"
# Make Sure that our bootstrapped dotnet cli is available in future steps of the Azure Pipelines build
Write-PipelinePrependPath -Path $dotnetRoot
Write-PipelineSetVariable -Name 'DOTNET_MULTILEVEL_LOOKUP' -Value '0'
Write-PipelineSetVariable -Name 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE' -Value '1'
if ($ci) {
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
Write-Host "##vso[task.prependpath]$dotnetRoot"
Write-Host "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
Write-Host "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
}
return $global:_DotNetInstallDir = $dotnetRoot
}
function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot 'dotnet-install.ps1'
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
$maxRetries = 5
$retries = 1
$uri = "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1"
while($true) {
try {
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
break
}
catch {
Write-Host "Failed to download '$uri'"
Write-Error $_.Exception.Message -ErrorAction Continue
}
if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
throw "Unable to download file in $maxRetries attempts."
}
}
Invoke-WebRequest "https://dot.net/v1/dotnet-install.ps1" -OutFile $installScript
}
return $installScript
}
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = '') {
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = "") {
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot,
[string] $version,
[string] $architecture = '',
[string] $runtime = '',
[bool] $skipNonVersionedFiles = $false,
[string] $runtimeSourceFeed = '',
[string] $runtimeSourceFeedKey = '') {
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) { $installScript = GetDotNetInstallScript $dotnetRoot
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
@@ -242,37 +182,15 @@ function InstallDotNet([string] $dotnetRoot,
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from public location."
# Only the runtime can be installed from a custom [private] location.
if ($runtime -and ($runtimeSourceFeed -or $runtimeSourceFeedKey)) {
if ($runtimeSourceFeed) { $installParameters.AzureFeed = $runtimeSourceFeed }
if ($runtimeSourceFeedKey) {
$decodedBytes = [System.Convert]::FromBase64String($runtimeSourceFeedKey)
$decodedString = [System.Text.Encoding]::UTF8.GetString($decodedBytes)
$installParameters.FeedCredential = $decodedString
}
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from custom location '$runtimeSourceFeed'."
ExitWithExitCode 1
}
} else {
ExitWithExitCode 1
}
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-Host "Failed to install dotnet cli (exit code '$lastExitCode')." -ForegroundColor Red
ExitWithExitCode $lastExitCode
}
}
#
# Locates Visual Studio MSBuild installation.
# Locates Visual Studio MSBuild installation.
# The preference order for MSBuild to use is as follows:
#
# 1. MSBuild from an active VS command prompt
@@ -288,16 +206,16 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { '15.9' }
$vsMinVersion = [Version]::new($vsMinVersionStr)
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
$msbuildCmd = Get-Command 'msbuild.exe' -ErrorAction SilentlyContinue
$msbuildCmd = Get-Command "msbuild.exe" -ErrorAction SilentlyContinue
if ($msbuildCmd -ne $null) {
# Workaround for https://github.com/dotnet/roslyn/issues/35793
# Due to this issue $msbuildCmd.Version returns 0.0.0.0 for msbuild.exe 16.2+
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split([char[]]@('-', '+'))[0])
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split(@('-', '+'))[0])
if ($msbuildVersion -ge $vsMinVersion) {
return $global:_MSBuildExe = $msbuildCmd.Path
@@ -317,7 +235,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
InitializeVisualStudioEnvironmentVariables $vsInstallDir $vsMajorVersion
} else {
if (Get-Member -InputObject $GlobalJson.tools -Name 'xcopy-msbuild') {
if (Get-Member -InputObject $GlobalJson.tools -Name "xcopy-msbuild") {
$xcopyMSBuildVersion = $GlobalJson.tools.'xcopy-msbuild'
$vsMajorVersion = $xcopyMSBuildVersion.Split('.')[0]
} else {
@@ -325,12 +243,9 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
$xcopyMSBuildVersion = "$vsMajorVersion.$($vsMinVersion.Minor).0-alpha"
}
$vsInstallDir = $null
if ($xcopyMSBuildVersion.Trim() -ine "none") {
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
}
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
if ($vsInstallDir -eq $null) {
throw 'Unable to find Visual Studio that has required version and components installed'
throw "Unable to find Visual Studio that has required version and components installed"
}
}
@@ -341,7 +256,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
function InitializeVisualStudioEnvironmentVariables([string] $vsInstallDir, [string] $vsMajorVersion) {
$env:VSINSTALLDIR = $vsInstallDir
Set-Item "env:VS$($vsMajorVersion)0COMNTOOLS" (Join-Path $vsInstallDir "Common7\Tools\")
$vsSdkInstallDir = Join-Path $vsInstallDir "VSSDK\"
if (Test-Path $vsSdkInstallDir) {
Set-Item "env:VSSDK$($vsMajorVersion)0Install" $vsSdkInstallDir
@@ -354,7 +269,7 @@ function InstallXCopyMSBuild([string]$packageVersion) {
}
function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
$packageName = 'RoslynTools.MSBuild'
$packageName = "RoslynTools.MSBuild"
$packageDir = Join-Path $ToolsDir "msbuild\$packageVersion"
$packagePath = Join-Path $packageDir "$packageName.$packageVersion.nupkg"
@@ -365,61 +280,55 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
Create-Directory $packageDir
Write-Host "Downloading $packageName $packageVersion"
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dotnet.myget.org/F/roslyn-tools/api/v2/package/$packageName/$packageVersion/" -OutFile $packagePath
Unzip $packagePath $packageDir
}
return Join-Path $packageDir 'tools'
return Join-Path $packageDir "tools"
}
#
# Locates Visual Studio instance that meets the minimal requirements specified by tools.vs object in global.json.
#
# The following properties of tools.vs are recognized:
# "version": "{major}.{minor}"
# "version": "{major}.{minor}"
# Two part minimal VS version, e.g. "15.9", "16.0", etc.
# "components": ["componentId1", "componentId2", ...]
# "components": ["componentId1", "componentId2", ...]
# Array of ids of workload components that must be available in the VS instance.
# See e.g. https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-enterprise?view=vs-2017
#
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
if (Get-Member -InputObject $GlobalJson.tools -Name 'vswhere') {
if (Get-Member -InputObject $GlobalJson.tools -Name "vswhere") {
$vswhereVersion = $GlobalJson.tools.vswhere
} else {
$vswhereVersion = '2.5.2'
$vswhereVersion = "2.5.2"
}
$vsWhereDir = Join-Path $ToolsDir "vswhere\$vswhereVersion"
$vsWhereExe = Join-Path $vsWhereDir 'vswhere.exe'
$vsWhereExe = Join-Path $vsWhereDir "vswhere.exe"
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host 'Downloading vswhere'
try {
Invoke-WebRequest "https://netcorenativeassets.blob.core.windows.net/resource-packages/external/windows/vswhere/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
}
Write-Host "Downloading vswhere"
Invoke-WebRequest "https://github.com/Microsoft/vswhere/releases/download/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @('-latest', '-prerelease', '-format', 'json', '-requires', 'Microsoft.Component.MSBuild', '-products', '*')
if (Get-Member -InputObject $vsRequirements -Name 'version') {
$args += '-version'
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild")
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
$args += $vsRequirements.version
}
if (Get-Member -InputObject $vsRequirements -Name 'components') {
if (Get-Member -InputObject $vsRequirements -Name "components") {
foreach ($component in $vsRequirements.components) {
$args += '-requires'
$args += "-requires"
$args += $component
}
}
}
$vsInfo =& $vsWhereExe $args | ConvertFrom-Json
@@ -443,27 +352,28 @@ function InitializeBuildTool() {
# Initialize dotnet cli if listed in 'tools'
$dotnetRoot = $null
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
$dotnetRoot = InitializeDotNetCli -install:$restore
}
if ($msbuildEngine -eq 'dotnet') {
if ($msbuildEngine -eq "dotnet") {
if (!$dotnetRoot) {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "/global.json must specify 'tools.dotnet'."
Write-Host "/global.json must specify 'tools.dotnet'." -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot 'dotnet.exe'; Command = 'msbuild'; Tool = 'dotnet'; Framework = 'netcoreapp2.1' }
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
Write-Host $_ -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
Write-Host "Unexpected value of -msbuildEngine: '$msbuildEngine'." -ForegroundColor Red
ExitWithExitCode 1
}
@@ -472,15 +382,15 @@ function InitializeBuildTool() {
function GetDefaultMSBuildEngine() {
# Presence of tools.vs indicates the repo needs to build using VS msbuild on Windows.
if (Get-Member -InputObject $GlobalJson.tools -Name 'vs') {
return 'vs'
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
}
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
return 'dotnet'
}
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
Write-Host "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'." -ForegroundColor Red
ExitWithExitCode 1
}
@@ -489,9 +399,9 @@ function GetNuGetPackageCachePath() {
# Use local cache on CI to ensure deterministic build,
# use global cache in dev builds to avoid cost of downloading packages.
if ($useGlobalNuGetCache) {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile '.nuget\packages'
$env:NUGET_PACKAGES = Join-Path $env:UserProfile ".nuget\packages"
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot '.packages'
$env:NUGET_PACKAGES = Join-Path $RepoRoot ".packages"
}
}
@@ -504,14 +414,12 @@ function GetSdkTaskProject([string]$taskName) {
}
function InitializeNativeTools() {
if (-Not (Test-Path variable:DisableNativeToolsetInstalls) -And (Get-Member -InputObject $GlobalJson -Name "native-tools")) {
$nativeArgs= @{}
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
$nativeArgs=""
if ($ci) {
$nativeArgs = @{
InstallDirectory = "$ToolsDir"
}
$nativeArgs = "-InstallDirectory $ToolsDir"
}
& "$PSScriptRoot/init-tools-native.ps1" @nativeArgs
Invoke-Expression "& `"$PSScriptRoot/init-tools-native.ps1`" $nativeArgs"
}
}
@@ -533,14 +441,14 @@ function InitializeToolset() {
}
if (-not $restore) {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Toolset version $toolsetVersion has not been restored."
Write-Host "Toolset version $toolsetVersion has not been restored." -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = InitializeBuildTool
$proj = Join-Path $ToolsetDir 'restore.proj'
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'ToolsetRestore.binlog') } else { '' }
$proj = Join-Path $ToolsetDir "restore.proj"
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "ToolsetRestore.binlog") } else { "" }
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Set-Content $proj
@@ -562,7 +470,7 @@ function ExitWithExitCode([int] $exitCode) {
}
function Stop-Processes() {
Write-Host 'Killing running build processes...'
Write-Host "Killing running build processes..."
foreach ($processName in $processesToStopOnExit) {
Get-Process -Name $processName -ErrorAction SilentlyContinue | Stop-Process
}
@@ -576,21 +484,9 @@ function Stop-Processes() {
function MSBuild() {
if ($pipelinesLog) {
$buildTool = InitializeBuildTool
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if ($ci -and $buildTool.Tool -eq 'dotnet') {
dotnet nuget locals http-cache -c
$env:NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS = 20
$env:NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS = 20
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS' -Value '20'
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS' -Value '20'
}
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework 'Microsoft.DotNet.Arcade.Sdk.dll')
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
$args += "/logger:$path"
}
@@ -605,13 +501,11 @@ function MSBuild() {
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Binary log must be enabled in CI build.'
ExitWithExitCode 1
throw "Binary log must be enabled in CI build."
}
if ($nodeReuse) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Node reuse must be disabled in CI build.'
ExitWithExitCode 1
throw "Node reuse must be disabled in CI build."
}
}
@@ -619,11 +513,8 @@ function MSBuild-Core() {
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += ' /warnaserror /p:TreatWarningsAsErrors=true'
}
else {
$cmdArgs += ' /p:TreatWarningsAsErrors=false'
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
}
foreach ($arg in $args) {
@@ -631,31 +522,31 @@ function MSBuild-Core() {
$cmdArgs += " `"$arg`""
}
}
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Build failed.'
Write-Host "Build failed." -ForegroundColor Red
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
}
ExitWithExitCode $exitCode
}
}
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith('/bl:', "OrdinalIgnoreCase")) {
return $arg.Substring('/bl:'.Length)
}
if ($arg.StartsWith('/binaryLogger:', 'OrdinalIgnoreCase')) {
return $arg.Substring('/binaryLogger:'.Length)
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
}
}
}
@@ -663,16 +554,14 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
return $null
}
. $PSScriptRoot\pipeline-logging-functions.ps1
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$ArtifactsDir = Join-Path $RepoRoot 'artifacts'
$ToolsetDir = Join-Path $ArtifactsDir 'toolset'
$ToolsDir = Join-Path $RepoRoot '.tools'
$LogDir = Join-Path (Join-Path $ArtifactsDir 'log') $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir 'tmp') $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot 'global.json') | ConvertFrom-Json
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$ToolsetDir = Join-Path $ArtifactsDir "toolset"
$ToolsDir = Join-Path $RepoRoot ".tools"
$LogDir = Join-Path (Join-Path $ArtifactsDir "log") $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir "tmp") $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot "global.json") | ConvertFrom-Json
# true if global.json contains a "runtimes" section
$globalJsonHasRuntimes = if ($GlobalJson.tools.PSObject.Properties.Name -Match 'runtimes') { $true } else { $false }
@@ -680,23 +569,11 @@ Create-Directory $ToolsetDir
Create-Directory $TempDir
Create-Directory $LogDir
Write-PipelineSetVariable -Name 'Artifacts' -Value $ArtifactsDir
Write-PipelineSetVariable -Name 'Artifacts.Toolset' -Value $ToolsetDir
Write-PipelineSetVariable -Name 'Artifacts.Log' -Value $LogDir
Write-PipelineSetVariable -Name 'TEMP' -Value $TempDir
Write-PipelineSetVariable -Name 'TMP' -Value $TempDir
if ($ci) {
Write-Host "##vso[task.setvariable variable=Artifacts]$ArtifactsDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Toolset]$ToolsetDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Log]$LogDir"
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
if (!$disableConfigureToolsetImport) {
$configureToolsetScript = Join-Path $EngRoot 'configure-toolset.ps1'
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
if ((Test-Path variable:failOnConfigureToolsetError) -And $failOnConfigureToolsetError) {
if ((Test-Path variable:LastExitCode) -And ($LastExitCode -ne 0)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'configure-toolset.ps1 returned a non-zero exit code'
ExitWithExitCode $LastExitCode
}
}
}
$env:TEMP = $TempDir
$env:TMP = $TempDir
}
Executable → Regular
+104 -116
View File
@@ -41,14 +41,10 @@ fi
# Configures warning treatment in msbuild.
warn_as_error=${warn_as_error:-true}
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.sh
dotnetInstallScriptVersion=${dotnetInstallScriptVersion:-'v1'}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
if [[ "$ci" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
@@ -56,6 +52,78 @@ else
use_global_nuget_cache=${use_global_nuget_cache:-true}
fi
function EmitError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
message_type="error"
sourcepath=''
linenumber=''
columnnumber=''
error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-l)
linenumber=$2
shift
;;
-columnnumber|-col)
columnnumber=$2
shift
;;
-code|-c)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
message='##vso[task.logissue'
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
else
message="$message;sourcepath=${BASH_SOURCE[1]}"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
else
message="$message;linenumber=${BASH_LINENO[0]}"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
# Resolve any symlinks in the given path.
function ResolvePath {
local path=$1
@@ -77,11 +145,11 @@ function ResolvePath {
function ReadGlobalVersion {
local key=$1
local line=$(awk "/$key/ {print; exit}" "$global_json_file")
local line=`grep -m 1 "$key" "$global_json_file"`
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
Write-PipelineTelemetryError -category 'Build' "Error: Cannot find \"$key\" in $global_json_file"
EmitError "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
@@ -142,7 +210,7 @@ function InitializeDotNetCli {
if [[ "$install" == true ]]; then
InstallDotNetSdk "$dotnet_root" "$dotnet_sdk_version"
else
Write-PipelineTelemetryError -category 'InitializeToolset' "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
EmitError "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
ExitWithExitCode 1
fi
fi
@@ -150,10 +218,14 @@ function InitializeDotNetCli {
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
Write-PipelinePrependPath -path "$dotnet_root"
export PATH="$dotnet_root:$PATH"
Write-PipelineSetVariable -name "DOTNET_MULTILEVEL_LOOKUP" -value "0"
Write-PipelineSetVariable -name "DOTNET_SKIP_FIRST_TIME_EXPERIENCE" -value "1"
if [[ $ci == true ]]; then
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
echo "##vso[task.prependpath]$dotnet_root"
echo "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
echo "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
fi
# return value
_InitializeDotNetCli="$dotnet_root"
@@ -172,7 +244,7 @@ function InstallDotNetSdk {
function InstallDotNet {
local root=$1
local version=$2
GetDotNetInstallScript "$root"
local install_script=$_GetDotNetInstallScript
@@ -191,59 +263,15 @@ function InstallDotNet {
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from public location (exit code '$exit_code')."
if [[ -n "$runtimeArg" ]]; then
local runtimeSourceFeed=''
if [[ -n "${6:-}" ]]; then
runtimeSourceFeed="--azure-feed $6"
fi
local runtimeSourceFeedKey=''
if [[ -n "${7:-}" ]]; then
decodedFeedKey=`echo $7 | base64 --decode`
runtimeSourceFeedKey="--feed-credential $decodedFeedKey"
fi
if [[ -n "$runtimeSourceFeed" || -n "$runtimeSourceFeedKey" ]]; then
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from custom location '$runtimeSourceFeed' (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
ExitWithExitCode $exit_code
fi
fi
EmitError "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
function with_retries {
local maxRetries=5
local retries=1
echo "Trying to run '$@' for maximum of $maxRetries attempts."
while [[ $((retries++)) -le $maxRetries ]]; do
"$@"
if [[ $? == 0 ]]; then
echo "Ran '$@' successfully."
return 0
fi
timeout=$((2**$retries-1))
echo "Failed to execute '$@'. Waiting $timeout seconds before next attempt ($retries out of $maxRetries)." 1>&2
sleep $timeout
done
echo "Failed to execute '$@' for $maxRetries times." 1>&2
return 1
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
local install_script_url="https://dot.net/$dotnetInstallScriptVersion/dotnet-install.sh"
local install_script_url="https://dot.net/v1/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
@@ -252,19 +280,12 @@ function GetDotNetInstallScript {
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
with_retries curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script"
else
with_retries wget -v -O "$install_script" "$install_script_url" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
wget -q -O "$install_script" "$install_script_url"
fi
fi
# return value
_GetDotNetInstallScript="$install_script"
}
@@ -273,11 +294,11 @@ function InitializeBuildTool {
if [[ -n "${_InitializeBuildTool:-}" ]]; then
return
fi
InitializeDotNetCli $restore
# return values
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildToolCommand="msbuild"
_InitializeBuildToolFramework="netcoreapp2.1"
}
@@ -296,14 +317,11 @@ function GetNuGetPackageCachePath {
}
function InitializeNativeTools() {
if [[ -n "${DisableNativeToolsetInstalls:-}" ]]; then
return
fi
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
if [[ "$ci" == true ]]; then
nativeArgs="--installDirectory $tools_dir"
nativeArgs="-InstallDirectory $tools_dir"
fi
"$_script_dir/init-tools-native.sh" $nativeArgs
fi
@@ -331,7 +349,7 @@ function InitializeToolset {
fi
if [[ "$restore" != true ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Toolset version $toolset_version has not been restored."
EmitError "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 2
fi
@@ -341,14 +359,14 @@ function InitializeToolset {
if [[ "$binary_log" == true ]]; then
bl="/bl:$log_dir/ToolsetRestore.binlog"
fi
echo '<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' > "$proj"
MSBuild-Core "$proj" $bl /t:__WriteToolsetLocation /clp:ErrorsOnly\;NoSummary /p:__ToolsetLocationOutputFile="$toolset_location_file"
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
Write-PipelineTelemetryError -category 'Build' "Invalid toolset path: $toolset_build_proj"
EmitError "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
@@ -375,18 +393,6 @@ function MSBuild {
if [[ "$pipelines_log" == true ]]; then
InitializeBuildTool
InitializeToolset
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if [[ "$ci" == true ]]; then
"$_InitializeBuildTool" nuget locals http-cache -c
export NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS=20
export NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS=20
Write-PipelineSetVariable -name "NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS" -value "20"
Write-PipelineSetVariable -name "NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS" -value "20"
fi
local toolset_dir="${_InitializeToolset%/*}"
local logger_path="$toolset_dir/$_InitializeBuildToolFramework/Microsoft.DotNet.Arcade.Sdk.dll"
args=( "${args[@]}" "-logger:$logger_path" )
@@ -398,12 +404,12 @@ function MSBuild {
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
Write-PipelineTelemetryError -category 'Build' "Binary log must be enabled in CI build."
EmitError "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
Write-PipelineTelemetryError -category 'Build' "Node reuse must be disabled in CI build."
EmitError "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
@@ -417,7 +423,7 @@ function MSBuild-Core {
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'Build' "Build failed (exit code '$exit_code')."
EmitError "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
@@ -425,8 +431,6 @@ function MSBuild-Core {
ResolvePath "${BASH_SOURCE[0]}"
_script_dir=`dirname "$_ResolvePath"`
. "$_script_dir/pipeline-logging-functions.sh"
eng_root=`cd -P "$_script_dir/.." && pwd`
repo_root=`cd -P "$_script_dir/../.." && pwd`
artifacts_dir="$repo_root/artifacts"
@@ -438,7 +442,7 @@ temp_dir="$artifacts_dir/tmp/$configuration"
global_json_file="$repo_root/global.json"
# determine if global.json contains a "runtimes" entry
global_json_has_runtimes=false
dotnetlocal_key=$(awk "/runtimes/ {print; exit}" "$global_json_file") || true
dotnetlocal_key=`grep -m 1 "runtimes" "$global_json_file"` || true
if [[ -n "$dotnetlocal_key" ]]; then
global_json_has_runtimes=true
fi
@@ -453,23 +457,7 @@ mkdir -p "$toolset_dir"
mkdir -p "$temp_dir"
mkdir -p "$log_dir"
Write-PipelineSetVariable -name "Artifacts" -value "$artifacts_dir"
Write-PipelineSetVariable -name "Artifacts.Toolset" -value "$toolset_dir"
Write-PipelineSetVariable -name "Artifacts.Log" -value "$log_dir"
Write-PipelineSetVariable -name "Temp" -value "$temp_dir"
Write-PipelineSetVariable -name "TMP" -value "$temp_dir"
# Import custom tools configuration, if present in the repo.
if [ -z "${disable_configure_toolset_import:-}" ]; then
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ $ci == true ]]; then
export TEMP="$temp_dir"
export TMP="$temp_dir"
fi
View File
File renamed without changes.
+51
View File
@@ -0,0 +1,51 @@
<Project DefaultTargets="PublishPackages">
<!-- TODO: move properties imported from here into a common props file -->
<!-- <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dir.props))\dir.props" /> -->
<!-- Use an explicit SDK import so that arcade uses the build
configuration computed in dir.common.props. -->
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
<Import Project="$(MSBuildThisFileDirectory)\..\.packages\microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<PropertyGroup>
<!-- Set the TargetFramework just to make the SDK happy -->
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<Target Name="PublishPackages">
<ItemGroup>
<PackagesToPublish Include="$(MSBuildThisFileDirectory)\..\artifacts\*.nupkg" />
</ItemGroup>
<ItemGroup>
<ItemsToPushToBlobFeed Include="@(PackagesToPublish)" />
</ItemGroup>
<Error Condition=" '$(AzureFeedUrl)' == '' " Text="AzureFeedUrl must be set" />
<Error Condition=" '$(AzureAccountKey)' == '' " Text="AzureAccountKey must be set" />
<Error Condition=" '$(BUILD_REPOSITORY_URI)' == '' " Text="BUILD_REPOSITORY_URI must be set" />
<Error Condition=" '$(BUILD_SOURCEBRANCH)' == '' " Text="BUILD_SOURCEBRANCH must be set" />
<Error Condition=" '$(BUILD_BUILDNUMBER)' == '' " Text="BUILD_BUILDNUMBER must be set" />
<Error Condition=" '$(BUILD_SOURCEVERSION)' == '' " Text="BUILD_SOURCEVERSION must be set" />
<PropertyGroup>
<AssetManifestFilePath>$(ArtifactsLogDir)AssetManifest\$(AssetManifest)</AssetManifestFilePath>
</PropertyGroup>
<Message Importance="Low" Text="BUILD_REPOSITORY_URI: $(BUILD_REPOSITORY_URI)" />
<PushToBlobFeed ExpectedFeedUrl="$(AzureFeedUrl)"
AccountKey="$(AzureAccountKey)"
ItemsToPush="@(ItemsToPushToBlobFeed)"
ManifestBuildData="Location=$(AzureFeedUrl)"
ManifestRepoUri="$(BUILD_REPOSITORY_URI)"
ManifestBranch="$(BUILD_SOURCEBRANCH)"
ManifestBuildId="$(BUILD_BUILDNUMBER)"
ManifestCommit="$(BUILD_SOURCEVERSION)"
AssetManifestPath="$(AssetManifestFilePath)" />
</Target>
</Project>
-1
Submodule external/bdwgc deleted from 6e8db07c8d.
Loaded 100 of 3566 files, more files were not shown because too many files have changed in this diff. Show more