Compare commits

..
215 Commits
Author SHA1 Message Date
Ryan Lucia 476d72b9e3 [IO] Remove read-only logic in mono_w32_get_disk_free_space
This brings us in line with GetDiskFreeSpaceExW on Windows as well as .NET Core.
2019-10-15 10:43:27 +02:00
Aleksey Kliger (λgeek) bdc557612b [2019-06][merp] Don't install SIGTERM handler in EnableMicrosoftTelemetry (#17296)
* [merp] Put thread into async context before running summarizer

[merp] Put thread into async context before running summarizer

Followup work for https://github.com/mono/mono/pull/17090

In order to get managed method info for unmanaged stacktraces, we call
`mini_jit_info_table_find_ext` which calls `decode_exception_debug_info` which may
allocate if it's not in async context.

Do the switch in `mono_threads_summarize_execute` so that non-supervising threads
all switch when they run the `sigterm_signal_handler`.  `mono_threads_summarize`
already does it for the supervisor thread.

Fixes https://github.com/mono/mono/issues/17180

Backport of #17194.

/cc @lambdageek

* [merp] Don't call install_state_summarizer in EnableMicrosoftTelemetry

install_state_summarizer installs a SIGTERM handler. Unfortunately
sigterm_signal_handler assumes that it will only be called when a crash is
already in progress (it assumes it won't be the supervisor, and so it doesn't
setup the memory for capturing a crash report among other things).

* [merp] Register SIGTERM handler in dump_native_stacktrace

Previously it was installed by EnableMicrosoftTelemetry, which is too early.
Now we get here without a handler installed even when mono_merp_enabled returns true.
2019-10-14 17:24:44 +02:00
Alexander Köplinger 1e2d850ec4 [tests] Bump corefx to get Azure testhost change (#17275)
* Bump corefx to get Azure testhost change

See https://github.com/mono/corefx/pull/360

* Update ClientWebSocketTest as well

(cherry picked from commit d457346e06)

# Conflicts:
#	external/corefx
2019-10-11 09:41:33 +02:00
Alexander Köplinger 703e9a2fad [2019-06] Add drawing type converters to mobile profiles (#17241)
* [2019-06] Add drawing type converters to mobile profiles

Needed for https://github.com/xamarin/xamarin-android/issues/3726

* Bump API snapshot submodule

* [csproj] Update project files
2019-10-09 12:16:23 +02:00
Thays Grazia 4ec605fc9b If there is a perform_wait_callback in the stack there will be another catch generated by the method called in the owner thread, so we don't need to throw as unhandled exception, we can continue and find the next catch.
Fixes #17083

Backport of #17106.
2019-10-09 09:07:21 +02:00
monojenkins 061c225621 [merp] exit_status is 0 if we ran the uploader successfully (#17186)
Not 1 (TRUE)
2019-10-05 11:36:29 -04:00
Alexis Christoforides 07c23f2ca4 [crashing] Remove Mono signal handlers when starting to handle a crash
Fixes https://github.com/mono/mono/issues/16803

Backport of https://github.com/mono/mono/pull/17002
2019-10-02 09:35:59 +02:00
Egor Bogatov 5608fe0abb [2019-06] Update NETStandard 2.1 APIs (#17081)
Contributes to #17064
2019-09-27 17:42:14 +02:00
Steve Pfister a5b8d6fcb5 [2019-06] Fix time zone issue when jumping into DST (#17062)
There is an issue with jumping into DST for some time zones when the incorrect date-time offset is returned for date-time in UTC (which comes from DateTime.Now). The fix is to just check if the incoming date-time is in UTC

Also I added set of tests for some time zones but they verify jumping into DST in general.

Fixes https://github.com/mono/mono/issues/16395

Backport of https://github.com/mono/mono/pull/16430
2019-09-26 20:03:02 +02:00
Marek Safar 390e6e124f Revert "[metadata] Fix leaks when handling a few attributes (#16675) (#16851)"
This reverts commit a6b5187d76.
2019-09-26 18:49:35 +02:00
Alexander Köplinger 459dd927df [2019-06] Bump bockbuild to get https://github.com/mono/bockbuild/pull/121 2019-09-25 20:00:56 +02:00
Rolf Bjarne Kvinge beb9a1b182 [sdks] Remove the mac32 build.
Xamarin.Mac doesn't need nor use it anymore.

This also removes the last part of the mono archive that required Xcode 9.4.
2019-09-25 16:52:49 +02:00
Alexander Köplinger 747a919a06 [ci] Make ios/mac sdks archive URL more predictable
Requested by xamarin-macios team.

(cherry picked from commit 0a3c9c451b)
2019-09-24 15:26:55 +02:00
Alexander Köplinger 114013096e [ci] Build iOS/Mac Mono sdks archive using Xcode 11
This was requested by the xamarin-macios team.

(cherry picked from commit 6ad929c317)
2019-09-23 17:37:21 +02:00
EgorBo 10a24f3ea1 Implement WriteCore and ReadCore in DeflateStream 2019-09-23 16:48:16 +02:00
Rolf Bjarne Kvinge a925846b1f [offsets-tool] Install clang into the user-specific python directory. (#16933)
Because the system location isn't world writeable on Catalina.
2019-09-19 17:43:35 +02:00
Larry Ewing fe64a4765e [2019-06] Bump msbuild and sdk versions to 3.0.1xx latest (#16870)
* [2019-06] Bump msbuild and sdk versions to 3.0.1xx latest

* Bump msbuild again to pull in the build updates

* pull in roslyn-binaries fix

* One more bump

* Fix the roslyn hash

* [msbuild] Bump to pick up a build fix from https://github.com/mono/msbuild/pull/138

* Pull in final update from msbuild of nuget and sdk
NuGet 5.3.0-rtm.6251
Sdk 3.0.100-rc2.19467.3

* Pick up packaging fix in https://github.com/mono/msbuild/pull/141
2019-09-18 19:13:51 +02:00
monojenkins 7293597b90 [corlib] Fix building nunit-lite twice (#16910)
We sometimes saw the following build error in Mono SDKs builds on Jenkins:

```
[2019-09-16T01:03:30.709Z] CSC     [xammac] nunit-lite-console.exe
[2019-09-16T01:03:30.709Z] error CS0009: Metadata file '/Users/builder/jenkins/workspace/archive-mono/2019-08/mac/release/mcs/class/lib/xammac/nunitlite.dll' could not be opened -- PE image doesn't contain managed metadata.
```

Taking a closer look we found out that nunit-lite was trying to be built twice for the same profile which caused a race where the file was corrupted.

The reason for this is that the corlib Makefile defines two test assemblies: the normal PROFILE_corlib_test.dll and BinarySerializationOverVersionsTest.dll and
integrating the BinarySerializationOverVersionsTest.dll via the `test-vts` target incorrectly overwrote the `test` target instead of using `test-local`.

Backport of #16894.
2019-09-18 13:39:36 +02:00
Rolf Bjarne Kvinge 1648e88687 Rename bundle identifier for the various Mono.frameworks we create for Xamarin.iOS. Fixes xamarin/xamarin-macios#7005. (#16896)
This is the equivalent of
https://github.com/xamarin/xamarin-macios/commit/a9abc07cb357f9f012e3da66c70fa09d96bf02f7,
just for the Mono.frameworks in the mono archive.

Fixes https://github.com/xamarin/xamarin-macios/issues/7005.
2019-09-17 21:16:29 +02:00
Vlad Brezae a6b5187d76 [metadata] Fix leaks when handling a few attributes (#16675) (#16851)
Callers of mono_reflection_create_custom_attr_data_args_noalloc were leaking some of the returned information. Accessed attributes are FixedBufferAttribute and UnmanagedFunctionPointerAttribute.
2019-09-16 14:16:01 +03:00
Steve Pfister 7da9a041b3 [2019-06] Bump to mono/corefx@e79cf5b
Backport of https://github.com/mono/mono/pull/16774
2019-09-12 09:08:26 +02:00
monojenkins 2b7050bdf3 [2019-06] Add RenamedEvent* to FSW sources from CoreFX (#16758)
Add RenamedEvent* to FSW sources from CoreFX

Fixes #16486 - This was only partially fixed until now

* [csproj] Update project files

* Add test files which test these code paths on corefx
2019-09-11 12:29:51 -04:00
Larry Ewing 4f5ed502c6 [msbuild] pick up p4 versions 2019-09-10 09:35:15 +02:00
Larry Ewing f04ee2219d [2019-06][msbuid][roslyn] Bump msbuild and roslyn-binaries to pick up dotnet 3.0.100-p9 toolset 2019-09-10 09:35:15 +02:00
Thays Grazia 6b4b99e571 Vtable [i] can be null so this should be check before use it. Fixes #16712 2019-09-09 19:17:01 +02:00
monojenkins 7af64d1ebe [2019-06] [android] remove text relocations in AOT/arm32 (#16660)
* Revert "[aot] partial revert of #14043 (#14135)"

This reverts commit 223e0bd78d.

* Revert "[aot] change method table generation so that it works for larger assemblies on ARM (#14043)"

This reverts commit 9ff3b0d65e.
2019-09-04 15:31:19 -05:00
monojenkins 2c3aeaf378 [2019-06] [corlib] Add hack for Hebrew calendar data (#16642) 2019-09-03 14:09:37 -04:00
monojenkins 3216a3c1e3 [2019-06] Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5. (#16575)
* Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5.

* [monodis] Add a missing cattr parent case.

Backport of #16567.
2019-08-30 12:30:03 +02:00
Alexander Köplinger 7ef99c046c Build pkg-config as 64bit only, libtiff as universal and remove gtk-quartz-engine.py (#16548)
See https://github.com/mono/bockbuild/pull/116

(cherry picked from commit 309cf1bcee)

# Conflicts:
#	external/bockbuild
2019-08-29 20:37:00 +02:00
monojenkins 1604b9df9d [2019-06] Use the right FileSystemWatcher sources (#16555)
* Use the right FileSystemWatcher sources

The corefx code for FileSystemEventArgs has some changes to support
constructing one with a null filename.

Fixes #16486

* Remove the mono sources for the newly replaced files

* [csproj] Update project files
2019-08-29 14:35:35 +02:00
monojenkins fd75f0f487 [2019-06] Implement Read and WriteAsyncMemory in Mono DeflateStream
Resolves https://github.com/mono/mono/issues/16122

Since we put in the corefx version of GZipStream, it is expected that
our version of DeflateStream actually implements Write+ReadAsyncMemory.

This change does that by just calling the plain WriteAsync & ReadAsync
methods until we decide (or not) to pull in the corefx DeflateStream.

Backport of https://github.com/mono/mono/pull/16523.
2019-08-29 08:33:44 -04:00
Steve Pfister 99c1602fcd Allow file:// based Uri's to be used in HttpClient when the BaseAddress is set
Fixes https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/967582

Bumps the HttpClient part in mono/corefx at: https://github.com/mono/corefx/commit/ee3acec16b37bec5016da129e1e3840097ca629b

The test that would have caught this failure was disabled previously via:
https://github.com/mono/mono/commit/2f411750acfd2132f9b4970ef8cf11d9a0f62df9

(cherry picked from commit 580defd967)
2019-08-27 20:04:13 +02:00
Alexander Köplinger 16c2464e24 [acceptance-tests] Bump roslyn test repo to remove a problematic nuget feed
(cherry picked from commit 8d422436f5)
2019-08-27 19:48:42 +02:00
monojenkins 5ecff82230 [2019-06] [w32socket] Translate some errno codes to WSA_ errors imprecisely. (#16489)
* [w32socket] Translate some errno codes to WSA_ errors imprecisely.

These aren't a perfect match, but better than falling into the `default` case
where Mono asserts and crashes.

Addresses part of https://github.com/mono/mono/issues/16024

* [w32error] Add WSA_INVALID_PARAMETER and WSA_INVALID_HANDLE

Backport of #16331.
2019-08-27 01:31:53 +02:00
monojenkins 8966e775a5 [utils] Don't use MAP_32BIT on Apple platforms, fixes crash with XCode 11 beta6 (#16481)
As of XCode 11 beta6 the MacOSX SDK defines the MAP_32BIT symbol.

This causes the `mono_valloc()` function to try to use the `MAP_32BIT` flag for `mmap()`.

However as mentioned in the comment for the symbol in mman.h it seems to require a special entitlement which isn't available/documented anywhere yet.
This in turn causes the mmap call to fail presumably because we're missing that entitlement.

Instead we now skip setting this flag on Apple platforms to make `mmap()` behave like it did before.

Backport of https://github.com/mono/mono/pull/16441
2019-08-26 23:07:37 +02:00
Larry Ewing 3608a49541 [2019-06][msbuild] bump msbuild to pick up latest p9 versions and enable shared compilation again (#16445)
* [2019-06] bump msbuild to stop disabling shared compilation

* Pull in nuget update for preview 9
2019-08-24 09:51:03 +02:00
Ankit Jain 049d9ecb3c [msbuild] Bump to track mono-2019-06 (#16428)
* Bump msbuild to track mono-2019-06

.. to get changes from https://github.com/mono/msbuild/pull/130
which tracks dotnet `release/3.0.100-preview9`.

Also, bump Roslyn to 3.3.1-beta3-19421-04 .

* Bump the roslyn reference
2019-08-23 22:15:13 +02:00
monojenkins 66d623d118 Fix issue in SafeHandle.DangerousReleaseInternal() that caused #16034. (#16424)
Using the CoreCLR version of this line fixed the problem and my investigations
also revealed why.

The `SafePipeHandle` contains a reference to both the Socket as well as the
`SafeSocketHandle`, taking ownership of them.  If there are no additional
references to either of them, so that both are collected in the same GC pass,
their finalizer may be invoked in any order.

If the Socket gets finalized first, it's `SafeSocketHandle` will have a `_mstate`
of 8 (two references) at the beginning of `DangerousReleaseInternal()` - which
will set it to 6 (one reference plus disposed flag).

Then when the `SafePipeHandle` gets collected, it will call `DangerousRelease ()`
on it's `SafeSocketHandle`.  As mentioned above, it's `_mstate` will be 6.
Now we own the handle, so we'll decrease it's reference count by one and set the
`Closed` flag - the new `_mstate` should thus be 3.

However, using

    new_state = (old_state & RefCount_Mask) - RefCount_One;

would set it to 1 - thus effectively clearing the `Disposed' flag.  And since it's
reference count is now also zero, the next finalize pass will then throw.

The CoreCLR version of this class uses

    new_state = old_state - RefCount_One;

which fixes this.

Backport of #16244.
2019-08-23 15:34:57 +02:00
Alexander Köplinger 3e76198aa4 Bump corefx to get https://github.com/mono/corefx/pull/336 2019-08-23 14:44:43 +02:00
Steve Pfister fb26dfd84f [2019-06] Add Compiler Server Workaround for 32-bit Linux (#16422)
Currently, if the process is 32-bit, roslyn tries to determine how much
memory is available. To do that, it tries to pinvoke into GlobalMemoryStatusEx,
which is not supported on mono.

Without it, the compiler server bombs on https://github.com/dotnet/roslyn/blob/0e63260c5afb3fb5b74c357dd250e500172bcd63/src/Compilers/Server/VBCSCompiler/NamedPipeClientConnection.cs#L55-L5

Backport of https://github.com/mono/mono/pull/16405
2019-08-23 12:08:30 +02:00
monojenkins 6434153d8b [android] mono_dl_open_file: use g_file_test only on absolute paths (#16392)
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is
the name of a shared library we want to open is to speed up probing for
non-existent libraries.

See https://github.com/mono/mono/pull/12074

The problem is that if file is just a simple "libdl.so" then `dlopen (file)`
doesn't just look for it in the current working directory, it will probe some
other paths too.  (For example on desktop linux you'd also look in all the
directories in LD_LIBRARY_PATH).  So the g_file_test() call is not a robust way
to avoid calling dlopen if the filename is relative.

But it actually broke more things: https://github.com/xamarin/xamarin-android/issues/3388

When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try
prepending some paths that it knows about and we end up calling
`dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security
restrictions on what code can dlopen something from /system/lib with an
absolute path.  Eventually mono_lookup_pinvoke_call will go back to trying the
bare "libdl.so" which hits `g_file_test` and returns NULL.

The new code only does the file test if we pass it an absolute path, which
gives Bionic's dlopen a chance to deal with relative paths however it needs to.
2019-08-22 13:36:13 -04:00
Alexander Köplinger cc0632e4f3 Bump corefx to get https://github.com/mono/corefx/pull/332 2019-08-20 20:52:51 +02:00
monojenkins ff5e4b074f [2019-06] Add missing Dispose() call on error in MonoTlsStream.CreateStream(). (#16327)
* Add missing `Dispose()` call on error in `MonoTlsStream.CreateStream()`.

* Remove the `finally` block to make it more easy to read.

Backport of #16233.
2019-08-20 19:45:59 +02:00
Alexander Köplinger 45c7df08f4 [pkg] Bump bockbuild to get gtksharp bump (#16346) 2019-08-20 11:51:03 +02:00
monojenkins e6d44f7ce4 [2019-06] [corlib] Fix sharing mode validation rules (#16280)
Fixes https://github.com/mono/mono/issues/16032

When we switched to corefx File.Copy on unix, the behavior changed where it opened a source and destination stream in order to pass the safe handles to Interop.Sys.CopyFile. This exposed a flaw in our sharing validation rules in MonoIO.Open, which threw an exception in this scenario:

new FileStream("source.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite, 2048, FileOptions.DeleteOnClose);
new FileStream("source.txt", FileMode.Open, FileAccess.Read, FileShare.Read, 2048, FileOptions.None);

Simplified the FileShare check down to:

1. If the caller specifies FileShare.None, then it's an exclusive lock and no
   sharing is allowed.

2. If the caller specifies some other FileShare option, then FileShare.None
    cannot be specified.
2019-08-16 10:23:09 -04:00
monojenkins 4a85530472 (mono.1) Update the MONO_THREADS_SUSPEND description (#16238) 2019-08-14 19:24:21 +02:00
monojenkins 6c97670c23 [sre] Outline typebuilder_setup_fields loop body (#16203)
The loop body uses coop handles, so wrap it in HANDLE_FUNCTION_ENTER/HANDLE_FUNCTION_RETURN

Addresses part of #16179

Backport of #16184.
2019-08-14 12:32:00 +02:00
Alexander Köplinger 4988ad56ad [pkg] Bump libgdiplus to 6.0.2
(cherry picked from commit b60aaea791)
2019-08-13 12:13:48 +02:00
Sebastien Pouliot 4365dce8b7 [2019-06][sdk] Enable the SRE runtime code inside iOS builds
This will allow the interpreter to use System.Reflection.Emit.

Note: There is a small size increase (about 50kb) for an application
built in release mode with the LLVM backend (even if the interpreter
is now used).

Backport of https://github.com/mono/mono/pull/16151
2019-08-12 21:43:22 +02:00
Alexander Köplinger a7919897e8 Bump corefx to fix build break 2019-08-10 23:47:19 +02:00
Alexander Köplinger 9e02c677aa Bump corefx to get https://github.com/mono/corefx/pull/327 2019-08-10 16:20:04 +02:00
Alexander Köplinger 3c1ee41b12 Bump corefx to get https://github.com/mono/corefx/pull/323 2019-08-09 16:17:39 +02:00
monojenkins 63d98369c6 [2019-06] Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements. (#16108)
* Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements.

https://github.com/mono/mono/issues/16010

The current behavior for GetConstructorCandidates when the name is null, "",
or "*" is to return the set of constructors for the type. For "", .NET Core and
.NET Framework return 0 elements.  This PR adds a check for "" to match that
behavior.

* Removed unncessary changes

* Unnecessary spaces

* Made the if check a bit more readable
2019-08-09 11:56:56 +02:00
Alexis Christoforides 893099f23f [2019-06] [System] Guard against attempting to creating out-of-range spans in m… (#16123)
* [System] Guard against attempting to creating out-of-range spans in macOS FileSystemWatcher

Backport of #16049
2019-08-09 00:38:29 +02:00
Ankit Jain 5d9caa2373 [msbuild] Bump to track mono-2019-06 and update roslyn (#16095)
Prompted by mono/msbuild#123 :

This includes:

Update to upstream vs16.3 branch
SDK update to track dotnet release/3.0.100-preview8
Roslyn updated to 3.3.0-beta2-19381-14

* Work around remoting1.exe test issue with new Roslyn
2019-08-08 20:07:00 +02:00
Larry Ewing 8c035c2144 Pull in the unbroken msbuild 2019-08-07 22:53:27 -05:00
Larry Ewing 3683f7d2b8 Bump msbuild cd5b32d36ad51cb5c21621bb5cfe6a7452214eaa to disable compiler server 2019-08-07 11:34:20 -05:00
monojenkins 91adcb5cd1 [2019-06] [arm] respect I1 and I2 on gsharedvt out transition (#16017)
* [arm] respect I1 and I2 on gsharedvt out transition

In order to trigger the problem, we must call into gsharedvt code ("in" transition) and then escape the gsharedvt world ("out" transition). The bug happens in the latter. Also some other stars need to align (I haven't fully understood it), thus the repro is quite long.

The provided repro only triggers with LLVM, which we don't have enabled on CI yet: https://github.com/mono/mono/issues/15999

Fixes https://github.com/mono/mono/issues/15261
Fixes https://github.com/mono/mono/issues/15262
Fixes https://github.com/mono/mono/issues/15263
Fixes https://github.com/mono/mono/issues/15307


Backport of #16000.
2019-08-07 13:28:18 +02:00
Thays Grazia b454f5a9d0 allow emitting DIM 2019-08-07 09:25:31 +02:00
Aleksey Kliger df37100e76 [w32socket] Translate ELOOP and ENAMETOOLONG
Translate to WSAELOOP and WSAENAMETOOLONG, respectively.

See https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2
2019-08-07 09:23:10 +02:00
Egor Bogatov 7581fffa8a Forgot to check IsInvalid state 2019-08-06 19:29:54 +02:00
monojenkins d53c952790 [corlib] Ignore TimeZoneTest.TestCtors on Android GMT (#16038)
TimeZone.CurrentTimeZone.DaylightName returns an empty value on Android
when the system timezone is set to GMT.

Backport of #15987.
2019-08-06 14:03:00 +02:00
monojenkins 6fbdb372b8 [2019-06] Add finalizer to DeflateStream (#16008)
Fixes #15825

Backport of #15879.
2019-08-05 13:21:53 +02:00
Jo Shields 47a3d9204d Bump libgdiplus to new release (#15996)
(cherry picked from commit 9a7b6fd5ee)
2019-08-03 00:50:36 +02:00
monojenkins 5dc37d7ee6 [2019-06] [marshal] Free delegates with target that are passed to native code. (#15966)
* [marshal] Always use gchandles in delegate_hash_table

Makes the code easier to follow and it also fixes race from https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b with boehm.

* [marshal] Free delegates with target that are passed to native code.

For static method delegates, we have a unique delegate_trampoline that is shared among all delegates. We always keep alive the first static method delegate passed to native, by creating a normal gchandle to it and storing it in delegate_hash_table. For instance methods, each delegate will create a separate wrapper and these wrappers are never shared (which was wrongly assumed in https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b). We shuldn't keep the delegate alive and this commit reverts the behavior for delegate with instance methods introduced in the mentioned commit.

Fixes https://github.com/mono/mono/issues/15751

Backport of #15935.
2019-08-01 20:39:33 +02:00
monojenkins 8b837cb9ca [2019-06] Bump corefx (#15962)
This PR brings changes from mono/corefx#313.
Fixes #15887.

Backport of #15932.
2019-08-01 18:39:40 +02:00
monojenkins 84bc821cdc Fix Socket.ConnectAsync(SocketAsyncEventArgs) behavior. (#15949)
We need to clearly distinguish between synchronous and asynchronous completions
and avoid any ambiguities and race conditions when it comes to error handling.

On synchronous completion, we must not invoke the async completion delegate.

In `System.Net.Sockets/Socket.cs`, the internal `BeginSConnect()` and `BeginMConnect()`
now return a `bool` indicating whether or not an async operation is pending.

If any error happens prior to starting the async operation, then we shall call
`SocketAsyncResult.Complete(..., true)` and return `false`.  And since this is
a synchronous completion, it's `AsyncCallback` will not be invoked.

In `referencesource/System/net/System/Net/Sockets/_MultipleConnectAsync.cs`,
I compared the entire file against the CoreFX version and then copied the
`AttemptConnection()` method from their implementation to adapt their behavior.

The difference is that on synchronous completion, the `InternalConnectCallback(null, args)`
should be invoked.

Backport of #15947.
2019-08-01 12:33:37 +02:00
monojenkins f93add7b16 [threads] Enter GC Unsafe before pumping HP queue in unregister_thread (#15940)
[2019-06] [threads] Enter GC Unsafe before pumping HP queue in unregister_thread

If the thread info TLS key dtor runs `unregister_thread` from a foreign thread and we have work in the hazard pointer queue, we need to first switch to GC Unsafe mode because some of the free methods passed to `mono_thread_hazardous_try_free` need to be coop-aware.

Also, added checked mode assertions that `conc_table_free` and `jit_info_table_free` are called from GC Unsafe mode.

Fixes https://github.com/mono/mono/issues/15878


Backport of #15890.

/cc @lambdageek
2019-08-01 02:35:52 -04:00
monojenkins a5e772c4a1 [2019-06] [sdks] remove 32bit llvm (#15924)
* [sdks] remove 32bit llvm

* remove MonoAotOffsetsDumper leftover

* fix typo
2019-07-31 11:27:33 +02:00
monojenkins 0c44ef150f HANDLE_FUNCTION_RETURN_VAL parameter should not use handles in the current frame. (#15880)
Backport of #15873.
2019-07-30 21:05:36 +02:00
Alexander Köplinger 9df9494bcf [sdks] Disable WASM build on 2019-06 2019-07-30 21:02:35 +02:00
Alexander Köplinger 216d2b2724 [sdks] Make sure python3-pip is installed for Linux Android builds
(cherry picked from commit ee636ee8d9)
2019-07-30 20:46:29 +02:00
Bernhard Urban 17ac7dcfc7 [android] switch to python offset tool (#15848)
Backport of #15834

Warning: Disables WebAssembly build on `2019-06`.
2019-07-29 14:55:21 +02:00
Ankit Jain cec37277df Update nuget.exe to 5.2.0, matching msbuild (#15851)
This is a bit delayed as the 5.2.0 `nuget.exe` wasn't available earlier.
2019-07-27 02:18:49 +02:00
Ankit Jain 1b250ede0a [msbuild] Bump to track HEAD of mono-2019-06
.. to pick up update to track upstream `vs16.3`.
https://github.com/mono/msbuild/pull/118
2019-07-26 18:32:55 -05:00
Ankit Jain 844d5d7c91 Bump msbuild to track mono-2019-06 2019-07-26 18:32:55 -05:00
monojenkins c16ec8c31a Fix http framework module that errors using SDK 2.1.801 (#15854)
- 2.1.801 seems to have a problem with:

```
						using (var uint8Buffer = Uint8Array.From(await request.Content.ReadAsByteArrayAsync ()))
```
2019-07-26 23:39:18 +02:00
Alexander Köplinger 761220d914 [sdks] Report xcode11 builds as a separate GitHub status 2019-07-25 23:45:15 +02:00
Alexander Köplinger f06da77a36 [sdks] Add special build job for building with Xcode 11 2019-07-25 17:13:23 +02:00
Alexander Köplinger c6949d7e3d Bump bockbuild to get pkg-config fix for https://github.com/mono/mono/issues/13804 2019-07-25 00:21:19 +02:00
Vlad Brezae 0cd93e984c [sre] Make creation of dynamic method synchronized (#15814)
[2019-06][sre] Make creation of dynamic method synchronized

Before, reflection_create_dynamic_method could end up being called by two threads simultaneously. At the end of the method the ilgen field of the MonoReflectionDynamicMethod object is cleared, to enable its collection since it is no longer needed. The problem with this is that it can confuse the other thread compiling the dynamic method, making it see that there is no body.

We could either bother making this clearing work with multiple threads (and I'm not 100% convinced that the rest of the code is thread safe), disable the clearing of ilgen (which would make us use more memory) or go with the simple approach of locking on this DynamicMethod so only one thread is compiling it at one time.

Backport of https://github.com/mono/mono/pull/15803
2019-07-24 13:35:56 -04:00
monojenkins d8286ce40b Bump mono/corefx@eb5ba09 (#15813)
Backport of #15799.
2019-07-24 17:30:52 +02:00
Zoltan Varga 4b7a92e09a [runtime] Fix the error message for exceptions created from MonoError.
Fixes https://github.com/mono/mono/issues/15692.
2019-07-23 19:08:44 +02:00
monojenkins 31bf75ed23 [2019-06] Fix ambiguous time at the end of DST period on Windo… (#15785)
As an addition to #15401 it should resolve XA-related issue.

Fixes #9664

Backport of #15548.
2019-07-23 11:15:04 +02:00
Alexander Köplinger 4f29e313aa Add more tests to tarball with EXTRA_DIST
(cherry picked from commit f497893a84)
2019-07-22 15:23:26 +02:00
Bernhard Urban afcf28a366 [2019-06][ios] Switch to python offset tool (#15756)
The C# version doesn't work on 64bit. It would require to update its dependencies, but Zoltan already added a Python version for WebAssembly which works fine.

Some notes:
* changed the semantics of `--xcode-path` in order to match the output of `xcode-select -p`
* switched to python3/pip3
* removed hack for arm64_32, it works properly now because the Python version uses stock `clang`
* removed some leftover usages of `XCODE32_DIR`.

Manually verified by comparing output with the C# version.

This will unblock the iOS team when building on Catalina/Xcode11, as they can't consume SDK archives there. See xamarin/xamarin-macios#6603 (comment)

Contributes to #9621
 master (#15744)
2019-07-19 23:14:17 +02:00
Alexander Köplinger 5bd33de631 [ci] Clean working directory after build in package and sdks build
(cherry picked from commit d159a38c47)
2019-07-19 23:02:19 +02:00
Alexander Köplinger a375e6a090 [packaging] Don't use dl.xamarin.com for gtksharp package anymore
(cherry picked from commit 6407d27000)
2019-07-19 23:02:09 +02:00
Alexander Köplinger 4db3b960c0 [sdks] Calculate sha1 of sdks archive and upload it
This was requested by the XA team.

(cherry picked from commit 49ed255707)
2019-07-19 20:06:31 +02:00
Alexander Köplinger ad226f5d37 [ci] Don't require master node for reportGitHubStatus method
In some jobs we use throttle() logic which would mean only one job can execute on master, blocking other jobs in the meantime.

We don't really care which node executes this so let Jenkins pick one.

(cherry picked from commit 7784f20a26)
2019-07-19 14:50:37 +02:00
Alexander Köplinger aef3c4c16b [sdks] Add linker sources to Android linux archive
(cherry picked from commit a02f3aaf4b)
2019-07-19 11:38:01 +02:00
monojenkins a2bcc639e1 [2019-06] [sdks] Add third party notices to sdks archives (#15755)
It takes all THIRD-PARTY-NOTICES.TXT and LICENSE files from the repo.
2019-07-19 11:09:57 +02:00
Bijan Tabatabai c636636084 Align mcs's tuple deconstruction behavior with csc
Before the code
(a, b) = (b, a);
was the equivalent of
a = b;
b = a;

now, it evaluates it to so something like
tmp1 = a;
tmp2 = b;
a = tmp2;
b = tmp1;

Signed-off-by: Bijan Tabatabai <[email protected]>
2019-07-19 05:01:11 -04:00
monojenkins 1c919ea11c [2019-06] Print two bytes at a time in mono_pe_file_map conversion error (#15726)
* Print two bytes at a time

* Iterate over gunichar2
2019-07-18 14:48:10 -07:00
monojenkins 4b15f9b3ee [mini] Runtime check if boxing is needed for DIM constrained calls (#15730)
This is to handle the following example.  The issue is that in Check() we have
a constrained call where the called method is IAdder`1<!!U>::PlusPlus() which
is a default interface method, and we need to determine at runtime whether to
box the this argument.

```
using System;

public interface IAdder<T> {
	int Add (int x);

	int PlusPlus () {
		return Add (1);
	}
}

interface IGen3<T> { }

struct Adder<T> : IAdder<IGen3<T>> {
	int _field;
	public int Add (int x) {
		_field = x + _field;
		return _field;
	}
}

public class P {
	public static int Check<T, U>(T t) where T : IAdder<U> {
		return t.PlusPlus () + t.PlusPlus ();
	}

	public static void Main () {
		var x = new Adder<object> ();
		int y = Check<Adder<object>, IGen3<object>> (x);
		Console.WriteLine ("expect 2, got {0}", y);
	}
}
```
2019-07-18 16:31:49 -04:00
Larry Ewing 17520a8622 Bump msbuild to msbuild:mono-2019-06 to stop disabling compiler server 2019-07-17 16:47:23 -04:00
monojenkins 048d95d86d Add com.apple.security.cs.allow-unsigned-executable-memory exception to hardened runtime entitlements (#15725)
This avoids the issue from https://github.com/mono/mono/issues/13445 when hardened runtime is enabled.
2019-07-17 20:50:23 +02:00
monojenkins c63edc338b [2019-06] [bcl] Sync recent NS2.1 changes (#15719)
1) Copied changes from corefx for System.Data (also mono/corefx#310)
2) New Tracing api is left empty (because the current API is empty too)
3) New AES types throw PNSE

Bump netstandard.dll AssemblyVersion to 2.1 (fixes #13412)

Backport of #15696.
2019-07-17 14:31:43 +02:00
Alexander Köplinger 74e1f4cbf9 Revert "[ci] Remove separate utils.groovy file" and run step on master instead
This reverts commit d732cef5fc.

The github commit status setter step requires a node context so we now just use on on master instead.

(cherry picked from commit 0509ceccb6)
2019-07-17 14:27:05 +02:00
monojenkins a79115dd7f [2019-06] [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools… (#15718)
* [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools profile

Requested by the XA team.

* [csproj] Update project files
2019-07-17 12:44:53 +02:00
Alexander Köplinger d410e07d8b [ci] Remove separate utils.groovy file
For some reason when using a separate file and loading the function
from there Jenkins still requires us to have a node() context when
calling the function which is not what we want.
2019-07-17 10:57:09 +02:00
Alexander Köplinger 058d207f58 [ci] Wait for signing outside of a node() block
We don't need to block the machine just for waiting on signing to finish.

(cherry picked from commit 39472e06a3)
2019-07-17 01:02:55 +02:00
monojenkins a5b77554e4 [2019-06] [System] Keep events from firing from a disposed 'De… (#15706)
Fixes https://github.com/mono/mono/issues/8747

Also try to stop other implementations from dispatching events past Dispose(), without guaranteeing it
2019-07-16 21:45:40 +02:00
Alexander Köplinger ce68c20971 [ci] Bump timeout for .pkg signing job
In preparation for notarizing our .pkg with Apple we need to bump the timeout
since it can take up to an hour for Apple's server to return a result.

(cherry picked from commit 7480830b36)
2019-07-16 21:32:31 +02:00
Alexander Köplinger edc9e70e04 [sdks] Use Azul Zulu OpenJDK for Android Linux build, since Debian 10 has no Java 8 2019-07-15 23:24:20 +02:00
Alexander Köplinger ea8da188ef [sdks] Update android linux builds to Debian 10
This was suggested by XA folks to fix the weird mingw llvm issues we've been seeing.

(cherry picked from commit 4b6f49bb06)
2019-07-15 21:36:27 +02:00
monojenkins fe2a0a8810 [2019-06] [debugger] Don't list finalizer gc thread if it's not executing managed code (#15683)
* The example that reproduces the bug does this: var thread = vm.GetThreads()[0]; greetingValue = program.InvokeMethod(thread, greeting, new Value[0]) as StringMirror;

But sometimes the Thread[0] was the MainThread of the program and it works, and other times was the GC Finalizer, but when it tries to run something on GC Finalizer thread, the thread is not suspended and throws an exception.
In the fix I removed the Finalizer thread of the list that is returned when CMD_VM_ALL_THREADS is called.

Fix #13311

* The example that reproduces the bug does this: var thread = vm.GetThreads()[0]; greetingValue = program.InvokeMethod(thread, greeting, new Value[0]) as StringMirror;

But sometimes the Thread[0] was the MainThread of the program and it works, and other times was the GC Finalizer, but when it tries to run something on GC Finalizer thread, the thread is not suspended and throws an exception.
In the fix I removed the Finalizer thread if ti's not executing a managed code of the list that is returned when CMD_VM_ALL_THREADS is called.

Fix #13311

* Changing what was suggested by Aleksey.

* Removing unused attribute.
2019-07-15 17:14:15 +02:00
monojenkins 7fd5a4da49 [2019-06] Fix marshalling bugs for StringBuilder parameters (#15685)
Fixes #15366 and #15306.

- Fixes NULL termination of `StringBuilder` to UTF-16 marshalling. Additionally an extra NULL is added as defense-in-depth protection for incorrect handling in native calls. This extra NULL adds protection when trying to marshal the NULL-terminated string back and the native function doesn't null-terminate at the size of the buffer.
- Fixes marshalling back from UTF-16/UTF-8 buffers into chunked `StringBuilder`.
- Fixes missing `mono_marshal_free` call causing a memory leak when marshalling `StringBuilder` to UTF-16 and back.
- Skips marshalling back to `StringBuilder` when the parameter doesn't have `[Out]` attribute.

Unit tests are still TBD. Not sure where to put them and how to structure them. Any help would be appreciated. /cc @vargaz @EgorBo 

Backport of #15392.
2019-07-15 16:45:55 +02:00
monojenkins f688049114 [2019-06] [ios] Nil Check to Ensure Local TimeZone Id is Set F… (#15690) 2019-07-15 16:06:45 +02:00
Alexander Köplinger a56786e89e [System] Remove FileSystemWatcher tests from monotouch/xammac
They were already excluded from watch and tv but they don't work on any mobile Apple profile.
2019-07-15 15:16:09 +02:00
monojenkins 675c744a7e [2019-06] [ios] Ensure Local TimeZone Id is Set For TimeZoneIn… (#15670)
* Fixes https://github.com/mono/mono/issues/14839

The local timezone name was not known and defaulted to "Local".
This change adds an sdk function that returns the local name.

Backport of #15667.
2019-07-15 15:11:19 +02:00
Alexander Köplinger 5098258cce [ci] Upload mac-entitlements.plist to azure when uploading .pkg
So it can be used by the codesigning job.

(cherry picked from commit e3d5b23ff9)
2019-07-12 19:48:08 +02:00
Jo Shields 4ec4736af4 Add hardened runtime testing to Catalina lane (#15589)
* Permit DYLD mangling, so p/invokes work from non-system paths

* Add com.apple.security.cs.disable-library-validation

This is required for the .dylibs used in p/invokes to work.

* Use adhoc signing

(cherry picked from commit 202dca075b)
2019-07-12 19:48:08 +02:00
monojenkins 5debe32373 [2019-06] Fix offsets tool for Android NDK r20 (#15650)
Android NDK r20 contains a header file (`ioctl.h`) with this code:

    int ioctl(int __fd, int __request, ...);

    #if !defined(BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD)
    /* enable_if(1) just exists to break overloading ties. */
    int ioctl(int __fd, unsigned __request, ...) __overloadable __enable_if(1, "") __RENAME(ioctl);
    #endif

The `_RENAME` macro is defined as `__asm__(x)` and serves the purpose of
replacing an existing symbol with the new one, decorated with the keyword.
Unfortunately, Mono's `offsets-tool` doesn't parse the header properly as
`CppSharp` doesn't understand `__asm__`:

    Processing triple: armv7-none-linux-androideabi
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/metadata/metadata-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/mini/mini-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    make: *** [/Users/rodo/git/xa2/external/mono/sdks/builds/android-cross-arm-release/armv7-linux-android.h] Error 1

The workaround, implemented in this commit, is to define the
`BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD` macro thus removing the "conflicting"
declaration from the picture, which changes nothing from the offset tool's point
of view and lets us process the headers in NDK r20 properly.
2019-07-11 20:39:42 +02:00
Alexander Köplinger 5898c9fd07 Bump bockbuild to fix gtk build break 2019-07-11 17:30:51 +02:00
Alexander Köplinger aed9c000ec Bump bockbuild to get https://github.com/mono/bockbuild/pull/105 2019-07-11 16:09:22 +02:00
monojenkins ba719acaa2 [2019-06] Add TypeForwards for FileSystemName & CryptographicOperations (#15657)
* Add TypeForwards for:

* System.IO.Enumeration.FileSystemName
* System.Security.Cryptography.CryptographicOperations

Since these were removed in NS 2.1

* Bump API snapshot submodule
2019-07-11 01:10:10 +02:00
Steve Pfister dfd02351f4 [2019-06] Bump mono/corefx@17dd540 (#15653) 2019-07-10 23:57:36 +02:00
monojenkins 4c710e041c [2019-06] [System]: Allow reading fully buffered `HttpWebRespo… (#15638)
[System]: Allow reading fully buffered `HttpWebResponse` body after close.

If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.

* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
   call down after argument checks and call a new protected abstract method
   called `TryReadFromBufferedContent ()` prior to it.

* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
  If the entire response body has been buffered, then `bufferedEntireContent`
  is true and `innerStream is BufferedReadStream` and we can call the new
  `BufferedReadStream.TryReadFromBuffer ()`.

* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
   after reading the entire response body.

* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.

Fixes #10645.

Backport of #15220.
2019-07-10 10:45:12 +02:00
monojenkins d99e13bdea [2019-06] [llvm] avoid FP elimination on iOS/armv7 (#15621)
[2019-06] [llvm] avoid FP elimination on iOS/armv7

Instead of
```
push    {r4, r5, r6, r7, r10, r11, lr}
```

LLVM will generate
```
push    {r4, r5, r6, r7, lr}
add     r7, sp, #12
push    {r9, r10}
```

Seems like this https://github.com/mono/llvm/commit/a04e9e4a0af16f15ace258e81448b7eeca5ff599 assumes that `-disable-fp-elim` is passed for iOS targets. I wondered why this wouldn't break things for them, but when you run `clang` from Xcode with `-v`, you will discover that its driver passes `-mdisable-fp-elim`.

Together with https://github.com/mono/llvm/pull/48 this fixes crashes with FullAOT+LLVM on iOS 32bit.

Fixes https://github.com/mono/mono/issues/15058

Contributes to https://github.com/mono/mono/issues/9621



Backport of #15617.

/cc @lewurm
2019-07-09 17:25:44 -04:00
Alexander Köplinger c608cf3eaf Make RemoteExecutor a no-op on all mobile profiles (#15611)
Starting another Mono process for the remote executor doesn't work on any of them.

Helps with https://github.com/mono/mono/issues/15328

(cherry picked from commit b2ba8e62ca)
2019-07-09 11:45:35 +02:00
monojenkins 90d32a9d62 Crashes on running the boring.il test from coreclr, the problem was that using [System.Runtime]System.String the type was not understood as the primitive type string. Using ilasm from coreclr and running on mono it works, because of that I decided to fix on ilasm and not on runtime. (#15609)
Fixes #13923
2019-07-08 17:48:55 -03:00
Alexander Köplinger 329145ee73 [sdks] Fix android sdks job, take 2
It needs to work for both debug and release configs.

(cherry picked from commit 784449a077)
2019-07-08 19:35:09 +02:00
Alexander Köplinger a48723b842 [sdks] Fix android sdks job
It would try to execute the .stamp-android-loader-path target in unrelated configurations.

(cherry picked from commit 305b2e76e6)
2019-07-08 18:06:22 +02:00
Nikita Voronchev 08dd9882ce [2019-06] [System.Windows.Forms] Fix crash of DataGridView in in case of assigning a custom Column Header. (#15461)
This PR fixes the crash of DataGridView in case of assigning a custom Column Header.

Backport of #15135
2019-07-05 17:42:07 +02:00
monojenkins 16bf40461d [2019-06] [android] switch cross compilers host from 32bit to… (#15581)
Untested, but we need to do it anyway soon. This will help us to understand the situation on Xamarin.Android while finishing the work for Xamarin.iOS.

Contributes to https://github.com/mono/mono/issues/9621

Backport of https://github.com/mono/mono/issues/15553.
2019-07-05 17:41:08 +02:00
monojenkins 91f98eb2c8 [2019-06] [sdks] Use @loader_path for mono and libmonosgen (#15587)
* [sdks] Use @loader_path for mono and libmonosgen

This way we are able to dynamically load libmonosgen and also use host
mono binary.

The library dependencies look like this after the change:
```
otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...

otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...
```
2019-07-05 17:40:30 +02:00
Steve Pfister 2629c6d8f8 [2019-06] Bump mono/corefx@9354dfd (#15547) 2019-07-05 14:34:55 +02:00
Ankit Jain 972d548394 [2019-06] Bump msbuild to track mono-2019-06 (#15525)
* Bump msbuild to track mono-2019-06

* Bump nuget.exe to 5.1.0 to match what msbuild has.
2019-07-03 10:57:13 +02:00
Alexander Köplinger 39e8153c00 [2019-06] Bump corefx (#15519)
* [2019-06] Bump corefx

* Bump API snapshot submodule
2019-07-03 01:50:47 +02:00
monojenkins 0f75b36c0f [2019-06] [System.Net.Http]: Delete broken `Send_Transfer_Enco… (#15496)
This test was checking a broken Mono-specific behavior, which has been
removed after we switched to the new corefx-based handler. It also does
not make much sense to use a `Transfer-Encoding` header with a GET request.
2019-07-02 01:29:29 +02:00
Alexander Köplinger ffbba36f45 [2019-06] Bump bockbuild to bring in fix for mono/mono#14957 2019-07-02 00:32:12 +02:00
Thays Grazia 567964aff5 Fixes #13925. (#15489)
Backport https://github.com/mono/mono/pull/15433 to 2019-06.
2019-07-01 19:01:09 -03:00
Aleksey Kliger f55e0a671c [loader] remap only System.Threading.Tasks.Dataflow older than 4.0.0.0
Allow through requests for newer versions unchanged.
2019-07-01 17:20:56 +02:00
monojenkins 3702e62c7d [jit] Avoid calling mini_jit_info_table_find () in mini_add_method_trampoline () unless needed. (#15414) 2019-06-30 19:07:07 -04:00
monojenkins 41aad78a9d CreateNLSocket and CloseNLSocket should return gpointer (#15431)
In managed, these functions are used as `static external IntPtr`. This means that previously on arm64 the top bits of the return value were garbage. A comparison with 64-bit -1 like in LinuxNetworkChange.EnsureSocket would never be true, which was causing us to hit other assertions in the runtime.
Co-authored-by: Aleksey Kliger <[email protected]>
2019-06-30 15:06:13 -04:00
Aleksey Kliger fc63d85a38 [sdks] dynamically link libmonosgen to mono-sgen for monodroid host builds 2019-06-28 21:46:35 -05:00
Aleksey Kliger 605f7f04fc [runtime] Mark mono_file_map_error as MONO_API, always
Fixed the `--with-static_mono=no` build on non-Win32
2019-06-28 21:46:35 -05:00
Aleksey Kliger d8e2a049fc [configure] Cosmetic AC_MSG_WARN fix 2019-06-28 21:46:35 -05:00
monojenkins 9fd0d07cfb Correct env vars used for Android directory override (#15471)
This fixes https://github.com/mono/mono/pull/15453

The issue is the env var names differ vs. Xamarin.Android
2019-06-28 15:33:20 -04:00
monojenkins 8c50c1a047 [2019-06] Use /mnt/scratch for Android SDK, to match https://g… (#15468) 2019-06-28 19:23:35 +02:00
monojenkins 89867fd4ea [2019-06] Fix year+ old MIPS compile break. (#15460)
https://github.com/mono/mono/issues/15441
2019-06-28 11:29:50 +02:00
monojenkins 25a7d8c97c [2019-06] Fix time zone transition out of DST (#15445)
* Time zone DST out fix

* Add unit tests for Europe/Vatican, Iran and Europe/Guernsey time zones

* Disable the tests on WINAOT; fixed Iran time zone id
2019-06-27 20:48:40 +02:00
monojenkins dec365d6fd [sgen] Fix crashes in sgen due to xdomain refs (#15426)
CADMessages are passed directly between appdomains. When using contained data, we need to make sure we don't create any cross domain references.

When domains are unloaded, all threads executing code in that appdomain are aborted and the entire heap is traversed and objects in that appdomain are cleared. If any objects in other domains are directly referencing objects in the unloaded domain, they will reference junk. If any, cross domain refs should only be allowed when the referenced object is in the root domain which is never unloaded.
2019-06-27 12:37:19 +02:00
lateralusX 68d1fceba9 Fix race related to final event signaling in RegisteredWaitHandle.
We have seen sporadic failures in thread pool tests on CI related to
test case:

System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
          at System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest () [0x00143] in D:\j\workspace\z3\label\w64\external\corefx\src\System.Threading.ThreadPool\tests\ThreadPoolTests.cs:337
          at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
          at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in D:\j\workspace\z3\label\w64\mcs\class\corlib\System.Reflection\RuntimeMethodInfo.cs:395

Finally managed to track down the issue to a race when using the same final
event in sequential calls to RegisterWaitForSingleObject. The event could, depending on
timing, get signaled multiple times for the same RegisterWaitHandle instance
causing a thread waiting for completion on another RegisterWaitHandle using same event
to be signaled before that callback was completed. That will cause the race seen in the
failing test above.

Fix make sure we only signal the final event one time per RegisterWaitHandle
instance.
2019-06-26 07:12:16 -05:00
monojenkins 9bfdc5c38b [2019-06] [jit] Fix interface cast (#15375)
* [jit] Throw exception if casting fails for special array interface

After the standard interface check (mini_emit_iface_cast), for special interfaces we need to do additional checks. We were skipping these checks if the object was not an array, without throwing a cast exception. This code path is reached for configurations where remoting is disabled.

Fixes https://github.com/mono/mono/issues/14729

* [tests] Add test for interface casting case

This test would have failed on fullaot lane for example.
2019-06-26 11:38:39 +02:00
Alexander Köplinger ce79aff5be [System] Fix monotouch_watch xunit tests build
Forgot to exclude one more file.

(cherry picked from commit 41767754d5)
2019-06-25 22:22:42 +02:00
monojenkins f481f794ce [2019-06] Exclude a few tests on monotouch_watch/xammac (#15409)
* [System.Core] Exclude AnonymousPipe tests on monotouch/xammac profiles

They should only be run on xammac_net_4_5.

Fixes https://github.com/mono/mono/issues/15329

* [System] Exclude WebClient and WebSocket tests on monotouch_watch

Fixes https://github.com/mono/mono/issues/15265.
2019-06-25 20:03:44 +02:00
Martin Baulig ed3321827e [test-helpers]Don't use named Mutexes on mobile. 2019-06-25 08:46:26 +02:00
Alexander Köplinger c5d9285ad3 [sdks] Do the WATCHOS5_VERSION -> WATCHOS64_32_VERSION rename in run-jenkins.sh too
It was missed in https://github.com/mono/mono/pull/15202
2019-06-24 19:14:14 +02:00
Bernhard Urban d707d745a6 [arm] set exception handling model to dwarf for iOS
We recently switched the AOT compiler for iOS armv7 from 32bit host to 64bit host. At the same time we _also_ switched from LLVM 3.6 to the newer LLVM 6.0 fork we support.

In a Xamarin.iOS Release build for iOS armv7 exception handling would crash. Specifically the problem was that the exception object wasn't properly injected into the handler block. The way it works is, that the exception handler machinery passes the pointer to the exception object via `r0`.

With the newer LLVM the generated code of the exception handling block
looks like this:
```
LBB1_15:                                @ %EH_CLAUSE0_BB3
Ltmp8:
	ldr	r0, [sp, #24]
	str	r0, [sp, #4]
	ldr	r0, [sp, #28]
	ldr	r0, [sp, #12]
	ldr	r0, [r0]
	cmp	r0, #0
	beq	LBB1_17
```
thus overwriting `r0`, so that the pointer to the exception object is lost.  With the older LLVM 3.6 the generated code looks like this:
```
LBB1_8:                                 @ %EH_CLAUSE0_BB3
Ltmp8:
	str	r0, [sp, #4]
	ldr	r0, [r6]
	cmp	r0, #0
	beq	LBB1_10
```
correctly storing the exception object into a stack slot for later usage.

After some time I figured out that there are _different_ exception handling models. Depending on the model, the exception pointer is passed in `r0` or not:
https://github.com/mono/llvm/blob/2bd2f1db1803f7b36687e5abf912c69baa848305/lib/Target/ARM/ARMISelLowering.cpp#L14416-L14428

`SjLj` stands for "SetJump / LongJump" and  means that exception handling is implemented with this. On iOS this is the default model that is used. We want `dwarf` instead, so we tell this `llc` now and the generated code handles it correctly.

So why was it working before? In our older LLVM 3.6 fork we hardcoded it:
https://github.com/mono/llvm/blob/f80899cb3eb75f7f5640b4519e83bd96991bffb8/lib/Target/ARM/ARMISelLowering.cpp#L756-L762

The `-exception-model=` option is also not available in LLVM 3.6.

Contributes to https://github.com/mono/mono/issues/15058 and https://github.com/mono/mono/issues/9621
2019-06-24 09:48:54 +02:00
Zoltan Varga d0294cf582 [aot] Fix a deadlock in init_plt () by calling create_trampoline () outside the lock.
Fixes https://github.com/mono/mono/issues/15330.
2019-06-24 09:33:44 +02:00
Vlad Brezae 3b576e58c7 [arm64] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-24 09:22:39 +02:00
Vlad Brezae 2878c33dfe [arm] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-24 09:22:39 +02:00
Vlad Brezae 1398a3d33a [amd64] Fix location saving for *this*
With static methods we were saving location for irrelevant argument.
2019-06-24 09:22:39 +02:00
Aleksey Kliger (λgeek) 6bed4fc2ff [ci] One more workaround for Git 2.22.0 (#15325)
Homebrew Git 2.22.0 misparses the submodule command and passes arguments like --hard and -xdff to git-submodule instead of to git-reset or git-clean.  Passing the entire command as a single argument seems to help.

Git mailing list bug report: https://public-inbox.org/git/[email protected]/
2019-06-21 10:57:25 -04:00
monojenkins f74dc9a395 [2019-06] [debugger] Debugger doesn't break on unhandled exceptions (#15286)
* When the catch found is in a MONO_WRAPPER_RUNTIME_INVOKE the exception should be treated as an UNHANDLED by the debugger.

* Adding unit test.

* trying to fix side effect.

* I've tested the MonoTests.DebuggerTests.UnhandledException_2 test on VsWin using .net Framework and I think the test is wrong because the first break is in throw and the next break will be on breakpoint. In our implementation we were ignoring the unhandled exception and stopping only on breakpoint.
2019-06-21 11:24:21 +02:00
Larry Ewing b2987f714b Merge pull request #15292 from monojenkins/backport-pr-15255-to-2019-06
[2019-06] [jit] Make alias analysis properly handle alias cloberring
2019-06-20 22:42:53 -05:00
Vlad Brezae b869c961ce [jit] Make alias analysis properly handle alias cloberring
Alias analysis attempts to remove indirection when using OP_LDADDR (which fetches the address of a variable). For example :

	ldaddr r1 <- r0
	load_mem r2 <- r1

is equivalent to

	mov r2 <- r0

The remembered r1 alias needs to be forgotten in case r1 is changed. We were handling this for direct assignment with MOV, but not for the more subtle case where an alias to r1 would be passed to a call, which could lead to changing r1.

https://github.com/mono/mono/issues/14872
2019-06-20 18:42:58 +00:00
monojenkins 3379668399 [csproj] Update project files 2019-06-20 18:49:25 +02:00
Marek Safar a49a3ea553 Bump linker & cecil 2019-06-20 18:49:25 +02:00
Bernhard Urban 71e0854e2a [arm] fix some more host/target pointer size issues
As suggested by Jay in https://github.com/mono/mono/issues/15058#issuecomment-501995400

I haven't seen fixing it any specific issue, but we should still backport it to `2019-06` in order to avoid hard-to-debug regressions.

Contributes to https://github.com/mono/mono/issues/9621
2019-06-19 12:58:52 +02:00
Alexis Christoforides 9d1153af22 Improve robustness of post-install script
We have several issues (e.g. https://github.com/mono/mono/issues/14181) where the installer occasionally fails in the postinstall script for unknown causes, on some hosts.

This seems to be sometimes the culpable section and this patch should help trace the issue or alleviate the common symptoms of it (see https://github.com/mono/mono/issues/14181)
2019-06-19 08:48:57 +02:00
Aleksey Kliger 6091b8f209 [coop] Mark a few functions with MONO_PROFILER_API 2019-06-18 19:44:16 +02:00
Aleksey Kliger 7fcdd9f318 [runtime] Don't do GC Unsafe transition in mono_gchandle_free
Xamarin.Mac can call back here from a native thread that isn't attached to the
runtime.  SGen gchandle code is lockfree, and the profiler event can fire
without involving Mono.
2019-06-18 19:44:16 +02:00
Aleksey Kliger 5c36c00915 [profiler] Don't switch to GC Safe when locking if thread isn't attached to the runtime.
Some profiler events (for example gc_handle_deleted) could be raised on threads
that aren't attached to the runtime.  Since the thread won't participate in
Mono's stop the world (since the runtime can't see it), there's no need to do
the transition to GC Safe while the thread is trying to grab the profiler
buffer lock.
2019-06-18 19:44:16 +02:00
monojenkins a099adcbce [2019-06] Implementing reabstraction of default interface methods. (#15229)
* Implementing reabstraction of default interface methods.

* Fixing side effect on unit tests. Tested with another example and changed the implementation: using System; public interface I1 { void M1(); /*void M2(){ System.Console.WriteLine("I1.M2"); }*/ }

public interface I2 : I1
{
    abstract void I1.M1();
    /* void M2(){
        System.Console.WriteLine("I2.M2");
    }*/
}

class Test1 : I2
{
    static void Main()
    {
        I1 i1 = new Test1();
        i1.M1();
        //i1.M2();
    }

    void I1.M1()
    {
        System.Console.WriteLine("Test1.M1");
    }
}

* Fixing side effect on unit tests.

* Fixing side effect on unit tests.

* Implementing unit test. Fixing side effect.

* Fix unit test.

* Fixing reabstraction.il coreclr test.

* Changing comments.

* Changing things pointed by Aleksey.

* Changing things pointed by Aleksey. Implementing generics test.

* Changing things pointed by Aleksey.
2019-06-18 19:43:25 +02:00
Bernhard Urban 70732b33a8 [watch] consistent naming for watchOS min version
See https://github.com/xamarin/xamarin-macios/pull/6307
2019-06-18 16:38:09 +02:00
Bernhard Urban d68b18c9c2 [arm] use target_mgreg_t instead of plain pointer type in sdb breakpoint
Fixes Debug mode for iOS 32bit.

Contributes to https://github.com/mono/mono/issues/15058 & https://github.com/mono/mono/issues/9621
2019-06-17 16:52:42 +02:00
Aleksey Kliger 02311a91c1 [debugger-agent] Ignore thread_end event if thread is detached
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set.  In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.

Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace

```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
  * frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
    frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
    frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
    frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
    frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
    frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
    frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
    frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
    frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
    frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
    frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
    frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
    frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
    frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
    frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
2019-06-15 09:31:20 +02:00
Aleksey Kliger 0d00315b97 don't use 'gpointer' in a public header 2019-06-15 09:29:36 +02:00
Aleksey Kliger f73b774483 [mini] Move some MONO_API functions to public header
And mark them all MONO_RT_EXTERNAL_ONLY.

mono_set_use_llvm is new - Xamarin.Android previously assigned to mono_use_llvm
directly.
2019-06-15 09:29:36 +02:00
Jay Krell 507c4d8c99 Add missing read barrier to mono_lazy_initialize().
This is unfortunate.

The general pattern of:

if initialized
  use the data
else:
  initialize the data, possibly with lots of locks and barriers
  mark initialized

is racy, because "use the data" can be scheduled ahead of "if initialized".

"Barriers come in pairs" generally, and this was missing one.

It depends somewhat. If the data is all pointers, initialized in the else path,
from null to non-null, and use includes dereferencing, which is a common
but not universal case, then it is ok on all non-Alpha processors, due
to "data dependency".

But if the data includes reading globals, then there is a race.

This is a bit of a slow down on fast paths on arm, and possibly
other architectures.

There are barrier-free ways to solve this, involving a thread local,
that should seriously be considered, and applied throughout the runtime.

The runtime has a lot of on-demand initialization and a lot of looks racy.

See www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2660.htm.
Which would have to be adopted to be coop-friendly which should not be difficult.

Notice that the existing "lazy" mechanism is also not coop-friendly.
That is not changed by this PR.
2019-06-14 23:53:49 +02:00
Jay Krell b60c60f2ab Fix bug and race in mono_lazy_initialize. 2019-06-14 23:53:49 +02:00
monojenkins c15b300298 [2019-06] [System.Net.Http]: Add hack to the legacy HttpClient to pass down Timeout property. #12577. (#15014)
* [System.Net.Http]Add hack to the legacy HttpClient to pass down Timeout property. #12577.

* `HttpClientHandler`, `IMonoHttpClientHandler`: add new internal `MonoSetTimeout (TimeSpan)` function.

* `HttpClient.SendAsyncWorker()`: if `handler` is `HttpClientHandler`, call the new internal
  `MonoSetTimeout()` function to pass down the timeout.

* `MonoWebRequestHandler.SendAsync()`: this is only used here to pass the timeout to the `HttpWebRequest`.

* Address feedback.

* Update HttpMessageInvoker.cs

* Make it build.

* Really make it build (forgot monotouch_watch).
2019-06-14 10:51:02 +02:00
monojenkins 6752005e64 [2019-06] Fix Japan's new era fallback isn't reflected in Mono (#14984)
* Fix Japan's new era fallback isn't reflected in Mono

refs: <https://github.com/dotnet/coreclr/pull/23614>

* Fix array length

* Fix MonoTests.System.Globalization.CalendarTest

* Fix MonoTests.System.Globalization.EastAsianLunisolarCalendarTest

* Fix era index

refs: 1715e1d0e738ce95382ba372c24502967002a2f0

* Amend fixing

refs: 05af6e6f75fcea76e3d9e9f2f95f0caedb753aa7
2019-06-14 10:50:09 +02:00
Ryan Lucia a2424f031e Actually print out *filename correctly 2019-06-14 10:49:46 +02:00
Ryan Lucia 1517a02c09 Naming: err -> error 2019-06-14 10:49:46 +02:00
Ryan Lucia c7a57444dc Switch everything to MonoError 2019-06-14 10:49:46 +02:00
Ryan Lucia 7cf9bf95d4 Remove todos 2019-06-14 10:49:46 +02:00
Ryan Lucia 1477c73eba Print input string when conversion fails 2019-06-14 10:49:46 +02:00
Ryan Lucia 897621178b Remove redundant goto 2019-06-14 10:49:46 +02:00
Ryan Lucia f85685911c Migrate things to _error version where sensible 2019-06-14 10:49:46 +02:00
Ryan Lucia 7490bdd567 Add internal version of mono_unicode_to_external to propagate error 2019-06-14 10:49:46 +02:00
Imran Hameed 23bac8c6e6 Move ternary SSA cfold test from basic.cs to objects.cs. 2019-06-13 23:48:21 -05:00
Imran Hameed e1ebf711eb Treat ternary operations as non-constant... (#13873)
...during constant evaluation and propagation.
2019-06-13 23:48:21 -05:00
monojenkins 46f7a64e38 Bump API snapshot submodule 2019-06-13 08:42:09 +02:00
Zoltan Varga a020195ec7 [corlib] Add missing Unsafe.Unbox () method.
Fixes https://github.com/mono/mono/issues/14871.
2019-06-13 08:42:09 +02:00
Aleksey Kliger 43f00b12ad [llvm] bump to mono/llvm@2bd2f1db18
Emit .fnend directives on arm even for functions which don't have a corresponding mono method.

Fixes https://github.com/mono/mono/issues/15048
2019-06-13 08:38:52 +02:00
Aleksey Kliger e126d67a6e [runtime] Switch to GC Unsafe for all GC external API functions
Fixes https://github.com/mono/mono/issues/14975
2019-06-12 12:55:27 -05:00
monojenkins 0ef4a3b0d9 Trying to unrevert https://github.com/mono/mono/pull/7390. Couldn't reproduce the same side effect as that caused before. (#14969)
This PR fixes #7377
2019-06-12 14:44:28 -03:00
Zoltan Varga 9a5bddb57d [runtime] Fix gshared support for BeginInvoke ()/EndInvoke () wrappers.
Fixes https://github.com/mono/mono/issues/14972.
2019-06-12 18:56:09 +02:00
Zoltan Varga 05e324bce7 [aot] Open AOT profile files using "rb" so it works on windows.
Fixes https://github.com/xamarin/xamarin-android/issues/3168.
2019-06-12 08:31:14 +02:00
monojenkins b96e503690 [2019-06] [llvm] Fix disabling direct calls and inlining (#14886)
* [llvm] Enable disabling inlining on llvm

* [llvm] Fix disabling direct calls on llvm

* [llvm] Rename mono_inited even when doing self_init (generated init functions refer to it)
2019-06-11 18:44:39 +02:00
monojenkins c2c9c2e012 [install] Include .exe extension when symlinking the mono binary (#14965)
Prevent the SDKs from making dangling symlinks when packaging the MXE
cross-compilers: should hav

    mono.exe -> mono-sgen.exe

rather than

    mono -> mono-sgen (dangling).
2019-06-11 07:31:29 -04:00
Martin Baulig 547b34f0ed Do the same thing in HttpListener. 2019-06-08 11:24:54 +02:00
Martin Baulig 63a2ab73ed Implement RSACertificateExtensions.CopyWithPrivateKey(). #14152.
* RSACertificateExtensions.CopyWithPrivateKey(): implement.

Since `RSACertificateExtensions` lives in `System.Core` and only `mscorlib`
internals (but not `System` internals) are visible to it, we need to use a
little trick here:

* System.Security.Cryptography.X509Certificates.X509CertificateImpl:
  add new abstract methods CopyWithPrivateKey() and CreateCertificate().

* System.Security.Cryptography.X509Certificates.X509Certificate2Impl:
  sealed implement them here.

The `httpcfg` tool, only checks whether the key is valid, so we don't really
need to call `CopyWithPrivateKey()` (which requires us to take a dependency
on `System.Core`).
2019-06-08 11:24:54 +02:00
monojenkins ddef564e34 [2019-06] [runtime] Make cross-appdomain wrappers throw NIE for methods with ref struct args (#14867)
* [AppDomain] Cross-appdomain wrapper throws NIE for methods with ref struct arguments

If we have a ref struct and a method that takes it as an argument:

```
[Serializable]
public ref struct R {
  public int i;
}

public interface I {
  public void M1 (R r, SomeClass o);
}

public class M : MarshalByRefObject, I {
  public void M1 (R r, SomeClass o) { ... }
}
```

If we create an instance of M in another domain and try to call it:
```
  I obj = (I)otherDomain.CreateInstanceAndUnwrap (typeof(M).Assembly.FullName,
  nameof(M));
  R r;
  r.i = 42;
  SomeClass o = ...;
  obj.M1 (r, o);
```

Mono will need to create a wrapper that can invoke M.M1 in the other domain.
The way the wrapper works in `mono_marshal_get_xappdomain_invoke` is by
creating an array of all the arguments that need to be serialized (see
`mono_get_xdomain_marshal_type`) and then serializing the whole array (by
calling out to `System.Runtime.Remoting.RemotingServices.SerializeCallData`).
In the example it would make a two element array and try to serialize `r` and
`o` into it. For valuetypes it normally does this by boxing the argument.
However since `R` is a ref-struct, boxing it is not allowed.

This works on .NET Framework.

However for Mono we would need to serilize `R` without boxing it, which seems
challenging with our current setup.

So for now we generate a wrapper that just throws a NotImplementedException if
it is ever called.

This fixes https://github.com/mono/mono/issues/14809 (in that marshalling a
TextWriter across domains works again) but in an unsatisfactory way (because
you can't call any of the ReadOnlySpan<char> overloads on the transparent proxy
object).

* [corlib] Add back Serializable attribute on some reflection classes

Tests that use https://www.nuget.org/packages/NUnit.ApplicationDomain/
need to serialize MethodInfo (and hence MethodBase).

* [corlib] Add test for transparent proxies for classes with methods that take ref struct args

Regression test for https://github.com/mono/mono/issues/14809

* Bump API snapshot submodule
2019-06-07 13:07:13 +02:00
monojenkins fed776b886 [ios] switch cross32/crosswatch to be executed on x86_64 and use newer LLVM (#14856)
Contributes to https://github.com/mono/mono/issues/9621
Fixes https://github.com/mono/mono/issues/14841

Note that it breaks iOS 32bit on devices. The Apple Watch _might_ still works (only tested on `2019-02`).  It unblocks some build related issues on the Xamarin.iOS `2019-06` integration, and we have to do it anyway. I'll come back to it to fix device related issues.
2019-06-07 00:01:18 +02:00
Bernhard Urban 23b831032a [mach] cast objc_msgSend{,Super} usages (#14836)
Fixes #14792

(cherry picked from commit 812357ec51)
2019-06-06 20:16:03 +02:00
monojenkins 2efb7058f5 [2019-06] [llvm] emit call to mini_llvmonly_throw_nullref_exception without exceptions in mind (#14846)
Fixes https://github.com/mono/mono/issues/14243
2019-06-06 19:24:26 +02:00
monojenkins efddb9595c [2019-06] [arm64] Set a llvm target layout. Assert in llvm if the target layout is not set, the defaults used by 'opt' are not correct. (#14844)
It is important to provide this, otherwise the `opt` command will make modifications of every GC safepoint load. For the SmartHotel360 Android app, this can result in `opt` taking 19 minutes and 20 seconds to finish the optimization of the Mono.Android.dll AOT modules.

### IR Example: 

Before Opt:
```
   >   %GC_SAFE_POINT_FLAG_7 = load i64*, i64** getelementptr inbounds ([166068 x i64*], [166068 x i64*]* @mono_aot_Mono_Android_llvm_got, i32 0, i32 7), !invariant.load !139597
    >   %4 = load volatile i64, i64* %GC_SAFE_POINT_FLAG_7
    >   %5 = icmp eq i64 %4, 0
    >   br i1 %5, label %CONT_BB2, label %POLL_BB1
```

After Opt:
```
    <   %GC_SAFE_POINT_FLAG_7 = load i64*, i64** getelementptr inbounds ([166068 x i64*], [166068 x i64*]* @mono_aot_Mono_Android_llvm_got, i64 0, i64 7), align 8, !invariant.load !139597
    <   %3 = load volatile i64, i64* %GC_SAFE_POINT_FLAG_7, align 4
    <   %4 = icmp eq i64 %3, 0
    <   br i1 %4, label %BB2, label %POLL_BB1
```

This has a huge impact on build time, giving InstCombine a huge slowdown. This change removes the need for opt to rewrite all of these basic blocks. 

Backport of #14714.
2019-06-06 18:48:49 +02:00
monojenkins c84d913f41 [2019-06] [sdks] Use 7z format for the android/ios/mac archives (#14838)
It has way better compression performance.
Kept WebAssembly on .zip for now since external users took dependencies on it [1].

Fixes https://github.com/mono/mono/issues/14215

[1] https://github.com/nventive/Uno.Wasm.Bootstrap/blob/e2bf8946e162642ca1bdf4f003f8de4a94954f73/Readme.md#updating-the-mono-sdk-build
2019-06-06 12:26:52 +02:00
Martin Baulig a59e941662 Bump CoreFX to pickup the fix for #14214.
HttpClient.SendAsync() should not attempt to read response body on a HEAD request.

* CoreFX PR https://github.com/dotnet/corefx/pull/38129.

* Backported in https://github.com/mono/corefx/pull/297.

* Fixes https://github.com/mono/mono/issues/14214.
2019-06-06 09:33:30 +02:00
Ankit Jain b33f230a80 Bump msbuild to track xplat-master (#14816) 2019-06-06 00:51:10 +02:00
monojenkins a2ffc3d3d5 [2019-06] Fix build on macOS 10.15 Catalina + xcode 11 (#14824)
1) CC_MD5_Init, CC_MD5_Update and CC_MD5_Final are deprecated.

```c
mono-md5.c:45:2: error: 'CC_MD5_Init' is deprecated: first deprecated in macOS 10.15 - This function is
      cryptographically broken and should not be used in security contexts. Clients should migrate to SHA256
      (or stronger). [-Werror,-Wdeprecated-declarations]
        CC_MD5_Init (ctx);
        ^
```

2) objc_msgSend is deprecated
```
mono-threads-mach-helper.c:95:17: error: too many arguments to function call, expected 0, have 2
                objc_msgSend (value, release);
                ~~~~~~~~~~~~  ^~~~~~~~~~~~~~
```
XCode has a special switch `Enable Strict Checking of objc_msgSend Calls` which is now `true` by default. It's fixed by `-DOBJC_OLD_DISPATCH_PROTOTYPES=1` flag to clang or the corresponding define in the source file.

3) Fixes https://github.com/mono/mono/issues/14793

It worked on pre 10.15 without `-framework GSS`, because it got pulled in through the dependencies. You can see it by running DYLD_PRINT_LIBRARIES=1 csharp and then trace it back. On Mojave /System/Library/Frameworks/GSS.framework/Versions/A/GSS got loaded as dependency of /usr/lib/libcups.2.dylib and so on. Unlike Windows, macOS will resolve the symbols to whatever is already available so it was just happy that someone loaded GSS before libmono-native[-compat/-unified].
2019-06-06 00:47:14 +02:00
monojenkins 8cfcd89a7a [2019-06] Reflect recent NS2.1 changes (#14822)
Due to few recent updates to dotnet/standard (NETStandard 2.1) we had 40 apicompat issues.
Mostly copy-pasted from dotnet/corefx.
Didn't implement Cryptography stuff yet.
Checked all profiles.

Implements #14609

Backport of #14679.
2019-06-05 21:47:53 +02:00
Alexander Köplinger f1fda814c2 [tests] Exclude more nunit categories on ios/mac (#14807)
A few more that we missed adding to the exclude file in the sdks archive.

(cherry picked from commit 3d9513efd4)
2019-06-05 16:01:44 +02:00
Aleksey Kliger 8ec87bbaec [runtime] Autoconf check for pthread_mutexattr_setprotocol
Android doesn't always declare that function.
2019-06-05 09:55:46 +02:00
monojenkins 1e6cd29e60 [2019-06] [Mono.Profiler.Log] Add profiled data reader APIs (#14763)
* [Mono.Profiler.Log] Add profiled data reader APIs

* Update API

* [csproj] Update project files
2019-06-05 01:24:46 +02:00
monojenkins 829949df54 [2019-06] [arm64] Correct exception filter stack frame size (#14770)
[2019-06] [arm64] Correct exception filter stack frame size

Fixes: https://github.com/mono/mono/issues/14170
Fixes: https://github.com/xamarin/xamarin-android/issues/3112

The `call_filter()` function generated by `mono_arch_get_call_filter()`
was overwriting a part of the previous stack frame because it was not
creating a large enough new stack frame for itself.  This had been
working by luck before the switch to building the runtime with Android
NDK r19.

I used a local build of [xamarin/xamarin-android/d16-1@87a80b][0] to
verify that this change resolved both of the issues linked above.  I
also confirmed that I was able to reintroduce the issues in my local
environment by removing the change and rebuilding.

After this change, the generated `call_filter()` function now starts by
reserving sufficient space on the stack to hold a 64-bit value at the
`ctx_offset` location.  The 64-bit `ctx` value is saved to the
`ctx_offset` location (offset 344 in the new stack frame) shortly
afterwards:

    stp	x29, x30, [sp,#-352]!
    mov	x29, sp
    str	x0, [x29,#344]

As expected, the invocation of `call_filter()` now no longer modifies
the top of the previous stack frame.

Top of the stack before `call_filter()`:

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

Top of the stack after `call_filter()` (unchanged):

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

<details>
<summary>Additional background information</summary>

### The original lucky, "good" behavior for [mono/mono/2018-08@725ba2a built with Android NDK r14][1]

 1. The `resume` parameter for `mono_handle_exception_internal()` is
    held in register `w23`.

 2. That register is saved into the current stack frame at offset 20 by
    a `str` instruction:

           0x00000000000bc1bc <+3012>:	str	w23, [sp,#20]

 3. `handle_exception_first_pass()` invokes `call_filter()` via a `blr`
    instruction:

        2279							filtered = call_filter (ctx, ei->data.filter);
           0x00000000000bc60c <+4116>:	add	x9, x22, x24, lsl #6
           0x00000000000bc610 <+4120>:	ldr	x8, [x8,#3120]
           0x00000000000bc614 <+4124>:	ldr	x1, [x9,#112]
           0x00000000000bc618 <+4128>:	add	x0, sp, #0x110
           0x00000000000bc61c <+4132>:	blr	x8

    Before the `blr` instruction, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x0a8b31d8	0x00000071
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

 4. `call_filter()` runs.  This function is generated by
    `mono_arch_get_call_filter()` and starts with:

           stp	x29, x30, [sp,#-336]!
           mov	x29, sp
           str	x0, [x29,#344]

    Note in particular how the first line subtracts 336 from `sp` to
    start a new stack frame, but the third line writes to a position
    that is 344 bytes beyond that, back within the *previous* stack
    frame.

 5. After the invocations of `call_filter()` and
    `handle_exception_first_pass()`, `w23` is restored from the stack:

           0x00000000000bc820 <+4648>:	ldr	w23, [sp,#20]

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

    Notice how `call_filter()` has overwritten bytes 8 through 15 of the
    stack frame.  In this original lucky scenario, this does not affect
    the value restored into register `w23` because that value starts at
    byte 20.

 6. `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000bb960 <+872>:	cbz	w23, 0xbb9c4 <mono_handle_exception_internal+972>

    Since `w23` is `0`, `mono_handle_exception_internal()` correctly
    continues into the `else` branch.

### The bad behavior for [mono/mono/2018-08@725ba2a built with Android NDK r19][2] works slightly differently

 1. As before, the local `resume` parameter starts in register `w23`.
 2. This time, the register is saved into the stack frame at offset 12
    (instead of 20):

           0x00000000000bed7c <+3200>:	str	w23, [sp,#12]

 3. As before, `handle_exception_first_pass()` invokes `call_filter()`
    via a `blr` instruction.

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

 4. `call_filter()` runs as before.  And the first few instructions of
    that function are the same as before:

        stp	x29, x30, [sp,#-336]!
        mov	x29, sp
        str	x0, [x29,#344]

 5. `w23` is again restored from the stack, but this time from offset 12:

           0x00000000000bf2c0 <+4548>:	ldr	w23, [sp,#12]

    The problem is that `call_filter()` has again overwritten bytes 8
    through 15 of the stack frame:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

    So after the `ldr` instruction, `w23` has an incorrect value of
    `0x7f` rather than the correct value `0`.

 6. As before, `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000be430 <+820>:	cbz	w23, 0xbe834 <mono_handle_exception_internal+1848>

    But this time because `w23` is *not* zero, execution continues
    *incorrectly* into the `if` branch rather than the `else` branch.
    The result is that `ji` is set to `0` rather than a valid
    `(MonoJitInfo *)` value.  This incorrect value for `ji` leads to a
    null pointer dereference, as seen in the bug reports.

[0]: https://github.com/xamarin/xamarin-android/tree/xamarin-android-9.3.0.22
[1]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1432/Azure/
[2]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1436/Azure/

</details>

Backport of #14757.

/cc @lambdageek @brendanzagaeski
2019-06-03 20:05:59 -04:00
Katelyn Gadd c28e96bda1 Bump roslyn to release 3.1.0 (#14775) 2019-06-03 22:37:04 +02:00
monojenkins 989433530a [sdks] Bump Android NDK, build tools and platform tools versions (#14758)
Try to match Xamarin.Android master

Build tools and Platform tools: https://github.com/xamarin/xamarin-android/blob/f9f5b6cd120484b13df74e94638e07411d8693fa/Configuration.props#L100-L102

NDK: https://github.com/xamarin/xamarin-android/blob/f9f5b6cd120484b13df74e94638e07411d8693fa/build-tools/scripts/Ndk.targets#L4
2019-06-03 12:50:33 +02:00
Alexis Christoforides 03bdf62922 [MacSDK] Properly handle PATHs with spaces during package post-install 2019-06-03 08:58:36 +02:00
Jo Shields da1fa28109 Bump version to 6.4.0 (#14727) 2019-05-31 12:53:23 -04:00
2907 changed files with 133219 additions and 116285 deletions

No files matched your search

-3
View File
@@ -1,6 +1,3 @@
# diff settings
*.cs diff=csharp
# ensure LF endings on all checkouts
configure.ac crlf=input
config.rpath crlf=input
+31 -40
View File
@@ -15,7 +15,7 @@
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger @kg
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/acceptance-tests/profiler-stress* @alexrp @akoeplinger
/data @directhex @akoeplinger
/data/gdb @lewurm @vargaz
@@ -23,7 +23,7 @@
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp @lewurm
/docs/sources/mono-api-profiler.html @BrzVlad
/docs/sources/mono-api-profiler.html @alexrp
/external @akoeplinger @marek-safar
/external/api-snapshot @marek-safar @akoeplinger
@@ -31,39 +31,39 @@
/external/bockbuild @akoeplinger @alexischr
/external/boringssl @baulig
/external/cecil @marek-safar @akoeplinger
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/corefx @akoeplinger @marek-safar @egorbo @alexischr
/external/corert @akoeplinger @marek-safar @egorbo @alexischr
/external/linker @marek-safar
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/llvm @alexanderkyte @vargaz
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
/external/roslyn-binaries @kg @marek-safar @akoeplinger
/libgc @BrzVlad @vargaz @naricc
/libgc @BrzVlad @vargaz
/llvm @vargaz @SamMonoRT @imhameed
/llvm @vargaz @alexanderkyte
/man @marek-safar @miguel
/man/*prof* @BrzVlad
/man/*prof* @alexrp
/mcs @marek-safar @akoeplinger @egorbo
/mcs/build @marek-safar @akoeplinger
/mcs/class @steveisok @akoeplinger
/mcs/class @marek-safar @akoeplinger
/mcs/class/corlib/System.Reflection*/ @vargaz @lambdageek
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/Mono.Btls.Interface @baulig @steveisok
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/Mono.Btls.Interface @bauligg @marek-safar
/mcs/class/Mono.Data.Tds @egorbo
/mcs/class/Mono.Debugger.Soft @vargaz @thaystg
/mcs/class/Mono.Options @jonpryor @marek-safar
/mcs/class/Mono.Profiler.Log @lambdageek
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @steveisok
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @marek-safar
/mcs/class/System/Mono.AppleTls @baulig @rolf
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @steveisok
/mcs/class/System/Mono.Security.Interface @baulig @steveisok
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/System.Data* @egorbo @steveisok
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @marek-safar
/mcs/class/System/Mono.Security.Interface @baulig @marek-safar
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/System.Data* @egorbo @marek-safar
/mcs/errors @marek-safar @lewurm
/mcs/ilasm @marek-safar @lewurm
@@ -71,10 +71,10 @@
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar @lewurm
/mcs/tools @marek-safar @akoeplinger @steveisok
/mcs/tools @marek-safar @akoeplinger
/mcs/tools/gensources @kg @akoeplinger
/mcs/tools/resx2sr @baulig
/mcs/tools/upload-to-sentry @directhex @akoeplinger
/mcs/tools/upload-to-sentry @alexanderkyte @akoeplinger
/mono/arch @vargaz @lewurm
@@ -82,37 +82,30 @@
/mono/dis @lambdageek @vargaz
/mono/eglib @vargaz @lambdageek
/mono/benchmark @SamMonoRT @lewurm
/mono/benchmark @alexanderkyte @lewurm
/mono/metadata @vargaz @lambdageek @thaystg
/mono/metadata/*profiler* @lambdageek
/mono/metadata/handle* @lambdageek @vargaz
/mono/metadata/monitor* @brzvlad @vargaz
/mono/metadata/sgen* @brzvlad @vargaz @naricc
/mono/metadata/sgen* @brzvlad @vargaz
/mono/metadata/thread* @lateralusX @lambdageek
/mono/metadata/w32* @lateralusX @lambdageek
/mono/metadata/*-win* @lateralusX @lambdageek
/mono/metadata/appdomain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/assembly* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/domain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/loader* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lewurm @lambdageek @SamMonoRT
/mono/mini @vargaz @lewurm @lambdageek
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz @BrzVlad
/mono/mini/*exceptions* @lewurm @vargaz
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
/mono/mini/*profiler* @alexrp @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas
/mono/native @baulig @egorbo @alexischr @marek-safar
/mono/profiler @BrzVlad @lambdageek
/mono/profiler @alexrp @lambdageek
/mono/sgen @BrzVlad @lambdageek
@@ -120,15 +113,13 @@
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
/mono/utils/mono-mem* @lewurm @vargaz
/mono/utils/mono-merp* @alexischr @lambdageek
/mono/utils/mono-state* @alexischr @lambdageek
/mono/utils/mono-threads* @lambdageek @vargaz
/mono/utils/*-win* @lateralusX @lambdageek
/msvc @lateralusX @kg @akoeplinger
/msvc/scripts @akoeplinger @kg
/netcore @marek-safar @akoeplinger @egorbo @vargaz @steveisok
/netcore @marek-safar @akoeplinger @egorbo
/packaging @alexischr @akoeplinger @directhex
/packaging/MacSDK/msbuild.py @radical @akoeplinger
@@ -136,13 +127,13 @@
/runtime @akoeplinger @marek-safar
/samples @lambdageek @lewurm
/samples/profiler @BrzVlad @lambdageek
/samples/profiler @alexrp @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
/scripts/ci @akoeplinger @directhex
/sdks @marek-safar @lewurm @vargaz @lambdageek
/sdks @marek-safar @lewurm @vargaz
/sdks/builds/android* @alexischr @jonpryor
/sdks/builds/mxe* @alexischr @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
@@ -158,4 +149,4 @@
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/sgen @brzvlad @vargaz @naricc
/tools/sgen @brzvlad @vargaz
+2 -4
View File
@@ -138,9 +138,7 @@ llvm/build/
llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
tools/offsets-tool-py/offtool/
tools/offsets-tool-py/clang/__pycache__
##############################################################################
# Arcade output directories
@@ -148,4 +146,4 @@ tools/offsets-tool-py/clang/__pycache__
.packages
artifacts
.dotnet
.dotnet
+4 -7
View File
@@ -58,16 +58,13 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/llvm"]
path = external/llvm
url = git://github.com/mono/llvm.git
branch = release_60
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+3 -8
View File
@@ -21,14 +21,13 @@ endif
if ENABLE_NETCORE
update_submodules =
SUBDIRS = mk llvm mono runtime netcore
else
update_submodules = update_submodules
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
endif
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore netcore/corerun
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore
all: $(update_submodules)
@@ -46,10 +45,9 @@ EXTRA_DIST= \
winconfig.h \
code_of_conduct.md \
external \
netcore/Makefile \
mcs/class/referencesource
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_RUNTIME=false
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_MCS=false EXTERNAL_RUNTIME=false
# Distribute the 'mcs' tree too
GIT_DIR ?= $(srcdir)/.git
@@ -67,9 +65,6 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
+1 -4
View File
@@ -30,8 +30,7 @@ The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.o
| Windows | i386 | [![windows-i386][17]][18] |
| CentOS | s390x (cs) | [![centos-s390x][19]][20] |
| Debian 9 | ppc64el (cs) | [![debian-9-ppc64el][21]][22]|
| AIX 6.1 | ppc64 (cs) | [![aix-ppc64][23]][24] |
| FreeBSD 12 | amd64 (cs) | [![freebsd-amd64][25]][26] |
| AIX | ppc64 (cs) | [![aix-ppc64][23]][24] |
_(cs) = community supported architecture_
@@ -59,8 +58,6 @@ _(cs) = community supported architecture_
[22]: https://jenkins.mono-project.com/job/test-mono-mainline-community-chroot/label=debian-9-ppc64el
[23]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64/badge/icon
[24]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64
[25]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64/badge/icon
[26]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64
Compilation and Installation
============================
+5 -172
View File
@@ -1,185 +1,18 @@
Mono uses third-party libraries or other resources that may be
distributed under licenses different than the Mono software.
distributed under licenses different than the .NET Core software.
Attributions and license notices for test cases originally authored by
Attributions and license notices for test cases originally authored by
third parties can be found in the respective test directories.
In the event that we accidentally failed to list a required notice, please
In the event that we accidentally failed to list a required notice, please
bring it to our attention. Post an issue or email us:
[email protected]
The attached notices are provided for information only.
License notice for Mono
-----------------------
Note: multiple license notices are provided that occure in source code.
They are equivalent forms of the the MIT license from various copyright holders.
// Dual licensed under the terms of the MIT X11 or GNU GPL
// Copyright 2003-2011 Novell, Inc.
// Copyright 2011 Xamarin Inc
/**
*
* Copyright 2001-2003 Ximian, Inc
* Copyright 2003-2010 Novell, Inc.
* Copyright (C) 2012 Xamarin Inc
*
* Licensed under the MIT license. See LICENSE file in the project root for full license information.
*/
/******************************************************************************
* The MIT License
* Copyright (c) 2003 Novell Inc. www.novell.com
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the Software), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*******************************************************************************/
License notice for Apple
License notice for Apple
-------------------------
Content: https://github.com/mono/mono/blob/435f968102d0820ca65d63e1e741e4130b4d1303/support/ios/net/route.h
License: https://opensource.apple.com/apsl/
License for GCC
---------------
License: http://gcc.gnu.org/onlinedocs/gcc-4.6.4/libstdc++/manual/manual/license.html
Content: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini
Example: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini/decompose.c#L967
GCC RUNTIME LIBRARY EXCEPTION
Version 3.1, 31 March 2009
Copyright (C) 2009 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
This GCC Runtime Library Exception ("Exception") is an additional
permission under section 7 of the GNU General Public License, version
3 ("GPLv3"). It applies to a given file (the "Runtime Library") that
bears a notice placed by the copyright holder of the file stating that
the file is governed by GPLv3 along with this Exception.
When you use GCC to compile a program, GCC may combine portions of
certain GCC header files and runtime libraries with the compiled
program. The purpose of this Exception is to allow compilation of
non-GPL (including proprietary) programs to use, in this way, the
header files and runtime libraries covered by this Exception.
0. Definitions.
A file is an "Independent Module" if it either requires the Runtime
Library for execution after a Compilation Process, or makes use of an
interface provided by the Runtime Library, but is not otherwise based
on the Runtime Library.
"GCC" means a version of the GNU Compiler Collection, with or without
modifications, governed by version 3 (or a specified later version) of
the GNU General Public License (GPL) with the option of using any
subsequent versions published by the FSF.
"GPL-compatible Software" is software whose conditions of propagation,
modification and use would permit combination with GCC in accord with
the license of GCC.
"Target Code" refers to output from any compiler for a real or virtual
target processor architecture, in executable form or suitable for
input to an assembler, loader, linker and/or execution
phase. Notwithstanding that, Target Code does not include data in any
format that is used as a compiler intermediate representation, or used
for producing a compiler intermediate representation.
The "Compilation Process" transforms code entirely represented in
non-intermediate languages designed for human-written code, and/or in
Java Virtual Machine byte code, into Target Code. Thus, for example,
use of source code generators and preprocessors need not be considered
part of the Compilation Process, since the Compilation Process can be
understood as starting with the output of the generators or
preprocessors.
A Compilation Process is "Eligible" if it is done using GCC, alone or
with other GPL-compatible software, or if it is done without using any
work based on GCC. For example, using non-GPL-compatible Software to
optimize any GCC intermediate representations would not qualify as an
Eligible Compilation Process.
1. Grant of Additional Permission.
You have permission to propagate a work of Target Code formed by
combining the Runtime Library with Independent Modules, even if such
propagation would otherwise violate the terms of GPLv3, provided that
all Target Code was generated by Eligible Compilation Processes. You
may then convey such a combination under terms of your choice,
consistent with the licensing of the Independent Modules.
2. No Weakening of GCC Copyleft.
The availability of this Exception does not imply any general
presumption that third-party software is unaffected by the copyleft
requirements of the license of GCC.
Hopefully that text is self-explanatory. If it isn't, you need to speak to your lawyer, or the Free Software Foundation.
License notice for Silicon Graphics and HP
------------------------------------------
/*
* Copyright (c) 1996-1997
* Silicon Graphics Computer Systems, Inc.
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Silicon Graphics makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*
* Copyright (c) 2002
* Hewlett-Packard Company
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Hewlett-Packard Company makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*/
/*
* This implements standard-conforming allocators that interact with
* the garbage collector. Gc_alloctor<T> allocates garbage-collectable
* objects of type T. Traceable_allocator<T> allocates objects that
* are not temselves garbage collected, but are scanned by the
* collector for pointers to collectable objects. Traceable_alloc
* should be used for explicitly managed STL containers that may
* point to collectable objects.
*
* This code was derived from an earlier version of the GNU C++ standard
* library, which itself was derived from the SGI STL implementation.
*/
Content: https://github.com/mono/mono/blob/d58dfa185af1d48fb93efdef1d046762f214f0a7/libgc/include/gc_allocator.h
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"rev": "13a40eedacc799a78710aba5c8e4b2dcc80a7e61",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+5 -17
View File
@@ -150,29 +150,17 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
if test -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
echo "MONO_EXTRA_CONFIGURE_FLAGS is $MONO_EXTRA_CONFIGURE_FLAGS"
fi
host_conf_flag=
build_uname_all=`(uname -a) 2>/dev/null`
case "$build_uname_all" in
CYGWIN*)
if [[ "$@" != *"--host="* ]]; then
echo "Missing --host parameter, configure using ./configure --host=i686-w64-mingw32 or --host=x86_64-w64-mingw32"
echo "Falling back using --host=x86_64-w64-mingw32 as default."
host_conf_flag="--host=x86_64-w64-mingw32"
fi
;;
esac
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings $host_conf_flag" #--enable-iso-c
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings" #--enable-iso-c
if test x$NOCONFIGURE = x; then
echo Running $srcdir/configure $conf_flags "$@" ...
+86 -94
View File
@@ -574,8 +574,6 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ilasm", "mcs/ilasm/ilasm.cs
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "al", "mcs/tools/al/al.csproj", "{0A20671F-CCA9-457D-886C-0828C7D64128}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "aprofutil", "mcs/tools/aprofutil/aprofutil.csproj", "{658B566B-CED1-4C42-9926-C1AA930BA368}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "monolinker", "mcs/tools/linker/monolinker.csproj", "{D60432F1-3C12-4235-A1CC-4CA5575403BD}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "culevel", "mcs/tools/culevel/culevel.csproj", "{E8E246BD-CD0C-4734-A3C2-7F44796EC47B}"
@@ -1536,8 +1534,10 @@ Global
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|unreal.Build.0 = Debug|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.ActiveCfg = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.Build.0 = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.Build.0 = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.Build.0 = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.ActiveCfg = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.Build.0 = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|winaot.ActiveCfg = Release|winaot
@@ -1662,8 +1662,10 @@ Global
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|unreal.Build.0 = Debug|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.ActiveCfg = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.Build.0 = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.Build.0 = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.Build.0 = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.ActiveCfg = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.Build.0 = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|winaot.ActiveCfg = Release|winaot
@@ -2210,8 +2212,10 @@ Global
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|unreal.Build.0 = Debug|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.ActiveCfg = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.Build.0 = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.Build.0 = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.Build.0 = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.ActiveCfg = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.Build.0 = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|winaot.ActiveCfg = Release|winaot
@@ -2270,8 +2274,10 @@ Global
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|unreal.Build.0 = Debug|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.ActiveCfg = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.Build.0 = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.Build.0 = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.Build.0 = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.ActiveCfg = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.Build.0 = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|winaot.ActiveCfg = Release|winaot
@@ -2374,8 +2380,10 @@ Global
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|unreal.Build.0 = Debug|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.ActiveCfg = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.Build.0 = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.Build.0 = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.Build.0 = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.ActiveCfg = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.Build.0 = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|winaot.ActiveCfg = Release|winaot
@@ -2566,8 +2574,10 @@ Global
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|unreal.Build.0 = Debug|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.ActiveCfg = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.Build.0 = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.Build.0 = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.Build.0 = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.ActiveCfg = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.Build.0 = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|winaot.ActiveCfg = Release|winaot
@@ -3632,8 +3642,10 @@ Global
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|unreal.Build.0 = Debug|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.ActiveCfg = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.Build.0 = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.Build.0 = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.Build.0 = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.ActiveCfg = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.Build.0 = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|winaot.ActiveCfg = Release|winaot
@@ -3956,8 +3968,10 @@ Global
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|unreal.Build.0 = Debug|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.ActiveCfg = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.Build.0 = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.Build.0 = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.Build.0 = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.ActiveCfg = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.Build.0 = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|winaot.ActiveCfg = Release|winaot
@@ -4014,42 +4028,24 @@ Global
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|net_4_x.Build.0 = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.Build.0 = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.Build.0 = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.Build.0 = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.Build.0 = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.Build.0 = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.Build.0 = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.Build.0 = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.Build.0 = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.Build.0 = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.Build.0 = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.Build.0 = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.Build.0 = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.Build.0 = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.Build.0 = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.Build.0 = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.Build.0 = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.Build.0 = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.Build.0 = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.Build.0 = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.Build.0 = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|net_4_x.ActiveCfg = Release|net_4_x
@@ -4078,8 +4074,10 @@ Global
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|unreal.Build.0 = Debug|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.ActiveCfg = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.Build.0 = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.Build.0 = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.Build.0 = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.ActiveCfg = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.Build.0 = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|winaot.ActiveCfg = Release|winaot
@@ -4200,8 +4198,10 @@ Global
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|unreal.Build.0 = Debug|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.ActiveCfg = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.Build.0 = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.Build.0 = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.Build.0 = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.ActiveCfg = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.Build.0 = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4238,8 +4238,10 @@ Global
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|unreal.Build.0 = Debug|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.ActiveCfg = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.Build.0 = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.Build.0 = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.Build.0 = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.ActiveCfg = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.Build.0 = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4496,8 +4498,10 @@ Global
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|unreal.Build.0 = Debug|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.ActiveCfg = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.Build.0 = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.Build.0 = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.Build.0 = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.ActiveCfg = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.Build.0 = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|winaot.ActiveCfg = Release|winaot
@@ -6012,8 +6016,10 @@ Global
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|unreal.Build.0 = Debug|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.ActiveCfg = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.Build.0 = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.Build.0 = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.Build.0 = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.ActiveCfg = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.Build.0 = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|winaot.ActiveCfg = Release|winaot
@@ -6050,8 +6056,10 @@ Global
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|unreal.Build.0 = Debug|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.ActiveCfg = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.Build.0 = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.Build.0 = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.Build.0 = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.ActiveCfg = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.Build.0 = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|winaot.ActiveCfg = Release|winaot
@@ -6528,8 +6536,10 @@ Global
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|unreal.Build.0 = Debug|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.ActiveCfg = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.Build.0 = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.Build.0 = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.Build.0 = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.ActiveCfg = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.Build.0 = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|winaot.ActiveCfg = Release|winaot
@@ -6566,8 +6576,10 @@ Global
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|unreal.Build.0 = Debug|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.ActiveCfg = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.Build.0 = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.Build.0 = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.Build.0 = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.ActiveCfg = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.Build.0 = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|winaot.ActiveCfg = Release|winaot
@@ -9844,8 +9856,10 @@ Global
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|unreal.Build.0 = Debug|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.ActiveCfg = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.Build.0 = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.Build.0 = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.Build.0 = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.ActiveCfg = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.Build.0 = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|winaot.ActiveCfg = Release|winaot
@@ -10120,28 +10134,6 @@ Global
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|winaot.ActiveCfg = Release|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Debug|xammac.ActiveCfg = Debug|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|xammac.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.Build.0 = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.Build.0 = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monodroid.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|orbis.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|orbis.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|unreal.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|unreal.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|wasm.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|wasm.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|winaot.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|winaot.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|xammac.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|xammac.ActiveCfg = Release|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.Build.0 = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Release|net_4_x.ActiveCfg = Release|net_4_x
+159 -466
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -6,7 +6,7 @@
<dllmap dll="libintl" name="bind_textdomain_codeset" target="@LIBC@" os="solaris"/>
<dllmap dll="libintl" target="@INTL@" os="!windows"/>
<dllmap dll="i:libxslt.dll" target="libxslt@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="@ODBC@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libodbc@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libiodbc.dylib" os="osx"/>
<dllmap dll="oci" target="libclntsh@libsuffix@" os="!windows"/>
<dllmap dll="db2cli" target="libdb2_36@libsuffix@" os="!windows"/>
+13 -13
View File
@@ -1,9 +1,8 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
print('%c thread #%i' % ('*' if current_thread else ' ', thread.idx))
print '%c thread #%i' % ('*' if current_thread else ' ', thread.idx)
if current_thread:
selected_frame = thread.GetSelectedFrame()
@@ -15,16 +14,17 @@ def print_frames(thread, num_frames, current_thread):
if function_name == "interp_exec_method_full":
try:
s = 'frame->imethod->method'
methoddesc = frame.EvaluateExpression('(char*) mono_method_full_name (' + s + ', 1)').summary[1:-1]
klassname = frame.EvaluateExpression('(char*) ' + s + '->klass->name').summary[1:-1]
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
ipoffset -= frame.EvaluateExpression('imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
var = '%s::%s @ %d %s || %s' % (klassname, methodname, ipoffset, insn, frame)
except Exception as e:
traceback.print_exc()
print "DBG: execfail:" + str(e)
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -32,7 +32,7 @@ def print_frames(thread, num_frames, current_thread):
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
var = 'transforming %s::%s || %s' % (klassname, methodname, frame)
except Exception as e:
print("DBG: transformfail:" + str(e))
print "DBG: transformfail:" + str(e)
elif pc[0] == '0':
try:
framestr = frame.EvaluateExpression('(char*)mono_pmip((void*)%s)' % pc).summary[1:-1]
@@ -40,7 +40,7 @@ def print_frames(thread, num_frames, current_thread):
except:
pass
print(' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var))
print ' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var)
def monobt(debugger, command, result, dict):
opts = {'all_bt': False, 'num_frames': None}
@@ -50,20 +50,20 @@ def monobt(debugger, command, result, dict):
elif command.isdigit():
opts['num_frames'] = int(command)
elif command != '':
print('error: monobt [<number>|all]')
print 'error: monobt [<number>|all]'
return
target = debugger.GetSelectedTarget()
process = target.process
if not process.IsValid():
print('error: invalid process')
print 'error: invalid process'
return
if opts['all_bt']:
for thread in process.threads:
print_frames(thread, len(thread), process.selected_thread == thread)
print('')
print ''
else:
thread = process.selected_thread
num_frames = len(thread) if opts['num_frames'] is None else opts['num_frames']
@@ -76,4 +76,4 @@ def __lldb_init_module (debugger, dict):
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f monobt.monobt monobt')
debugger.HandleCommand('command alias mbt monobt')
print('"monobt" command installed')
print '"monobt" command installed'
-23
View File
@@ -18,27 +18,6 @@ convert.exe: $(srcdir)/convert.cs AgilityPack.dll
AgilityPack.dll:
$(CSCOMPILE) -r:$(topdir)/class/lib/$(PROFILE)/mscorlib.dll -r:$(topdir)/class/lib/$(PROFILE)/System.dll -r:$(topdir)/class/lib/$(PROFILE)/System.Xml.dll -target:library -out:$@ $(srcdir)/HtmlAgilityPack/*.cs
ifdef MCS_MODE
monoapi.zip:
touch $@
monoapi.tree: monoapi.zip
touch $@
mono-tools.zip:
touch $@
mono-tools.tree: mono-tools.zip
touch $@
mono-file-formats.zip:
touch $@
mono-file-formats.tree: mono-file-formats.zip
touch $@
else
monoapi.zip: monoapi.tree
@test -f $@ || { rm -f $< && $(MAKE) $<; }
@@ -57,7 +36,5 @@ mono-file-formats.zip: mono-file-formats.tree
mono-file-formats.tree: $(srcdir)/mono-file-formats.config $(srcdir)/docs.make
$(MDOC) assemble -o mono-file-formats -f man $<
endif
.doc-stamp:
+39 -61
View File
@@ -2,58 +2,36 @@
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documentation
## Useful RISC-V documents
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications/)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa/)
* [RISC-V Debug Specification Standard](https://github.com/riscv/riscv-debug-spec/blob/master/riscv-debug-spec.pdf)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual/blob/master/riscv-asm.md)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Linux](https://github.com/riscv/riscv-linux)
* [RISC-V LLD](https://github.com/riscv/riscv-lld)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V QEMU](https://github.com/riscv/riscv-qemu)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
* [lowRISC Organization](https://github.com/lowrisc)
* [RISC-V LLVM](https://github.com/lowrisc/riscv-llvm)
## Setting up a cross environment
### Debian/Ubuntu packages
Setting up a cross environment with a Linux toolchain and QEMU is quite easy.
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
First, add these to your `.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
@@ -61,42 +39,47 @@ export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Next, install some dependencies needed to build the toolchain:
Install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev libusb-1.0-0-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev device-tree-compiler pkg-config libexpat-dev libglib2.0-dev libpixman-1-dev
```
Finally, build the toolchain:
Now you can build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ git clone --recursive [email protected]:riscv/riscv-tools.git
$ cd riscv-tools
$ ./build.sh
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ ./configure --prefix=$RISCV --enable-multilib
$ make linux
$ cd ../..
$ git clone --recursive [email protected]:riscv/riscv-qemu.git
$ cd riscv-qemu
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --disable-werror
$ make
$ make install
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
@@ -106,24 +89,19 @@ Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 ./mono/mini/mono hello.exe
$ qemu-riscv64 $RISCV/sysroot/bin/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
$ qemu-riscv64 -g 12345 ./mono/mini/mono-sgen --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono-sgen
```
## Things to be done
Things that need to be done beyond the basic 64-bit port, in no particular
order:
In no particular order:
* Complete the soft float port.
* Complete the 32-bit port.
-6
View File
@@ -109,12 +109,6 @@ typedef struct {
<h3>Write Barriers</h3>
<p>SGen is a concurrent and generational GC, features which require
tracking changes to the state of the heap. This is achieved through
write barriers. Whenever native code is changing the state of the
heap by storing references into another managed object, it needs to
do it using this write barrier API.
<h4><a name="api:mono_gc_wbarrier_arrayref_copy">mono_gc_wbarrier_arrayref_copy</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_nostore">mono_gc_wbarrier_generic_nostore</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_store">mono_gc_wbarrier_generic_store</a></h4>
+4 -16
View File
@@ -1,27 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>39d545dc14bc4d5a546e5e2f58d1ebaca72fcd61</Sha>
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.19567.2">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>928efaa081ffd17b37f6c662fee44775f2d0b90f</Sha>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.19567.2">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>928efaa081ffd17b37f6c662fee44775f2d0b90f</Sha>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
+1 -8
View File
@@ -1,14 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>5.0.0</VersionPrefix>
<VersionPrefix>1.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
</Project>
-11
View File
@@ -1,11 +0,0 @@
@{
IncludeRules=@('PSAvoidUsingCmdletAliases',
'PSAvoidUsingWMICmdlet',
'PSAvoidUsingPositionalParameters',
'PSAvoidUsingInvokeExpression',
'PSUseDeclaredVarsMoreThanAssignments',
'PSUseCmdletCorrectly',
'PSStandardDSCFunctionsInResource',
'PSUseIdenticalMandatoryParametersForDSC',
'PSUseIdenticalParametersForDSC')
}
+2 -4
View File
@@ -37,8 +37,6 @@
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymWeb Condition="'$(PublishToSymWeb)' == ''">true</PublishToSymWeb>
<PublishToMSDL Condition="'$(PublishToMSDL)' == ''">true</PublishToMSDL>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
@@ -58,7 +56,7 @@
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer) and $(PublishToMSDL)"/>
Condition="$(PublishToSymbolServer)"/>
<!--
Symbol Uploader: SymWeb
@@ -75,7 +73,7 @@
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer) and $(PublishToSymWeb)"/>
Condition="$(PublishToSymbolServer)"/>
</Target>
<ItemGroup>
-127
View File
@@ -1,127 +0,0 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile ${Env:BUILD_SOURCESDIRECTORY}/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-Host "Couldn't find the file NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
$doc.Save($filename)
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $BUILD_SOURCESDIRECTORY/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
echo "Couldn't find the file NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
# Ensure dotnet3-internal and dotnet3-internal-transport is in the packageSources
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" NuGet.config
fi
# Ensure dotnet3-internal and dotnet3-internal-transport is in the packageSources
grep -i "<add key=\"dotnet3-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" NuGet.config
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
PackageSources+=('dotnet3-internal')
PackageSources+=('dotnet3-internal-transport')
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" NuGet.config
fi
done
+1 -1
View File
@@ -3,7 +3,7 @@
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the SignCheck tool to validate that packages about to
to invoke the the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
+10 -16
View File
@@ -18,7 +18,6 @@ Param(
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
@@ -49,7 +48,6 @@ function Print-Usage() {
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
@@ -87,10 +85,6 @@ function Build {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
@@ -114,14 +108,6 @@ function Build {
@properties
}
if ($clean) {
if(Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host "Artifacts directory deleted."
}
exit 0
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
@@ -133,15 +119,23 @@ try {
$nodeReuse = $false
}
if ($restore) {
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
InitializeNativeTools
}
Build
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
ExitWithExitCode 1
}
+12 -12
View File
@@ -26,7 +26,6 @@ usage()
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
@@ -63,7 +62,6 @@ publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
@@ -84,9 +82,6 @@ while [[ $# > 0 ]]; do
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
@@ -201,15 +196,20 @@ function Build {
ExitWithExitCode 0
}
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
if [[ "$restore" == true ]]; then
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
InitializeNativeTools
fi
+3 -3
View File
@@ -157,15 +157,15 @@ fetch_tizen_pkgs()
Inform "Initialize arm base"
fetch_tizen_pkgs_init standard base
Inform "fetch common packages"
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel libatomic
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel
fetch_tizen_pkgs noarch linux-glibc-devel
Inform "fetch coreclr packages"
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel tizen-release lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l libcom_err libcom_err-devel zlib zlib-devel libopenssl libopenssl-devel krb5 krb5-devel libcurl libcurl-devel
Inform "Initialize standard unified"
fetch_tizen_pkgs_init standard unified
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l gssdp gssdp-devel tizen-release
fetch_tizen_pkgs armv7l gssdp gssdp-devel
+1 -2
View File
@@ -181,7 +181,7 @@ if [ -z "$__RootfsDir" ] && [ ! -z "$ROOTFS_DIR" ]; then
fi
if [ -z "$__RootfsDir" ]; then
__RootfsDir="$__CrossDir/../../../.tools/rootfs/$__BuildArch"
__RootfsDir="$__CrossDir/rootfs/$__BuildArch"
fi
if [ -d "$__RootfsDir" ]; then
@@ -203,7 +203,6 @@ if [[ "$__LinuxCodeName" == "alpine" ]]; then
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
rm -r $__ApkToolsDir
+34 -15
View File
@@ -31,10 +31,6 @@ else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
@@ -43,25 +39,48 @@ if(TARGET_ARCH_NAME STREQUAL "armel")
endif()
endif()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
endif()
endforeach()
endmacro()
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_link_options(-m32)
add_compile_param(CROSS_LINK_FLAGS "-m32")
endif()
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
@@ -84,7 +103,7 @@ elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths for builds that need lldb.
# Set LLDB include and library paths
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
@@ -112,7 +131,7 @@ if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+7 -12
View File
@@ -1,10 +1,9 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16",
$verbosity = "m",
$toolpath = $null
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
)
$verbosity = "m"
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
@@ -12,10 +11,10 @@ function InstallDarcCli ($darcVersion) {
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list -g
$toolList = Invoke-Expression "& `"$dotnet`" tool list -g"
if ($toolList -like "*$darcCliPackageName*") {
& "$dotnet" tool uninstall $darcCliPackageName -g
Invoke-Expression "& `"$dotnet`" tool uninstall $darcCliPackageName -g"
}
# If the user didn't explicitly specify the darc version,
@@ -23,16 +22,12 @@ function InstallDarcCli ($darcVersion) {
if (-not $darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
$arcadeServicesSource = 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
if (-not $toolpath) {
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
Invoke-Expression "& `"$dotnet`" tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
}
InstallDarcCli $darcVersion
+7 -25
View File
@@ -3,7 +3,6 @@
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
verbosity=m
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
@@ -16,14 +15,6 @@ while [[ $# > 0 ]]; do
versionEndpoint=$2
shift
;;
--verbosity)
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
@@ -43,6 +34,7 @@ while [[ -h "$source" ]]; do
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity=m
. "$scriptroot/tools.sh"
@@ -56,27 +48,17 @@ function InstallDarcCli {
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
local arcadeServicesSource="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
}
InstallDarcCli
+2 -4
View File
@@ -3,9 +3,7 @@ Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet",
[string] $RuntimeSourceFeed = "",
[string] $RuntimeSourceFeedKey = ""
[string] $runtime = "dotnet"
)
. $PSScriptRoot\tools.ps1
@@ -17,7 +15,7 @@ try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
InstallDotNet $installdir $version $architecture $runtime $true
}
catch {
Write-Host $_
+2 -41
View File
@@ -14,8 +14,6 @@ scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
@@ -31,55 +29,18 @@ while [[ $# > 0 ]]; do
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv7l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
. "$scriptroot/tools.sh"
dotnetRoot="$repo_root/.dotnet"
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
@@ -1,6 +0,0 @@
param(
[string] $token
)
Write-Host "##vso[task.setvariable variable=VSS_NUGET_ACCESSTOKEN]$token"
Write-Host "##vso[task.setvariable variable=VSS_NUGET_URI_PREFIXES]https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/"
+10 -10
View File
@@ -25,7 +25,7 @@ function CheckExitCode ([string]$stage)
try {
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
@@ -40,9 +40,9 @@ try {
$darcExe = "$env:USERPROFILE\.dotnet\tools"
$darcExe = Resolve-Path "$darcExe\darc.exe"
Create-Directory $outputFolder
# Generate 3 graph descriptions:
# 1. Flat with coherency information
# 2. Graphviz (dot) file
@@ -51,26 +51,26 @@ try {
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = @( "--github-pat", "$gitHubPat", "--azdev-pat", "$azdoPat", "--password", "$barToken" )
$baseOptions = "get-dependency-graph --github-pat $gitHubPat --azdev-pat $azdoPat --password $barToken"
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += @( "--include-toolset" )
$baseOptions += " --include-toolset"
}
Write-Host "Generating standard dependency graph..."
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
Invoke-Expression "& `"$darcExe`" $baseOptions --output-file $normalGraphFilePath"
CheckExitCode "Generating normal dependency graph"
Write-Host "Generating flat dependency graph and graphviz file..."
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
Invoke-Expression "& `"$darcExe`" $baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath"
CheckExitCode "Generating flat and graphviz dependency graph"
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
Invoke-Expression "& `"$dotFilePath`" -Tpng -o'$graphVizImageFilePath' `"$graphVizFilePath`""
CheckExitCode "Generating graphviz image"
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
+12 -18
View File
@@ -79,37 +79,32 @@ try {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
$ToolVersion = $_.Value
$LocalInstallerArguments = @{ ToolName = "$ToolName" }
$LocalInstallerArguments += @{ InstallPath = "$InstallBin" }
$LocalInstallerArguments += @{ BaseUri = "$BaseUri" }
$LocalInstallerArguments += @{ CommonLibraryDirectory = "$EngCommonBaseDir" }
$LocalInstallerArguments += @{ Version = "$ToolVersion" }
$LocalInstallerCommand = $InstallerPath
$LocalInstallerCommand += " -ToolName $ToolName"
$LocalInstallerCommand += " -InstallPath $InstallBin"
$LocalInstallerCommand += " -BaseUri $BaseUri"
$LocalInstallerCommand += " -CommonLibraryDirectory $EngCommonBaseDir"
$LocalInstallerCommand += " -Version $ToolVersion"
if ($Verbose) {
$LocalInstallerArguments += @{ Verbose = $True }
$LocalInstallerCommand += " -Verbose"
}
if (Get-Variable 'Force' -ErrorAction 'SilentlyContinue') {
if($Force) {
$LocalInstallerArguments += @{ Force = $True }
$LocalInstallerCommand += " -Force"
}
}
if ($Clean) {
$LocalInstallerArguments += @{ Clean = $True }
$LocalInstallerCommand += " -Clean"
}
Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$InstallerPath $($LocalInstallerArguments.Keys.ForEach({"-$_ '$($LocalInstallerArguments.$_)'"}) -join ' ')'"
& $InstallerPath @LocalInstallerArguments
Write-Verbose "Executing '$LocalInstallerCommand'"
Invoke-Expression "$LocalInstallerCommand"
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
$showNativeToolsWarning = $true
if ((Get-Variable 'DoNotDisplayNativeToolsInstallationWarnings' -ErrorAction 'SilentlyContinue') -and $DoNotDisplayNativeToolsInstallationWarnings) {
$showNativeToolsWarning = $false
}
if ($showNativeToolsWarning) {
Write-Warning $errMsg
}
Write-Warning $errMsg
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
@@ -133,7 +128,6 @@ try {
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
+2 -2
View File
@@ -70,7 +70,8 @@ function ReadGlobalJsonNativeTools {
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=$( echo "$native_tools_list" | sed 's/\s//g' | sed 's/,/\n/g' )
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[:space:]')"
local old_IFS=$IFS
while read -r line; do
@@ -107,7 +108,6 @@ else
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
echo $installer_command
if [[ $force = true ]]; then
installer_command+=" --force"
-135
View File
@@ -1,135 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $Operation,
[string] $AuthToken,
[string] $CommitSha,
[string] $RepoName,
[switch] $IsFeedPrivate
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables. This should ONLY be called from identified
# internal builds
function SetupCredProvider {
param(
[string] $AuthToken
)
# Install the Cred Provider NuGet plugin
Write-Host "Setting up Cred Provider NuGet plugin in the agent..."
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host "Installing plugin..."
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-Host "CredProvider plugin was not installed correctly!"
ExitWithExitCode 1
}
else {
Write-Host "CredProvider plugin was installed correctly!"
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-Host "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
}
$endpoints = New-Object System.Collections.ArrayList
$nugetConfigPackageSources = Select-Xml -Path $nugetConfigPath -XPath "//packageSources/add[contains(@key, 'darc-int-')]/@value" | foreach{$_.Node.Value}
if (($nugetConfigPackageSources | Measure-Object).Count -gt 0 ) {
foreach ($stableRestoreResource in $nugetConfigPackageSources) {
$trimmedResource = ([string]$stableRestoreResource).Trim()
[void]$endpoints.Add(@{endpoint="$trimmedResource"; password="$AuthToken"})
}
}
if (($endpoints | Measure-Object).Count -gt 0) {
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
$endpointCredentials = @{endpointCredentials=$endpoints} | ConvertTo-Json -Compress
# Create the environment variables the AzDo way
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $endpointCredentials -Properties @{
'variable' = 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS'
'issecret' = 'false'
}
# We don't want sessions cached since we will be updating the endpoints quite frequently
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data 'False' -Properties @{
'variable' = 'NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED'
'issecret' = 'false'
}
}
else
{
Write-Host "No internal endpoints found in NuGet.config"
}
}
#Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
$dotnetTempDir = "$RepoRoot\dotnet"
$dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
$dotnet = "$dotnetTempDir\dotnet.exe"
$restoreProjPath = "$PSScriptRoot\restore.proj"
Write-Host "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Out-File "$restoreProjPath"
& $dotnet restore $restoreProjPath
Write-Host "Arcade SDK restored!"
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
}
if (Test-Path -Path $dotnetTempDir) {
Remove-Item $dotnetTempDir -Recurse
}
}
try {
Push-Location $PSScriptRoot
if ($Operation -like "setup") {
SetupCredProvider $AuthToken
}
elseif ($Operation -like "install-restore") {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-Host "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
finally {
Pop-Location
}
-142
View File
@@ -1,142 +0,0 @@
#!/usr/bin/env bash
set -e
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables.
# This should ONLY be called from identified internal builds
function SetupCredProvider {
local authToken=$1
# Install the Cred Provider NuGet plugin
echo "Setting up Cred Provider NuGet plugin in the agent..."...
echo "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
local url="https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh"
echo "Writing the contents of 'installcredprovider.ps1' locally..."
local installcredproviderPath="installcredprovider.sh"
if command -v curl > /dev/null; then
curl $url > "$installcredproviderPath"
else
wget -q -O "$installcredproviderPath" "$url"
fi
echo "Installing plugin..."
. "$installcredproviderPath"
echo "Deleting local copy of 'installcredprovider.sh'..."
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
echo "CredProvider plugin was not installed correctly!"
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
fi
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
echo "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
local endpoints='['
local nugetConfigPackageValues=`cat "$nugetConfigPath" | grep "key=\"darc-int-"`
local pattern="value=\"(.*)\""
for value in $nugetConfigPackageValues
do
if [[ $value =~ $pattern ]]; then
local endpoint="${BASH_REMATCH[1]}"
endpoints+="{\"endpoint\": \"$endpoint\", \"password\": \"$authToken\"},"
fi
done
endpoints=${endpoints%?}
endpoints+=']'
if [ ${#endpoints} -gt 2 ]; then
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
local endpointCredentials="{\"endpointCredentials\": "$endpoints"}"
echo "##vso[task.setvariable variable=VSS_NUGET_EXTERNAL_FEED_ENDPOINTS]$endpointCredentials"
echo "##vso[task.setvariable variable=NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED]False"
else
echo "No internal endpoints found in NuGet.config"
fi
}
# Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
local dotnetTempDir="$repo_root/dotnet"
local dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
local restoreProjPath="$repo_root/eng/common/restore.proj"
echo "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
echo "<Project Sdk=\"Microsoft.DotNet.Arcade.Sdk\"/>" > "$restoreProjPath"
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
local res=`$dotnetTempDir/dotnet restore $restoreProjPath`
echo "Arcade SDK restored!"
# Cleanup
if [ "$restoreProjPath" ]; then
rm "$restoreProjPath"
fi
if [ "$dotnetTempDir" ]; then
rm -r $dotnetTempDir
fi
}
source="${BASH_SOURCE[0]}"
operation=''
authToken=''
repoName=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--operation)
operation=$2
shift
;;
--authtoken)
authToken=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ "$operation" = "setup" ]; then
SetupCredProvider $authToken
elif [ "$operation" = "install-restore" ]; then
InstallDotNetSdkAndRestoreArcade
else
echo "Unknown operation '$operation'!"
fi
+3 -34
View File
@@ -59,38 +59,9 @@ function DownloadAndExtract {
-Verbose:$Verbose
if ($UnzipStatus -Eq $False) {
# Retry Download one more time with Force=true
$DownloadRetryStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries 1 `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$True `
-Verbose:$Verbose
if ($DownloadRetryStatus -Eq $False) {
Write-Error "Last attempt of download failed as well"
return $False
}
# Retry unzip again one more time with Force=true
$UnzipRetryStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$True `
-Verbose:$Verbose
if ($UnzipRetryStatus -Eq $False)
{
Write-Error "Last attempt of unzip failed as well"
# Clean up partial zips and extracts
if (Test-Path $TempToolPath) {
Remove-Item $TempToolPath -Force
}
if (Test-Path $InstallDirectory) {
Remove-Item $InstallDirectory -Force -Recurse
}
return $False
}
Write-Error "Unzip failed"
return $False
}
return $True
}
@@ -152,8 +123,6 @@ function Get-File {
}
else {
Write-Verbose "Downloading $Uri"
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
while($Attempt -Lt $DownloadRetries)
{
try {
@@ -240,7 +209,7 @@ function New-ScriptShim {
Remove-Item (Join-Path $ShimDirectory "$ShimName.exe")
}
& "$ShimDirectory\WinShimmer\winshimmer.exe" $ShimName $ToolFilePath $ShimDirectory
Invoke-Expression "$ShimDirectory\WinShimmer\winshimmer.exe $ShimName $ToolFilePath $ShimDirectory"
return $True
}
catch {
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake-test"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
exit 1
fi
exit 0
+1 -1
View File
@@ -69,7 +69,7 @@ tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
uri="${base_uri}/$tool_folder/cmake/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
@@ -1,102 +0,0 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<WorkItemCommand>%HELIX_CORRELATION_PAYLOAD%\performance\scripts\benchmarks_ci.py --csproj %HELIX_CORRELATION_PAYLOAD%\performance\$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
<DotnetExe>%HELIX_CORRELATION_PAYLOAD%\performance\tools\dotnet\$(Architecture)\dotnet.exe</DotnetExe>
<Percent>%25%25</Percent>
<XMLResults>%HELIX_WORKITEM_ROOT%\testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
<BaseDirectory>$HELIX_CORRELATION_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)/performance</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'true'">
<BaseDirectory>$HELIX_WORKITEM_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT'">
<WorkItemCommand>$(PerformanceDirectory)/scripts/benchmarks_ci.py --csproj $(PerformanceDirectory)/$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<BaselineCoreRun>$(BaseDirectory)/Baseline_Core_Root/corerun</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>$(PerformanceDirectory)/src/tools/ResultsComparer/ResultsComparer.csproj</ResultsComparer>
<DotnetExe>$(PerformanceDirectory)/tools/dotnet/$(Architecture)/dotnet</DotnetExe>
<Percent>%25</Percent>
<XMLResults>$HELIX_WORKITEM_ROOT/testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(UseBaselineCoreRun)' == 'true'">
<BaselineCoreRunArgument>--corerun $(BaselineCoreRun)</BaselineCoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>
<PropertyGroup Condition="'$(_Framework)' != 'net461'">
<WorkItemCommand>$(WorkItemCommand) $(CliArguments)</WorkItemCommand>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<PropertyGroup>
<PartitionCount>5</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="$(BuildConfig).Partition0" Index="0" />
<Partition Include="$(BuildConfig).Partition1" Index="1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>
<PropertyGroup Condition="'$(Compare)' == 'true'">
<FailOnTestFailure>false</FailOnTestFailure>
</PropertyGroup>
<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults);$(FinalCommand)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
</Project>
@@ -1,106 +0,0 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $BaselineCoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp5.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="coreclr corefx",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)
$UseBaselineCoreRun = ($BaselineCoreRootDirectory -ne [string]::Empty)
$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
$WorkItemDirectory = (Join-Path $SourceDirectory "workitem")
$ExtraBenchmarkDotNetArguments = "--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
$Creator = $env:BUILD_DEFINITIONNAME
$PerfLabArguments = ""
$HelixSourcePrefix = "pr"
$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"
if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS5.Open"
}
if ($Compare) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf.Open"
$PerfLabArguments = ""
$ExtraBenchmarkDotNetArguments = ""
}
if ($Internal) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}
$CommonSetupArguments="--frameworks $Framework --queue $Queue --build-number $BuildNumber --build-configs $Configurations"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"
robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}
if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}
if ($UseBaselineCoreRun) {
$NewBaselineCoreRoot = (Join-Path $PayloadDirectory "Baseline_Core_Root")
Move-Item -Path $BaselineCoreRootDirectory -Destination $NewBaselineCoreRoot
}
$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory
# Set variables that we will need to have in future steps
$ci = $true
. "$PSScriptRoot\..\pipeline-logging-functions.ps1"
# Directories
Write-PipelineSetVariable -Name 'PayloadDirectory' -Value "$PayloadDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerformanceDirectory' -Value "$PerformanceDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'WorkItemDirectory' -Value "$WorkItemDirectory" -IsMultiJobVariable $false
# Script Arguments
Write-PipelineSetVariable -Name 'Python' -Value "py -3" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'ExtraBenchmarkDotNetArguments' -Value "$ExtraBenchmarkDotNetArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'SetupArguments' -Value "$SetupArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerfLabArguments' -Value "$PerfLabArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'BDNCategories' -Value "$RunCategories" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseBaselineCoreRun' -Value "$UseBaselineCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Compare' -Value "$Compare" -IsMultiJobVariable $false
# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Queue' -Value "$Queue" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'HelixSourcePrefix' -Value "$HelixSourcePrefix" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name '_BuildConfig' -Value "$Architecture.$Kind.$Framework" -IsMultiJobVariable $false
exit 0
-216
View File
@@ -1,216 +0,0 @@
#!/usr/bin/env bash
source_directory=$BUILD_SOURCESDIRECTORY
core_root_directory=
baseline_core_root_directory=
architecture=x64
framework=netcoreapp5.0
compilation_mode=tiered
repository=$BUILD_REPOSITORY_NAME
branch=$BUILD_SOURCEBRANCH
commit_sha=$BUILD_SOURCEVERSION
build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="coreclr corefx"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations=
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--sourcedirectory)
source_directory=$2
shift 2
;;
--corerootdirectory)
core_root_directory=$2
shift 2
;;
--baselinecorerootdirectory)
baseline_core_root_directory=$2
shift 2
;;
--architecture)
architecture=$2
shift 2
;;
--framework)
framework=$2
shift 2
;;
--compilationmode)
compilation_mode=$2
shift 2
;;
--repository)
repository=$2
shift 2
;;
--branch)
branch=$2
shift 2
;;
--commitsha)
commit_sha=$2
shift 2
;;
--buildnumber)
build_number=$2
shift 2
;;
--kind)
kind=$2
shift 2
;;
--runcategories)
run_categories=$2
shift 2
;;
--csproj)
csproj=$2
shift 2
;;
--internal)
internal=true
shift 1
;;
--compare)
compare=true
shift 1
;;
--configurations)
configurations=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --corerootdirectory <value> Directory where Core_Root exists, if running perf testing with --corerun"
echo " --architecture <value> Architecture of the testing being run"
echo " --configurations <value> List of key=value pairs that will be passed to perf testing infrastructure."
echo " ex: --configurations \"CompilationMode=Tiered OptimzationLevel=PGO\""
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --framework <value> The framework to run, if not running in master"
echo " --compliationmode <value> The compilation mode if not passing --configurations"
echo " --sourcedirectory <value> The directory of the sources. Defaults to env:BUILD_SOURCESDIRECTORY"
echo " --repository <value> The name of the repository in the <owner>/<repository name> format. Defaults to env:BUILD_REPOSITORY_NAME"
echo " --branch <value> The name of the branch. Defaults to env:BUILD_SOURCEBRANCH"
echo " --commitsha <value> The commit sha1 to run against. Defaults to env:BUILD_SOURCEVERSION"
echo " --buildnumber <value> The build number currently running. Defaults to env:BUILD_BUILDNUMBER"
echo " --csproj The relative path to the benchmark csproj whose tests should be run. Defaults to src\benchmarks\micro\MicroBenchmarks.csproj"
echo " --kind <value> Related to csproj. The kind of benchmarks that should be run. Defaults to micro"
echo " --runcategories <value> Related to csproj. Categories of benchmarks to run. Defaults to \"coreclr corefx\""
echo " --internal If the benchmarks are running as an official job."
echo ""
exit 0
;;
esac
done
if [ "$repository" == "dotnet/performance" ] || [ "$repository" == "dotnet-performance" ]; then
run_from_perf_repo=true
fi
if [ -z "$configurations" ]; then
configurations="CompliationMode=$compilation_mode"
fi
if [ -z "$core_root_directory" ]; then
use_core_run=false
fi
if [ -z "$baseline_core_root_directory" ]; then
use_baseline_core_run=false
fi
payload_directory=$source_directory/Payload
performance_directory=$payload_directory/performance
workitem_directory=$source_directory/workitem
extra_benchmark_dotnet_arguments="--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
perflab_arguments=
queue=Ubuntu.1804.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"
if [[ "$compare" == true ]]; then
extra_benchmark_dotnet_arguments=
perflab_arguments=
# No open queues for arm64
if [[ "$architecture" = "arm64" ]]; then
echo "Compare not available for arm64"
exit 1
fi
queue=Ubuntu.1804.Amd64.Tiger.Perf.Open
fi
if [[ "$internal" == true ]]; then
perflab_arguments="--upload-to-perflab-container"
helix_source_prefix="official"
creator=
extra_benchmark_dotnet_arguments=
if [[ "$architecture" = "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
queue=Ubuntu.1804.Amd64.Tiger.Perf
fi
fi
common_setup_arguments="--frameworks $framework --queue $queue --build-number $build_number --build-configs $configurations"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
payload_directory=
workitem_directory=$source_directory
performance_directory=$workitem_directory
setup_arguments="--perf-hash $commit_sha $common_setup_arguments"
else
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $performance_directory
docs_directory=$performance_directory/docs
mv $docs_directory $workitem_directory
fi
if [[ "$use_core_run" = true ]]; then
new_core_root=$payload_directory/Core_Root
mv $core_root_directory $new_core_root
fi
if [[ "$use_baseline_core_run" = true ]]; then
new_baseline_core_root=$payload_directory/Baseline_Core_Root
mv $baseline_core_root_directory $new_baseline_core_root
fi
ci=true
_script_dir=$(pwd)/eng/common
. "$_script_dir/pipeline-logging-functions.sh"
# Make sure all of our variables are available for future steps
Write-PipelineSetVariable -name "UseCoreRun" -value "$use_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "UseBaselineCoreRun" -value "$use_baseline_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "Architecture" -value "$architecture" -is_multi_job_variable false
Write-PipelineSetVariable -name "PayloadDirectory" -value "$payload_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerformanceDirectory" -value "$performance_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "WorkItemDirectory" -value "$workitem_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "Queue" -value "$queue" -is_multi_job_variable false
Write-PipelineSetVariable -name "SetupArguments" -value "$setup_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "Python" -value "$python3" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerfLabArguments" -value "$perflab_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "ExtraBenchmarkDotNetArguments" -value "$extra_benchmark_dotnet_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "BDNCategories" -value "$run_categories" -is_multi_job_variable false
Write-PipelineSetVariable -name "TargetCsproj" -value "$csproj" -is_multi_job_variable false
Write-PipelineSetVariable -name "RunFromPerfRepo" -value "$run_from_perf_repo" -is_multi_job_variable false
Write-PipelineSetVariable -name "Creator" -value "$creator" -is_multi_job_variable false
Write-PipelineSetVariable -name "HelixSourcePrefix" -value "$helix_source_prefix" -is_multi_job_variable false
Write-PipelineSetVariable -name "Kind" -value "$kind" -is_multi_job_variable false
Write-PipelineSetVariable -name "_BuildConfig" -value "$architecture.$kind.$framework" -is_multi_job_variable false
Write-PipelineSetVariable -name "Compare" -value "$compare" -is_multi_job_variable false
-234
View File
@@ -1,234 +0,0 @@
# Source for this file was taken from https://github.com/microsoft/azure-pipelines-task-lib/blob/11c9439d4af17e6475d9fe058e6b2e03914d17e6/powershell/VstsTaskSdk/LoggingCommandFunctions.ps1 and modified.
# NOTE: You should not be calling these method directly as they are likely to change. Instead you should be calling the Write-Pipeline* functions defined in tools.ps1
$script:loggingCommandPrefix = '##vso['
$script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"?
New-Object psobject -Property @{ Token = ';' ; Replacement = '%3B' }
New-Object psobject -Property @{ Token = "`r" ; Replacement = '%0D' }
New-Object psobject -Property @{ Token = "`n" ; Replacement = '%0A' }
New-Object psobject -Property @{ Token = "]" ; Replacement = '%5D' }
)
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Category,
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
$PSBoundParameters.Remove("Category") | Out-Null
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
$PSBoundParameters.Remove("Message") | Out-Null
$PSBoundParameters.Add("Message", $Message)
Write-PipelineTaskError @PSBoundParameters
}
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
if(!$ci) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Name,
[string]$Value,
[switch]$Secret,
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
'isOutput' = $IsMultiJobVariable
} -AsOutput:$AsOutput
}
}
function Write-PipelinePrependPath {
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
<########################################
# Private functions.
########################################>
function Format-LoggingCommandData {
[CmdletBinding()]
param([string]$Value, [switch]$Reverse)
if (!$Value) {
return ''
}
if (!$Reverse) {
foreach ($mapping in $script:loggingCommandEscapeMappings) {
$Value = $Value.Replace($mapping.Token, $mapping.Replacement)
}
} else {
for ($i = $script:loggingCommandEscapeMappings.Length - 1 ; $i -ge 0 ; $i--) {
$mapping = $script:loggingCommandEscapeMappings[$i]
$Value = $Value.Replace($mapping.Replacement, $mapping.Token)
}
}
return $Value
}
function Format-LoggingCommand {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Area,
[Parameter(Mandatory = $true)]
[string]$Event,
[string]$Data,
[hashtable]$Properties)
# Append the preamble.
[System.Text.StringBuilder]$sb = New-Object -TypeName System.Text.StringBuilder
$null = $sb.Append($script:loggingCommandPrefix).Append($Area).Append('.').Append($Event)
# Append the properties.
if ($Properties) {
$first = $true
foreach ($key in $Properties.Keys) {
[string]$value = Format-LoggingCommandData $Properties[$key]
if ($value) {
if ($first) {
$null = $sb.Append(' ')
$first = $false
} else {
$null = $sb.Append(';')
}
$null = $sb.Append("$key=$value")
}
}
}
# Append the tail and output the value.
$Data = Format-LoggingCommandData $Data
$sb.Append(']').Append($Data).ToString()
}
function Write-LoggingCommand {
[CmdletBinding(DefaultParameterSetName = 'Parameters')]
param(
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Area,
[Parameter(Mandatory = $true, ParameterSetName = 'Parameters')]
[string]$Event,
[Parameter(ParameterSetName = 'Parameters')]
[string]$Data,
[Parameter(ParameterSetName = 'Parameters')]
[hashtable]$Properties,
[Parameter(Mandatory = $true, ParameterSetName = 'Object')]
$Command,
[switch]$AsOutput)
if ($PSCmdlet.ParameterSetName -eq 'Object') {
Write-LoggingCommand -Area $Command.Area -Event $Command.Event -Data $Command.Data -Properties $Command.Properties -AsOutput:$AsOutput
return
}
$command = Format-LoggingCommand -Area $Area -Event $Event -Data $Data -Properties $Properties
if ($AsOutput) {
$command
} else {
Write-Host $command
}
}
function Write-LogIssue {
[CmdletBinding()]
param(
[ValidateSet('warning', 'error')]
[Parameter(Mandatory = $true)]
[string]$Type,
[string]$Message,
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
$command = Format-LoggingCommand -Area 'task' -Event 'logissue' -Data $Message -Properties @{
'type' = $Type
'code' = $ErrCode
'sourcepath' = $SourcePath
'linenumber' = $LineNumber
'columnnumber' = $ColumnNumber
}
if ($AsOutput) {
return $command
}
if ($Type -eq 'error') {
$foregroundColor = $host.PrivateData.ErrorForegroundColor
$backgroundColor = $host.PrivateData.ErrorBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Red
$backgroundColor = [System.ConsoleColor]::Black
}
} else {
$foregroundColor = $host.PrivateData.WarningForegroundColor
$backgroundColor = $host.PrivateData.WarningBackgroundColor
if ($foregroundColor -isnot [System.ConsoleColor] -or $backgroundColor -isnot [System.ConsoleColor]) {
$foregroundColor = [System.ConsoleColor]::Yellow
$backgroundColor = [System.ConsoleColor]::Black
}
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
function Write-PipelineTelemetryError {
local telemetry_category=''
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-category|-c)
telemetry_category=$2
shift
;;
-*)
function_args+=("$1 $2")
shift
;;
*)
message=$*
;;
esac
shift
done
if [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
local message_type="error"
local sourcepath=''
local linenumber=''
local columnnumber=''
local error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-ln)
linenumber=$2
shift
;;
-columnnumber|-cn)
columnnumber=$2
shift
;;
-errcode|-e)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
local message="##vso[task.logissue"
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
function Write-PipelineSetVariable {
if [[ "$ci" != true ]]; then
return
fi
local name=''
local value=''
local secret=false
local as_output=false
local is_multi_job_variable=true
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-name|-n)
name=$2
shift
;;
-value|-v)
value=$2
shift
;;
-secret|-s)
secret=true
;;
-as_output|-a)
as_output=true
;;
-is_multi_job_variable|-i)
is_multi_job_variable=$2
shift
;;
esac
shift
done
value=${value/;/%3B}
value=${value/\\r/%0D}
value=${value/\\n/%0A}
value=${value/]/%5D}
local message="##vso[task.setvariable variable=$name;isSecret=$secret;isOutput=$is_multi_job_variable]$value"
if [[ "$as_output" == true ]]; then
$message
else
echo "$message"
fi
}
function Write-PipelinePrependPath {
local prepend_path=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-path|-p)
prepend_path=$2
shift
;;
esac
shift
done
export PATH="$prepend_path:$PATH"
if [[ "$ci" == true ]]; then
echo "##vso[task.prependpath]$prepend_path"
fi
}
@@ -1,45 +0,0 @@
param(
[Parameter(Mandatory=$true)][int] $BarBuildId, # ID of the build which assets should be downloaded
[Parameter(Mandatory=$true)][string] $DropLocation, # Where the assets should be downloaded to
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken, # Token used to access Maestro API
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com", # Maestro API URL
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16" # Version of Maestro API to use
)
. $PSScriptRoot\post-build-utils.ps1
try {
Write-Host "Installing DARC ..."
. $PSScriptRoot\..\darc-init.ps1
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-PipelineTaskError "Something failed while running 'darc-init.ps1'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
# For now, only use a dry run.
# Ideally we would change darc to enable a quick request that
# would check whether the file exists that you can download it,
# and that it won't conflict with other files.
# https://github.com/dotnet/arcade/issues/3674
# Right now we can't remove continue-on-error because we ocassionally will have
# dependencies that have no associated builds (e.g. an old dependency).
# We need to add an option to baseline specific dependencies away, or add them manually
# to the BAR.
darc gather-drop --non-shipping `
--dry-run `
--continue-on-error `
--id $BarBuildId `
--output-dir $DropLocation `
--bar-uri $MaestroApiEndpoint `
--password $MaestroApiAccessToken `
--latest-location
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
@@ -1,25 +0,0 @@
# This script validates NuGet package metadata information using this
# tool: https://github.com/NuGet/NuGetGallery/tree/jver-verify/src/VerifyMicrosoftPackage
param(
[Parameter(Mandatory=$true)][string] $PackagesPath, # Path to where the packages to be validated are
[Parameter(Mandatory=$true)][string] $ToolDestinationPath # Where the validation tool should be downloaded to
)
. $PSScriptRoot\post-build-utils.ps1
try {
$url = "https://raw.githubusercontent.com/NuGet/NuGetGallery/jver-verify/src/VerifyMicrosoftPackage/verify.ps1"
New-Item -ItemType "directory" -Path ${ToolDestinationPath} -Force
Invoke-WebRequest $url -OutFile ${ToolDestinationPath}\verify.ps1
& ${ToolDestinationPath}\verify.ps1 ${PackagesPath}\*.nupkg
}
catch {
Write-PipelineTaskError "NuGet package validation failed. Please check error logs."
Write-Host $_
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
@@ -1,91 +0,0 @@
# Most of the functions in this file require the variables `MaestroApiEndPoint`,
# `MaestroApiVersion` and `MaestroApiAccessToken` to be globally available.
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = "true"
. $PSScriptRoot\..\tools.ps1
function Create-MaestroApiRequestHeaders([string]$ContentType = "application/json") {
Validate-MaestroVars
$headers = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$headers.Add('Accept', $ContentType)
$headers.Add('Authorization',"Bearer $MaestroApiAccessToken")
return $headers
}
function Get-MaestroChannel([int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}?api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroBuild([int]$BuildId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/builds/${BuildId}?api-version=$MaestroApiVersion"
$result = try { return Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroSubscriptions([string]$SourceRepository, [int]$ChannelId) {
Validate-MaestroVars
$SourceRepository = [System.Web.HttpUtility]::UrlEncode($SourceRepository)
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions?sourceRepository=$SourceRepository&channelId=$ChannelId&api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}/builds/${BuildId}?api-version=$MaestroApiVersion"
Invoke-WebRequest -Method Post -Uri $apiEndpoint -Headers $apiHeaders | Out-Null
}
function Validate-MaestroVars {
try {
Get-Variable MaestroApiEndPoint -Scope Global | Out-Null
Get-Variable MaestroApiVersion -Scope Global | Out-Null
Get-Variable MaestroApiAccessToken -Scope Global | Out-Null
if (!($MaestroApiEndPoint -Match "^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$")) {
Write-PipelineTaskError "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
ExitWithExitCode 1
}
if (!($MaestroApiVersion -Match "^[0-9]{4}-[0-9]{2}-[0-9]{2}$")) {
Write-PipelineTaskError "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
ExitWithExitCode 1
}
}
catch {
Write-PipelineTaskError "Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script."
Write-Host $_
ExitWithExitCode 1
}
}
-48
View File
@@ -1,48 +0,0 @@
param(
[Parameter(Mandatory=$true)][int] $BuildId,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
)
. $PSScriptRoot\post-build-utils.ps1
try {
# Check that the channel we are going to promote the build to exist
$channelInfo = Get-MaestroChannel -ChannelId $ChannelId
if (!$channelInfo) {
Write-Host "Channel with BAR ID $ChannelId was not found in BAR!"
ExitWithExitCode 1
}
# Get info about which channels the build has already been promoted to
$buildInfo = Get-MaestroBuild -BuildId $BuildId
if (!$buildInfo) {
Write-Host "Build with BAR ID $BuildId was not found in BAR!"
ExitWithExitCode 1
}
# Find whether the build is already assigned to the channel or not
if ($buildInfo.channels) {
foreach ($channel in $buildInfo.channels) {
if ($channel.Id -eq $ChannelId) {
Write-Host "The build with BAR ID $BuildId is already on channel $ChannelId!"
ExitWithExitCode 0
}
}
}
Write-Host "Promoting build '$BuildId' to channel '$ChannelId'."
Assign-BuildToChannel -BuildId $BuildId -ChannelId $ChannelId
Write-Host "done."
}
catch {
Write-Host "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
Write-Host $_
Write-Host $_.ScriptStackTrace
}
@@ -1,257 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$false)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$false)][string] $GHCommit, # GitHub commit SHA used to build the packages
[Parameter(Mandatory=$true)][string] $SourcelinkCliVersion # Version of SourceLink CLI to use
)
. $PSScriptRoot\post-build-utils.ps1
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
# Maximum number of jobs to run in parallel
$MaxParallelJobs = 6
# Wait time between check for system load
$SecondsBetweenLoadChecks = 10
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
. $using:PSScriptRoot\..\tools.ps1
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-Host "Input file does not exist: $PackagePath"
return 1
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath) | Out-Null
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
$sourcelinkExe = "$env:USERPROFILE\.dotnet\tools"
$sourcelinkExe = Resolve-Path "$sourcelinkExe\sourcelink.exe"
$SourceLinkInfos = & $sourcelinkExe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = "https://raw.githubusercontent.com/${using:GHRepoName}/${using:GHCommit}/"
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match "github" -or $Uri.Host -match "githubusercontent")) {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
write-host $_
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
}
catch {
}
finally {
$zip.Dispose()
}
if ($FailedFiles -eq 0) {
Write-Host "Passed."
return 0
}
else {
Write-Host "$PackagePath has broken SourceLink links."
return 1
}
}
function ValidateSourceLinkLinks {
if ($GHRepoName -ne "" -and !($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
if (!($GHRepoName -Match "^[^\s-]+-[^\s]+$")) {
Write-PipelineTaskError "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
ExitWithExitCode 1
}
else {
$GHRepoName = $GHRepoName -replace '^([^\s-]+)-([^\s]+)$', '$1/$2';
}
}
if ($GHCommit -ne "" -and !($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-PipelineTaskError "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
ExitWithExitCode 1
}
if ($GHRepoName -ne "" -and $GHCommit -ne "") {
$RepoTreeURL = -Join("http://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL -UseBasicParsing | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL . Execution will proceed without caching."
}
}
elseif ($GHRepoName -ne "" -or $GHCommit -ne "") {
Write-Host "For using the http caching mechanism both GHRepoName and GHCommit should be informed."
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName | Out-Null
$NumJobs = @(Get-Job -State 'Running').Count
while ($NumJobs -ge $MaxParallelJobs) {
Write-Host "There are $NumJobs validation jobs running right now. Waiting $SecondsBetweenLoadChecks seconds to check again."
sleep $SecondsBetweenLoadChecks
$NumJobs = @(Get-Job -State 'Running').Count
}
foreach ($Job in @(Get-Job -State 'Completed')) {
Receive-Job -Id $Job.Id
Remove-Job -Id $Job.Id
}
}
$ValidationFailures = 0
foreach ($Job in @(Get-Job)) {
$jobResult = Wait-Job -Id $Job.Id | Receive-Job
if ($jobResult -ne "0") {
$ValidationFailures++
}
}
if ($ValidationFailures -gt 0) {
Write-PipelineTaskError " $ValidationFailures package(s) failed validation."
ExitWithExitCode 1
}
}
function InstallSourcelinkCli {
$sourcelinkCliPackageName = "sourcelink"
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$sourcelinkCliPackageName*") -and ($toolList -like "*$sourcelinkCliVersion*")) {
Write-Host "SourceLink CLI version $sourcelinkCliVersion is already installed."
}
else {
Write-Host "Installing SourceLink CLI version $sourcelinkCliVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
& "$dotnet" tool install $sourcelinkCliPackageName --version $sourcelinkCliVersion --verbosity "minimal" --global
}
}
try {
InstallSourcelinkCli
ValidateSourceLinkLinks
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
@@ -1,189 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $DotnetSymbolVersion # Version of dotnet symbol to use
)
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $TargetServerParam, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $SymbolsPath
)
$FileName = [System.IO.Path]::GetFileName($FullPath)
$Extension = [System.IO.Path]::GetExtension($FullPath)
# Those below are potential symbol files that the `dotnet symbol` might
# return. Which one will be returned depend on the type of file we are
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$dotnetSymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetSymbolExe = Resolve-Path "$dotnetSymbolExe\dotnet-symbol.exe"
& $dotnetSymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
}
elseif (Test-Path $SymbolPath) {
return "Module"
}
else {
return $null
}
}
function CountMissingSymbols {
param(
[string] $PackagePath # Path to a NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
# How many files are missing symbol information
$MissingSymbols = 0
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
}
else {
Write-Host "No symbols found on SymWeb!"
}
}
}
}
Pop-Location
return $MissingSymbols
}
function CheckSymbolsAvailable {
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-PipelineTaskError "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
}
Write-Host
}
}
function InstallDotnetSymbol {
$dotnetSymbolPackageName = "dotnet-symbol"
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$dotnetSymbolPackageName*") -and ($toolList -like "*$dotnetSymbolVersion*")) {
Write-Host "dotnet-symbol version $dotnetSymbolVersion is already installed."
}
else {
Write-Host "Installing dotnet-symbol version $dotnetSymbolVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
& "$dotnet" tool install $dotnetSymbolPackageName --version $dotnetSymbolVersion --verbosity "minimal" --global
}
}
try {
InstallDotnetSymbol
CheckSymbolsAvailable
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
@@ -1,57 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $SourceRepo,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = "https://maestro-prod.westus2.cloudapp.azure.com",
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = "2019-01-16"
)
. $PSScriptRoot\post-build-utils.ps1
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
if (!$subscriptions) {
Write-Host "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: $subscription.id"
[void]$subscriptionsToTrigger.Add($subscription.id)
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host "done."
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-Host "At least one subscription failed to be triggered..."
ExitWithExitCode 1
}
else {
Write-Host "All subscriptions were triggered successfully!"
}
-13
View File
@@ -1,13 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
<packageSources>
<clear />
<add key="guardian" value="https://securitytools.pkgs.visualstudio.com/_packaging/Guardian/nuget/v3/index.json" />
</packageSources>
<disabledPackageSources>
<clear />
</disabledPackageSources>
</configuration>
-100
View File
@@ -1,100 +0,0 @@
Param(
[string] $GuardianPackageName, # Required: the name of guardian CLI package (not needed if GuardianCliLocation is specified)
[string] $NugetPackageDirectory, # Required: directory where NuGet packages are installed (not needed if GuardianCliLocation is specified)
[string] $GuardianCliLocation, # Optional: Direct location of Guardian CLI executable if GuardianPackageName & NugetPackageDirectory are not specified
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ("artifacts")), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
[bool] $TsaPublish=$False, # Optional: true will publish results to TSA; only set to true after onboarding to TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaBranchName=$env:BUILD_SOURCEBRANCH, # Optional: required for TSA publish; defaults to $(Build.SourceBranchName); TSA is the automated framework used to upload test results as bugs.
[string] $TsaRepositoryName=$env:BUILD_REPOSITORY_NAME, # Optional: TSA repository name; will be generated automatically if not submitted; TSA is the automated framework used to upload test results as bugs.
[string] $BuildNumber=$env:BUILD_BUILDNUMBER, # Optional: required for TSA publish; defaults to $(Build.BuildNumber)
[bool] $UpdateBaseline=$False, # Optional: if true, will update the baseline in the repository; should only be run after fixing any issues which need to be fixed
[bool] $TsaOnboard=$False, # Optional: if true, will onboard the repository to TSA; should only be run once; TSA is the automated framework used to upload test results as bugs.
[string] $TsaInstanceUrl, # Optional: only needed if TsaOnboard or TsaPublish is true; the instance-url registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the codebase registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaProjectName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the project registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaNotificationEmail, # Optional: only needed if TsaOnboard is true; the email(s) which will receive notifications of TSA bug filings (e.g. [email protected]); TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel="Standard", # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path "tools" "guardian.cmd"))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
exit 1
}
& $(Join-Path $PSScriptRoot "init-sdl.ps1") -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory ".gdn"
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not onboard to TSA -- not all required values ($$TsaCodebaseName, $$TsaNotificationEmail, $$TsaCodebaseAdmin, $$TsaBugAreaPath) were specified."
exit 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot "run-sdl.ps1") -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Update baseline"
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian tsa-publish failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
} else {
Write-Host "Could not publish to TSA -- not all required values ($$TsaBranchName, $$BuildNumber) were specified."
exit 1
}
}
@@ -1,78 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where artifact packages are stored
[Parameter(Mandatory=$true)][string] $ExtractPath # Full path to directory where the packages will be extracted
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = "true"
. $PSScriptRoot\..\tools.ps1
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Extracting" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
}
finally {
$zip.Dispose()
}
}
function ExtractArtifacts {
if (!(Test-Path $InputPath)) {
Write-Host "Input Path does not exist: $InputPath"
ExitWithExitCode 0
}
$Jobs = @()
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ExtractPackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
try {
Measure-Command { ExtractArtifacts }
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
-51
View File
@@ -1,51 +0,0 @@
Param(
[string] $GuardianCliLocation,
[string] $Repository,
[string] $BranchName="master",
[string] $WorkingDirectory,
[string] $AzureDevOpsAccessToken,
[string] $GuardianLoggerLevel="Standard"
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
# Construct basic auth from AzDO access token; construct URI to the repository's gdn folder stored in that repository; construct location of zip file
$encodedPat = [Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$AzureDevOpsAccessToken"))
$escapedRepository = [Uri]::EscapeDataString("/$Repository/$BranchName/.gdn")
$uri = "https://dev.azure.com/dnceng/internal/_apis/git/repositories/sdl-tool-cfg/Items?path=$escapedRepository&versionDescriptor[versionOptions]=0&`$format=zip&api-version=5.0-preview.1"
$zipFile = "$WorkingDirectory/gdn.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
$gdnFolder = (Join-Path $WorkingDirectory ".gdn")
Try
{
# We try to download the zip; if the request fails (e.g. the file doesn't exist), we catch it and init guardian instead
Write-Host "Downloading gdn folder from internal config repostiory..."
Invoke-WebRequest -Headers @{ "Accept"="application/zip"; "Authorization"="Basic $encodedPat" } -Uri $uri -OutFile $zipFile
if (Test-Path $gdnFolder) {
# Remove the gdn folder if it exists (it shouldn't unless there's too much caching; this is just in case)
Remove-Item -Force -Recurse $gdnFolder
}
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipFile, $WorkingDirectory)
Write-Host $gdnFolder
} Catch [System.Net.WebException] {
# if the folder does not exist, we'll do a guardian init and push it to the remote repository
Write-Host "Initializing Guardian..."
Write-Host "$GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel"
& $GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian init failed with exit code $LASTEXITCODE."
}
# We create the mainbaseline so it can be edited later
Write-Host "$GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline"
& $GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline
if ($LASTEXITCODE -ne 0) {
Write-Error "Guardian baseline failed with exit code $LASTEXITCODE."
}
& $(Join-Path $PSScriptRoot "push-gdn.ps1") -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason "Initialize gdn folder"
}
-4
View File
@@ -1,4 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Guardian.Cli" version="0.7.2"/>
</packages>
-51
View File
@@ -1,51 +0,0 @@
Param(
[string] $Repository,
[string] $BranchName="master",
[string] $GdnFolder,
[string] $AzureDevOpsAccessToken,
[string] $PushReason
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP "sdl"
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-Error "Git clone failed with exit code $LASTEXITCODE."
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) ".gdn"
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host "git add ."
git add .
if ($LASTEXITCODE -ne 0) {
Write-Error "Git add failed with exit code $LASTEXITCODE."
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-Error "Git commit failed with exit code $LASTEXITCODE."
}
Write-Host "git push"
git push
if ($LASTEXITCODE -ne 0) {
Write-Error "Git push failed with exit code $LASTEXITCODE."
}
# Return to the original directory
Pop-Location
-59
View File
@@ -1,59 +0,0 @@
Param(
[string] $GuardianCliLocation,
[string] $WorkingDirectory,
[string] $TargetDirectory,
[string] $GdnFolder,
[string[]] $ToolsList,
[string] $UpdateBaseline,
[string] $GuardianLoggerLevel="Standard",
[string[]] $CrScanAdditionalRunConfigParams,
[string[]] $PoliCheckAdditionalRunConfigParams
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
$LASTEXITCODE = 0
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder "r"
$ValidPath = Test-Path $GuardianCliLocation
if ($ValidPath -eq $False)
{
Write-Host "Invalid Guardian CLI Location."
exit 1
}
$configParam = @("--config")
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq "credscan") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
}
if ($tool -eq "policheck") {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian configure for $tool failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
}
$configParam+=$gdnConfigFile
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-Host "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
exit $LASTEXITCODE
}
-73
View File
@@ -1,73 +0,0 @@
parameters:
overrideParameters: '' # Optional: to override values for parameters.
additionalParameters: '' # Optional: parameters that need user specific values eg: '-SourceToolsList @("abc","def") -ArtifactToolsList @("ghi","jkl")'
# There is some sort of bug (has been reported) in Azure DevOps where if this parameter is named
# 'continueOnError', the parameter value is not correctly picked up.
# This can also be remedied by the caller (post-build.yml) if it does not use a nested parameter
sdlContinueOnError: false # optional: determines whether to continue the build if the step errors;
dependsOn: '' # Optional: dependencies of the job
artifactNames: '' # Optional: patterns supplied to DownloadBuildArtifacts
# Usage:
# artifactNames:
# - 'BlobArtifacts'
# - 'Artifacts_Windows_NT_Release'
jobs:
- job: Run_SDL
dependsOn: ${{ parameters.dependsOn }}
displayName: Run SDL tool
variables:
- group: DotNet-VSTS-Bot
pool:
name: Hosted VS2017
steps:
- checkout: self
clean: true
- ${{ if ne(parameters.artifactNames, '') }}:
- ${{ each artifactName in parameters.artifactNames }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: current
artifactName: ${{ artifactName }}
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if eq(parameters.artifactNames, '') }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: current
downloadType: specific files
itemPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
displayName: Extract Blob Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
displayName: Extract Package Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
- task: NuGetCommand@2
displayName: 'Install Guardian'
inputs:
restoreSolution: $(Build.SourcesDirectory)\eng\common\sdl\packages.config
feedsToUse: config
nugetConfigPath: $(Build.SourcesDirectory)\eng\common\sdl\NuGet.config
externalFeedCredentials: GuardianConnect
restoreDirectory: $(Build.SourcesDirectory)\.packages
- ${{ if ne(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1 ${{ parameters.overrideParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
- ${{ if eq(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1
-GuardianPackageName Microsoft.Guardian.Cli.0.7.2
-NugetPackageDirectory $(Build.SourcesDirectory)\.packages
-AzureDevOpsAccessToken $(dn-bot-dotnet-build-rw-code-rw)
${{ parameters.additionalParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
+64 -75
View File
@@ -1,33 +1,64 @@
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
parameters:
# Job schema parameters - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
cancelTimeoutInMinutes: ''
condition: ''
container: ''
continueOnError: false
container: ''
dependsOn: ''
displayName: ''
pool: ''
steps: []
pool: ''
strategy: ''
timeoutInMinutes: ''
variables: []
workspace: ''
# Job base template specific parameters
# See schema documentation - https://github.com/dotnet/arcade/blob/master/Documentation/AzureDevOps/TemplateSchema.md
artifacts: ''
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
enablePublishUsingPipelines: false
# Optional: enable sending telemetry
enableTelemetry: false
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: define the helix type for telemetry (example: 'build/product/')
helixType: ''
# Required: name of the job
name: ''
preSteps: []
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- job: ${{ parameters.name }}
@@ -59,7 +90,7 @@ jobs:
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ if ne(parameters.enableTelemetry, 'false') }}:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- ${{ each variable in parameters.variables }}:
@@ -91,12 +122,21 @@ jobs:
workspace: ${{ parameters.workspace }}
steps:
- ${{ if ne(parameters.preSteps, '') }}:
- ${{ each preStep in parameters.preSteps }}:
- ${{ preStep }}
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendStartTelemetry@0
displayName: 'Send Helix Start Telemetry'
inputs:
helixRepo: ${{ parameters.helixRepo }}
${{ if ne(parameters.helixType, '') }}:
helixType: ${{ parameters.helixType }}
buildConfig: $(_BuildConfig)
runAsPublic: ${{ parameters.runAsPublic }}
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
@@ -108,16 +148,6 @@ jobs:
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- task: NuGetAuthenticate@0
- ${{ if or(eq(parameters.artifacts.download, 'true'), ne(parameters.artifacts.download, '')) }}:
- task: DownloadPipelineArtifact@2
inputs:
buildType: current
artifactName: ${{ coalesce(parameters.artifacts.download.name, 'Artifacts_$(Agent.OS)_$(_BuildConfig)') }}
targetPath: ${{ coalesce(parameters.artifacts.download.path, 'artifacts') }}
itemPattern: ${{ coalesce(parameters.artifacts.download.pattern, '**') }}
- ${{ each step in parameters.steps }}:
- ${{ step }}
@@ -130,60 +160,20 @@ jobs:
env:
TeamName: $(_TeamName)
- ${{ if ne(parameters.artifacts.publish, '') }}:
- ${{ if or(eq(parameters.artifacts.publish.artifacts, 'true'), ne(parameters.artifacts.publish.artifacts, '')) }}:
- task: CopyFiles@2
displayName: Gather binaries for publish to artifacts
inputs:
SourceFolder: 'artifacts/bin'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/bin'
- task: CopyFiles@2
displayName: Gather packages for publish to artifacts
inputs:
SourceFolder: 'artifacts/packages'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/packages'
- task: PublishBuildArtifacts@1
displayName: Publish pipeline artifacts
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.logs, 'true'), ne(parameters.artifacts.publish.logs, '')) }}:
- publish: artifacts/log
artifact: ${{ coalesce(parameters.artifacts.publish.logs.name, 'Logs_Build_$(Agent.Os)_$(_BuildConfig)') }}
displayName: Publish logs
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- ${{ if and(ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.ArtifactStagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendEndTelemetry@0
displayName: 'Send Helix End Telemetry'
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if ne(parameters.enablePublishBuildArtifacts, 'false') }}:
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
ArtifactName: $(Agent.Os)_$(Agent.JobName)
continueOnError: true
condition: always()
@@ -197,7 +187,7 @@ jobs:
continueOnError: true
condition: always()
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(variables['_PublishUsingPipelines'], 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
@@ -205,7 +195,6 @@ jobs:
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
-95
View File
@@ -1,95 +0,0 @@
parameters:
steps: [] # optional -- any additional steps that need to happen before pulling down the performance repo and sending the performance benchmarks to helix (ie building your repo)
variables: [] # optional -- list of additional variables to send to the template
jobName: '' # required -- job name
displayName: '' # optional -- display name for the job. Will use jobName if not passed
pool: '' # required -- name of the Build pool
container: '' # required -- name of the container
osGroup: '' # required -- operating system for the job
extraSetupParameters: '' # optional -- extra arguments to pass to the setup script
frameworks: ['netcoreapp3.0'] # optional -- list of frameworks to run against
continueOnError: 'false' # optional -- determines whether to continue the build if the step errors
dependsOn: '' # optional -- dependencies of the job
timeoutInMinutes: 320 # optional -- timeout for the job
enableTelemetry: false # optional -- enable for telemetry
jobs:
- template: ../jobs/jobs.yml
parameters:
dependsOn: ${{ parameters.dependsOn }}
enableTelemetry: ${{ parameters.enableTelemetry }}
enablePublishBuildArtifacts: true
continueOnError: ${{ parameters.continueOnError }}
jobs:
- job: '${{ parameters.jobName }}'
${{ if ne(parameters.displayName, '') }}:
displayName: '${{ parameters.displayName }}'
${{ if eq(parameters.displayName, '') }}:
displayName: '${{ parameters.jobName }}'
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ each variable in parameters.variables }}:
- ${{ if ne(variable.name, '') }}:
- name: ${{ variable.name }}
value: ${{ variable.value }}
- ${{ if ne(variable.group, '') }}:
- group: ${{ variable.group }}
- IsInternal: ''
- HelixApiAccessToken: ''
- HelixPreCommand: ''
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq( parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'set "PERFLAB_UPLOAD_TOKEN=$(PerfCommandUploadToken)"'
- IsInternal: -Internal
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'export PERFLAB_UPLOAD_TOKEN="$(PerfCommandUploadTokenLinux)"'
- IsInternal: --internal
- group: DotNet-HelixApi-Access
- group: dotnet-benchview
workspace:
clean: all
pool:
${{ parameters.pool }}
container: ${{ parameters.container }}
strategy:
matrix:
${{ each framework in parameters.frameworks }}:
${{ framework }}:
_Framework: ${{ framework }}
steps:
- checkout: self
clean: true
# Run all of the steps to setup repo
- ${{ each step in parameters.steps }}:
- ${{ step }}
- powershell: $(Build.SourcesDirectory)\eng\common\performance\performance-setup.ps1 $(IsInternal) -Framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Windows)
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Build.SourcesDirectory)/eng/common/performance/performance-setup.sh $(IsInternal) --framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Unix)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments)
displayName: Run ci setup script
# Run perf testing in helix
- template: /eng/common/templates/steps/perf-send-to-helix.yml
parameters:
HelixSource: '$(HelixSourcePrefix)/$(Build.Repository.Name)/$(Build.SourceBranch)' # sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'test/performance/$(Kind)/$(_Framework)/$(Architecture)'
HelixAccessToken: $(HelixApiAccessToken)
HelixTargetQueues: $(Queue)
HelixPreCommands: $(HelixPreCommand)
Creator: $(Creator)
WorkItemTimeout: 4:00 # 4 hours
WorkItemDirectory: '$(WorkItemDirectory)' # WorkItemDirectory can not be empty, so we send it some docs to keep it happy
CorrelationPayloadDirectory: '$(PayloadDirectory)' # it gets checked out to a folder with shorter path than WorkItemDirectory so we can avoid file name too long exceptions
@@ -47,10 +47,6 @@ jobs:
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: NuGetAuthenticate@0
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
@@ -63,23 +59,6 @@ jobs:
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: powershell@2
displayName: Create ReleaseConfigs Artifact
inputs:
targetType: inline
script: |
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(BARBuildId)
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value "$(DefaultChannels)"
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(IsStableBuild)
- task: PublishBuildArtifacts@1
displayName: Publish ReleaseConfigs Artifact
inputs:
PathtoPublish: '$(Build.StagingDirectory)/ReleaseConfigs.txt'
PublishLocation: Container
ArtifactName: ReleaseConfigs
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
+46 -28
View File
@@ -1,10 +1,19 @@
parameters:
# See schema documentation in /Documentation/AzureDevOps/TemplateSchema.md
# Optional: 'true' if failures in job.yml job should not fail the job
continueOnError: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Enable publishing using release pipelines
enablePublishUsingPipelines: false
@@ -14,9 +23,19 @@ parameters:
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
# if enabled then the 'helixRepo' parameter should also be specified
enableTelemetry: false
# Required: A collection of jobs to run - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
jobs: []
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: Override automatically derived dependsOn value for "publish build assets" job
publishBuildAssetsDependsOn: ''
@@ -43,30 +62,29 @@ jobs:
name: ${{ job.job }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if eq(parameters.graphFileGeneration.enabled, true) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if and(eq(parameters.graphFileGeneration.enabled, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
@@ -1,160 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} ))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
/p:PublishToMSDL=false
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=true
/p:ChecksumsTargetStaticFeed=$(InternalChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(InternalChecksumsBlobFeedKey)
/p:InstallersTargetStaticFeed=$(InternalInstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(InternalInstallersBlobFeedKey)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:PublishToMSDL=false
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,160 +0,0 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} ))
variables:
- group: DotNet-Symbol-Server-Pats
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
artifactName: 'BlobArtifacts'
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download PDB Artifacts
inputs:
artifactName: 'PDBArtifacts'
continueOnError: true
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
continueOnError: true
- task: DownloadBuildArtifacts@0
displayName: Download Asset Manifests
inputs:
buildType: current
artifactName: AssetManifests
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(_DotNetArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/promote-build.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -1,88 +0,0 @@
variables:
- group: AzureDevOps-Artifact-Feeds-Pats
- group: DotNet-Blob-Feed
- group: DotNet-DotNetCli-Storage
- group: DotNet-MSRC-Storage
- group: Publish-Build-Assets
# .NET Core 3.1 Dev
- name: PublicDevRelease_31_Channel_Id
value: 128
# .NET Core 5 Dev
- name: NetCore_5_Dev_Channel_Id
value: 131
# .NET Tools - Validation
- name: NetCore_Tools_Validation_Channel_Id
value: 9
# .NET Tools - Latest
- name: NetCore_Tools_Latest_Channel_Id
value: 2
# .NET 3 Tools - Validation
- name: NETCore_3_Tools_Validation_Channel_Id
value: 390
# .NET 3 Tools - Latest
- name: NetCore_3_Tools_Channel_Id
value: 344
# .NET Core 3.0 Internal Servicing
- name: InternalServicing_30_Channel_Id
value: 184
# .NET Core 3.0 Release
- name: PublicRelease_30_Channel_Id
value: 19
# .NET Core 3.1 Release
- name: PublicRelease_31_Channel_Id
value: 129
# General Testing
- name: GeneralTesting_Channel_Id
value: 529
# .NET Core 3.1 Blazor Features
- name: NetCore_31_Blazor_Features_Channel_Id
value: 531
# Whether the build is internal or not
- name: IsInternalBuild
value: ${{ and(ne(variables['System.TeamProject'], 'public'), contains(variables['Build.SourceBranch'], 'internal')) }}
# Default Maestro++ API Endpoint and API Version
- name: MaestroApiEndPoint
value: "https://maestro-prod.westus2.cloudapp.azure.com"
- name: MaestroApiAccessToken
value: $(MaestroAccessToken)
- name: MaestroApiVersion
value: "2019-01-16"
- name: SourceLinkCLIVersion
value: 3.0.0
- name: SymbolToolVersion
value: 1.0.1
# Feed Configurations
# These should include the suffix "/index.json"
# Default locations for Installers and checksums
# Public Locations
- name: ChecksumsBlobFeedUrl
value: https://dotnetclichecksums.blob.core.windows.net/dotnet/index.json
- name: InstallersBlobFeedUrl
value: https://dotnetcli.blob.core.windows.net/dotnet/index.json
# Private Locations
- name: InternalChecksumsBlobFeedUrl
value: https://dotnetclichecksumsmsrc.blob.core.windows.net/dotnet/index.json
- name: InternalChecksumsBlobFeedKey
value: $(dotnetclichecksumsmsrc-storage-key)
- name: InternalInstallersBlobFeedUrl
value: https://dotnetclimsrc.blob.core.windows.net/dotnet/index.json
- name: InternalInstallersBlobFeedKey
value: $(dotnetclimsrc-access-key)
@@ -1,23 +0,0 @@
parameters:
ChannelId: 0
jobs:
- job: gatherDrop
displayName: Gather Drop
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.ChannelId }}))
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Darc gather-drop
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/darc-gather-drop.ps1
arguments: -BarBuildId $(BARBuildId)
-DropLocation $(Agent.BuildDirectory)/Temp/Drop/
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,288 +0,0 @@
parameters:
enableSourceLinkValidation: false
enableSigningValidation: true
enableSymbolValidation: false
enableNugetValidation: true
publishInstallersAndChecksums: false
SDLValidationParameters:
enable: false
continueOnError: false
params: ''
artifactNames: ''
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
symbolPublishingAdditionalParameters: ''
artifactsPublishingAdditionalParameters: ''
signingValidationAdditionalParameters: ''
# Which stages should finish execution before post-build stages start
validateDependsOn:
- build
publishDependsOn:
- Validate
stages:
- stage: Validate
dependsOn: ${{ parameters.validateDependsOn }}
displayName: Validate
jobs:
- ${{ if eq(parameters.enableNugetValidation, 'true') }}:
- job:
displayName: NuGet Validation
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- ${{ if eq(parameters.enableSigningValidation, 'true') }}:
- job:
displayName: Signing Validation
variables:
- template: common-variables.yml
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: current
artifactName: PackageArtifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
${{ parameters.signingValidationAdditionalParameters }}
- template: ../steps/publish-logs.yml
parameters:
StageLabel: 'Validation'
JobLabel: 'Signing'
- ${{ if eq(parameters.enableSourceLinkValidation, 'true') }}:
- job:
displayName: SourceLink Validation
variables:
- template: common-variables.yml
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: current
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- ${{ if eq(parameters.SDLValidationParameters.enable, 'true') }}:
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
artifactNames: ${{ parameters.SDLValidationParameters.artifactNames }}
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev5_Publish'
channelName: '.NET Core 5 Dev'
channelId: 131
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev31_Publish'
channelName: '.NET Core 3.1 Dev'
channelId: 128
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Tools_Latest_Publish'
channelName: '.NET Tools - Latest'
channelId: 2
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'PVR_Publish'
channelName: '.NET Tools - Validation'
channelId: 9
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_3_Tools_Validation_Publish'
channelName: '.NET 3 Tools - Validation'
channelId: 390
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_3_Tools_Publish'
channelName: '.NET 3 Tools'
channelId: 344
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Release30_Publish'
channelName: '.NET Core 3.0 Release'
channelId: 19
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Release31_Publish'
channelName: '.NET Core 3.1 Release'
channelId: 129
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Blazor31_Features_Publish'
channelName: '.NET Core 3.1 Blazor Features'
channelId: 531
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-blazor/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-blazor/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-blazor-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_30_Internal_Servicing_Publishing'
channelName: '.NET Core 3.0 Internal Servicing'
channelId: 184
transportFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'General_Testing_Publish'
channelName: 'General Testing'
channelId: 529
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Dev_Publishing'
channelName: '.NET Core Tooling Dev'
channelId: 548
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Release_Publishing'
channelName: '.NET Core Tooling Release'
channelId: 549
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
@@ -1,25 +0,0 @@
parameters:
ChannelId: 0
jobs:
- job:
displayName: Promote Build
dependsOn: setupMaestroVars
condition: contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.ChannelId }}))
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: ChannelId
value: ${{ parameters.ChannelId }}
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/promote-build.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId $(ChannelId)
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,40 +0,0 @@
jobs:
- job: setupMaestroVars
displayName: Setup Maestro Vars
pool:
vmImage: 'windows-2019'
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
displayName: Download Release Configs
inputs:
buildType: current
artifactName: ReleaseConfigs
- task: PowerShell@2
name: setReleaseVars
displayName: Set Release Configs Vars
inputs:
targetType: inline
script: |
try {
$Content = Get-Content $(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
Write-Host "##vso[task.setvariable variable=BARBuildId;isOutput=true]$BarId"
Write-Host "##vso[task.setvariable variable=InitialChannels;isOutput=true]$Channels"
Write-Host "##vso[task.setvariable variable=IsStableBuild;isOutput=true]$IsStableBuild"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
exit 1
}
@@ -1,13 +0,0 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Triggering subscriptions
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/trigger-subscriptions.ps1
arguments: -SourceRepo $(Build.Repository.Uri)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,66 +0,0 @@
# Please remember to update the documentation if you make changes to these parameters!
parameters:
HelixSource: 'pr/default' # required -- sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'tests/default/' # required -- Helix telemetry which identifies what type of data this is; should include "test" for clarity and must end in '/'
HelixBuild: $(Build.BuildNumber) # required -- the build number Helix will use to identify this -- automatically set to the AzDO build number
HelixTargetQueues: '' # required -- semicolon delimited list of Helix queues to test on; see https://helix.dot.net/ for a list of queues
HelixAccessToken: '' # required -- access token to make Helix API requests; should be provided by the appropriate variable group
HelixPreCommands: '' # optional -- commands to run before Helix work item execution
HelixPostCommands: '' # optional -- commands to run after Helix work item execution
WorkItemDirectory: '' # optional -- a payload directory to zip up and send to Helix; requires WorkItemCommand; incompatible with XUnitProjects
CorrelationPayloadDirectory: '' # optional -- a directory to zip up and send to Helix as a correlation payload
IncludeDotNetCli: false # optional -- true will download a version of the .NET CLI onto the Helix machine as a correlation payload; requires DotNetCliPackageType and DotNetCliVersion
DotNetCliPackageType: '' # optional -- either 'sdk' or 'runtime'; determines whether the sdk or runtime will be sent to Helix; see https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
DotNetCliVersion: '' # optional -- version of the CLI to send to Helix; based on this: https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Send job to Helix' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
continueOnError: false # optional -- determines whether to continue the build if the step errors; defaults to false
steps:
- powershell: $(Build.SourcesDirectory)\eng\common\msbuild.ps1 $(Build.SourcesDirectory)\eng\common\performance\perfhelixpublish.proj /restore /t:Test /bl:$(Build.SourcesDirectory)\artifacts\log\$env:BuildConfig\SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Windows)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $BUILD_SOURCESDIRECTORY/eng/common/msbuild.sh $BUILD_SOURCESDIRECTORY/eng/common/performance/perfhelixpublish.proj /restore /t:Test /bl:$BUILD_SOURCESDIRECTORY/artifacts/log/$BuildConfig/SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Unix)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
@@ -1,13 +0,0 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/promote-build.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -1,23 +0,0 @@
parameters:
StageLabel: ''
JobLabel: ''
steps:
- task: Powershell@2
displayName: Prepare Binlogs to Upload
inputs:
targetType: inline
script: |
New-Item -ItemType Directory $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
Move-Item -Path $(Build.SourcesDirectory)/artifacts/log/Debug/* $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
continueOnError: true
condition: always()
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/PostBuildLogs'
PublishLocation: Container
ArtifactName: PostBuilLogs
continueOnError: true
condition: always()
@@ -23,7 +23,6 @@ parameters:
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
HelixBaseUri: 'https://helix.dot.net/' # optional -- sets the Helix API base URI (allows targeting int)
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
@@ -56,7 +55,6 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
@@ -87,7 +85,6 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
+53 -112
View File
@@ -35,22 +35,16 @@
# Specifies which msbuild engine to use for build: 'vs', 'dotnet' or unspecified (determined based on presence of tools.vs in global.json).
[string]$msbuildEngine = if (Test-Path variable:msbuildEngine) { $msbuildEngine } else { $null }
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.ps1
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { "v1" }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
$disableConfigureToolsetImport = if (Test-Path variable:disableConfigureToolsetImport) { $disableConfigureToolsetImport } else { $null }
set-strictmode -version 2.0
$ErrorActionPreference = "Stop"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
@@ -82,7 +76,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
$finished = $false
try {
while (-not $process.WaitForExit(100)) {
while (-not $process.WaitForExit(100)) {
# Non-blocking loop done to allow ctr-c interrupts
}
@@ -90,7 +84,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
return $global:LASTEXITCODE = $process.ExitCode
}
finally {
# If we didn't finish then an error occurred or the user hit ctrl-c. Either
# If we didn't finish then an error occured or the user hit ctrl-c. Either
# way kill the process
if (-not $finished) {
$process.Kill()
@@ -140,7 +134,7 @@ function InitializeDotNetCli([bool]$install) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
Write-Host "Unable to find dotnet with SDK version '$dotnetSdkVersion'" -ForegroundColor Red
ExitWithExitCode 1
}
}
@@ -153,11 +147,12 @@ function InitializeDotNetCli([bool]$install) {
# It also ensures that VS msbuild will use the downloaded sdk targets.
$env:PATH = "$dotnetRoot;$env:PATH"
# Make Sure that our bootstrapped dotnet cli is available in future steps of the Azure Pipelines build
Write-PipelinePrependPath -Path $dotnetRoot
Write-PipelineSetVariable -Name 'DOTNET_MULTILEVEL_LOOKUP' -Value '0'
Write-PipelineSetVariable -Name 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE' -Value '1'
if ($ci) {
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
Write-Host "##vso[task.prependpath]$dotnetRoot"
Write-Host "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
Write-Host "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
}
return $global:_DotNetInstallDir = $dotnetRoot
}
@@ -166,8 +161,7 @@ function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1" -OutFile $installScript
Invoke-WebRequest "https://dot.net/v1/dotnet-install.ps1" -OutFile $installScript
}
return $installScript
@@ -177,14 +171,7 @@ function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $arc
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot,
[string] $version,
[string] $architecture = "",
[string] $runtime = "",
[bool] $skipNonVersionedFiles = $false,
[string] $runtimeSourceFeed = "",
[string] $runtimeSourceFeedKey = "") {
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) { $installScript = GetDotNetInstallScript $dotnetRoot
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
@@ -195,34 +182,15 @@ function InstallDotNet([string] $dotnetRoot,
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Failed to install dotnet runtime '$runtime' from public location."
# Only the runtime can be installed from a custom [private] location.
if ($runtime -and ($runtimeSourceFeed -or $runtimeSourceFeedKey)) {
if ($runtimeSourceFeed) { $installParameters.AzureFeed = $runtimeSourceFeed }
if ($runtimeSourceFeedKey) {
$decodedBytes = [System.Convert]::FromBase64String($runtimeSourceFeedKey)
$decodedString = [System.Text.Encoding]::UTF8.GetString($decodedBytes)
$installParameters.FeedCredential = $decodedString
}
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Failed to install dotnet runtime '$runtime' from custom location '$runtimeSourceFeed'."
}
}
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-Host "Failed to install dotnet cli (exit code '$lastExitCode')." -ForegroundColor Red
ExitWithExitCode $lastExitCode
}
}
#
# Locates Visual Studio MSBuild installation.
# Locates Visual Studio MSBuild installation.
# The preference order for MSBuild to use is as follows:
#
# 1. MSBuild from an active VS command prompt
@@ -239,7 +207,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersion = [Version]::new($vsMinVersionStr)
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
@@ -288,7 +256,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
function InitializeVisualStudioEnvironmentVariables([string] $vsInstallDir, [string] $vsMajorVersion) {
$env:VSINSTALLDIR = $vsInstallDir
Set-Item "env:VS$($vsMajorVersion)0COMNTOOLS" (Join-Path $vsInstallDir "Common7\Tools\")
$vsSdkInstallDir = Join-Path $vsInstallDir "VSSDK\"
if (Test-Path $vsSdkInstallDir) {
Set-Item "env:VSSDK$($vsMajorVersion)0Install" $vsSdkInstallDir
@@ -312,7 +280,6 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
Create-Directory $packageDir
Write-Host "Downloading $packageName $packageVersion"
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dotnet.myget.org/F/roslyn-tools/api/v2/package/$packageName/$packageVersion/" -OutFile $packagePath
Unzip $packagePath $packageDir
}
@@ -324,13 +291,13 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
# Locates Visual Studio instance that meets the minimal requirements specified by tools.vs object in global.json.
#
# The following properties of tools.vs are recognized:
# "version": "{major}.{minor}"
# "version": "{major}.{minor}"
# Two part minimal VS version, e.g. "15.9", "16.0", etc.
# "components": ["componentId1", "componentId2", ...]
# "components": ["componentId1", "componentId2", ...]
# Array of ids of workload components that must be available in the VS instance.
# See e.g. https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-enterprise?view=vs-2017
#
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
@@ -350,8 +317,8 @@ function LocateVisualStudio([object]$vsRequirements = $null){
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild", "-products", "*")
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild")
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
$args += $vsRequirements.version
@@ -361,7 +328,7 @@ function LocateVisualStudio([object]$vsRequirements = $null){
foreach ($component in $vsRequirements.components) {
$args += "-requires"
$args += $component
}
}
}
$vsInfo =& $vsWhereExe $args | ConvertFrom-Json
@@ -391,21 +358,22 @@ function InitializeBuildTool() {
if ($msbuildEngine -eq "dotnet") {
if (!$dotnetRoot) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "/global.json must specify 'tools.dotnet'."
Write-Host "/global.json must specify 'tools.dotnet'." -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message $_
Write-Host $_ -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
Write-Host "Unexpected value of -msbuildEngine: '$msbuildEngine'." -ForegroundColor Red
ExitWithExitCode 1
}
@@ -417,12 +385,12 @@ function GetDefaultMSBuildEngine() {
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
}
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
Write-Host "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'." -ForegroundColor Red
ExitWithExitCode 1
}
@@ -446,14 +414,12 @@ function GetSdkTaskProject([string]$taskName) {
}
function InitializeNativeTools() {
if (-Not (Test-Path variable:DisableNativeToolsetInstalls) -And (Get-Member -InputObject $GlobalJson -Name "native-tools")) {
$nativeArgs= @{}
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
$nativeArgs=""
if ($ci) {
$nativeArgs = @{
InstallDirectory = "$ToolsDir"
}
$nativeArgs = "-InstallDirectory $ToolsDir"
}
& "$PSScriptRoot/init-tools-native.ps1" @nativeArgs
Invoke-Expression "& `"$PSScriptRoot/init-tools-native.ps1`" $nativeArgs"
}
}
@@ -475,7 +441,7 @@ function InitializeToolset() {
}
if (-not $restore) {
Write-PipelineTelemetryError -Category "InitializeToolset" -Message "Toolset version $toolsetVersion has not been restored."
Write-Host "Toolset version $toolsetVersion has not been restored." -ForegroundColor Red
ExitWithExitCode 1
}
@@ -518,18 +484,6 @@ function Stop-Processes() {
function MSBuild() {
if ($pipelinesLog) {
$buildTool = InitializeBuildTool
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if ($ci -and $buildTool.Tool -eq "dotnet") {
dotnet nuget locals http-cache -c
$env:NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS = 20
$env:NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS = 20
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS' -Value '20'
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS' -Value '20'
}
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
@@ -547,13 +501,11 @@ function MSBuild() {
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
Write-PipelineTaskError -Message "Binary log must be enabled in CI build."
ExitWithExitCode 1
throw "Binary log must be enabled in CI build."
}
if ($nodeReuse) {
Write-PipelineTaskError -Message "Node reuse must be disabled in CI build."
ExitWithExitCode 1
throw "Node reuse must be disabled in CI build."
}
}
@@ -561,11 +513,8 @@ function MSBuild-Core() {
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
}
else {
$cmdArgs += " /p:TreatWarningsAsErrors=false"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
}
foreach ($arg in $args) {
@@ -573,29 +522,29 @@ function MSBuild-Core() {
$cmdArgs += " `"$arg`""
}
}
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-PipelineTaskError -Message "Build failed."
Write-Host "Build failed." -ForegroundColor Red
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
}
ExitWithExitCode $exitCode
}
}
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
}
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
}
@@ -605,8 +554,6 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
return $null
}
. $PSScriptRoot\pipeline-logging-functions.ps1
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
@@ -622,17 +569,11 @@ Create-Directory $ToolsetDir
Create-Directory $TempDir
Create-Directory $LogDir
Write-PipelineSetVariable -Name 'Artifacts' -Value $ArtifactsDir
Write-PipelineSetVariable -Name 'Artifacts.Toolset' -Value $ToolsetDir
Write-PipelineSetVariable -Name 'Artifacts.Log' -Value $LogDir
Write-PipelineSetVariable -Name 'TEMP' -Value $TempDir
Write-PipelineSetVariable -Name 'TMP' -Value $TempDir
if ($ci) {
Write-Host "##vso[task.setvariable variable=Artifacts]$ArtifactsDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Toolset]$ToolsetDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Log]$LogDir"
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
if (!$disableConfigureToolsetImport) {
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
$env:TEMP = $TempDir
$env:TMP = $TempDir
}
Executable → Regular
+98 -87
View File
@@ -4,7 +4,6 @@
# CI mode - set to true on CI server for PR validation build or official build.
ci=${ci:-false}
disable_configure_toolset_import=${disable_configure_toolset_import:-}
# Set to true to use the pipelines logger which will enable Azure logging output.
# https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md
@@ -46,10 +45,6 @@ warn_as_error=${warn_as_error:-true}
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.sh
dotnetInstallScriptVersion=${dotnetInstallScriptVersion:-'v1'}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
if [[ "$ci" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
@@ -57,6 +52,78 @@ else
use_global_nuget_cache=${use_global_nuget_cache:-true}
fi
function EmitError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
message_type="error"
sourcepath=''
linenumber=''
columnnumber=''
error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-l)
linenumber=$2
shift
;;
-columnnumber|-col)
columnnumber=$2
shift
;;
-code|-c)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
message='##vso[task.logissue'
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
else
message="$message;sourcepath=${BASH_SOURCE[1]}"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
else
message="$message;linenumber=${BASH_LINENO[0]}"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
# Resolve any symlinks in the given path.
function ResolvePath {
local path=$1
@@ -82,7 +149,7 @@ function ReadGlobalVersion {
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Error: Cannot find \"$key\" in $global_json_file"
EmitError "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
@@ -143,7 +210,7 @@ function InitializeDotNetCli {
if [[ "$install" == true ]]; then
InstallDotNetSdk "$dotnet_root" "$dotnet_sdk_version"
else
Write-PipelineTelemetryError -category 'InitializeToolset' "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
EmitError "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
ExitWithExitCode 1
fi
fi
@@ -151,10 +218,14 @@ function InitializeDotNetCli {
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
Write-PipelinePrependPath -path "$dotnet_root"
export PATH="$dotnet_root:$PATH"
Write-PipelineSetVariable -name "DOTNET_MULTILEVEL_LOOKUP" -value "0"
Write-PipelineSetVariable -name "DOTNET_SKIP_FIRST_TIME_EXPERIENCE" -value "1"
if [[ $ci == true ]]; then
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
echo "##vso[task.prependpath]$dotnet_root"
echo "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
echo "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
fi
# return value
_InitializeDotNetCli="$dotnet_root"
@@ -192,35 +263,15 @@ function InstallDotNet {
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from public location (exit code '$exit_code')."
if [[ -n "$runtimeArg" ]]; then
local runtimeSourceFeed=''
if [[ -n "${6:-}" ]]; then
runtimeSourceFeed="--azure-feed $6"
fi
local runtimeSourceFeedKey=''
if [[ -n "${7:-}" ]]; then
decodedFeedKey=`echo $7 | base64 --decode`
runtimeSourceFeedKey="--feed-credential $decodedFeedKey"
fi
if [[ -n "$runtimeSourceFeed" || -n "$runtimeSourceFeedKey" ]]; then
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from custom location '$runtimeSourceFeed' (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
fi
fi
EmitError "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
local install_script_url="https://dot.net/$dotnetInstallScriptVersion/dotnet-install.sh"
local install_script_url="https://dot.net/v1/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
@@ -229,19 +280,12 @@ function GetDotNetInstallScript {
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
wget -q -O "$install_script" "$install_script_url" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script"
else
wget -q -O "$install_script" "$install_script_url"
fi
fi
# return value
_GetDotNetInstallScript="$install_script"
}
@@ -273,14 +317,11 @@ function GetNuGetPackageCachePath {
}
function InitializeNativeTools() {
if [[ -n "${DisableNativeToolsetInstalls:-}" ]]; then
return
fi
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
if [[ "$ci" == true ]]; then
nativeArgs="--installDirectory $tools_dir"
nativeArgs="-InstallDirectory $tools_dir"
fi
"$_script_dir/init-tools-native.sh" $nativeArgs
fi
@@ -308,7 +349,7 @@ function InitializeToolset {
fi
if [[ "$restore" != true ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Toolset version $toolset_version has not been restored."
EmitError "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 2
fi
@@ -325,7 +366,7 @@ function InitializeToolset {
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
Write-PipelineTelemetryError -category 'InitializeToolset' "Invalid toolset path: $toolset_build_proj"
EmitError "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
@@ -352,18 +393,6 @@ function MSBuild {
if [[ "$pipelines_log" == true ]]; then
InitializeBuildTool
InitializeToolset
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if [[ "$ci" == true ]]; then
"$_InitializeBuildTool" nuget locals http-cache -c
export NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS=20
export NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS=20
Write-PipelineSetVariable -name "NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS" -value "20"
Write-PipelineSetVariable -name "NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS" -value "20"
fi
local toolset_dir="${_InitializeToolset%/*}"
local logger_path="$toolset_dir/$_InitializeBuildToolFramework/Microsoft.DotNet.Arcade.Sdk.dll"
args=( "${args[@]}" "-logger:$logger_path" )
@@ -375,12 +404,12 @@ function MSBuild {
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
Write-PipelineTaskError "Binary log must be enabled in CI build."
EmitError "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
Write-PipelineTaskError "Node reuse must be disabled in CI build."
EmitError "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
@@ -394,7 +423,7 @@ function MSBuild-Core {
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
Write-PipelineTaskError "Build failed (exit code '$exit_code')."
EmitError "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
@@ -402,8 +431,6 @@ function MSBuild-Core {
ResolvePath "${BASH_SOURCE[0]}"
_script_dir=`dirname "$_ResolvePath"`
. "$_script_dir/pipeline-logging-functions.sh"
eng_root=`cd -P "$_script_dir/.." && pwd`
repo_root=`cd -P "$_script_dir/../.." && pwd`
artifacts_dir="$repo_root/artifacts"
@@ -430,23 +457,7 @@ mkdir -p "$toolset_dir"
mkdir -p "$temp_dir"
mkdir -p "$log_dir"
Write-PipelineSetVariable -name "Artifacts" -value "$artifacts_dir"
Write-PipelineSetVariable -name "Artifacts.Toolset" -value "$toolset_dir"
Write-PipelineSetVariable -name "Artifacts.Log" -value "$log_dir"
Write-PipelineSetVariable -name "Temp" -value "$temp_dir"
Write-PipelineSetVariable -name "TMP" -value "$temp_dir"
# Import custom tools configuration, if present in the repo.
if [[ -z "$disable_configure_toolset_import" ]]; then
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ $ci == true ]]; then
export TEMP="$temp_dir"
export TMP="$temp_dir"
fi
View File
File renamed without changes.
+51
View File
@@ -0,0 +1,51 @@
<Project DefaultTargets="PublishPackages">
<!-- TODO: move properties imported from here into a common props file -->
<!-- <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dir.props))\dir.props" /> -->
<!-- Use an explicit SDK import so that arcade uses the build
configuration computed in dir.common.props. -->
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
<Import Project="$(MSBuildThisFileDirectory)\..\.packages\microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<PropertyGroup>
<!-- Set the TargetFramework just to make the SDK happy -->
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<Target Name="PublishPackages">
<ItemGroup>
<PackagesToPublish Include="$(MSBuildThisFileDirectory)\..\artifacts\*.nupkg" />
</ItemGroup>
<ItemGroup>
<ItemsToPushToBlobFeed Include="@(PackagesToPublish)" />
</ItemGroup>
<Error Condition=" '$(AzureFeedUrl)' == '' " Text="AzureFeedUrl must be set" />
<Error Condition=" '$(AzureAccountKey)' == '' " Text="AzureAccountKey must be set" />
<Error Condition=" '$(BUILD_REPOSITORY_URI)' == '' " Text="BUILD_REPOSITORY_URI must be set" />
<Error Condition=" '$(BUILD_SOURCEBRANCH)' == '' " Text="BUILD_SOURCEBRANCH must be set" />
<Error Condition=" '$(BUILD_BUILDNUMBER)' == '' " Text="BUILD_BUILDNUMBER must be set" />
<Error Condition=" '$(BUILD_SOURCEVERSION)' == '' " Text="BUILD_SOURCEVERSION must be set" />
<PropertyGroup>
<AssetManifestFilePath>$(ArtifactsLogDir)AssetManifest\$(AssetManifest)</AssetManifestFilePath>
</PropertyGroup>
<Message Importance="Low" Text="BUILD_REPOSITORY_URI: $(BUILD_REPOSITORY_URI)" />
<PushToBlobFeed ExpectedFeedUrl="$(AzureFeedUrl)"
AccountKey="$(AzureAccountKey)"
ItemsToPush="@(ItemsToPushToBlobFeed)"
ManifestBuildData="Location=$(AzureFeedUrl)"
ManifestRepoUri="$(BUILD_REPOSITORY_URI)"
ManifestBranch="$(BUILD_SOURCEBRANCH)"
ManifestBuildId="$(BUILD_BUILDNUMBER)"
ManifestCommit="$(BUILD_SOURCEVERSION)"
AssetManifestPath="$(AssetManifestFilePath)" />
</Target>
</Project>
-1
Submodule external/bdwgc deleted from 6e8db07c8d.
+1 -1
+1 -1
+1 -1
+1 -1
Vendored Submodule
+1
Submodule external/llvm added at 29f36fd315.
Submodule external/llvm-project deleted from 0a3f49c40d.
+3 -3
View File
@@ -1,9 +1,9 @@
{
"tools": {
"dotnet": "5.0.100-alpha1-014854"
"dotnet": "3.0.100-preview5-011568"
},
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "5.0.0-beta.19567.2",
"Microsoft.DotNet.Helix.Sdk": "5.0.0-beta.19567.2"
"Microsoft.DotNet.Arcade.Sdk": "1.0.0-beta.19279.5",
"Microsoft.DotNet.Helix.Sdk": "2.0.0-beta.19279.5"
}
}
+4
View File
@@ -0,0 +1,4 @@
ltconfig crlf=input
install-sh crlf=input
/BCC_MAKEFILE -crlf
/digimars.mak -crlf
+25
View File
@@ -0,0 +1,25 @@
/aclocal.m4
/config.cache
/config.guess
/config.h
/config.h.in
/config.log
/config.status
/config.sub
/configure
/libtool
/ltmain.sh
/ltconfig
/Makefile
/Makefile.in
/stamp-h
/stamp-h.in
/*.lo
/.deps
/.libs
/*.la
/*.o
/autom4te.cache
/libgc-mono.pc
/libgc-mono-*.tar.gz
dolt*
+623
View File
@@ -0,0 +1,623 @@
/******************************************************************
AmigaOS-spesific routines for GC.
This file is normally included from os_dep.c
******************************************************************/
#if !defined(GC_AMIGA_DEF) && !defined(GC_AMIGA_SB) && !defined(GC_AMIGA_DS) && !defined(GC_AMIGA_AM)
# include "gc_priv.h"
# include <stdio.h>
# include <signal.h>
# define GC_AMIGA_DEF
# define GC_AMIGA_SB
# define GC_AMIGA_DS
# define GC_AMIGA_AM
#endif
#ifdef GC_AMIGA_DEF
# ifndef __GNUC__
# include <exec/exec.h>
# endif
# include <proto/exec.h>
# include <proto/dos.h>
# include <dos/dosextens.h>
# include <workbench/startup.h>
#endif
#ifdef GC_AMIGA_SB
/******************************************************************
Find the base of the stack.
******************************************************************/
ptr_t GC_get_stack_base()
{
struct Process *proc = (struct Process*)SysBase->ThisTask;
/* Reference: Amiga Guru Book Pages: 42,567,574 */
if (proc->pr_Task.tc_Node.ln_Type==NT_PROCESS
&& proc->pr_CLI != NULL) {
/* first ULONG is StackSize */
/*longPtr = proc->pr_ReturnAddr;
size = longPtr[0];*/
return (char *)proc->pr_ReturnAddr + sizeof(ULONG);
} else {
return (char *)proc->pr_Task.tc_SPUpper;
}
}
#if 0 /* old version */
ptr_t GC_get_stack_base()
{
extern struct WBStartup *_WBenchMsg;
extern long __base;
extern long __stack;
struct Task *task;
struct Process *proc;
struct CommandLineInterface *cli;
long size;
if ((task = FindTask(0)) == 0) {
GC_err_puts("Cannot find own task structure\n");
ABORT("task missing");
}
proc = (struct Process *)task;
cli = BADDR(proc->pr_CLI);
if (_WBenchMsg != 0 || cli == 0) {
size = (char *)task->tc_SPUpper - (char *)task->tc_SPLower;
} else {
size = cli->cli_DefaultStack * 4;
}
return (ptr_t)(__base + GC_max(size, __stack));
}
#endif
#endif
#ifdef GC_AMIGA_DS
/******************************************************************
Register data segments.
******************************************************************/
void GC_register_data_segments()
{
struct Process *proc;
struct CommandLineInterface *cli;
BPTR myseglist;
ULONG *data;
int num;
# ifdef __GNUC__
ULONG dataSegSize;
GC_bool found_segment = FALSE;
extern char __data_size[];
dataSegSize=__data_size+8;
/* Can`t find the Location of __data_size, because
it`s possible that is it, inside the segment. */
# endif
proc= (struct Process*)SysBase->ThisTask;
/* Reference: Amiga Guru Book Pages: 538ff,565,573
and XOper.asm */
if (proc->pr_Task.tc_Node.ln_Type==NT_PROCESS) {
if (proc->pr_CLI == NULL) {
myseglist = proc->pr_SegList;
} else {
/* ProcLoaded 'Loaded as a command: '*/
cli = BADDR(proc->pr_CLI);
myseglist = cli->cli_Module;
}
} else {
ABORT("Not a Process.");
}
if (myseglist == NULL) {
ABORT("Arrrgh.. can't find segments, aborting");
}
/* xoper hunks Shell Process */
num=0;
for (data = (ULONG *)BADDR(myseglist); data != NULL;
data = (ULONG *)BADDR(data[0])) {
if (((ULONG) GC_register_data_segments < (ULONG) &data[1]) ||
((ULONG) GC_register_data_segments > (ULONG) &data[1] + data[-1])) {
# ifdef __GNUC__
if (dataSegSize == data[-1]) {
found_segment = TRUE;
}
# endif
GC_add_roots_inner((char *)&data[1],
((char *)&data[1]) + data[-1], FALSE);
}
++num;
} /* for */
# ifdef __GNUC__
if (!found_segment) {
ABORT("Can`t find correct Segments.\nSolution: Use an newer version of ixemul.library");
}
# endif
}
#if 0 /* old version */
void GC_register_data_segments()
{
extern struct WBStartup *_WBenchMsg;
struct Process *proc;
struct CommandLineInterface *cli;
BPTR myseglist;
ULONG *data;
if ( _WBenchMsg != 0 ) {
if ((myseglist = _WBenchMsg->sm_Segment) == 0) {
GC_err_puts("No seglist from workbench\n");
return;
}
} else {
if ((proc = (struct Process *)FindTask(0)) == 0) {
GC_err_puts("Cannot find process structure\n");
return;
}
if ((cli = BADDR(proc->pr_CLI)) == 0) {
GC_err_puts("No CLI\n");
return;
}
if ((myseglist = cli->cli_Module) == 0) {
GC_err_puts("No seglist from CLI\n");
return;
}
}
for (data = (ULONG *)BADDR(myseglist); data != 0;
data = (ULONG *)BADDR(data[0])) {
# ifdef AMIGA_SKIP_SEG
if (((ULONG) GC_register_data_segments < (ULONG) &data[1]) ||
((ULONG) GC_register_data_segments > (ULONG) &data[1] + data[-1])) {
# else
{
# endif /* AMIGA_SKIP_SEG */
GC_add_roots_inner((char *)&data[1],
((char *)&data[1]) + data[-1], FALSE);
}
}
}
#endif /* old version */
#endif
#ifdef GC_AMIGA_AM
#ifndef GC_AMIGA_FASTALLOC
void *GC_amiga_allocwrapper(size_t size,void *(*AllocFunction)(size_t size2)){
return (*AllocFunction)(size);
}
void *(*GC_amiga_allocwrapper_do)(size_t size,void *(*AllocFunction)(size_t size2))
=GC_amiga_allocwrapper;
#else
void *GC_amiga_allocwrapper_firsttime(size_t size,void *(*AllocFunction)(size_t size2));
void *(*GC_amiga_allocwrapper_do)(size_t size,void *(*AllocFunction)(size_t size2))
=GC_amiga_allocwrapper_firsttime;
/******************************************************************
Amiga-spesific routines to obtain memory, and force GC to give
back fast-mem whenever possible.
These hacks makes gc-programs go many times faster when
the amiga is low on memory, and are therefore strictly necesarry.
-Kjetil S. Matheussen, 2000.
******************************************************************/
/* List-header for all allocated memory. */
struct GC_Amiga_AllocedMemoryHeader{
ULONG size;
struct GC_Amiga_AllocedMemoryHeader *next;
};
struct GC_Amiga_AllocedMemoryHeader *GC_AMIGAMEM=(struct GC_Amiga_AllocedMemoryHeader *)(int)~(NULL);
/* Type of memory. Once in the execution of a program, this might change to MEMF_ANY|MEMF_CLEAR */
ULONG GC_AMIGA_MEMF = MEMF_FAST | MEMF_CLEAR;
/* Prevents GC_amiga_get_mem from allocating memory if this one is TRUE. */
#ifndef GC_AMIGA_ONLYFAST
BOOL GC_amiga_dontalloc=FALSE;
#endif
#ifdef GC_AMIGA_PRINTSTATS
int succ=0,succ2=0;
int nsucc=0,nsucc2=0;
int nullretries=0;
int numcollects=0;
int chipa=0;
int allochip=0;
int allocfast=0;
int cur0=0;
int cur1=0;
int cur10=0;
int cur50=0;
int cur150=0;
int cur151=0;
int ncur0=0;
int ncur1=0;
int ncur10=0;
int ncur50=0;
int ncur150=0;
int ncur151=0;
#endif
/* Free everything at program-end. */
void GC_amiga_free_all_mem(void){
struct GC_Amiga_AllocedMemoryHeader *gc_am=(struct GC_Amiga_AllocedMemoryHeader *)(~(int)(GC_AMIGAMEM));
struct GC_Amiga_AllocedMemoryHeader *temp;
#ifdef GC_AMIGA_PRINTSTATS
printf("\n\n"
"%d bytes of chip-mem, and %d bytes of fast-mem where allocated from the OS.\n",
allochip,allocfast
);
printf(
"%d bytes of chip-mem were returned from the GC_AMIGA_FASTALLOC supported allocating functions.\n",
chipa
);
printf("\n");
printf("GC_gcollect was called %d times to avoid returning NULL or start allocating with the MEMF_ANY flag.\n",numcollects);
printf("%d of them was a success. (the others had to use allocation from the OS.)\n",nullretries);
printf("\n");
printf("Succeded forcing %d gc-allocations (%d bytes) of chip-mem to be fast-mem.\n",succ,succ2);
printf("Failed forcing %d gc-allocations (%d bytes) of chip-mem to be fast-mem.\n",nsucc,nsucc2);
printf("\n");
printf(
"Number of retries before succeding a chip->fast force:\n"
"0: %d, 1: %d, 2-9: %d, 10-49: %d, 50-149: %d, >150: %d\n",
cur0,cur1,cur10,cur50,cur150,cur151
);
printf(
"Number of retries before giving up a chip->fast force:\n"
"0: %d, 1: %d, 2-9: %d, 10-49: %d, 50-149: %d, >150: %d\n",
ncur0,ncur1,ncur10,ncur50,ncur150,ncur151
);
#endif
while(gc_am!=NULL){
temp=gc_am->next;
FreeMem(gc_am,gc_am->size);
gc_am=(struct GC_Amiga_AllocedMemoryHeader *)(~(int)(temp));
}
}
#ifndef GC_AMIGA_ONLYFAST
/* All memory with address lower than this one is chip-mem. */
char *chipmax;
/*
* Allways set to the last size of memory tried to be allocated.
* Needed to ensure allocation when the size is bigger than 100000.
*
*/
size_t latestsize;
#endif
/*
* The actual function that is called with the GET_MEM macro.
*
*/
void *GC_amiga_get_mem(size_t size){
struct GC_Amiga_AllocedMemoryHeader *gc_am;
#ifndef GC_AMIGA_ONLYFAST
if(GC_amiga_dontalloc==TRUE){
// printf("rejected, size: %d, latestsize: %d\n",size,latestsize);
return NULL;
}
// We really don't want to use chip-mem, but if we must, then as little as possible.
if(GC_AMIGA_MEMF==(MEMF_ANY|MEMF_CLEAR) && size>100000 && latestsize<50000) return NULL;
#endif
gc_am=AllocMem((ULONG)(size + sizeof(struct GC_Amiga_AllocedMemoryHeader)),GC_AMIGA_MEMF);
if(gc_am==NULL) return NULL;
gc_am->next=GC_AMIGAMEM;
gc_am->size=size + sizeof(struct GC_Amiga_AllocedMemoryHeader);
GC_AMIGAMEM=(struct GC_Amiga_AllocedMemoryHeader *)(~(int)(gc_am));
// printf("Allocated %d (%d) bytes at address: %x. Latest: %d\n",size,tot,gc_am,latestsize);
#ifdef GC_AMIGA_PRINTSTATS
if((char *)gc_am<chipmax){
allochip+=size;
}else{
allocfast+=size;
}
#endif
return gc_am+1;
}
#ifndef GC_AMIGA_ONLYFAST
/* Tries very hard to force GC to find fast-mem to return. Done recursively
* to hold the rejected memory-pointers reachable from the collector in an
* easy way.
*
*/
#ifdef GC_AMIGA_RETRY
void *GC_amiga_rec_alloc(size_t size,void *(*AllocFunction)(size_t size2),const int rec){
void *ret;
ret=(*AllocFunction)(size);
#ifdef GC_AMIGA_PRINTSTATS
if((char *)ret>chipmax || ret==NULL){
if(ret==NULL){
nsucc++;
nsucc2+=size;
if(rec==0) ncur0++;
if(rec==1) ncur1++;
if(rec>1 && rec<10) ncur10++;
if(rec>=10 && rec<50) ncur50++;
if(rec>=50 && rec<150) ncur150++;
if(rec>=150) ncur151++;
}else{
succ++;
succ2+=size;
if(rec==0) cur0++;
if(rec==1) cur1++;
if(rec>1 && rec<10) cur10++;
if(rec>=10 && rec<50) cur50++;
if(rec>=50 && rec<150) cur150++;
if(rec>=150) cur151++;
}
}
#endif
if (((char *)ret)<=chipmax && ret!=NULL && (rec<(size>500000?9:size/5000))){
ret=GC_amiga_rec_alloc(size,AllocFunction,rec+1);
// GC_free(ret2);
}
return ret;
}
#endif
/* The allocating-functions defined inside the amiga-blocks in gc.h is called
* via these functions.
*/
void *GC_amiga_allocwrapper_any(size_t size,void *(*AllocFunction)(size_t size2)){
void *ret,*ret2;
GC_amiga_dontalloc=TRUE; // Pretty tough thing to do, but its indeed necesarry.
latestsize=size;
ret=(*AllocFunction)(size);
if(((char *)ret) <= chipmax){
if(ret==NULL){
//Give GC access to allocate memory.
#ifdef GC_AMIGA_GC
if(!GC_dont_gc){
GC_gcollect();
#ifdef GC_AMIGA_PRINTSTATS
numcollects++;
#endif
ret=(*AllocFunction)(size);
}
#endif
if(ret==NULL){
GC_amiga_dontalloc=FALSE;
ret=(*AllocFunction)(size);
if(ret==NULL){
WARN("Out of Memory! Returning NIL!\n", 0);
}
}
#ifdef GC_AMIGA_PRINTSTATS
else{
nullretries++;
}
if(ret!=NULL && (char *)ret<=chipmax) chipa+=size;
#endif
}
#ifdef GC_AMIGA_RETRY
else{
/* We got chip-mem. Better try again and again and again etc., we might get fast-mem sooner or later... */
/* Using gctest to check the effectiviness of doing this, does seldom give a very good result. */
/* However, real programs doesn't normally rapidly allocate and deallocate. */
// printf("trying to force... %d bytes... ",size);
if(
AllocFunction!=GC_malloc_uncollectable
#ifdef ATOMIC_UNCOLLECTABLE
&& AllocFunction!=GC_malloc_atomic_uncollectable
#endif
){
ret2=GC_amiga_rec_alloc(size,AllocFunction,0);
}else{
ret2=(*AllocFunction)(size);
#ifdef GC_AMIGA_PRINTSTATS
if((char *)ret2<chipmax || ret2==NULL){
nsucc++;
nsucc2+=size;
ncur0++;
}else{
succ++;
succ2+=size;
cur0++;
}
#endif
}
if(((char *)ret2)>chipmax){
// printf("Succeeded.\n");
GC_free(ret);
ret=ret2;
}else{
GC_free(ret2);
// printf("But did not succeed.\n");
}
}
#endif
}
GC_amiga_dontalloc=FALSE;
return ret;
}
void (*GC_amiga_toany)(void)=NULL;
void GC_amiga_set_toany(void (*func)(void)){
GC_amiga_toany=func;
}
#endif // !GC_AMIGA_ONLYFAST
void *GC_amiga_allocwrapper_fast(size_t size,void *(*AllocFunction)(size_t size2)){
void *ret;
ret=(*AllocFunction)(size);
if(ret==NULL){
// Enable chip-mem allocation.
// printf("ret==NULL\n");
#ifdef GC_AMIGA_GC
if(!GC_dont_gc){
GC_gcollect();
#ifdef GC_AMIGA_PRINTSTATS
numcollects++;
#endif
ret=(*AllocFunction)(size);
}
#endif
if(ret==NULL){
#ifndef GC_AMIGA_ONLYFAST
GC_AMIGA_MEMF=MEMF_ANY | MEMF_CLEAR;
if(GC_amiga_toany!=NULL) (*GC_amiga_toany)();
GC_amiga_allocwrapper_do=GC_amiga_allocwrapper_any;
return GC_amiga_allocwrapper_any(size,AllocFunction);
#endif
}
#ifdef GC_AMIGA_PRINTSTATS
else{
nullretries++;
}
#endif
}
return ret;
}
void *GC_amiga_allocwrapper_firsttime(size_t size,void *(*AllocFunction)(size_t size2)){
atexit(&GC_amiga_free_all_mem);
chipmax=(char *)SysBase->MaxLocMem; // For people still having SysBase in chip-mem, this might speed up a bit.
GC_amiga_allocwrapper_do=GC_amiga_allocwrapper_fast;
return GC_amiga_allocwrapper_fast(size,AllocFunction);
}
#endif //GC_AMIGA_FASTALLOC
/*
* The wrapped realloc function.
*
*/
void *GC_amiga_realloc(void *old_object,size_t new_size_in_bytes){
#ifndef GC_AMIGA_FASTALLOC
return GC_realloc(old_object,new_size_in_bytes);
#else
void *ret;
latestsize=new_size_in_bytes;
ret=GC_realloc(old_object,new_size_in_bytes);
if(ret==NULL && GC_AMIGA_MEMF==(MEMF_FAST | MEMF_CLEAR)){
/* Out of fast-mem. */
#ifdef GC_AMIGA_GC
if(!GC_dont_gc){
GC_gcollect();
#ifdef GC_AMIGA_PRINTSTATS
numcollects++;
#endif
ret=GC_realloc(old_object,new_size_in_bytes);
}
#endif
if(ret==NULL){
#ifndef GC_AMIGA_ONLYFAST
GC_AMIGA_MEMF=MEMF_ANY | MEMF_CLEAR;
if(GC_amiga_toany!=NULL) (*GC_amiga_toany)();
GC_amiga_allocwrapper_do=GC_amiga_allocwrapper_any;
ret=GC_realloc(old_object,new_size_in_bytes);
#endif
}
#ifdef GC_AMIGA_PRINTSTATS
else{
nullretries++;
}
#endif
}
if(ret==NULL){
WARN("Out of Memory! Returning NIL!\n", 0);
}
#ifdef GC_AMIGA_PRINTSTATS
if(((char *)ret)<chipmax && ret!=NULL){
chipa+=new_size_in_bytes;
}
#endif
return ret;
#endif
}
#endif //GC_AMIGA_AM
+88
View File
@@ -0,0 +1,88 @@
# Makefile for Borland C++ 5.5 on NT
# If you have the Borland assembler, remove "-DUSE_GENERIC"
#
bc= c:\Borland\BCC55
bcbin= $(bc)\bin
bclib= $(bc)\lib
bcinclude= $(bc)\include
gcinclude1 = $(bc)\gc6.2\include
gcinclude2 = $(bc)\gc6.2\cord
cc= $(bcbin)\bcc32
rc= $(bcbin)\brc32
lib= $(bcbin)\tlib
link= $(bcbin)\ilink32
cflags= -O2 -R -v- -vi -H -H=gc.csm -I$(bcinclude);$(gcinclude1);$(gcinclude2) -L$(bclib) \
-w-pro -w-aus -w-par -w-ccc -w-rch -a4 -D__STDC__=0
#defines= -DSILENT
defines= -DSILENT -DALL_INTERIOR_POINTERS -DUSE_GENERIC -DNO_GETENV -DJAVA_FINALIZATION -DGC_OPERATOR_NEW_ARRAY
.c.obj:
$(cc) @&&|
$(cdebug) $(cflags) $(cvars) $(defines) -o$* -c $*.c
|
.cpp.obj:
$(cc) @&&|
$(cdebug) $(cflags) $(cvars) $(defines) -o$* -c $*.cpp
|
.rc.res:
$(rc) -i$(bcinclude) -r -fo$* $*.rc
XXXOBJS= XXXalloc.obj XXXreclaim.obj XXXallchblk.obj XXXmisc.obj \
XXXmach_dep.obj XXXos_dep.obj XXXmark_rts.obj XXXheaders.obj XXXmark.obj \
XXXobj_map.obj XXXblacklst.obj XXXfinalize.obj XXXnew_hblk.obj \
XXXdbg_mlc.obj XXXmalloc.obj XXXstubborn.obj XXXdyn_load.obj \
XXXtypd_mlc.obj XXXptr_chck.obj XXXgc_cpp.obj XXXmallocx.obj
OBJS= $(XXXOBJS:XXX=)
all: gctest.exe cord\de.exe test_cpp.exe
$(OBJS) test.obj: include\private\gc_priv.h include\private\gc_hdrs.h include\gc.h include\private\gcconfig.h MAKEFILE
gc.lib: $(OBJS)
del gc.lib
$(lib) $* @&&|
$(XXXOBJS:XXX=+)
|
gctest.exe: tests\test.obj gc.lib
$(cc) @&&|
$(cflags) -W -e$* tests\test.obj gc.lib
|
cord\de.obj cord\de_win.obj: include\cord.h include\private\cord_pos.h cord\de_win.h \
cord\de_cmds.h
cord\de.exe: cord\cordbscs.obj cord\cordxtra.obj cord\de.obj cord\de_win.obj \
cord\de_win.res gc.lib
$(cc) @&&|
$(cflags) -W -e$* cord\cordbscs.obj cord\cordxtra.obj \
cord\de.obj cord\de_win.obj gc.lib
|
$(rc) cord\de_win.res cord\de.exe
gc_cpp.obj: include\gc_cpp.h include\gc.h
gc_cpp.cpp: gc_cpp.cc
copy gc_cpp.cc gc_cpp.cpp
test_cpp.cpp: tests\test_cpp.cc
copy tests\test_cpp.cc test_cpp.cpp
test_cpp.exe: test_cpp.obj include\gc_cpp.h include\gc.h gc.lib
$(cc) @&&|
$(cflags) -W -e$* test_cpp.obj gc.lib
|
scratch:
-del *.obj *.res *.exe *.csm cord\*.obj cord\*.res cord\*.exe cord\*.csm
clean:
del gc.lib
del *.obj
del tests\test.obj
Loaded 100 of 2907 files, more files were not shown because too many files have changed in this diff. Show more