Compare commits

..
100 Commits
Author SHA1 Message Date
Marius UngureanuandMartin Baulig df99fccf69 Apply suggestions from code review
Co-Authored-By: Martin Baulig <[email protected]>
2019-05-31 12:08:46 +02:00
Marius Ungureanu 27554bf06b Plug leak of CustomAttribute initializer name
All the CustomAttribute initializer names (fields, property) except the last were leaked. Ensure all names are freed.

Fixes #14543
2019-05-31 12:08:46 +02:00
Vlad Brezae ddf393026a [marshal] Fix race between delegate marshaling and finalization
When passing a delegate to native code, we pass the delegate_trampoline which can be used to invoke the delegate from native code and we store this mapping (delegate_trampoline->delegate) in a hashtable so we can get a delegate for the delegate_trampoline, when marshalling the opposite way. If we are trying to marshal a ftnptr (for managed code) to a delegate and we don't find this mapping we can crash. This could happen if the finalization for some other delegate that uses the same ftnptr removes this entry from the table. We solve this by never freeing these entries. The first delegate-ftnptr pair will always be alive. This is easier, faster and potentially uses less memory than tracking in the hashtable every pair. There should be a small and limited set of methods marshalled to native.

Fixes https://github.com/mono/mono/issues/13231
2019-05-31 12:05:50 +02:00
Alexander Köplinger 4181954955 [bcl] Add nunit exclude categories for monotouch/monodroid profiles (#14695)
We didn't set all the categories that the products used in our profiles.

Now that the products can consume the exclude categories from mono we
need to unify the values.

Removed ValueAdd since it's not used anymore and combined MacNotWorking
into NotOnMac.

(cherry picked from commit f46191114b)
2019-05-29 17:00:41 +02:00
Alexander Köplinger 0ca2187eb3 [System] Fix MultiCastOptionTest on WatchOS
With https://github.com/mono/mono/commit/cf69b4725976e51416bfdff22f3e1834006af00a it started doing DNS lookups for localhost and we don't have sockets on the watch.

(cherry picked from commit 4fcbbd8fb7)
2019-05-29 16:45:47 +02:00
Alexander Köplinger 197461e034 [sdks] Only package android-sources on macOS
rsync might not be installed on other platforms and we only need android-sources in one sdks archive.

(cherry picked from commit edd6c013b0)
2019-05-29 00:30:56 +02:00
Alexander Köplinger 356964caf2 [sdks] Don't package android-sources on Windows
We don't need it there.

(cherry picked from commit a2f66ddca1)
2019-05-28 19:50:49 +02:00
Alexander Köplinger 271b64409d [sdks] Add linker sources to android sdks archive (#14686)
(cherry picked from commit c96878fb16)
2019-05-28 18:56:11 +02:00
Alexander Köplinger 1e2119b6df [bcl] Write newline-separated nunit/xunit excludes file for sdks archive (#14681)
During XI integration we noticed that just using the verbatim commandline
-exclude or -notrait is not ideal since XI uses the nunit/xunit APIs to
pass that information and not the command line parser.

Instead we now write the exclude categories/traits as a newline-separated file.

(cherry picked from commit 07c58ee2ed)
2019-05-28 18:51:43 +02:00
monojenkins da671d0b3c [2019-04] [mscorlib] Ensure monotouch[_runtime] API are identical (#14680)
* [mscorlib] Ensure monotouch[_runtime] API are identical

Ensure the API surface for `mscorlib.Core.dll` is identical between the
`monotouch` and `monotouch_runtime` (used for REPL and the interpreter).

This ensure code compiled against one will be fine when building against
the other (e.g. avoiding linker errors).

In practice this means:

1. Expose the API needed for `System.Reflection.Emit` in XI

	- throwing `PlatformNotSupportedException` **by default**; or
	- calling the mono runtime when the interpreter is enabled;

2. Do not expose additional, not supported, API (e.g. COM related)

	- `monotouch_runtime` configuration was built without `FULL_AOT_RUNTIME`
	  and that brings some other stuff that is not AOT related

Follow up to #14606

Existing diff

Removed methods:

```csharp
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access);
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, System.Collections.Generic.IEnumerable<Reflection.Emit.CustomAttributeBuilder> assemblyAttributes);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.Policy.Evidence evidence);
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir);
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, System.Collections.Generic.IEnumerable<Reflection.Emit.CustomAttributeBuilder> assemblyAttributes, Security.SecurityContextSource securityContextSource);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, bool isSynchronized, System.Collections.Generic.IEnumerable<Reflection.Emit.CustomAttributeBuilder> assemblyAttributes);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions, bool isSynchronized);

[Obsolete ("Declarative security for assembly level is no longer enforced")]
public Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions, bool isSynchronized, System.Collections.Generic.IEnumerable<Reflection.Emit.CustomAttributeBuilder> assemblyAttributes);
```

Removed methods:

```csharp
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.Policy.Evidence evidence);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions);
public virtual Reflection.Emit.AssemblyBuilder DefineDynamicAssembly (Reflection.AssemblyName name, Reflection.Emit.AssemblyBuilderAccess access, string dir, Security.Policy.Evidence evidence, Security.PermissionSet requiredPermissions, Security.PermissionSet optionalPermissions, Security.PermissionSet refusedPermissions, bool isSynchronized);
```

Added constructor:

```csharp
public AssemblyBuilder ();
```

Removed properties:

```csharp
public override string CodeBase { get; }
public override System.Reflection.MethodInfo EntryPoint { get; }
public override string EscapedCodeBase { get; }
public override System.Security.Policy.Evidence Evidence { get; }
public override string FullName { get; }
public override bool GlobalAssemblyCache { get; }
public override string ImageRuntimeVersion { get; }
public override bool IsDynamic { get; }
public override string Location { get; }
public override System.Reflection.Module ManifestModule { get; }
public override bool ReflectionOnly { get; }
```

Removed methods:

```csharp
public void AddResourceFile (string name, string fileName);
public void AddResourceFile (string name, string fileName, System.Reflection.ResourceAttributes attribute);
public ModuleBuilder DefineDynamicModule (string name, bool emitSymbolInfo);
public ModuleBuilder DefineDynamicModule (string name, string fileName);
public ModuleBuilder DefineDynamicModule (string name, string fileName, bool emitSymbolInfo);
public System.Resources.IResourceWriter DefineResource (string name, string description, string fileName);
public System.Resources.IResourceWriter DefineResource (string name, string description, string fileName, System.Reflection.ResourceAttributes attribute);
public void DefineUnmanagedResource (byte[] resource);
public void DefineUnmanagedResource (string resourceFileName);
public void DefineVersionInfoResource ();
public void DefineVersionInfoResource (string product, string productVersion, string company, string copyright, string trademark);
public override bool Equals (object obj);
public override object[] GetCustomAttributes (bool inherit);
public override object[] GetCustomAttributes (System.Type attributeType, bool inherit);
public override System.Type[] GetExportedTypes ();
public override System.IO.FileStream GetFile (string name);
public override System.IO.FileStream[] GetFiles (bool getResourceModules);
public override int GetHashCode ();
public override System.Reflection.Module[] GetLoadedModules (bool getResourceModules);
public override System.Reflection.ManifestResourceInfo GetManifestResourceInfo (string resourceName);
public override string[] GetManifestResourceNames ();
public override System.IO.Stream GetManifestResourceStream (string name);
public override System.IO.Stream GetManifestResourceStream (System.Type type, string name);
public override System.Reflection.Module GetModule (string name);
public override System.Reflection.Module[] GetModules (bool getResourceModules);
public override System.Reflection.AssemblyName GetName (bool copiedName);
public override System.Reflection.AssemblyName[] GetReferencedAssemblies ();
public override System.Reflection.Assembly GetSatelliteAssembly (System.Globalization.CultureInfo culture);
public override System.Reflection.Assembly GetSatelliteAssembly (System.Globalization.CultureInfo culture, System.Version version);
public override System.Type GetType (string name, bool throwOnError, bool ignoreCase);
public override bool IsDefined (System.Type attributeType, bool inherit);
public void Save (string assemblyFileName);
public void Save (string assemblyFileName, System.Reflection.PortableExecutableKinds portableExecutableKind, System.Reflection.ImageFileMachine imageFileMachine);
public void SetEntryPoint (System.Reflection.MethodInfo entryMethod, PEFileKinds fileKind);
public override string ToString ();
```

Added constructor:

```csharp
public ConstructorBuilder ();
```

Removed properties:

```csharp
public override System.Reflection.CallingConventions CallingConvention { get; }
public override System.Reflection.Module Module { get; }

[Obsolete]
public System.Type ReturnType { get; }
```

Removed methods:

```csharp
public void AddDeclarativeSecurity (System.Security.Permissions.SecurityAction action, System.Security.PermissionSet pset);
public void SetSymCustomAttribute (string name, byte[] data);
public override string ToString ();
```

Removed property:

```csharp
public override System.Reflection.Module Module { get; }
```

Added property:

```csharp
public override System.Reflection.MethodImplAttributes MethodImplementationFlags { get; }
```

Removed methods:

```csharp
public override System.Delegate CreateDelegate (System.Type delegateType);
public override System.Delegate CreateDelegate (System.Type delegateType, object target);
public override string ToString ();
```

Added properties:

```csharp
public override bool IsByRefLike { get; }
public override bool IsSZArray { get; }
public override bool IsVariableBoundArray { get; }
```

Removed method:

```csharp
public override bool IsAssignableFrom (System.Reflection.TypeInfo typeInfo);
```

Added constructor:

```csharp
public EventBuilder ();
```

Removed properties:

```csharp
public override int MetadataToken { get; }
public override System.Reflection.Module Module { get; }
```

Removed method:

```csharp
[Obsolete ("An alternate API is available: Emit the MarshalAs custom attribute instead.")]
public void SetMarshal (UnmanagedMarshal unmanagedMarshal);
```

Added properties:

```csharp
public override bool IsByRefLike { get; }
public override bool IsConstructedGenericType { get; }
public override bool IsSZArray { get; }
public override bool IsTypeDefinition { get; }
public override bool IsVariableBoundArray { get; }
```

Removed methods:

```csharp
public override System.Type[] GetGenericParameterConstraints ();
public override bool IsAssignableFrom (System.Reflection.TypeInfo typeInfo);
public override bool IsInstanceOfType (object o);
```

Modified properties:

```diff
-public virtual int ILOffset { get; }
+public int ILOffset { get; }
```

Removed method:

```csharp
public virtual void MarkSequencePoint (System.Diagnostics.SymbolStore.ISymbolDocumentWriter document, int startLine, int startColumn, int endLine, int endColumn);
```

Added property:

```csharp
public override bool IsConstructedGenericMethod { get; }
```

Removed methods:

```csharp
public void AddDeclarativeSecurity (System.Security.Permissions.SecurityAction action, System.Security.PermissionSet pset);

[Obsolete ("An alternate API is available: Emit the MarshalAs custom attribute instead.")]
public void SetMarshal (UnmanagedMarshal unmanagedMarshal);
public void SetSymCustomAttribute (string name, byte[] data);
```

Removed properties:

```csharp
public override int MetadataToken { get; }
public override System.Guid ModuleVersionId { get; }
public override string ScopeName { get; }
```

Removed methods:

```csharp
public System.Diagnostics.SymbolStore.ISymbolDocumentWriter DefineDocument (string url, System.Guid language, System.Guid languageVendor, System.Guid documentType);
public void DefineManifestResource (string name, System.IO.Stream stream, System.Reflection.ResourceAttributes attribute);
public System.Resources.IResourceWriter DefineResource (string name, string description);
public System.Resources.IResourceWriter DefineResource (string name, string description, System.Reflection.ResourceAttributes attribute);
public void DefineUnmanagedResource (byte[] resource);
public void DefineUnmanagedResource (string resourceFileName);
public override object[] GetCustomAttributes (bool inherit);
public override object[] GetCustomAttributes (System.Type attributeType, bool inherit);
public override System.Reflection.FieldInfo GetField (string name, System.Reflection.BindingFlags bindingAttr);
public override System.Reflection.FieldInfo[] GetFields (System.Reflection.BindingFlags bindingFlags);
protected override System.Reflection.MethodInfo GetMethodImpl (string name, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, System.Reflection.CallingConventions callConvention, System.Type[] types, System.Reflection.ParameterModifier[] modifiers);
public override System.Reflection.MethodInfo[] GetMethods (System.Reflection.BindingFlags bindingFlags);
public System.Diagnostics.SymbolStore.ISymbolWriter GetSymWriter ();
public override System.Type GetType (string className);
public override System.Type GetType (string className, bool ignoreCase);
public override System.Type GetType (string className, bool throwOnError, bool ignoreCase);
public override System.Type[] GetTypes ();
public override bool IsDefined (System.Type attributeType, bool inherit);
public override bool IsResource ();
public override System.Reflection.FieldInfo ResolveField (int metadataToken, System.Type[] genericTypeArguments, System.Type[] genericMethodArguments);
public override System.Reflection.MemberInfo ResolveMember (int metadataToken, System.Type[] genericTypeArguments, System.Type[] genericMethodArguments);
public override System.Reflection.MethodBase ResolveMethod (int metadataToken, System.Type[] genericTypeArguments, System.Type[] genericMethodArguments);
public override byte[] ResolveSignature (int metadataToken);
public override string ResolveString (int metadataToken);
public override System.Type ResolveType (int metadataToken, System.Type[] genericTypeArguments, System.Type[] genericMethodArguments);
public void SetSymCustomAttribute (string name, byte[] data);
```

Modified properties:

```diff
-public virtual int Attributes { get; }
+public int Attributes { get; }
-public virtual string Name { get; }
+public string Name { get; }
-public virtual int Position { get; }
+public int Position { get; }
```

Removed method:

```csharp
[Obsolete ("An alternate API is available: Emit the MarshalAs custom attribute instead.")]
public virtual void SetMarshal (UnmanagedMarshal unmanagedMarshal);
```

Removed interface:

```csharp
System.Runtime.InteropServices._SignatureHelper
```

Removed methods:

```csharp
public override bool Equals (object obj);
public override int GetHashCode ();
public override string ToString ();
```

Removed property:

```csharp
public override bool ContainsGenericParameters { get; }
```

Added properties:

```csharp
public override bool IsByRefLike { get; }
public override bool IsSZArray { get; }
public override bool IsSecurityCritical { get; }
public override bool IsSecuritySafeCritical { get; }
public override bool IsSecurityTransparent { get; }
public override bool IsVariableBoundArray { get; }
```

Removed methods:

```csharp
public void AddDeclarativeSecurity (System.Security.Permissions.SecurityAction action, System.Security.PermissionSet pset);
public override bool IsAssignableFrom (System.Reflection.TypeInfo typeInfo);
protected override bool IsValueTypeImpl ();
```

Removed methods:

```csharp
public static object GetActiveObject (string progID);
public static IntPtr GetComInterfaceForObjectInContext (object o, System.Type t);
public static int GetComSlotForMethodInfo (System.Reflection.MemberInfo m);
public static int GetEndComSlot (System.Type t);
public static IntPtr GetExceptionPointers ();
public static IntPtr GetIDispatchForObjectInContext (object o);
public static IntPtr GetITypeInfoForType (System.Type t);
public static IntPtr GetIUnknownForObjectInContext (object o);

[Obsolete ("This method has been deprecated")]
public static IntPtr GetManagedThunkForUnmanagedMethodPtr (IntPtr pfnMethodToWrap, IntPtr pbSignature, int cbSignature);
public static System.Reflection.MemberInfo GetMethodInfoForComSlot (System.Type t, int slot, ref ComMemberType memberType);

[Obsolete ("This method has been deprecated")]
public static System.Threading.Thread GetThreadFromFiberCookie (int cookie);
public static System.Type GetTypeForITypeInfo (IntPtr piTypeInfo);

[Obsolete]
public static string GetTypeInfoName (UCOMITypeInfo pTI);
public static System.Guid GetTypeLibGuid (ComTypes.ITypeLib typelib);

[Obsolete]
public static System.Guid GetTypeLibGuid (UCOMITypeLib pTLB);
public static System.Guid GetTypeLibGuidForAssembly (System.Reflection.Assembly asm);
public static int GetTypeLibLcid (ComTypes.ITypeLib typelib);

[Obsolete]
public static int GetTypeLibLcid (UCOMITypeLib pTLB);
public static string GetTypeLibName (ComTypes.ITypeLib typelib);

[Obsolete]
public static string GetTypeLibName (UCOMITypeLib pTLB);
public static void GetTypeLibVersionForAssembly (System.Reflection.Assembly inputAssembly, out int majorVersion, out int minorVersion);

[Obsolete ("This method has been deprecated")]
public static IntPtr GetUnmanagedThunkForManagedMethodPtr (IntPtr pfnMethodToWrap, IntPtr pbSignature, int cbSignature);
public static bool IsTypeVisibleFromCom (System.Type t);
public static int NumParamBytes (System.Reflection.MethodInfo m);

[Obsolete]
public static void ReleaseThreadCache ();
```

* [csproj] Update project files

* Bump API snapshot submodule
2019-05-28 14:48:24 +02:00
Alexander Köplinger af47bd50b2 [sdks] Explicitly set version of the mac/ios/watchos/tvos platform SDKs (#14663)
Otherwise the Xcode linker tries to guess the version from the path,
and since we use Xcode101.app it'd use sdk=101 as the version which is wrong.

By explicitly specifying e.g. `$(XCODE_DIR)/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`
as the sysroot it correctly guesses sdk=12.1 instead.

See also https://github.com/xamarin/xamarin-macios/commit/834c0a9fd7062c0c25011721fd7d524b8a60b41e

(cherry picked from commit 7ca8e89ce7)
2019-05-28 01:39:17 +02:00
monojenkins 4766e2227a [2019-04] [System] Fix monotouch_runtime repl assemblies (#14638)
They were missing some exclude files that caused unnecessary p/invokes to be retained.

Fixes https://github.com/mono/mono/issues/13526
2019-05-27 14:01:57 +02:00
monojenkins 02d008c624 [2019-04] [System.Core] Ensure monotouch[_runtime] API are identical (#14617)
* [System.Core] Ensure monotouch[_runtime] API are identical

Ensure the API surface for `System.Core.dll` is identical between the
`monotouch` and `monotouch_runtime` (used for REPL and the interpreter).

This ensure code compiled against one will be fine when building against
the other (e.g. avoiding linker errors).

Removed methods:

```csharp
public void CompileToMethod (System.Reflection.Emit.MethodBuilder method);
public void CompileToMethod (System.Reflection.Emit.MethodBuilder method, System.Runtime.CompilerServices.DebugInfoGenerator debugInfoGenerator);
```

Removed methods:

```csharp
[Obsolete ("do not use this method")]
public static IRuntimeVariables MergeRuntimeVariables (IRuntimeVariables first, IRuntimeVariables second, int[] indexes);

[Obsolete ("do not use this method")]
public static System.Linq.Expressions.Expression Quote (System.Linq.Expressions.Expression expression, object hoistedLocals, object[] locals);
```

* [csproj] Update project files

* Bump API snapshot submodule
2019-05-24 15:39:50 +02:00
monojenkins 0c0df14755 [2019-04] Throw ObjectDisposedException instead of crashing when Cairo.Context handle == IntPtr.Zero (#14626)
* Throw ObjectDisposedException instead of crashing when Cairo.Context.handle == IntPtr.Zero

We are seeing instances where VS for Mac is crashing with a SIGSEGV when
calling into certain Cairo.Context functions.  Looking at the code, it's clear
that it's possible to have Context.handle == IntPtr.Zero after disposing the
object.

From research, it looks like passing a nullptr to raw cairo functions can
result in a SIGSEGV:
    https://bugs.launchpad.net/ubuntu/+source/compiz/+bug/93807

This change adds check to see if Conext.handle == IntPtr.Zero in Context
painting, font, and path methods.  This will turn uncatchable SIGSEGVs
into ObjectDisposedException.

* Update Context.cs

* Ensure SetTarget results in a valid handle
2019-05-24 15:20:08 +02:00
Alexander Köplinger 9a3a2665f0 [sdks] Fix min OSX version of runtime files in Mac SDK (#14615)
It was using an unescaped makefile variable in the template, i.e. ended up empty which meant the build just took the host version as the min version.

Fixed the variable and also moved it to CFLAGS/CXXFLAGS instead of CPPFLAGS to be consistent with ios.mk

(cherry picked from commit 82a9931248)
2019-05-24 14:11:21 +02:00
Alexander Köplinger c01bc7bca9 [tests] Add xunit.execution.dotnet.dll to tests directory (#14595)
So it gets included in the sdks archive

(cherry picked from commit f5f3c6aabe)
2019-05-23 20:18:42 +02:00
Bernhard Urban 9d909ca613 [arm] one more attempt to fix slotsize issue on llvmonly (#14444) (#14534)
[2019-04] [arm] one more attempt to fix slotsize issue on llvmonly (#14444)

Backport of https://github.com/mono/mono/pull/14444
2019-05-23 10:00:37 -04:00
Marek Habersack 26b9087b44 [Android] Fix runtime loading of DSOs for 64-bit processes
Context: https://github.com/xamarin/xamarin-android/issues/2780

On Android the executable for the application is going to be
`/system/bin/app_process{32,64}` depending on the application's architecture.
However, libraries for the different architectures live in different
subdirectories of `/system`: `lib` for 32-bit apps and `lib64` for 64-bit ones.
Thus appending `/lib` below will fail to load the DSO for a 64-bit app, even if
it exists there, because it will have a different architecture. The quickest fix
is to use `lib64` explicitly for 64-bit Android apps.
2019-05-23 10:46:30 +02:00
Alexander Köplinger a3d0f993c2 [tests] Add Xunit.NetCore.Extensions.dll to tests directory (#14571)
So it gets included in the sdks archive

(cherry picked from commit 2c76de1ee1)
2019-05-22 19:48:58 +02:00
Alexander Köplinger 92e73f22ea [2019-04] Bump gtk-sharp in bockbuild
Brings in mono/gtk-sharp#273.

Cherry-pick of https://github.com/mono/mono/pull/14537
2019-05-21 11:39:07 +02:00
monojenkins 4caae35744 [interp] disable inlining for built-in types (#14530)
[2019-04] [interp] disable inlining for built-in types

Fixes https://github.com/mono/mono/issues/14246

Backport of #14501.

/cc @lewurm
2019-05-20 16:16:11 -04:00
Alexander Köplinger d4e20f02e8 [sdks] Lipo arm64_32 binaries for watchOS
(cherry picked from commit ca5e8f207b)
2019-05-12 17:48:09 -07:00
monojenkins f506b6556b [2019-04] [WinForms] fix EditingCellFormattedValue getter and setter for bool value (#14436)
* [WinForms] If DataGridViewCheckBoxCell not in threeState mode EditingCellFormattedValue must return true or false, instead of value of CheckState type

* [WinForms] DataGrid: fix type check in EditingCellFormattedValue setter. When set bool value to EditingCellFormattedValue there will be exception. Type equality must be done through IsAssignableFrom not direct

* [WinForms] DataGrid: Add test for EditingCellFormattedValue getter and setter.
2019-05-12 05:39:39 -07:00
monojenkins cfd117ef41 [2019-04] Revert "[System] Add FSEvent FileSystemWatcher to monotouch BCL profile" (#14447)
* Revert "[System] Add FSEvent FileSystemWatcher to 'monotouch' BCL profile"

This reverts commit 46fc5352f3dfe0027e684a0f61e71130fccd559f.

While iOS FileSystemWatcher works, we now find that it is used unofficially-available  symbols and cannot publish apps using those.

Fixes https://github.com/mono/mono/issues/14290

* [csproj] Update project files
2019-05-12 05:21:40 -07:00
monojenkins 719fd63c43 [2019-04] [runtime] Use MAP_JIT automatically when running under a hardened runtime on osx. (#14414)
* [runtime] Use MAP_JIT automatically when running under a hardened runtime on osx.

* Fix memory ordering.
2019-05-12 05:19:22 -07:00
monojenkins 7bcbc7c2bf [2019-04] [corlib] Implement AssemblyBuilder.get_ReflectionOnly() (#14457)
* [corlib] Implement AssemblyBuilder.get_ReflectionOnly()

Fixes #13028

* Bump API snapshot submodule
2019-05-12 05:11:27 -07:00
monojenkins 3ac7efc336 [arm64] set MONO_ARCH_HAVE_UNWIND_BACKTRACE for watch4 (#14453)
[2019-04] [arm64] set MONO_ARCH_HAVE_UNWIND_BACKTRACE for watch4

Fixes a couple of mscorlib test failures, failing with
> Stack walks are not supported on this platform


Backport of #14440.

/cc @lewurm
2019-05-11 09:15:44 -04:00
monojenkins 482111976a [metadata] use handle stack in mono_runtime_object_init_handle (#14446)
[2019-04] [metadata] use handle stack in mono_runtime_object_init_handle

Fixes this crash on watchOS with llvmonly:
https://gist.github.com/lewurm/921b77a238c6fd60757a5ee9f54279d1

The hint here was that the crash happened in `mono_handle_stack_scan ()` and inspecting the handle stack there. After enabling `MONO_HANDLE_TRACK_OWNER` and `MONO_HANDLE_TRACK_SP` in `handle.h`, I got this:
https://gist.github.com/lewurm/71f936a126879ef2b96d00399e6b2bd9


@lambdageek suggested this fix after showing this trace to him. As far as I understand we do _not_ know the root cause yet. The handle leak happens in this trace:
https://gist.github.com/lewurm/b76fbc391f22d897bfd532ac1b67e2aa
`mono_runtime_object_init_checked ()` already should take care of the handle stack 😕


Contributes to #13006

Backport of #14399.

/cc @akoeplinger @lewurm
2019-05-10 18:06:04 -04:00
Egor Bogatov 7a8db67341 Update CultureInfoTest.cs 2019-05-10 13:54:54 +02:00
Egor Bogatov dce3386dbb Fix TextView 2019-05-10 13:54:54 +02:00
lateralusX ad01218417 Mutex memory leak on sgen GC (nursery, major).
During GC we allocate a gray queue protected by a mutex, sgen_gray_object_queue_init.
This mutex is never destroyed when clearing up the object containing it,
sgen_gray_object_queue_dispose, so it will leak. Overtime this could cause
OOM on some systems depending on how memory related to system primitives are
handled and partitioned.
2019-05-10 11:08:31 +02:00
Bernhard Urban b8a6096bb1 [2019-04] Do not allocate handles in ReleaseMutex. (#14429)
[2019-04] Do not allocate handles in ReleaseMutex.

Backport of https://github.com/mono/mono/pull/14403
2019-05-10 04:32:27 -04:00
monojenkins c35f4c3e5f Usually do not create a handle in mono_runtime_invoke_handle. Use mono_runtime_invoke_handle_void instead. (#14422)
[2019-04] Usually do not create a handle in mono_runtime_invoke_handle.

Use mono_runtime_invoke_handle_void instead.


Backport of #14404.

/cc @lewurm @jaykrell
2019-05-09 20:15:36 -04:00
Ankit Jain 08d062aca7 Bump msbuild to track mono-2019-04 (#14287) 2019-05-09 12:38:50 -04:00
monojenkins 82601ba547 [llvmonly] make local copy of trace_ips pointer from exception object (#14410)
[2019-04] [llvmonly] make local copy of trace_ips pointer from exception object

Fixes a race in llvmonly (watchOS):
```
Process 397 stopped
* thread %10, name = 'tid_8e2f', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
    frame %0: 0x0177f690 mscorlibtests`mono_llvm_load_exception at mini-exceptions.c:3886:23
   3883                 size_t upper = mono_array_length_internal (mono_ex->trace_ips);
   3884
   3885                 for (int i = 0; i < upper; i += TRACE_IP_ENTRY_SIZE) {
-> 3886                         gpointer curr_ip = mono_array_get_internal (mono_ex->trace_ips, gpointer, i);
   3887                         for (int j = 0; j < TRACE_IP_ENTRY_SIZE; ++j) {
   3888                                 gpointer p = mono_array_get_internal (mono_ex->trace_ips, gpointer, i + j);
   3889                                 trace_ips = g_list_append (trace_ips, p);
Target 0: (mscorlibtests) stopped.

(lldb) mbt
* thread %10
  * frame %0: 0x0177f690 mscorlibtests`mono_llvm_load_exception at mini-exceptions.c:3886:23
    frame %1: 0x00e23b7a mscorlibtests`mscorlibtests1_MonoTests_System_LazyTest__c__DisplayClass16_0__ConcurrentInitializationb__1 + 254
    frame %2: 0x0087af98 mscorlibtests`mscorlib_wrapper_delegate_invoke__Module_invoke_void + 610
    frame %3: 0x00368856 mscorlibtests`mscorlib_System_Threading_ThreadHelper_ThreadStart_Context_object + 448
  [...]
(lldb) p/x mono_ex
(MonoException *) $2 = 0x01d9e968
(lldb) p/x mono_ex->trace_ips
(MonoArray *) $3 = 0x00000000
(lldb) thread list
Process 397 stopped
  thread %1: tid = 0x38772, 0x018156aa mscorlibtests`jit_info_table_copy_and_split_chunk(table=0x17635a00, chunk=0x17b767f0) at jit-info.c:483:28, name = 'tid_303', queue = 'com.apple.main-thread'
  thread %3: tid = 0x387ad, 0x1c4f9764 libsystem_kernel.dylib`__psynch_cvwait + 24, name = 'SGen worker'
  thread %4: tid = 0x387b0, 0x1c4f1224 libsystem_kernel.dylib`semaphore_wait_trap + 8, name = 'Finalizer'
  thread %5: tid = 0x387b5, 0x1c4fa6cc libsystem_kernel.dylib`__workq_kernreturn + 8
  thread %6: tid = 0x387b6, 0x1c4f11d4 libsystem_kernel.dylib`mach_msg_trap + 20, name = 'com.apple.uikit.eventfetch-thread'
* thread %10: tid = 0x389a6, 0x0177f690 mscorlibtests`mono_llvm_load_exception at mini-exceptions.c:3886:23, name = 'tid_8e2f', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
  thread %11: tid = 0x38985, 0x1c4f9764 libsystem_kernel.dylib`__psynch_cvwait + 24, name = 'tid_8113'
  thread %12: tid = 0x38986, 0x1c4f123c libsystem_kernel.dylib`semaphore_timedwait_trap + 8, name = 'Thread Pool Worker'
  thread %13: tid = 0x38987, 0x1c4f123c libsystem_kernel.dylib`semaphore_timedwait_trap + 8, name = 'Thread Pool Worker'
  thread %14: tid = 0x389a3, 0x1c4f123c libsystem_kernel.dylib`semaphore_timedwait_trap + 8, name = 'Thread Pool Worker'
  thread %15: tid = 0x389a4, 0x1c4f123c libsystem_kernel.dylib`semaphore_timedwait_trap + 8, name = 'Thread Pool Worker'
  thread %16: tid = 0x389a5, 0x1c4f123c libsystem_kernel.dylib`semaphore_timedwait_trap + 8, name = 'Thread Pool Worker'
(lldb) thread select 1
* thread %1, name = 'tid_303', queue = 'com.apple.main-thread'
    frame %0: 0x018156aa mscorlibtests`jit_info_table_copy_and_split_chunk(table=0x17635a00, chunk=0x17b767f0) at jit-info.c:483:28
   480                          jit_info_table_split_chunk (chunk, &new_table->chunks [j], &new_table->chunks [j + 1]);
   481                          j += 2;
   482                  } else {
-> 483                          new_table->chunks [j] = table->chunks [i];
   484                          ++new_table->chunks [j]->refcount;
   485                          ++j;
   486                  }
(lldb) mbt
* thread %1
  * frame %0: 0x018156aa mscorlibtests`jit_info_table_copy_and_split_chunk(table=0x17635a00, chunk=0x17b767f0) at jit-info.c:483:28
    frame %1: 0x01815304 mscorlibtests`jit_info_table_chunk_overflow(table=0x17635a00, chunk=0x17b767f0) at jit-info.c:579:10
    frame %2: 0x018146ee mscorlibtests`jit_info_table_add(domain=0x16624cc0, table_ptr=0x16624d9c, ji=0x176f4428) at jit-info.c:614:33
    frame %3: 0x01814662 mscorlibtests`mono_jit_info_table_add(domain=0x16624cc0, ji=0x176f4428) at jit-info.c:674:2
    frame %4: 0x0175f5da mscorlibtests`mono_aot_find_jit_info(domain=0x16624cc0, image=0x16c07200, addr=0x00271ff1) at aot-runtime.c:3710:3
    frame %5: 0x0181443e mscorlibtests`mono_jit_info_table_find_internal(domain=0x16624cc0, addr=0x00271ff1, try_aot=1, allow_trampolines=0) at jit-info.c:306:9
    frame %6: 0x018145e0 mscorlibtests`mono_jit_info_table_find(domain=0x16624cc0, addr=0x00271ff1) at jit-info.c:335:9
    frame %7: 0x01780212 mscorlibtests`build_stack_trace(frame_ctx=0x01c2fb18, state=0x01c2ffb8) at mini-exceptions.c:392:31
    frame %8: 0x1c598aa2 libunwind.dylib`_Unwind_Backtrace + 232
    frame %9: 0x0177f4fc mscorlibtests`throw_exception(ex=0x01d9e968, rethrow=0) at mini-exceptions.c:3800:3
    frame %10: 0x0177f3c8 mscorlibtests`mono_llvm_throw_exception(ex=0x01d9e968) at mini-exceptions.c:3823:2
    frame %11: 0x00439ed0 mscorlibtests`mscorlib_System_Runtime_ExceptionServices_ExceptionDispatchInfo_Throw + 116
    frame %12: 0x00271ff0 mscorlibtests`mscorlib_System_LazyHelper_ThrowException + 98
    frame %13: 0x002730c2 mscorlibtests`mscorlib_System_Lazy_1_T_REF_CreateValue + 194
    frame %14: 0x00273390 mscorlibtests`mscorlib_System_Lazy_1_T_REF_get_Value + 128
    frame %15: 0x0027307a mscorlibtests`mscorlib_System_Lazy_1_T_REF_CreateValue + 122
    frame %16: 0x00273390 mscorlibtests`mscorlib_System_Lazy_1_T_REF_get_Value + 128
    frame %17: 0x00e22eb0 mscorlibtests`mscorlibtests1_MonoTests_System_LazyTest_ConcurrentInitialization + 1422
    frame %18: 0x008948e2 mscorlibtests`aot_wrapper_gsharedvt_out_sig_pinvoke_void_this_ + 28
    frame %19: 0x008664a2 mscorlibtests`mscorlib_wrapper_runtime_invoke_object_runtime_invoke_sig_void_intptr_intptr_object_intptr_intptr_intptr + 312
```


Contributes to #13006

Backport of #14400.

/cc @lewurm
2019-05-09 12:06:28 -04:00
Alexander Köplinger 7d5deb616d [sdks] Add .cs source files to ios archive
XI/XM install the Mono sources as part of their package.

(cherry picked from commit a597cee5b0)
2019-05-09 17:25:43 +02:00
monojenkins a0ae702117 A deadlock happens when during a process_suspend there is a pending invoke and then when it's calling the invoke there is another call to process_suspend in the same thread. (#14388) 2019-05-09 10:24:45 -03:00
Alexander Köplinger aee4ff6671 [sdks] Make sure the mono-version.txt directory exists
(cherry picked from commit 8370fe318b)
2019-05-09 11:49:24 +02:00
Alexander Köplinger 30e8a2329b Bump helix-binaries
Helix recently started using Python 3 so we need to have a compatible xunit-reporter.py script.

(cherry picked from commit be571243e4)
2019-05-09 10:40:02 +02:00
Alexander Köplinger 1e96a5cfe6 [sdks] Add Mono version to ios/mac archive 2019-05-08 17:09:10 +02:00
Alexander Köplinger a98c697a6d [sdks] Add Mono binaries to mac archive 2019-05-08 12:45:07 +02:00
monojenkins a5c39e9ac4 [2019-04] [loader] Don't store through a null ptr (#14112)
* [loader] Dont't store through a null ptr

If 'status' is null, don't try to write to it.

Fixes https://github.com/mono/mono/issues/13941

* [metadata] Protect writes to MonoImageOpenStatus* in a few places.

When possible assert that `MonoImageOpenStatus *status` arguments to various
functions in the runtime aren't null.  There are a few places (MONO_API) where
it's possible that callers are passing null.  In that case, use a local
MonoImageOpenStatus, or check for null before doing assignments.
2019-05-07 20:41:23 +02:00
Alexis Christoforides 11e10a07ad [corlib] Fix exception 'replacement' in IsolatedStorageFile.MoveFile()
This was caused by changes in File.Move coming from https://github.com/mono/mono/pull/12000
2019-05-07 14:34:03 -04:00
monojenkins 10db3b9dd0 [debugger-agent] Ignore thread_end event if thread is detached (#14003)
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set.  In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.

Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace

```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
  * frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
    frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
    frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
    frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
    frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
    frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
    frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
    frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
    frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
    frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
    frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
    frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
    frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
    frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
    frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
2019-05-07 20:22:46 +02:00
monojenkins 186a3b768e [2019-04] [corlib] Fix a few NSLogWriter issues. (#13952)
* [corlib] Make NSLogWriter thread-safe.

* [corlib] Simplify newline checking in NSLogWriter.

Checking the StringBuilder buffer after writing to it instead of the input
string/char[] makes the code somewhat simpler.

* [corlib] Fix NSLogWriter.Write to take into account the index and count parameters.
2019-05-07 20:21:33 +02:00
monojenkins cd42fa383f [arm] fix armv7k regression on struct passing (#14380)
[2019-04] [arm] fix armv7k regression on struct passing

We can't use `slotsize=8` on `armv7k` as it leads to stack corruption.

Note that `mtouch` already calls the AOT compiler with `--aot=mtriple=arm64_32-ios`, so no changes are required there.

Regression introduced by https://github.com/mono/mono/pull/12992

Fixes https://github.com/mono/mono/issues/13454


Backport of #14362.

/cc @lewurm
2019-05-07 11:37:30 -04:00
Alexander Köplinger 3134b554ed [sdks] Add Mono libraries to mac archive 2019-05-07 17:19:43 +02:00
Alexander Köplinger c716c2200b [sdks] Add mono-2.pc to mac archive
Xamarin.Mac needs it.
2019-05-07 15:59:38 +02:00
Alexander Köplinger 4616c1c2d6 [sdks] Add Mono libraries and frameworks to iOS archive 2019-05-07 15:59:38 +02:00
Alexander Köplinger 6f7e90d908 [sdks] Remove --with-monotouch from configure arguments
They are only needed for BCL builds and those are disabled by --disable-mcs-build
2019-05-07 15:59:38 +02:00
Alexander Köplinger bec31562b2 [sdks] Fix README.md 2019-05-07 15:59:38 +02:00
monojenkins 18b454c267 Bump API snapshot submodule 2019-05-06 15:02:14 +02:00
Marek Safar 5ce3f8f62b [Facades] Include missing enums in System.Memory 2019-05-06 15:02:14 +02:00
Alexander Köplinger 6c135b00e3 [tests] Write nunit/xunit exclude categories to a file (#14281)
So we can include it in the SDKs archive.

(cherry picked from commit 4eea0870e1)
2019-05-06 14:01:19 +02:00
monojenkins 1a105af0bc [arm] amend stack pointer properly in exception trampoline (#14314)
[2019-04] [arm] amend stack pointer properly in exception trampoline

The previous attempt ( https://github.com/mono/mono/pull/14078 ) only fixed it for ABIs where the stack pointer alignment is 8 bytes. ABIs, such as watchOS, require 16 byte alignment.

Fixes https://github.com/mono/mono/issues/14223

Backport of #14310.

/cc @lewurm
2019-05-03 13:56:18 -04:00
Zoltan Varga fd4b429551 [aot] Avoid passing -march=arm to llc when using thumb, it forces arm32 code generation.
Fixes https://github.com/mono/mono/issues/14247.
2019-05-02 15:46:37 +02:00
monojenkins 705aa0209b [2019-04] [corlib] Add API that was dropped during the coreFX System.IO integra… (#14275)
* [corlib] Add API that was dropped during the coreFX System.IO integration

* Bump API snapshot submodule
2019-05-01 14:23:21 -04:00
Jay Krell 9091e89cbf Remove libgcc_s_seh-1.dll dependency when targeting Win32 with gcc. (#14251)
Don't use popcount intrinsic.
Alternative but not exactly contradictory to https://github.com/mono/mono/pull/14248.

Note that this is a single inlinable instruction on x86,
if you assume at least SSE4 from circa 2007, which I am avoiding here.

(cherry picked from commit 90a91429ae)
2019-04-29 16:14:35 +02:00
Alexander Köplinger 64630570f7 [nunit-lite] Use .NOTPARALLEL to possibly fix some races
We're seeing occasional build errors on CI that seem to be some kind
of race when compiling nunit-lite and nunit-lite-console.

(cherry picked from commit c3541fc543)
2019-04-29 14:12:29 +02:00
Vlad Brezae dd06ea25fa [interp] Fix GetFunctionPointer (#14176)
This method returns a function pointer that can be called with a calli instruction. On interpreter we use a pointer to InterpMethod while on jit we use the native code address. Normally, GetFunctionPointer should return the InterpMethod pointer if called from interp or the native code address if called from jit. Since we don't have any information about the execution engine of the caller, we solve this by intrinsifying all these calls, that happen in the interpreter.

Passing such function pointers between jitted and interp code is probably still unreliable.

Fixes https://github.com/mono/mono/issues/13654
2019-04-26 21:10:08 +02:00
monojenkins c1b2d24d3b [2019-04] Fix arm32 tailcall_reg/tailcall_membase. (#14236)
* When moving parameters for tailcall, and having pushed ip, and adjusting the offset to account for that, split the adjustment between sp and frame pointer, as frame pointer may or may not be sp. This fixes https://github.com/mono/mono/issues/11489.

* Revert "[Tailcall] [arm] [jit] Fix moving tailcall parameters. (#12701)"

This reverts commit 257efc6374.
2019-04-26 10:53:21 -04:00
monojenkins b7cc391243 [interp] fix op_explicit for cast from nfloat to nint and vice versa (#14222)
[2019-04] [interp] fix op_explicit for cast from nfloat to nint and vice versa

Fixes https://github.com/xamarin/xamarin-macios/issues/5809

/cc @spouliot 

Backport of #14201.

/cc @lewurm
2019-04-25 05:36:18 -04:00
monojenkins 28a266a434 [2019-04] [interp] Fix interp strmfld vt (#14212)
* [interp] Fix store remote value type field

Looks like the culprit for some random mcs tests failures

* [interp] Delete dummy code
2019-04-25 12:07:08 +03:00
monojenkins f103f5ce57 [2019-04] [acceptance-tests] Run roslyn tests using in-tree mono instead of system mono (#14198)
We were inadvertently running the roslyn tests against the system mono in CI.
Instead we now install mono to a custom prefix, prepend that to PATH and run
the roslyn tests against the correct mono we just built.
2019-04-24 11:30:46 +02:00
monojenkins 525877cfe1 [aot] partial revert of #14043 (#14138)
[2019-04] [aot] partial revert of #14043

We need to emit a real call here. On arm/android we would emit this now:
```asm
ldr pc, =label
.ltorg         # place address of `label` here.
```
which is a jump to `label`, not a call.

We need a different solution to make larger assemblies work, but for now
I revert it so the `2019-02` intergration is unblocked.

Partial revert of #14043


Backport of #14135.

/cc @lewurm
2019-04-23 11:46:27 -04:00
monojenkins 3d2df4c76e [interp] Fix buffer overflow when incrementing sp (#14148)
Fixes https://github.com/mono/mono/issues/14079
2019-04-23 16:46:03 +03:00
monojenkins fc25025f64 [2019-04] Protecting boxing a null value (#14115)
When a static method is called the obj is not used, so it can be null and it doesn't need to be boxed. Adding an assert in the method to prevent other crash in this method if a NULL is received.
2019-04-18 11:39:40 -03:00
Jay Krell 1c08f4db80 Increase gsharedvt trampolines by 10% from 4000 to 4400.
There are many minor arguments here.

Where this matters most -- iOS -- the pool is already dynamic.

Other platforms could use a dynamic pool or JIT (despite FullAOT).
Using JIT would defeat the test coverage.

Mitigates https://github.com/mono/mono/issues/13888.
If this feels too tenative, go bigger.
2019-04-18 14:02:32 +02:00
monojenkins 030986fed7 [2019-04] [aot] change method table generation so that it works for larger assemblies on ARM (#14053)
[2019-04] [aot] change method table generation so that it works for larger assemblies on ARM

At the end of the AOT compilation, we dump a table containing all method
entrypoints using the toolchain linker to resolve the addresses. Previously
we used `bl` for that, however on larger assemblies the offset might
overflow the 24bit immediate (thus, the linker bailing out with
"Error: branch out of range"). Instead, using `ldr pc,=<label>` followed
by `.ltorg` emits the full 32bit address to a method. That takes 8 bytes
for each method instead of 4 bytes.


( I couldn't reopen https://github.com/mono/mono/pull/3486 anymore)

Backport of #14043.

/cc @marek-safar @lewurm
2019-04-16 17:55:54 -04:00
monojenkins 0f956be2e8 [arm] align stack pointer in throw trampoline (#14085)
[2019-04] [arm] align stack pointer in throw trampoline

This is a regression introduced by e46fa20466

Fixes https://github.com/mono/mono/issues/13672

Backport of #14078.

/cc @akoeplinger @lewurm
2019-04-16 16:16:20 -04:00
Alexis Christoforides 68eea797de [MacSDK] Bump F# to 4.5 (#10919) (#14058)
* Use a custom prefix instead of trying to infer it, which doesn't work under bockbuild.

The following line is problematic in the scope of building the Mac SDK package (using bockbuild):

https://github.com/fsharp/fsharp/blob/ccfccbacfdb16b2ab143cf83f5ece20f738fd864/mono/config.make#L5

It's problematic because where `mono` lives in this case would be something like `/macsdk-staging-dir/bin/mono` which is not the actual prefix (prefix here being the final destination, `/Library/Frameworks/Mono.framework/...`) but a staging area that bockbuild uses (this is how we build on system prefixes without installing on CI)

For the F# build, bockbuild expects things to end up  in `/fsharp-staging-dir/$(prefix)` (because that's what most `make install`s do when given `DESTDIR=/fsharp-staging-dir/`). If things end up in _any other_ directory in `/fsharp-staging-dir/`  (like `/fsharp-staging-dir/macsdk-staging-dir` in our case) that would cause us to miss packaging those files (this has happened before, with packages that have broken or different use of `DESTDIR`). So there is an added write-protection for anywhere in `/fsharp-staging-dir` that *isn't*  `/fsharp-staging-dir/$(prefix)` , because a write-protection error, while obtuse, is much preferable to missing the fact that some files are not packaged (as would have happened here)

So the fix is to stop trying to infer the prefix, and return to providing it explicitly to the build system (this used to be the case via the `configure` step)

* Also, if we are doing an override of the 'make' command, we need to take care of passing DESTDIR ourselves too

* Since now we perform 'make install' on the make step, make the 'install' step into a no-operation.

* Look for sn where mono is, not in the final destination (which again, may not be the same)
2019-04-16 09:43:19 -04:00
sancheolz 73430f2f78 [WinForms] Fix displayed image in TreeNode when ImageIndex set to -2 When ImageIndex in TreeNode set to -2 must not displayed any image 2019-04-15 14:26:45 +02:00
Alexander Köplinger c3c1df05ba [corlib] Disable FileTest.SymLinkStats() on TVOS
Symlinks on TVOS run into sandboxing restrictions. We've already disabled a similar test.

This test was added recently as part of 26e0b1deec.

(cherry picked from commit b85fdbdb10)
2019-04-12 14:27:32 +02:00
Maxim Lipnin a184b94276 Remove #if-def for TypedReference.MakeTypedReference readonly restriction 2019-04-12 10:54:24 +02:00
Maxim Lipnin a5c373ff40 [w32error] Add ENONET error case 2019-04-12 08:29:15 +02:00
Zoltan Varga 453c277692 [jit] Make dlmalloc use mono_valloc so it picks up the MAP_JIT flag on osx.
Fixes https://github.com/mono/mono/issues/13878.
2019-04-12 08:28:18 +02:00
Aleksey Kliger 432e2032c9 [utils] stub mono_setmmapjit on Win32 and Wasm 2019-04-12 08:27:34 +02:00
Miguel de Icaza 31efbed2fe Do not use MAP_JIT by default, instead make this something that is toggled by Xamarin.Mac which needs it 2019-04-12 08:27:34 +02:00
Ankit Jain a0d6820d94 [nuget] Bump to nuget 5.0.0 2019-04-11 13:39:15 +02:00
Alexander Köplinger d7431c205e [cil-strip] Fix reading of undefined NativeType values in old Cecil
cil-strip crashed when processing this method: https://github.com/mono/corefx/blob/f54e64b341a7b114c3502e197b46670cacb28639/src/Microsoft.CSharp/tests/DefaultParameterTests.cs#L88

It uses `MarshalAs((UnmanagedType)2000)`, i.e. an undefined value for the
native type. The old Cecil embedded in cil-strip tried to read more than one
byte which blows up in this case. Upstream Cecil just reads a single byte:
https://github.com/jbevain/cecil/blob/1b79b96d29f1e8e9ba832191b1b42ea9cb750d20/Mono.Cecil/AssemblyReader.cs#L3700-L3703

Fixes https://github.com/mono/mono/issues/13639
2019-04-11 13:38:06 +02:00
Alexander Köplinger 342edb8e52 [corlib] Disable a ParameterInfoTest test on MONOTOUCH
It needs SRE and even with interpreter config this doesn't work yet.

(cherry picked from commit 766b3204c3)
2019-04-10 21:00:11 +02:00
Alexander Köplinger aecfd90d2a [configure] Add failsafe for missing submodules (#13957)
GitHub provides a way to download a .zip/.tar.gz of the repository content in their UI, however the archive doesn't contain git submodules.

The mono build then fails with confusing "missing file" errors.

Since GitHub doesn't let us disable the archive download we should at least add some logic to detect this broken case and bail out.

(cherry picked from commit e8caaa1dbd)
2019-04-10 15:26:48 +02:00
Alexander Köplinger 1e53aaa2b3 [System.Net.Http] Make some tests specific to HttpClientHandler
Otherwise they throw PNSE on e.g. WatchOS.

(cherry picked from commit 0b6dbfa349)
2019-04-10 15:03:59 +02:00
monojenkins d9d7943901 [metadata] Don't free_aggregate_modifiers in mono_metadata_free_type (#13796)
The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes https://github.com/mono/mono/issues/13775
2019-04-09 18:51:21 -04:00
Bernhard Urban 5fd5f4ab42 [AOT] fix winaot LLVM profile (#13909) (#13944)
[2019-04] [AOT] fix winaot LLVM profile

Regression due to https://github.com/mono/mono/pull/13877

Backport of #13909
2019-04-09 15:05:52 -04:00
monojenkins 5ab8c20e76 [2019-04] [merp] Restore non-assertion behavior in debugger-engine.c (#13932)
In my batch of changes here: https://github.com/mono/mono/commit/c3fe80a0c43be780e35f3a4091d3dc1023b35a00#diff-dca5f20c7ac28a5a4487805c49836fdbL340

I changed

```
			if (!seq_points)
				/* Could be AOT code */
				continue;
```

into

```
			seq_points = (MonoSeqPointInfo *) ji->seq_points;

			if (!seq_points) {
				MonoMethod *jmethod = jinfo_get_method (ji);
				if (jmethod->is_inflated) {
					MonoJitInfo *seq_ji;
					MonoMethod *declaring = mono_method_get_declaring_generic_method (jmethod);
					mono_jit_search_all_backends_for_jit_info (domain, declaring, &seq_ji);
					seq_points = (MonoSeqPointInfo *) seq_ji->seq_points;
				}
			}
```

which doesn't account for the case when it can't find the seq_points. When seq_points is NULL at the end of that block, we don't `continue;` as we did before. Instead we assert, and crash the runtime.

This PR undoes this change, making us continue if `!seq_points`.

Backport of #13865.
2019-04-09 11:29:06 +02:00
Alexander Köplinger 56ce963113 [System.Data] Remove unnecessary file from xunit tests (#13935)
It contained DllImports to some Windows APIs which causes issues on xamarin-macios which expects all pinvokes to exist.

Turns out we don't even need that file.

(cherry picked from commit 3903a50e43)
2019-04-08 23:52:00 +02:00
monojenkins 8d6a92f914 [AOT] remove confusing llcopts flag (#13911)
[2019-04] [AOT] remove confusing llcopts flag

Introduced by
https://github.com/mono/mono/commit/2eeaf888c1d0a681c8705bd609f76196d8fb82ed



Backport of #13877.

/cc @lewurm
2019-04-08 13:56:05 -04:00
Alexander Köplinger b3ff0883cc Bump bockbuild to get GTK fix (#13912)
https://github.com/mono/bockbuild/pull/93
2019-04-08 15:49:13 +02:00
Aleksey Kliger 982adf1ac4 [mono-dl] Don't do thread state transitions in mono_dl_fallback_register
1. it doesn't touch the managed runtime

2. it may be called (arguably it ought to be called only) before the runtime is
initialized (and in particular before the current thread is attached).
2019-04-05 16:04:41 -04:00
monojenkins daf75c61ca [gsharedvt] Fix sizeof opcode (#13832)
[2019-04] [gsharedvt] Fix sizeof opcode

We were setting the stack type by using another global local named ins.

This lead to missing methods in the aot image due to invalid IL. Fixes https://github.com/mono/mono/issues/13479.

Backport of #13817.

/cc @lewurm @BrzVlad
2019-04-04 09:55:23 -04:00
Alexander Köplinger ba238de63b Update csproj file
(cherry picked from commit e75ba60277)
2019-04-04 11:10:15 +02:00
Alexander Köplinger 21e76d610e [corlib] Disable EncodingTest.AllEncodingsAreSerializable() on XM4.5 too
(cherry picked from commit 9eb2dbf4cc)
2019-04-04 01:57:10 +02:00
Alexander Köplinger 84aa2ebf15 [MacSDK] Bump bockbuild for fix to #13804 2019-04-04 00:14:25 +02:00
Alexander Köplinger 4d67a0f9f2 [2019-04] Bump linker to latest master (#13818)
(cherry picked from commit 6717c3422a)
2019-04-03 23:40:32 +02:00
Alexander Köplinger 0796cd6829 [System] Clear MONO_CFG_DIR before starting mcs in CodeDOM CSharpCodeGenerator for XM4.5 (#13742)
* [System] Clear MONO_CFG_DIR before starting mcs in CodeDOM CSharpCodeGenerator

This showed up in the xamarin-macios 2019-02 integration.
When you used Xslt with a custom C# script in a Xamarin.Mac Full application
we'd try to launch mcs via CodeDOM to compile the script.

This started failing in 2019-02 because System.Native (mono-native) wasn't found,
the reason is that XM sets MONO_CFG_DIR to point to the Contents/MonoBundle
in the app bundle where it packages the global config file with dllmaps.

However the started Mono wasn't able to find mono-native since the config
there uses a different name for the file.

To prevent that we now clear MONO_CFG_DIR so it uses the default value for the
mono process we're starting (note that setting it to String.Empty doesn't work)

* Make it specific to XM 4.5

(cherry picked from commit 448cf31a8d)
2019-04-02 10:46:33 +02:00
Jo Shields 2046cf5d26 Fix tarball builds
(cherry picked from commit 262a543a17)
2019-04-01 15:52:05 -04:00
Alexander Köplinger 0f705a4acd Apply API snapshot and csproj changes 2019-04-01 21:51:24 +02:00
Jo Shields f683efc4a6 Merge pull request #13766 from directhex/2019-04
Bump version to 6.2
2019-04-01 11:40:49 -04:00
Jo Shields 7a7bebbfb2 Bump version to 6.2 2019-04-01 10:33:13 -04:00
1977 changed files with 24171 additions and 61490 deletions

No files matched your search

+1 -13
View File
@@ -24,6 +24,7 @@
*.tlh
*.bak
*.ilk
*.log
*.lib
*.sbr
*.sdf
@@ -128,22 +129,9 @@ docs/perlmod*
compile
mono.h
mono-*.tar.*
mono_crash*
tmpinst-dir.stamp
msvc/scripts/inputs/
extensions-config.h
*.dylib.dSYM
*.mlpd
llvm/build/
llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
##############################################################################
# Arcade output directories
##############################################################################
.packages
artifacts
.dotnet
+8 -7
View File
@@ -3,8 +3,11 @@
{
"name": "Mac",
"includePath": [
"${default}",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1",
"/usr/local/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include",
"/usr/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
"${workspaceRoot}/mono/eglib",
@@ -14,8 +17,11 @@
"intelliSenseMode": "clang-x64",
"browse": {
"path": [
"${default}",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1",
"/usr/local/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include",
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include",
"/usr/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
"${workspaceRoot}/mono/eglib",
@@ -25,7 +31,6 @@
"databaseFilename": ""
},
"macFrameworkPath": [
"${default}",
"/System/Library/Frameworks",
"/Library/Frameworks"
]
@@ -33,7 +38,6 @@
{
"name": "Linux",
"includePath": [
"${default}",
"/usr/include",
"/usr/local/include",
"${workspaceRoot}",
@@ -45,7 +49,6 @@
"intelliSenseMode": "clang-x64",
"browse": {
"path": [
"${default}",
"/usr/include",
"/usr/local/include",
"${workspaceRoot}",
@@ -60,7 +63,6 @@
{
"name": "Win32",
"includePath": [
"${default}",
"C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/include",
"${workspaceRoot}",
"${workspaceRoot}/mono",
@@ -74,7 +76,6 @@
"intelliSenseMode": "msvc-x64",
"browse": {
"path": [
"${default}",
"C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/include/*",
"${workspaceRoot}",
"${workspaceRoot}/mono",
+1 -7
View File
@@ -19,17 +19,11 @@ build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
else
update_submodules = update_submodules
endif
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore
all: $(update_submodules)
all: update_submodules
update_submodules:
@cd $(srcdir) && scripts/update_submodules.sh
-8
View File
@@ -1,8 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
-3
View File
@@ -306,9 +306,6 @@ faster under the Xen virtualization system.
* `--with-large-heap=yes,no` - Enable support for GC heaps larger than 3GB.
* This only applies only to the Boehm garbage collector, the SGen garbage
collector does not use this configuration option.
* This defaults to `no`.
* `--enable-small-config=yes,no` - Enable some tweaks to reduce memory usage
-18
View File
@@ -1,18 +0,0 @@
Mono uses third-party libraries or other resources that may be
distributed under licenses different than the .NET Core software.
Attributions and license notices for test cases originally authored by
third parties can be found in the respective test directories.
In the event that we accidentally failed to list a required notice, please
bring it to our attention. Post an issue or email us:
[email protected]
The attached notices are provided for information only.
License notice for Apple
-------------------------
Content: https://github.com/mono/mono/blob/435f968102d0820ca65d63e1e741e4130b4d1303/support/ios/net/route.h
License: https://opensource.apple.com/apsl/
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "13a40eedacc799a78710aba5c8e4b2dcc80a7e61",
"rev": "7a840bd5697ef6ea41f554a09d837636fba9d88d",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+12 -86
View File
@@ -4,7 +4,7 @@
# This line is parsed by tools besides autoconf, such as msvc/mono.winconfig.targets.
# It should remain in the format they expect.
#
AC_INIT(mono, [6.4.0],
AC_INIT(mono, [6.2.0],
[https://github.com/mono/mono/issues/new])
AC_CONFIG_SRCDIR([README.md])
@@ -56,13 +56,10 @@ MONO_VERSION_BUILD=`echo $VERSION | cut -d . -f 3`
# Generate it with uuidgen. For example:
# perl -pi.bak -e "s/^MONO_CORLIB_VERSION=\S+/MONO_CORLIB_VERSION=`uuidgen`/" configure.ac
#
# There is no ordering of corlib versions, no old or new,
# an exact match is required between corlib and runtime.
#
# This line is parsed by tools besides autoconf, such as msvc/mono.winconfig.targets.
# It should remain in the format they expect.
#
MONO_CORLIB_VERSION=62731146-81CF-4D61-826D-9A8DDED14432
MONO_CORLIB_VERSION=48B64EB1-FC8C-4707-A363-84A1FE488DCA
#
# Put a quoted #define in config.h.
@@ -162,22 +159,19 @@ case "$host" in
fi
HOST_CC="gcc"
RID="win-x86"
CORETARGETS="-p:TargetsWindows=true"
COREARCH="x86"
# Boehm not supported on 64-bit Windows.
case "$host" in
x86_64-*-* | amd64-*-*)
support_boehm=no
with_gc=sgen
RID="win-x64"
COREARCH="x64"
;;
esac
# Windows 7 or later is required
WIN32_CPPFLAGS="-DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -D_UNICODE -DUNICODE -DWIN32_THREADS -DFD_SETSIZE=1024"
CPPFLAGS="$CPPFLAGS $WIN32_CPPFLAGS"
WIN32_LDFLAGS="-lbcrypt -lmswsock -lws2_32 -lole32 -loleaut32 -lpsapi -lversion -ladvapi32 -lwinmm -lkernel32 -liphlpapi -static-libgcc"
WIN32_LDFLAGS="-lbcrypt -lmswsock -lws2_32 -lole32 -loleaut32 -lpsapi -lversion -ladvapi32 -lwinmm -lkernel32 -liphlpapi"
LDFLAGS="$LDFLAGS $WIN32_LDFLAGS"
libmono_cflags="-mms-bitfields -mwindows"
libmono_ldflags="-mms-bitfields -mwindows"
@@ -312,7 +306,6 @@ case "$host" in
libmono_cflags="-D_REENTRANT"
libdl="-ldl"
libgc_threads=pthreads
CORETARGETS="-p:TargetsUnix=true"
use_sigposix=yes
if test "x$cross_compiling" != "xno"; then
# to bypass the underscore linker check, not
@@ -336,11 +329,9 @@ case "$host" in
;;
x86-*)
RID="linux-x86"
COREARCH="x86"
;;
x86_64-*)
RID="linux-x64"
COREARCH="x64"
;;
arm-*)
# deal with this in the FPU detection section, since
@@ -351,7 +342,6 @@ case "$host" in
support_boehm=no
with_gc=sgen
RID="linux-arm64"
COREARCH="arm64"
;;
powerpc*-*-linux*)
# https://bugzilla.novell.com/show_bug.cgi?id=504411
@@ -404,7 +394,6 @@ case "$host" in
need_link_unlink=yes
AC_DEFINE(PTHREAD_POINTER_ID)
AC_DEFINE(USE_MACH_SEMA, 1, [...])
CORETARGETS="-p:TargetsUnix=true -p:TargetsOSX=true"
libdl=
libgc_threads=pthreads
has_dtrace=yes
@@ -427,12 +416,10 @@ case "$host" in
CFLAGS_FOR_LIBGC="$CFLAGS_FOR_LIBGC $BROKEN_DARWIN_FLAGS"
with_sgen_default_concurrent=yes
RID="osx-x86"
COREARCH="x86"
;;
x*64-*-darwin*)
with_sgen_default_concurrent=yes
RID="osx-x64"
COREARCH="x64"
;;
arm*-darwin*)
platform_ios=yes
@@ -492,7 +479,6 @@ case "$host" in
dnl ppc Linux is the same? test further
disable_munmap=yes
RID="aix-ppc64"
CORETARGETS="-p:TargetsUnix=true"
;;
*)
AC_MSG_WARN([*** Please add $host to configure.ac checks!])
@@ -535,21 +521,17 @@ AM_CONDITIONAL(HOST_WASM, test x$platform_wasm = xyes)
if test -z "$HOST_DARWIN_TRUE"; then :
PLATFORM_AOT_SUFFIX=.dylib
PLATFORM_AOT_PREFIX=lib
fi
if test -z "$HOST_LINUX_TRUE"; then :
PLATFORM_AOT_SUFFIX=.so
PLATFORM_AOT_PREFIX=lib
fi
if test -z "$HOST_WIN32_TRUE"; then :
PLATFORM_AOT_SUFFIX=.dll
PLATFORM_AOT_PREFIX=
fi
AC_SUBST(PLATFORM_AOT_SUFFIX)
AC_SUBST(PLATFORM_AOT_PREFIX)
if test -z "$HOST_WASM_TRUE"; then :
AC_DEFINE(HAVE_UTIME)
@@ -573,10 +555,6 @@ dnl We should use AM_PROG_AS, but it's not available on automake/aclocal 1.4
AC_SUBST(CCAS)
AC_SUBST(CCASFLAGS)
if test "x$CXX" = "xno"; then
AC_MSG_ERROR([No c++ compiler found. You need to install a c++ compiler])
fi
# AC_PROG_CXX helpfully sets CXX to g++ even if no c++ compiler is found so check
# GXX instead. See http://lists.gnu.org/archive/html/bug-autoconf/2002-04/msg00056.html
if test "x$CXX" = "xg++"; then
@@ -1274,8 +1252,6 @@ INVARIANT_AOT_OPTIONS=nimt-trampolines=2000,ntrampolines=10000,nrgctx-fetch-tram
AOT_BUILD_ATTRS=$INVARIANT_AOT_OPTIONS
MONO_LLVM_PATH_OPTION=llvm-path="`pwd`/llvm/usr/bin"
if test x$cross_compiling = xyes -o x$enable_mcs_build = xno; then
DISABLE_MCS_DOCS_default=yes
elif test x$with_runtime_preset = xnetcore; then
@@ -1620,13 +1596,6 @@ if test x$USE_NLS = xprofile_default; then
AC_MSG_RESULT([$USE_NLS])
fi
AC_ARG_ENABLE(static-gcc-libs, [ --enable-static-gcc-libs Statically link GCC support libs (for MinGW32)])
if test "x$enable_static_gcc_libs" = "xyes"; then
# llvm/build.mk doesn't use automake, so make a regular make variable for it.
AC_SUBST(STATIC_GCC_LIBS,1)
fi
AM_CONDITIONAL(ENABLE_STATIC_GCC_LIBS, test "x$enable_static_gcc_libs" = "xyes")
AC_ARG_ENABLE(minimal, [ --enable-minimal=LIST drop support for LIST subsystems.
LIST is a comma-separated list from: aot, profiler, decimal, pinvoke, debug, appdomains, verifier,
reflection_emit, reflection_emit_save, large_code, logging, com, ssa, generics, attach, jit, interpreter, simd, soft_debug, perfcounters, normalization, desktop_loader, shared_perfcounters, remoting,
@@ -2646,8 +2615,6 @@ if test x$host_win32 = xno; then
AC_CHECK_FUNCS(pthread_attr_setstacksize)
AC_CHECK_FUNCS(pthread_attr_getstack pthread_attr_getstacksize)
AC_CHECK_FUNCS(pthread_get_stacksize_np pthread_get_stackaddr_np)
dnl check that pthread_mutexattr_setprotocol is declared
AC_CHECK_DECLS([pthread_mutexattr_setprotocol], [], [], [[#include <pthread.h>]])
AC_CHECK_FUNCS(mincore mlock munlock)
dnl ***********************************
@@ -4955,15 +4922,14 @@ AC_SEARCH_LIBS(dlopen, dl)
old_ldflags="${LDFLAGS}"
# GNU specific option, this confuses IBM ld, but do offer alternatives when possible
if test $lt_cv_prog_gnu_ld = yes; then
LDFLAGS="${LDFLAGS} -Wl,-export-dynamic,-Bsymbolic"
LDFLAGS="${LDFLAGS} -Wl,-export-dynamic"
else
case $host in
*-*-aix*|*-*-os400*)
LDFLAGS="${LDFLAGS} -bsymbolic"
;;
*)
dnl Try to use export-dynamic anyways
LDFLAGS="${LDFLAGS} -Wl,-export-dynamic,-Bsymbolic"
LDFLAGS="${LDFLAGS} -Wl,-export-dynamic"
;;
esac
fi
@@ -5268,17 +5234,13 @@ if test ${TARGET} = ARM; then
if test x$host_linux = xyes; then
RID="linux-arm"
COREARCH="arm"
if test x$fpu = xNONE; then
RID="linux-armel"
COREARCH="armel"
fi
fi
fi
AC_SUBST(RID)
AC_SUBST(COREARCH)
AC_SUBST(CORETARGETS)
if test ${TARGET} = RISCV32 -o ${TARGET} = RISCV64; then
AC_ARG_WITH([riscv-fpabi], [ --with-riscv-fpabi=auto,double,soft Select RISC-V floating point ABI (auto)], [fpabi=$withval], [fpabi=double])
@@ -5867,8 +5829,8 @@ fi
AC_SUBST(mono_runtime)
AC_SUBST(mono_runtime_wrapper)
CSC_LOCATION=`cd $srcdir && pwd`/external/roslyn-binaries/Microsoft.Net.Compilers/3.3.1/csc.exe
VBCS_LOCATION=`cd $srcdir && pwd`/external/roslyn-binaries/Microsoft.Net.Compilers/3.3.1/VBCSCompiler.exe
CSC_LOCATION=`cd $srcdir && pwd`/external/roslyn-binaries/Microsoft.Net.Compilers/3.1.0/csc.exe
VBCS_LOCATION=`cd $srcdir && pwd`/external/roslyn-binaries/Microsoft.Net.Compilers/3.1.0/VBCSCompiler.exe
if test $csc_compiler = mcs; then
CSC=$mcs_topdir/class/lib/build/mcs.exe
@@ -6112,10 +6074,6 @@ elif test x$target_osx = xyes; then
MONO_NATIVE_PLATFORM=macos
MONO_NATIVE_PLATFORM_TYPE="MONO_NATIVE_PLATFORM_TYPE_MACOS"
if test x$enable_gss = xyes; then
MONO_NATIVE_LDFLAGS="$MONO_NATIVE_LDFLAGS -framework GSS"
fi
AC_MONO_APPLE_AVAILABLE(mono_native_compat, [whether we need the compatibility layer],
[!(MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_X_VERSION_10_12)])
@@ -6430,9 +6388,7 @@ AC_SUBST(CPPFLAGS)
AC_SUBST(LDFLAGS)
# Update all submodules recursively to ensure everything is checked out
if test "x$with_core" != "xonly"; then
(cd $srcdir && scripts/update_submodules.sh)
fi
(cd $srcdir && scripts/update_submodules.sh)
AC_OUTPUT([
Makefile
@@ -6478,6 +6434,7 @@ mono/profiler/Makefile
mono/eglib/Makefile
mono/eglib/eglib-config.h
mono/eglib/test/Makefile
netcore/Makefile
m4/Makefile
ikvm-native/Makefile
scripts/Makefile
@@ -6535,26 +6492,6 @@ if test x$host_darwin = xyes; then
sed -e 's,lock_old_archive_extraction=yes,lock_old_archive_extraction=no,g' < libtool > libtool.new && mv libtool.new libtool && chmod +x libtool
fi
AC_MSG_CHECKING([compiler server])
AC_ARG_WITH(compiler-server, [ --with-compiler-server=yes,no,default Enables or disables compiler server],[
if test x$withval = xyes; then
enable_compiler_server=yes;
elif test x$withval = xno; then
enable_compiler_server=no;
elif test x$withval = xdefault; then
enable_compiler_server=no;
else
AC_MSG_ERROR([You must supply one of "yes", "no" or "default" to the --with-compiler-server option])
fi
],[enable_compiler_server=no])
if test x$enable_compiler_server = xyes; then
if test x$csc_compiler = xmcs; then
AC_MSG_WARN([mcs does not support the compiler server])
enable_compiler_server=no;
fi
fi
(
case $prefix in
NONE) prefix=$ac_default_prefix ;;
@@ -6581,10 +6518,6 @@ fi
echo "RUNTIME = $mono_build_root/runtime/mono-wrapper" >> $mcs_topdir/build/config.make
echo "ILDISASM = $mono_build_root/runtime/monodis-wrapper" >> $mcs_topdir/build/config.make
echo "JAY_CFLAGS = $JAY_CFLAGS" >> $mcs_topdir/build/config.make
echo "RID = $RID" >> $mcs_topdir/build/config.make
echo "COREARCH = $COREARCH" >> $mcs_topdir/build/config.make
echo "CORETARGETS = $CORETARGETS" >> $mcs_topdir/build/config.make
echo "VERSION = $VERSION" >> $mcs_topdir/build/config.make
case $INSTALL in
[[\\/$]]* | ?:[[\\/]]* ) mcs_INSTALL=$INSTALL ;;
@@ -6623,7 +6556,6 @@ fi
echo "XTEST_PLATFORM ?= unix" >> $mcs_topdir/build/config.make
fi
echo "PLATFORM_AOT_PREFIX = $PLATFORM_AOT_PREFIX" >> $mcs_topdir/build/config.make
if test "x$PLATFORM_AOT_SUFFIX" != "x"; then
echo "PLATFORM_AOT_SUFFIX = $PLATFORM_AOT_SUFFIX" >> $mcs_topdir/build/config.make
fi
@@ -6660,13 +6592,8 @@ fi
if test "x$AOT_BUILD_FLAGS" != "x" ; then
echo "AOT_RUN_FLAGS=$AOT_RUN_FLAGS" >> $srcdir/$mcsdir/build/config.make
echo "AOT_BUILD_FLAGS=$AOT_BUILD_FLAGS" >> $srcdir/$mcsdir/build/config.make
echo "AOT_BUILD_ATTRS=$AOT_BUILD_ATTRS" >> $srcdir/$mcsdir/build/config.make
if test "x$internal_llvm" != "xno"; then
echo "AOT_BUILD_FLAGS=$AOT_BUILD_FLAGS,$MONO_LLVM_PATH_OPTION" >> $srcdir/$mcsdir/build/config.make
else
echo "AOT_BUILD_FLAGS=$AOT_BUILD_FLAGS" >> $srcdir/$mcsdir/build/config.make
fi
fi
if test "x$AOT_MODE" != "x" ; then
@@ -6725,9 +6652,8 @@ if test x$buildsgen = xyes; then
fi
echo "
mcs source: $mcsdir
mcs source: $mcsdir
C# Compiler: $csc_compiler
CompilerServer:$enable_compiler_server
Engine:
Host: $host
@@ -6765,5 +6691,5 @@ echo "
$disabled
"
if test x$with_static_mono = xno -a "x$host_win32" != "xyes"; then
AC_MSG_WARN([Turning off static Mono is a risk, you might run into unexpected bugs])
AC_MSG_WARN(Turning off static Mono is a risk, you might run into unexpected bugs)
fi
-15
View File
@@ -1,15 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19279.5">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>fb62c6377a6bd163af2a7516260f064498942585</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
-7
View File
@@ -1,7 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>1.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
</PropertyGroup>
</Project>
-2
View File
@@ -1,2 +0,0 @@
@echo off
powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0Build.ps1""" -restore -build -test -sign -pack -publish -ci %*"
-158
View File
@@ -1,158 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SymbolToolPath # Full path to directory where dotnet symbol-tool was installed
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $TargetServerParam, # Parameter to pass to `Symbol Tool` indicating the server to lookup for symbols
[string] $SymbolsPath
)
$FileName = [System.IO.Path]::GetFileName($FullPath)
$Extension = [System.IO.Path]::GetExtension($FullPath)
# Those below are potential symbol files that the `dotnet symbol` might
# return. Which one will be returned depend on the type of file we are
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
}
elseif (Test-Path $SymbolPath) {
return "Module"
}
else {
return $null
}
}
function CountMissingSymbols {
param(
[string] $PackagePath # Path to a NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
# How many files are missing symbol information
$MissingSymbols = 0
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
# Makes easier to reference `symbol tool`
Push-Location $SymbolToolPath
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
}
else {
Write-Host "No symbols found on SymWeb!"
}
}
}
}
Pop-Location
return $MissingSymbols
}
function CheckSymbolsAvailable {
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-Error "Missing symbols for $Status modules in the package $FileName"
}
Write-Host
}
}
CheckSymbolsAvailable
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PushToStaticFeed task that will read the build asset
manifest and publish the assets described in the manifest to
informed target feeds.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(MSBuildThisFileDirectory)DefaultVersions.props" Condition="Exists('$(MSBuildThisFileDirectory)DefaultVersions.props')" />
<!--
This won't be necessary once we solve this issue:
https://github.com/dotnet/arcade/issues/2266
-->
<Import Project="$(MSBuildThisFileDirectory)ArtifactsCategory.props" Condition="Exists('$(MSBuildThisFileDirectory)ArtifactsCategory.props')" />
<Import Project="$(NuGetPackageRoot)microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<Target Name="PublishToFeed">
<Error Condition="'$(ArtifactsCategory)' == ''" Text="ArtifactsCategory: The artifacts' category produced by the build wasn't provided." />
<Error Condition="'$(AccountKeyToStaticFeed)' == ''" Text="AccountKeyToStaticFeed: Account key for target feed wasn't provided." />
<Error Condition="'$(ManifestsBasePath)' == ''" Text="Full path to asset manifests directory wasn't provided." />
<Error Condition="'$(BlobBasePath)' == '' AND '$(PackageBasePath)' == ''" Text="A valid full path to BlobBasePath of PackageBasePath is required." />
<ItemGroup>
<!-- Include all manifests found in the manifest folder. -->
<ManifestFiles Include="$(ManifestsBasePath)*.xml" />
</ItemGroup>
<Error Condition="'@(ManifestFiles)' == ''" Text="No manifest file was found in the provided path: $(ManifestsBasePath)" />
<!--
For now the type of packages being published will be informed for the whole build.
Eventually this will be specified on a per package basis:
TODO: https://github.com/dotnet/arcade/issues/2266
-->
<PropertyGroup>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCORE'">https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCOREVALIDATION'">https://dotnetfeed.blob.core.windows.net/arcade-validation/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORETOOLING'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ENTITYFRAMEWORKCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETEXTENSIONS'">https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORECLR'">https://dotnetfeed.blob.core.windows.net/dotnet-coreclr/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORESDK'">https://dotnetfeed.blob.core.windows.net/dotnet-sdk/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSINTERNAL'">https://dotnetfeed.blob.core.windows.net/dotnet-tools-internal/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSET'">https://dotnetfeed.blob.core.windows.net/dotnet-toolset/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'WINDOWSDESKTOP'">https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'NUGETCLIENT'">https://dotnetfeed.blob.core.windows.net/nuget-nugetclient/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETENTITYFRAMEWORK6'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframework6/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETBLAZOR'">https://dotnetfeed.blob.core.windows.net/aspnet-blazor/index.json</TargetStaticFeed>
</PropertyGroup>
<Error
Condition="'$(TargetStaticFeed)' == ''"
Text="'$(ArtifactsCategory)' wasn't recognized as a valid artifact category. Valid categories are: '.NetCore' and '.NetCoreValidation'" />
<!-- Iterate publishing assets from each manifest file. -->
<PushArtifactsInManifestToFeed
ExpectedFeedUrl="$(TargetStaticFeed)"
AccountKey="$(AccountKeyToStaticFeed)"
BARBuildId="$(BARBuildId)"
MaestroApiEndpoint="$(MaestroApiEndpoint)"
BuildAssetRegistryToken="$(BuildAssetRegistryToken)"
Overwrite="$(OverrideAssetsWithSameName)"
PassIfExistingItemIdentical="$(PassIfExistingItemIdentical)"
MaxClients="$(MaxParallelUploads)"
UploadTimeoutInMinutes="$(MaxUploadTimeoutInMinutes)"
AssetManifestPath="%(ManifestFiles.Identity)"
BlobAssetsBasePath="$(BlobBasePath)"
PackageAssetsBasePath="$(PackageBasePath)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
</ItemGroup>
</Project>
-82
View File
@@ -1,82 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PublishSymbols tasks to publish symbols to MSDL and SymWeb.
Parameters:
- PDBArtifactsDirectory : Full path to directory containing PDB files to be published.
- BlobBasePath : Full path containing *.symbols.nupkg packages to be published.
- DotNetSymbolServerTokenMsdl : PAT to access MSDL.
- DotNetSymbolServerTokenSymWeb : PAT to access SymWeb.
- DotNetSymbolExpirationInDays : Expiration days for published packages. Default is 3650.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(NuGetPackageRoot)microsoft.symboluploader.build.task\$(SymbolUploaderVersion)\build\PublishSymbols.targets" />
<Target Name="PublishSymbols">
<ItemGroup>
<FilesToPublishToSymbolServer Include="$(PDBArtifactsDirectory)\*.pdb"/>
<PackagesToPublishToSymbolServer Include="$(BlobBasePath)\*.symbols.nupkg"/>
<!--
These packages from Arcade-Services include some native libraries that
our current symbol uploader can't handle. Below is a workaround until
we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
-->
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Darc.*" />
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Maestro.Tasks.*" />
</ItemGroup>
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
<Message
Importance="High"
Text="No symbol package(s) were found to publish."
Condition="$(PublishToSymbolServer) == false" />
<!-- Symbol Uploader: MSDL -->
<Message Importance="High" Text="Publishing symbol packages to MSDL ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenMsdl)"
SymbolServerPath="https://microsoftpublicsymbols.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
<!--
Symbol Uploader: SymWeb
Watson, VS insertion testings and the typical internal dev usage require SymWeb.
Currently we need to call the task twice (https://github.com/dotnet/core-eng/issues/3489).
-->
<Message Importance="High" Text="Publishing symbol packages to SymWeb ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenSymWeb)"
SymbolServerPath="https://microsoft.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.SymbolUploader.Build.Task" Version="$(SymbolUploaderVersion)" />
</ItemGroup>
</Project>
-28
View File
@@ -1,28 +0,0 @@
# Don't touch this folder
uuuuuuuuuuuuuuuuuuuu
u" uuuuuuuuuuuuuuuuuu "u
u" u$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
u" u$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$u "u
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
$ $$$" ... "$... ...$" ... "$$$ ... "$$$ $
$ $$$u `"$$$$$$$ $$$ $$$$$ $$ $$$ $$$ $
$ $$$$$$uu "$$$$ $$$ $$$$$ $$ """ u$$$ $
$ $$$""$$$ $$$$ $$$u "$$$" u$$ $$$$$$$$ $
$ $$$$....,$$$$$..$$$$$....,$$$$..$$$$$$$$ $
$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$$$$$" u"
"u "$$$$$$$$$$$$$$$$$$$$" u"
"u """""""""""""""""" u"
""""""""""""""""""""
!!! Changes made in this directory are subject to being overwritten by automation !!!
The files in this directory are shared by all Arcade repos and managed by automation. If you need to make changes to these files, open an issue or submit a pull request to https://github.com/dotnet/arcade first.
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
- PackageBasePath : Directory containing all files that need to be validated.
- SignCheckVersion : Version of SignCheck package to be used.
- SignValidationExclusionList : ItemGroup containing exclusion list to be forwarded to SignCheck.
- EnableJarSigningCheck : Whether .jar files should be validated.
- EnableStrongNameCheck : Whether strong name check should be performed.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<!--
From 'Signing.props' we import $(SignValidationExclusionList)
-->
<Import Project="$(MSBuildThisFileDirectory)Signing.props" Condition="Exists('$(MSBuildThisFileDirectory)Signing.props')" />
<Target Name="ValidateSigning">
<PropertyGroup>
<SignCheckToolPath>$(NuGetPackageRoot)Microsoft.DotNet.SignCheck\$(SignCheckVersion)\tools\Microsoft.DotNet.SignCheck.exe</SignCheckToolPath>
<SignCheckInputDir>$(PackageBasePath)</SignCheckInputDir>
<SignCheckLog>signcheck.log</SignCheckLog>
<SignCheckErrorLog>signcheck.errors.log</SignCheckErrorLog>
<SignCheckExclusionsFile>signcheck.exclusions.txt</SignCheckExclusionsFile>
</PropertyGroup>
<ItemGroup>
<!--
Documentation for these arguments is available here:
https://github.com/dotnet/arcade/tree/master/src/SignCheck
-->
<SignCheckArgs Include="--recursive" />
<SignCheckArgs Include="--traverse-subfolders" />
<SignCheckArgs Include="--file-status AllFiles" />
<SignCheckArgs Include="--log-file $(SignCheckLog)" />
<SignCheckArgs Include="--error-log-file $(SignCheckErrorLog)" />
<SignCheckArgs Include="--input-files $(SignCheckInputDir)" />
<SignCheckArgs Include="--exclusions-file $(SignCheckExclusionsFile)" Condition="'@(SignValidationExclusionList)' != ''" />
<SignCheckArgs Include="--verify-jar" Condition="'$(EnableJarSigningCheck)' == 'true'" />
<SignCheckArgs Include="--verify-strongname" Condition="'$(EnableStrongNameCheck)' == 'true'" />
</ItemGroup>
<WriteLinesToFile
File="$(SignCheckExclusionsFile)"
Lines="@(SignValidationExclusionList)"
Condition="'@(SignValidationExclusionList)' != ''"
Overwrite="true"
Encoding="Unicode"/>
<!--
IgnoreExitCode='true' because the tool doesn't return '0' on success.
-->
<Exec
Command="&quot;$(SignCheckToolPath)&quot; @(SignCheckArgs, ' ')"
IgnoreExitCode='true'
ConsoleToMsBuild="false"
StandardErrorImportance="high" />
<Error
Text="Signing validation failed. Check $(SignCheckErrorLog) for more information."
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
<Message
Text="##vso[artifact.upload containerfolder=LogFiles;artifactname=LogFiles]{SignCheckErrorLog}"
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.SignCheck" Version="$(SignCheckVersion)" />
</ItemGroup>
</Project>
-184
View File
@@ -1,184 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SourceLinkToolPath, # Full path to directory where dotnet SourceLink CLI was installed
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
$RepoTreeURL = -Join("https://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
Measure-Command { ValidateSourceLinkLinks }
-142
View File
@@ -1,142 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string][Alias('c')]$configuration = "Debug",
[string]$platform = $null,
[string] $projects,
[string][Alias('v')]$verbosity = "minimal",
[string] $msbuildEngine = $null,
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch][Alias('r')]$restore,
[switch] $deployDeps,
[switch][Alias('b')]$build,
[switch] $rebuild,
[switch] $deploy,
[switch][Alias('t')]$test,
[switch] $integrationTest,
[switch] $performanceTest,
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
[switch] $help,
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot "restore-toolset.ps1"
if (Test-Path $script) {
. $script
}
}
function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
MSBuild $toolsetBuildProj `
$bl `
$platformArg `
/p:Configuration=$configuration `
/p:RepoRoot=$RepoRoot `
/p:Restore=$restore `
/p:DeployDeps=$deployDeps `
/p:Build=$build `
/p:Rebuild=$rebuild `
/p:Deploy=$deploy `
/p:Test=$test `
/p:Pack=$pack `
/p:IntegrationTest=$integrationTest `
/p:PerformanceTest=$performanceTest `
/p:Sign=$sign `
/p:Publish=$publish `
@properties
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
exit 0
}
if ($ci) {
$binaryLog = $true
$nodeReuse = $false
}
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
InitializeNativeTools
}
Build
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
-216
View File
@@ -1,216 +0,0 @@
#!/usr/bin/env bash
# Stop script if unbound variable found (use ${var:-} if intentional)
set -u
# Stop script if command returns non-zero exit code.
# Prevents hidden errors caused by missing error code propagation.
set -e
usage()
{
echo "Common settings:"
echo " --configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
echo " --verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
echo " --binaryLog Create MSBuild binary log (short: -bl)"
echo " --help Print help and exit (short: -h)"
echo ""
echo "Actions:"
echo " --restore Restore dependencies (short: -r)"
echo " --build Build solution (short: -b)"
echo " --rebuild Rebuild solution"
echo " --test Run all unit tests in the solution (short: -t)"
echo " --integrationTest Run all integration tests in the solution"
echo " --performanceTest Run all performance tests in the solution"
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo ""
echo "Advanced settings:"
echo " --projects <value> Project or solution file(s) to build"
echo " --ci Set when running on CI server"
echo " --prepareMachine Prepare machine for CI run, clean up processes after build"
echo " --nodeReuse <value> Sets nodereuse msbuild parameter ('true' or 'false')"
echo " --warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
echo ""
echo "Command line arguments not listed above are passed thru to msbuild."
echo "Arguments can also be passed in with a single hyphen."
}
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
restore=false
build=false
rebuild=false
test=false
integration_test=false
performance_test=false
pack=false
publish=false
sign=false
public=false
ci=false
warn_as_error=true
node_reuse=true
binary_log=false
pipelines_log=false
projects=''
configuration='Debug'
prepare_machine=false
verbosity='minimal'
properties=''
while [[ $# > 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-help|-h)
usage
exit 0
;;
-configuration|-c)
configuration=$2
shift
;;
-verbosity|-v)
verbosity=$2
shift
;;
-binarylog|-bl)
binary_log=true
;;
-pipelineslog|-pl)
pipelines_log=true
;;
-restore|-r)
restore=true
;;
-build|-b)
build=true
;;
-rebuild)
rebuild=true
;;
-pack)
pack=true
;;
-test|-t)
test=true
;;
-integrationtest)
integration_test=true
;;
-performancetest)
performance_test=true
;;
-sign)
sign=true
;;
-publish)
publish=true
;;
-preparemachine)
prepare_machine=true
;;
-projects)
projects=$2
shift
;;
-ci)
ci=true
;;
-warnaserror)
warn_as_error=$2
shift
;;
-nodereuse)
node_reuse=$2
shift
;;
*)
properties="$properties $1"
;;
esac
shift
done
if [[ "$ci" == true ]]; then
pipelines_log=true
binary_log=true
node_reuse=false
fi
. "$scriptroot/tools.sh"
function InitializeCustomToolset {
local script="$eng_root/restore-toolset.sh"
if [[ -a "$script" ]]; then
. "$script"
fi
}
function Build {
InitializeToolset
InitializeCustomToolset
if [[ ! -z "$projects" ]]; then
properties="$properties /p:Projects=$projects"
fi
local bl=""
if [[ "$binary_log" == true ]]; then
bl="/bl:\"$log_dir/Build.binlog\""
fi
MSBuild $_InitializeToolset \
$bl \
/p:Configuration=$configuration \
/p:RepoRoot="$repo_root" \
/p:Restore=$restore \
/p:Build=$build \
/p:Rebuild=$rebuild \
/p:Test=$test \
/p:Pack=$pack \
/p:IntegrationTest=$integration_test \
/p:PerformanceTest=$performance_test \
/p:Sign=$sign \
/p:Publish=$publish \
$properties
ExitWithExitCode 0
}
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
InitializeNativeTools
fi
Build
-16
View File
@@ -1,16 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $SOURCE until the file is no longer a symlink
while [[ -h $source ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where
# the symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/build.sh" --restore --build --test --pack --publish --ci $@
@@ -1,41 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)
## Specify the toolchain
set(TOOLCHAIN "arm-linux-androideabi")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
add_compile_options(-mfloat-abi=soft)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/arm-linux-androideabi/)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
@@ -1,42 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
## Specify the toolchain
set(TOOLCHAIN "aarch64-linux-android")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
## Needed for Android or bionic specific conditionals
add_compile_options(-D__ANDROID__)
add_compile_options(-D__BIONIC__)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
-3
View File
@@ -1,3 +0,0 @@
# Debian (sid) # UNSTABLE
deb http://ftp.debian.org/debian/ sid main contrib non-free
deb-src http://ftp.debian.org/debian/ sid main contrib non-free
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
@@ -1,71 +0,0 @@
From e72c9d7ead60e3317bd6d1fade995c07021c947b Mon Sep 17 00:00:00 2001
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 7 May 2015 13:25:04 -0400
Subject: [PATCH] Fix: building probe providers with C++ compiler
Robert Daniels wrote:
> > I'm attempting to use lttng userspace tracing with a C++ application
> > on an ARM platform. I'm using GCC 4.8.4 on Linux 3.14 with the 2.6
> > release of lttng. I've compiled lttng-modules, lttng-ust, and
> > lttng-tools and have been able to get a simple test working with C
> > code. When I attempt to run the hello.cxx test on my target it will
> > segfault.
>
>
> I spent a little time digging into this issue and finally discovered the
> cause of my segfault with ARM C++ tracepoints.
>
> There is a struct called 'lttng_event' in ust-events.h which contains an
> empty union 'u'. This was the cause of my issue. Under C, this empty union
> compiles to a zero byte member while under C++ it compiles to a one byte
> member, and in my case was four-byte aligned which caused my C++ code to
> have the 'cds_list_head node' offset incorrectly by four bytes. This lead
> to an incorrect linked list structure which caused my issue.
>
> Since this union is empty, I simply removed it from the struct and everything
> worked correctly.
>
> I don't know the history or purpose behind this empty union so I'd like to
> know if this is a safe fix. If it is I can submit a patch with the union
> removed.
That's a very nice catch!
We do not support building tracepoint probe provider with
g++ yet, as stated in lttng-ust(3):
"- Note for C++ support: although an application instrumented with
tracepoints can be compiled with g++, tracepoint probes should be
compiled with gcc (only tested with gcc so far)."
However, if it works fine with this fix, then I'm tempted to take it,
especially because removing the empty union does not appear to affect
the layout of struct lttng_event as seen from liblttng-ust, which must
be compiled with a C compiler, and from probe providers compiled with
a C compiler. So all we are changing is the layout of a probe provider
compiled with a C++ compiler, which is anyway buggy at the moment,
because it is not compatible with the layout expected by liblttng-ust
compiled with a C compiler.
Reported-by: Robert Daniels <[email protected]>
Signed-off-by: Mathieu Desnoyers <[email protected]>
---
include/lttng/ust-events.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/usr/include/lttng/ust-events.h b/usr/include/lttng/ust-events.h
index 328a875..3d7a274 100644
--- a/usr/include/lttng/ust-events.h
+++ b/usr/include/lttng/ust-events.h
@@ -407,8 +407,6 @@ struct lttng_event {
void *_deprecated1;
struct lttng_ctx *ctx;
enum lttng_ust_instrumentation instrumentation;
- union {
- } u;
struct cds_list_head node; /* Event list in session */
struct cds_list_head _deprecated2;
void *_deprecated3;
--
2.7.4
-97
View File
@@ -1,97 +0,0 @@
diff -u -r a/usr/include/urcu/uatomic/generic.h b/usr/include/urcu/uatomic/generic.h
--- a/usr/include/urcu/uatomic/generic.h 2014-03-28 06:04:42.000000000 +0900
+++ b/usr/include/urcu/uatomic/generic.h 2017-02-13 10:35:21.189927116 +0900
@@ -65,17 +65,17 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- return __sync_val_compare_and_swap_1(addr, old, _new);
+ return __sync_val_compare_and_swap_1((uint8_t *) addr, old, _new);
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- return __sync_val_compare_and_swap_2(addr, old, _new);
+ return __sync_val_compare_and_swap_2((uint16_t *) addr, old, _new);
#endif
case 4:
- return __sync_val_compare_and_swap_4(addr, old, _new);
+ return __sync_val_compare_and_swap_4((uint32_t *) addr, old, _new);
#if (CAA_BITS_PER_LONG == 64)
case 8:
- return __sync_val_compare_and_swap_8(addr, old, _new);
+ return __sync_val_compare_and_swap_8((uint64_t *) addr, old, _new);
#endif
}
_uatomic_link_error();
@@ -100,20 +100,20 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- __sync_and_and_fetch_1(addr, val);
+ __sync_and_and_fetch_1((uint8_t *) addr, val);
return;
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- __sync_and_and_fetch_2(addr, val);
+ __sync_and_and_fetch_2((uint16_t *) addr, val);
return;
#endif
case 4:
- __sync_and_and_fetch_4(addr, val);
+ __sync_and_and_fetch_4((uint32_t *) addr, val);
return;
#if (CAA_BITS_PER_LONG == 64)
case 8:
- __sync_and_and_fetch_8(addr, val);
+ __sync_and_and_fetch_8((uint64_t *) addr, val);
return;
#endif
}
@@ -139,20 +139,20 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- __sync_or_and_fetch_1(addr, val);
+ __sync_or_and_fetch_1((uint8_t *) addr, val);
return;
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- __sync_or_and_fetch_2(addr, val);
+ __sync_or_and_fetch_2((uint16_t *) addr, val);
return;
#endif
case 4:
- __sync_or_and_fetch_4(addr, val);
+ __sync_or_and_fetch_4((uint32_t *) addr, val);
return;
#if (CAA_BITS_PER_LONG == 64)
case 8:
- __sync_or_and_fetch_8(addr, val);
+ __sync_or_and_fetch_8((uint64_t *) addr, val);
return;
#endif
}
@@ -180,17 +180,17 @@
switch (len) {
#ifdef UATOMIC_HAS_ATOMIC_BYTE
case 1:
- return __sync_add_and_fetch_1(addr, val);
+ return __sync_add_and_fetch_1((uint8_t *) addr, val);
#endif
#ifdef UATOMIC_HAS_ATOMIC_SHORT
case 2:
- return __sync_add_and_fetch_2(addr, val);
+ return __sync_add_and_fetch_2((uint16_t *) addr, val);
#endif
case 4:
- return __sync_add_and_fetch_4(addr, val);
+ return __sync_add_and_fetch_4((uint32_t *) addr, val);
#if (CAA_BITS_PER_LONG == 64)
case 8:
- return __sync_add_and_fetch_8(addr, val);
+ return __sync_add_and_fetch_8((uint64_t *) addr, val);
#endif
}
_uatomic_link_error();
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-security main restricted universe multiverse
@@ -1,11 +0,0 @@
deb http://deb.debian.org/debian buster main
deb-src http://deb.debian.org/debian buster main
deb http://deb.debian.org/debian-security/ buster/updates main
deb-src http://deb.debian.org/debian-security/ buster/updates main
deb http://deb.debian.org/debian buster-updates main
deb-src http://deb.debian.org/debian buster-updates main
deb http://deb.debian.org/debian buster-backports main contrib non-free
deb-src http://deb.debian.org/debian buster-backports main contrib non-free
@@ -1,12 +0,0 @@
deb http://deb.debian.org/debian stretch main
deb-src http://deb.debian.org/debian stretch main
deb http://deb.debian.org/debian-security/ stretch/updates main
deb-src http://deb.debian.org/debian-security/ stretch/updates main
deb http://deb.debian.org/debian stretch-updates main
deb-src http://deb.debian.org/debian stretch-updates main
deb http://deb.debian.org/debian stretch-backports main contrib non-free
deb-src http://deb.debian.org/debian stretch-backports main contrib non-free
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ trusty-security main restricted universe multiverse
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-updates main restricted universe
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-backports main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ zesty-security main restricted universe multiverse
@@ -1,3 +0,0 @@
# Debian (jessie) # Stable
deb http://ftp.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.debian.org/debian/ jessie main contrib non-free
@@ -1,44 +0,0 @@
#!/usr/bin/env bash
set -e
__ARM_SOFTFP_CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__TIZEN_CROSSDIR="$__ARM_SOFTFP_CrossDir/tizen"
if [[ -z "$ROOTFS_DIR" ]]; then
echo "ROOTFS_DIR is not defined."
exit 1;
fi
# Clean-up (TODO-Cleanup: We may already delete $ROOTFS_DIR at ./cross/build-rootfs.sh.)
# hk0110
if [ -d "$ROOTFS_DIR" ]; then
umount $ROOTFS_DIR/*
rm -rf $ROOTFS_DIR
fi
TIZEN_TMP_DIR=$ROOTFS_DIR/tizen_tmp
mkdir -p $TIZEN_TMP_DIR
# Download files
echo ">>Start downloading files"
VERBOSE=1 $__ARM_SOFTFP_CrossDir/tizen-fetch.sh $TIZEN_TMP_DIR
echo "<<Finish downloading files"
echo ">>Start constructing Tizen rootfs"
TIZEN_RPM_FILES=`ls $TIZEN_TMP_DIR/*.rpm`
cd $ROOTFS_DIR
for f in $TIZEN_RPM_FILES; do
rpm2cpio $f | cpio -idm --quiet
done
echo "<<Finish constructing Tizen rootfs"
# Cleanup tmp
rm -rf $TIZEN_TMP_DIR
# Configure Tizen rootfs
echo ">>Start configuring Tizen rootfs"
rm ./usr/lib/libunwind.so
ln -s libunwind.so.8 ./usr/lib/libunwind.so
ln -sfn asm-arm ./usr/include/asm
patch -p1 < $__TIZEN_CROSSDIR/tizen.patch
echo "<<Finish configuring Tizen rootfs"
-171
View File
@@ -1,171 +0,0 @@
#!/usr/bin/env bash
set -e
if [[ -z "${VERBOSE// }" ]] || [ "$VERBOSE" -ne "$VERBOSE" ] 2>/dev/null; then
VERBOSE=0
fi
Log()
{
if [ $VERBOSE -ge $1 ]; then
echo ${@:2}
fi
}
Inform()
{
Log 1 -e "\x1B[0;34m$@\x1B[m"
}
Debug()
{
Log 2 -e "\x1B[0;32m$@\x1B[m"
}
Error()
{
>&2 Log 0 -e "\x1B[0;31m$@\x1B[m"
}
Fetch()
{
URL=$1
FILE=$2
PROGRESS=$3
if [ $VERBOSE -ge 1 ] && [ $PROGRESS ]; then
CURL_OPT="--progress-bar"
else
CURL_OPT="--silent"
fi
curl $CURL_OPT $URL > $FILE
}
hash curl 2> /dev/null || { Error "Require 'curl' Aborting."; exit 1; }
hash xmllint 2> /dev/null || { Error "Require 'xmllint' Aborting."; exit 1; }
hash sha256sum 2> /dev/null || { Error "Require 'sha256sum' Aborting."; exit 1; }
TMPDIR=$1
if [ ! -d $TMPDIR ]; then
TMPDIR=./tizen_tmp
Debug "Create temporary directory : $TMPDIR"
mkdir -p $TMPDIR
fi
TIZEN_URL=http://download.tizen.org/releases/milestone/tizen
BUILD_XML=build.xml
REPOMD_XML=repomd.xml
PRIMARY_XML=primary.xml
TARGET_URL="http://__not_initialized"
Xpath_get()
{
XPATH_RESULT=''
XPATH=$1
XML_FILE=$2
RESULT=$(xmllint --xpath $XPATH $XML_FILE)
if [[ -z ${RESULT// } ]]; then
Error "Can not find target from $XML_FILE"
Debug "Xpath = $XPATH"
exit 1
fi
XPATH_RESULT=$RESULT
}
fetch_tizen_pkgs_init()
{
TARGET=$1
PROFILE=$2
Debug "Initialize TARGET=$TARGET, PROFILE=$PROFILE"
TMP_PKG_DIR=$TMPDIR/tizen_${PROFILE}_pkgs
if [ -d $TMP_PKG_DIR ]; then rm -rf $TMP_PKG_DIR; fi
mkdir -p $TMP_PKG_DIR
PKG_URL=$TIZEN_URL/$PROFILE/latest
BUILD_XML_URL=$PKG_URL/$BUILD_XML
TMP_BUILD=$TMP_PKG_DIR/$BUILD_XML
TMP_REPOMD=$TMP_PKG_DIR/$REPOMD_XML
TMP_PRIMARY=$TMP_PKG_DIR/$PRIMARY_XML
TMP_PRIMARYGZ=${TMP_PRIMARY}.gz
Fetch $BUILD_XML_URL $TMP_BUILD
Debug "fetch $BUILD_XML_URL to $TMP_BUILD"
TARGET_XPATH="//build/buildtargets/buildtarget[@name=\"$TARGET\"]/repo[@type=\"binary\"]/text()"
Xpath_get $TARGET_XPATH $TMP_BUILD
TARGET_PATH=$XPATH_RESULT
TARGET_URL=$PKG_URL/$TARGET_PATH
REPOMD_URL=$TARGET_URL/repodata/repomd.xml
PRIMARY_XPATH='string(//*[local-name()="data"][@type="primary"]/*[local-name()="location"]/@href)'
Fetch $REPOMD_URL $TMP_REPOMD
Debug "fetch $REPOMD_URL to $TMP_REPOMD"
Xpath_get $PRIMARY_XPATH $TMP_REPOMD
PRIMARY_XML_PATH=$XPATH_RESULT
PRIMARY_URL=$TARGET_URL/$PRIMARY_XML_PATH
Fetch $PRIMARY_URL $TMP_PRIMARYGZ
Debug "fetch $PRIMARY_URL to $TMP_PRIMARYGZ"
gunzip $TMP_PRIMARYGZ
Debug "unzip $TMP_PRIMARYGZ to $TMP_PRIMARY"
}
fetch_tizen_pkgs()
{
ARCH=$1
PACKAGE_XPATH_TPL='string(//*[local-name()="metadata"]/*[local-name()="package"][*[local-name()="name"][text()="_PKG_"]][*[local-name()="arch"][text()="_ARCH_"]]/*[local-name()="location"]/@href)'
PACKAGE_CHECKSUM_XPATH_TPL='string(//*[local-name()="metadata"]/*[local-name()="package"][*[local-name()="name"][text()="_PKG_"]][*[local-name()="arch"][text()="_ARCH_"]]/*[local-name()="checksum"]/text())'
for pkg in ${@:2}
do
Inform "Fetching... $pkg"
XPATH=${PACKAGE_XPATH_TPL/_PKG_/$pkg}
XPATH=${XPATH/_ARCH_/$ARCH}
Xpath_get $XPATH $TMP_PRIMARY
PKG_PATH=$XPATH_RESULT
XPATH=${PACKAGE_CHECKSUM_XPATH_TPL/_PKG_/$pkg}
XPATH=${XPATH/_ARCH_/$ARCH}
Xpath_get $XPATH $TMP_PRIMARY
CHECKSUM=$XPATH_RESULT
PKG_URL=$TARGET_URL/$PKG_PATH
PKG_FILE=$(basename $PKG_PATH)
PKG_PATH=$TMPDIR/$PKG_FILE
Debug "Download $PKG_URL to $PKG_PATH"
Fetch $PKG_URL $PKG_PATH true
echo "$CHECKSUM $PKG_PATH" | sha256sum -c - > /dev/null
if [ $? -ne 0 ]; then
Error "Fail to fetch $PKG_URL to $PKG_PATH"
Debug "Checksum = $CHECKSUM"
exit 1
fi
done
}
Inform "Initialize arm base"
fetch_tizen_pkgs_init standard base
Inform "fetch common packages"
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel
fetch_tizen_pkgs noarch linux-glibc-devel
Inform "fetch coreclr packages"
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel tizen-release lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l libcom_err libcom_err-devel zlib zlib-devel libopenssl libopenssl-devel krb5 krb5-devel libcurl libcurl-devel
Inform "Initialize standard unified"
fetch_tizen_pkgs_init standard unified
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l gssdp gssdp-devel
@@ -1,50 +0,0 @@
lang en_US.UTF-8
keyboard us
timezone --utc Asia/Seoul
part / --fstype="ext4" --size=3500 --ondisk=mmcblk0 --label rootfs --fsoptions=defaults,noatime
rootpw tizen
desktop --autologinuser=root
user --name root --groups audio,video --password 'tizen'
repo --name=standard --baseurl=http://download.tizen.org/releases/milestone/tizen/unified/latest/repos/standard/packages/ --ssl_verify=no
repo --name=base --baseurl=http://download.tizen.org/releases/milestone/tizen/base/latest/repos/standard/packages/ --ssl_verify=no
%packages
tar
gzip
sed
grep
gawk
perl
binutils
findutils
util-linux
lttng-ust
userspace-rcu
procps-ng
tzdata
ca-certificates
### Core FX
libicu
libunwind
iputils
zlib
krb5
libcurl
libopenssl
%end
%post
### Update /tmp privilege
chmod 777 /tmp
####################################
%end
-18
View File
@@ -1,18 +0,0 @@
diff -u -r a/usr/lib/libc.so b/usr/lib/libc.so
--- a/usr/lib/libc.so 2016-12-30 23:00:08.284951863 +0900
+++ b/usr/lib/libc.so 2016-12-30 23:00:32.140951815 +0900
@@ -2,4 +2,4 @@
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
-GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.3 ) )
+GROUP ( libc.so.6 libc_nonshared.a AS_NEEDED ( ld-linux.so.3 ) )
diff -u -r a/usr/lib/libpthread.so b/usr/lib/libpthread.so
--- a/usr/lib/libpthread.so 2016-12-30 23:00:19.408951841 +0900
+++ b/usr/lib/libpthread.so 2016-12-30 23:00:39.068951801 +0900
@@ -2,4 +2,4 @@
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
-GROUP ( /lib/libpthread.so.0 /usr/lib/libpthread_nonshared.a )
+GROUP ( libpthread.so.0 libpthread_nonshared.a )
-137
View File
@@ -1,137 +0,0 @@
#!/usr/bin/env bash
set -e
__NDK_Version=r14
usage()
{
echo "Creates a toolchain and sysroot used for cross-compiling for Android."
echo.
echo "Usage: $0 [BuildArch] [ApiLevel]"
echo.
echo "BuildArch is the target architecture of Android. Currently only arm64 is supported."
echo "ApiLevel is the target Android API level. API levels usually match to Android releases. See https://source.android.com/source/build-numbers.html"
echo.
echo "By default, the toolchain and sysroot will be generated in cross/android-rootfs/toolchain/[BuildArch]. You can change this behavior"
echo "by setting the TOOLCHAIN_DIR environment variable"
echo.
echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"
echo "you can set the NDK_DIR environment variable to have this script use that installation of the NDK."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.21-arm64. This file is to replace '/etc/os-release', which is not available for Android."
exit 1
}
__ApiLevel=21 # The minimum platform for arm64 is API level 21
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
for i in "$@"
do
lowerI="$(echo $i | awk '{print tolower($0)}')"
case $lowerI in
-?|-h|--help)
usage
exit 1
;;
arm64)
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
;;
arm)
__BuildArch=arm
__AndroidArch=arm
__AndroidToolchain=arm-linux-androideabi
;;
*[0-9])
__ApiLevel=$i
;;
*)
__UnprocessedBuildArgs="$__UnprocessedBuildArgs $i"
;;
esac
done
# Obtain the location of the bash script to figure out where the root of the repo is.
__CrossDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__Android_Cross_Dir="$__CrossDir/android-rootfs"
__NDK_Dir="$__Android_Cross_Dir/android-ndk-$__NDK_Version"
__libunwind_Dir="$__Android_Cross_Dir/libunwind"
__lldb_Dir="$__Android_Cross_Dir/lldb"
__ToolchainDir="$__Android_Cross_Dir/toolchain/$__BuildArch"
if [[ -n "$TOOLCHAIN_DIR" ]]; then
__ToolchainDir=$TOOLCHAIN_DIR
fi
if [[ -n "$NDK_DIR" ]]; then
__NDK_Dir=$NDK_DIR
fi
echo "Target API level: $__ApiLevel"
echo "Target architecture: $__BuildArch"
echo "NDK location: $__NDK_Dir"
echo "Target Toolchain location: $__ToolchainDir"
# Download the NDK if required
if [ ! -d $__NDK_Dir ]; then
echo Downloading the NDK into $__NDK_Dir
mkdir -p $__NDK_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__Android_Cross_Dir
fi
if [ ! -d $__lldb_Dir ]; then
mkdir -p $__lldb_Dir
echo Downloading LLDB into $__lldb_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
fi
# Create the RootFS for both arm64 as well as aarch
rm -rf $__Android_Cross_Dir/toolchain
echo Generating the $__BuildArch toolchain
$__NDK_Dir/build/tools/make_standalone_toolchain.py --arch $__BuildArch --api $__ApiLevel --install-dir $__ToolchainDir
# Install the required packages into the toolchain
# TODO: Add logic to get latest pkg version instead of specific version number
rm -rf $__Android_Cross_Dir/deb/
rm -rf $__Android_Cross_Dir/tmp
mkdir -p $__Android_Cross_Dir/deb/
mkdir -p $__Android_Cross_Dir/tmp/$arch/
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu-dev_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob-dev_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support-dev_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma-dev_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind-dev_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb
echo Unpacking Termux packages
dpkg -x $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
cp -R $__Android_Cross_Dir/tmp/$__AndroidArch/data/data/com.termux/files/usr/* $__ToolchainDir/sysroot/usr/
# Generate platform file for build.sh script to assign to __DistroRid
echo "Generating platform file..."
echo "RID=android.21-arm64" > $__ToolchainDir/sysroot/android_platform
echo Now run:
echo CONFIG_DIR=\`realpath cross/android/$__BuildArch\` ROOTFS_DIR=\`realpath $__ToolchainDir/sysroot\` ./build.sh cross $__BuildArch skipgenerateversion skipnuget cmakeargs -DENABLE_LLDBPLUGIN=0
-233
View File
@@ -1,233 +0,0 @@
#!/usr/bin/env bash
usage()
{
echo "Usage: $0 [BuildArch] [LinuxCodeName] [lldbx.y] [--skipunmount] --rootfsdir <directory>]"
echo "BuildArch can be: arm(default), armel, arm64, x86"
echo "LinuxCodeName - optional, Code name for Linux, can be: trusty, xenial(default), zesty, bionic, alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen."
echo "lldbx.y - optional, LLDB version, can be: lldb3.9(default), lldb4.0, lldb5.0, lldb6.0 no-lldb. Ignored for alpine"
echo "--skipunmount - optional, will skip the unmount of rootfs folder."
exit 1
}
__LinuxCodeName=xenial
__CrossDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
__InitialDir=$PWD
__BuildArch=arm
__UbuntuArch=armhf
__UbuntuRepo="http://ports.ubuntu.com/"
__LLDB_Package="liblldb-3.9-dev"
__SkipUnmount=0
# base development support
__UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
# symlinks fixer
__UbuntuPackages+=" symlinks"
# CoreCLR and CoreFX dependencies
__UbuntuPackages+=" libicu-dev"
__UbuntuPackages+=" liblttng-ust-dev"
__UbuntuPackages+=" libunwind8-dev"
__AlpinePackages+=" gettext-dev"
__AlpinePackages+=" icu-dev"
__AlpinePackages+=" libunwind-dev"
__AlpinePackages+=" lttng-ust-dev"
# CoreFX dependencies
__UbuntuPackages+=" libcurl4-openssl-dev"
__UbuntuPackages+=" libkrb5-dev"
__UbuntuPackages+=" libssl-dev"
__UbuntuPackages+=" zlib1g-dev"
__AlpinePackages+=" curl-dev"
__AlpinePackages+=" krb5-dev"
__AlpinePackages+=" openssl-dev"
__AlpinePackages+=" zlib-dev"
__UnprocessedBuildArgs=
while :; do
if [ $# -le 0 ]; then
break
fi
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
-?|-h|--help)
usage
exit 1
;;
arm)
__BuildArch=arm
__UbuntuArch=armhf
__AlpineArch=armhf
__QEMUArch=arm
;;
arm64)
__BuildArch=arm64
__UbuntuArch=arm64
__AlpineArch=aarch64
__QEMUArch=aarch64
;;
armel)
__BuildArch=armel
__UbuntuArch=armel
__UbuntuRepo="http://ftp.debian.org/debian/"
__LinuxCodeName=jessie
;;
x86)
__BuildArch=x86
__UbuntuArch=i386
__UbuntuRepo="http://archive.ubuntu.com/ubuntu/"
;;
lldb3.6)
__LLDB_Package="lldb-3.6-dev"
;;
lldb3.8)
__LLDB_Package="lldb-3.8-dev"
;;
lldb3.9)
__LLDB_Package="liblldb-3.9-dev"
;;
lldb4.0)
__LLDB_Package="liblldb-4.0-dev"
;;
lldb5.0)
__LLDB_Package="liblldb-5.0-dev"
;;
lldb6.0)
__LLDB_Package="liblldb-6.0-dev"
;;
no-lldb)
unset __LLDB_Package
;;
trusty) # Ubuntu 14.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=trusty
fi
;;
xenial) # Ubuntu 16.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=xenial
fi
;;
zesty) # Ubuntu 17.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=zesty
fi
;;
bionic) # Ubuntu 18.04
if [ "$__LinuxCodeName" != "jessie" ]; then
__LinuxCodeName=bionic
fi
;;
jessie) # Debian 8
__LinuxCodeName=jessie
__UbuntuRepo="http://ftp.debian.org/debian/"
;;
stretch) # Debian 9
__LinuxCodeName=stretch
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
buster) # Debian 10
__LinuxCodeName=buster
__UbuntuRepo="http://ftp.debian.org/debian/"
__LLDB_Package="liblldb-6.0-dev"
;;
tizen)
if [ "$__BuildArch" != "armel" ]; then
echo "Tizen is available only for armel."
usage;
exit 1;
fi
__LinuxCodeName=
__UbuntuRepo=
__Tizen=tizen
;;
alpine)
__LinuxCodeName=alpine
__UbuntuRepo=
;;
--skipunmount)
__SkipUnmount=1
;;
--rootfsdir|-rootfsdir)
shift
__RootfsDir=$1
;;
*)
__UnprocessedBuildArgs="$__UnprocessedBuildArgs $1"
;;
esac
shift
done
if [ "$__BuildArch" == "armel" ]; then
__LLDB_Package="lldb-3.5-dev"
fi
__UbuntuPackages+=" ${__LLDB_Package:-}"
if [ -z "$__RootfsDir" ] && [ ! -z "$ROOTFS_DIR" ]; then
__RootfsDir=$ROOTFS_DIR
fi
if [ -z "$__RootfsDir" ]; then
__RootfsDir="$__CrossDir/rootfs/$__BuildArch"
fi
if [ -d "$__RootfsDir" ]; then
if [ $__SkipUnmount == 0 ]; then
umount $__RootfsDir/*
fi
rm -rf $__RootfsDir
fi
if [[ "$__LinuxCodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.7
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
rm -r $__ApkToolsDir
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
cp $__CrossDir/$__BuildArch/sources.list.$__LinuxCodeName $__RootfsDir/etc/apt/sources.list
chroot $__RootfsDir apt-get update
chroot $__RootfsDir apt-get -f -y install
chroot $__RootfsDir apt-get -y install $__UbuntuPackages
chroot $__RootfsDir symlinks -cr /usr
if [ $__SkipUnmount == 0 ]; then
umount $__RootfsDir/*
fi
if [[ "$__BuildArch" == "arm" && "$__LinuxCodeName" == "trusty" ]]; then
pushd $__RootfsDir
patch -p1 < $__CrossDir/$__BuildArch/trusty.patch
patch -p1 < $__CrossDir/$__BuildArch/trusty-lttng-2.4.patch
popd
fi
elif [ "$__Tizen" == "tizen" ]; then
ROOTFS_DIR=$__RootfsDir $__CrossDir/$__BuildArch/tizen-build-rootfs.sh
else
echo "Unsupported target platform."
usage;
exit 1
fi
-138
View File
@@ -1,138 +0,0 @@
set(CROSS_ROOTFS $ENV{ROOTFS_DIR})
set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
if(TARGET_ARCH_NAME STREQUAL "armel")
set(CMAKE_SYSTEM_PROCESSOR armv7l)
set(TOOLCHAIN "arm-linux-gnueabi")
if("$ENV{__DistroRid}" MATCHES "tizen.*")
set(TIZEN_TOOLCHAIN "armv7l-tizen-linux-gnueabi/6.2.1")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "arm")
set(CMAKE_SYSTEM_PROCESSOR armv7l)
if(EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/armv6-alpine-linux-musleabihf)
set(TOOLCHAIN "armv6-alpine-linux-musleabihf")
else()
set(TOOLCHAIN "arm-linux-gnueabihf")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "arm64")
set(CMAKE_SYSTEM_PROCESSOR aarch64)
if(EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/aarch64-alpine-linux-musl)
set(TOOLCHAIN "aarch64-alpine-linux-musl")
else()
set(TOOLCHAIN "aarch64-linux-gnu")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
set(CMAKE_SYSTEM_PROCESSOR i686)
set(TOOLCHAIN "i686-linux-gnu")
else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
include_directories(SYSTEM ${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}/include/c++/)
include_directories(SYSTEM ${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}/include/c++/armv7l-tizen-linux-gnueabi)
endif()
endif()
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
endif()
endforeach()
endmacro()
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_param(CROSS_LINK_FLAGS "-m32")
endif()
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
endif()
if(TARGET_ARCH_NAME MATCHES "^(arm|armel)$")
add_compile_options(-mthumb)
add_compile_options(-mfpu=vfpv3)
if(TARGET_ARCH_NAME STREQUAL "armel")
add_compile_options(-mfloat-abi=softfp)
if(DEFINED TIZEN_TOOLCHAIN)
add_compile_options(-Wno-deprecated-declarations) # compile-time option
add_compile_options(-D__extern_always_inline=inline) # compile-time option
endif()
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-m32)
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
else() # arm/armel case
set(LLVM_CROSS_DIR "$ENV{LLVM_ARM_HOME}")
endif()
if(LLVM_CROSS_DIR)
set(WITH_LLDB_LIBS "${LLVM_CROSS_DIR}/lib/" CACHE STRING "")
set(WITH_LLDB_INCLUDES "${LLVM_CROSS_DIR}/include" CACHE STRING "")
set(LLDB_H "${WITH_LLDB_INCLUDES}" CACHE STRING "")
set(LLDB "${LLVM_CROSS_DIR}/lib/liblldb.so" CACHE STRING "")
else()
if(TARGET_ARCH_NAME STREQUAL "x86")
set(WITH_LLDB_LIBS "${CROSS_ROOTFS}/usr/lib/i386-linux-gnu" CACHE STRING "")
set(CHECK_LLVM_DIR "${CROSS_ROOTFS}/usr/lib/llvm-3.8/include")
if(EXISTS "${CHECK_LLVM_DIR}" AND IS_DIRECTORY "${CHECK_LLVM_DIR}")
set(WITH_LLDB_INCLUDES "${CHECK_LLVM_DIR}")
else()
set(WITH_LLDB_INCLUDES "${CROSS_ROOTFS}/usr/lib/llvm-3.6/include")
endif()
else() # arm/armel case
set(WITH_LLDB_LIBS "${CROSS_ROOTFS}/usr/lib/${TOOLCHAIN}" CACHE STRING "")
set(WITH_LLDB_INCLUDES "${CROSS_ROOTFS}/usr/lib/llvm-3.6/include" CACHE STRING "")
endif()
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ bionic main restricted universe
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
-11
View File
@@ -1,11 +0,0 @@
deb http://archive.ubuntu.com/ubuntu/ xenial main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ xenial main restricted universe
deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe
deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe
deb http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted
deb-src http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
-33
View File
@@ -1,33 +0,0 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
)
$verbosity = "m"
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = Invoke-Expression "& `"$dotnet`" tool list -g"
if ($toolList -like "*$darcCliPackageName*") {
Invoke-Expression "& `"$dotnet`" tool uninstall $darcCliPackageName -g"
}
# If the user didn't explicitly specify the darc version,
# query the Maestro API for the correct version of darc to install.
if (-not $darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Invoke-Expression "& `"$dotnet`" tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
}
InstallDarcCli $darcVersion
-64
View File
@@ -1,64 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--darcversion)
darcVersion=$2
shift
;;
--versionendpoint)
versionEndpoint=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity=m
. "$scriptroot/tools.sh"
if [ -z "$darcVersion" ]; then
darcVersion=$(curl -X GET "$versionEndpoint" -H "accept: text/plain")
fi
function InstallDarcCli {
local darc_cli_package_name="microsoft.dotnet.darc"
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
local arcadeServicesSource="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
}
InstallDarcCli
-2
View File
@@ -1,2 +0,0 @@
@echo off
powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0dotnet-install.ps1""" %*"
-27
View File
@@ -1,27 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet"
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
}
InstallDotNet $installdir $version $architecture $runtime $true
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
-49
View File
@@ -1,49 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
version='Latest'
architecture=''
runtime='dotnet'
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
-version|-v)
shift
version="$1"
;;
-architecture|-a)
shift
architecture="$1"
;;
-runtime|-r)
shift
runtime="$1"
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
. "$scriptroot/tools.sh"
dotnetRoot="$repo_root/.dotnet"
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
ExitWithExitCode 0
-87
View File
@@ -1,87 +0,0 @@
Param(
[Parameter(Mandatory=$true)][string] $barToken, # Token generated at https://maestro-prod.westus2.cloudapp.azure.com/Account/Tokens
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
Write-Host "Installing dot..."
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
$darcExe = Resolve-Path "$darcExe\darc.exe"
Create-Directory $outputFolder
# Generate 3 graph descriptions:
# 1. Flat with coherency information
# 2. Graphviz (dot) file
# 3. Standard dependency graph
$graphVizFilePath = "$outputFolder\graphviz.txt"
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = "get-dependency-graph --github-pat $gitHubPat --azdev-pat $azdoPat --password $barToken"
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += " --include-toolset"
}
Write-Host "Generating standard dependency graph..."
Invoke-Expression "& `"$darcExe`" $baseOptions --output-file $normalGraphFilePath"
CheckExitCode "Generating normal dependency graph"
Write-Host "Generating flat dependency graph and graphviz file..."
Invoke-Expression "& `"$darcExe`" $baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath"
CheckExitCode "Generating flat and graphviz dependency graph"
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
Invoke-Expression "& `"$dotFilePath`" -Tpng -o'$graphVizImageFilePath' `"$graphVizFilePath`""
CheckExitCode "Generating graphviz image"
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
} finally {
Pop-Location
}
-26
View File
@@ -1,26 +0,0 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup>
<Language>msbuild</Language>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<ItemGroup>
<HelixWorkItem Include="WorkItem" Condition="'$(WorkItemDirectory)' != ''">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand)</Command>
<Timeout Condition="'$(WorkItemTimeout)' != ''">$(WorkItemTimeout)</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup>
<XUnitProject Include="$(XUnitProjects.Split(';'))">
<Arguments />
</XUnitProject>
</ItemGroup>
</Project>
-3
View File
@@ -1,3 +0,0 @@
@echo off
powershell -NoProfile -NoLogo -ExecutionPolicy ByPass -command "& """%~dp0init-tools-native.ps1""" %*"
exit /b %ErrorLevel%
-142
View File
@@ -1,142 +0,0 @@
<#
.SYNOPSIS
Entry point script for installing native tools
.DESCRIPTION
Reads $RepoRoot\global.json file to determine native assets to install
and executes installers for those tools
.PARAMETER BaseUri
Base file directory or Url from which to acquire tool archives
.PARAMETER InstallDirectory
Directory to install native toolset. This is a command-line override for the default
Install directory precedence order:
- InstallDirectory command-line override
- NETCOREENG_INSTALL_DIRECTORY environment variable
- (default) %USERPROFILE%/.netcoreeng/native
.PARAMETER Clean
Switch specifying to not install anything, but cleanup native asset folders
.PARAMETER Force
Clean and then install tools
.PARAMETER DownloadRetries
Total number of retry attempts
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds
.PARAMETER GlobalJsonFile
File path to global.json file
.NOTES
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30,
[string] $GlobalJsonFile
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
}
Set-StrictMode -version 2.0
$ErrorActionPreference="Stop"
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
# Process tools list
Write-Host "Processing $GlobalJsonFile"
If (-Not (Test-Path $GlobalJsonFile)) {
Write-Host "Unable to find '$GlobalJsonFile'"
exit 0
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
$ToolVersion = $_.Value
$LocalInstallerCommand = $InstallerPath
$LocalInstallerCommand += " -ToolName $ToolName"
$LocalInstallerCommand += " -InstallPath $InstallBin"
$LocalInstallerCommand += " -BaseUri $BaseUri"
$LocalInstallerCommand += " -CommonLibraryDirectory $EngCommonBaseDir"
$LocalInstallerCommand += " -Version $ToolVersion"
if ($Verbose) {
$LocalInstallerCommand += " -Verbose"
}
if (Get-Variable 'Force' -ErrorAction 'SilentlyContinue') {
if($Force) {
$LocalInstallerCommand += " -Force"
}
}
if ($Clean) {
$LocalInstallerCommand += " -Clean"
}
Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$LocalInstallerCommand'"
Invoke-Expression "$LocalInstallerCommand"
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
Write-Warning $errMsg
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
exit 1
}
}
else {
Write-Host "No native tools defined in global.json"
exit 0
}
if ($Clean) {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
exit 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
}
-141
View File
@@ -1,141 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
base_uri='https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
install_directory=''
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/native/common-library.sh
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installdirectory)
install_directory=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --installdirectory Directory to install native toolset."
echo " This is a command-line override for the default"
echo " Install directory precedence order:"
echo " - InstallDirectory command-line override"
echo " - NETCOREENG_INSTALL_DIRECTORY environment variable"
echo " - (default) %USERPROFILE%/.netcoreeng/native"
echo ""
echo " --clean Switch specifying not to install anything, but cleanup native asset folders"
echo " --force Clean and then install tools"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --baseuri <value> Base URI for where to download native tools from"
echo " --downloadretries <value> Number of times a download should be attempted"
echo " --retrywaittimeseconds <value> Wait time between download attempts"
echo ""
exit 0
;;
esac
done
function ReadGlobalJsonNativeTools {
# Get the native-tools section from the global.json.
local native_tools_section=$(cat $global_json_file | awk '/"native-tools"/,/}/')
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[:space:]')"
local old_IFS=$IFS
while read -r line; do
# Lines are of the form: 'tool:version'
IFS=:
while read -r key value; do
native_assets[$key]=$value
done <<< "$line"
done <<< "$native_tools_list"
IFS=$old_IFS
return 0;
}
native_base_dir=$install_directory
if [[ -z $install_directory ]]; then
native_base_dir=$(GetNativeInstallDirectory)
fi
install_bin="${native_base_dir}/bin"
ReadGlobalJsonNativeTools
if [[ ${#native_assets[@]} -eq 0 ]]; then
echo "No native tools defined in global.json"
exit 0;
else
native_installer_dir="$scriptroot/native"
for tool in "${!native_assets[@]}"
do
tool_version=${native_assets[$tool]}
installer_name="install-$tool.sh"
installer_command="$native_installer_dir/$installer_name"
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
if [[ $force = true ]]; then
installer_command+=" --force"
fi
if [[ $clean = true ]]; then
installer_command+=" --clean"
fi
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
exit 1
fi
done
fi
if [[ $clean = true ]]; then
exit 0
fi
if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
echo "Native tools install directory does not exist, installation failed" >&2
exit 1
fi
exit 0
@@ -1,4 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project>
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
</Project>
-27
View File
@@ -1,27 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) Microsoft. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information. -->
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net472</TargetFramework>
<ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
</PropertyGroup>
<ItemGroup>
<!-- Clear references, the SDK may add some depending on UsuingToolXxx settings, but we only want to restore the following -->
<PackageReference Remove="@(PackageReference)"/>
<PackageReference Include="Microsoft.DotNet.IBCMerge" Version="$(MicrosoftDotNetIBCMergeVersion)" Condition="'$(UsingToolIbcOptimization)' == 'true'" />
<PackageReference Include="Drop.App" Version="$(DropAppVersion)" ExcludeAssets="all" Condition="'$(UsingToolVisualStudioIbcTraining)' == 'true'"/>
</ItemGroup>
<PropertyGroup>
<RestoreSources></RestoreSources>
<RestoreSources Condition="'$(UsingToolIbcOptimization)' == 'true'">
https://devdiv.pkgs.visualstudio.com/_packaging/dotnet-core-internal-tooling/nuget/v3/index.json;
</RestoreSources>
<RestoreSources Condition="'$(UsingToolVisualStudioIbcTraining)' == 'true'">
$(RestoreSources);
https://devdiv.pkgs.visualstudio.com/_packaging/VS/nuget/v3/index.json;
</RestoreSources>
</PropertyGroup>
<!-- Repository extensibility point -->
<Import Project="$(RepositoryEngineeringDir)InternalTools.props" Condition="Exists('$(RepositoryEngineeringDir)InternalTools.props')" />
</Project>
-27
View File
@@ -1,27 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
[switch] $prepareMachine,
[Parameter(ValueFromRemainingArguments=$true)][String[]]$extraArgs
)
. $PSScriptRoot\tools.ps1
try {
if ($ci) {
$nodeReuse = $false
}
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity='minimal'
warn_as_error=true
node_reuse=true
prepare_machine=false
extra_args=''
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--verbosity)
verbosity=$2
shift 2
;;
--warnaserror)
warn_as_error=$2
shift 2
;;
--nodereuse)
node_reuse=$2
shift 2
;;
--ci)
ci=true
shift 1
;;
--preparemachine)
prepare_machine=true
shift 1
;;
*)
extra_args="$extra_args $1"
shift 1
;;
esac
done
. "$scriptroot/tools.sh"
if [[ "$ci" == true ]]; then
node_reuse=false
fi
MSBuild $extra_args
ExitWithExitCode 0
-358
View File
@@ -1,358 +0,0 @@
<#
.SYNOPSIS
Helper module to install an archive to a directory
.DESCRIPTION
Helper module to download and extract an archive to a specified directory
.PARAMETER Uri
Uri of artifact to download
.PARAMETER InstallDirectory
Directory to extract artifact contents to
.PARAMETER Force
Force download / extraction if file or contents already exist. Default = False
.PARAMETER DownloadRetries
Total number of retry attempts. Default = 5
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds. Default = 30
.NOTES
Returns False if download or extraction fail, True otherwise
#>
function DownloadAndExtract {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Uri,
[Parameter(Mandatory=$True)]
[string] $InstallDirectory,
[switch] $Force = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30
)
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$TempToolPath = CommonLibrary\Get-TempPathFilename -Path $Uri
# Download native tool
$DownloadStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries $DownloadRetries `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$Force `
-Verbose:$Verbose
if ($DownloadStatus -Eq $False) {
Write-Error "Download failed"
return $False
}
# Extract native tool
$UnzipStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$Force `
-Verbose:$Verbose
if ($UnzipStatus -Eq $False) {
Write-Error "Unzip failed"
return $False
}
return $True
}
<#
.SYNOPSIS
Download a file, retry on failure
.DESCRIPTION
Download specified file and retry if attempt fails
.PARAMETER Uri
Uri of file to download. If Uri is a local path, the file will be copied instead of downloaded
.PARAMETER Path
Path to download or copy uri file to
.PARAMETER Force
Overwrite existing file if present. Default = False
.PARAMETER DownloadRetries
Total number of retry attempts. Default = 5
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds Default = 30
#>
function Get-File {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Uri,
[Parameter(Mandatory=$True)]
[string] $Path,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30,
[switch] $Force = $False
)
$Attempt = 0
if ($Force) {
if (Test-Path $Path) {
Remove-Item $Path -Force
}
}
if (Test-Path $Path) {
Write-Host "File '$Path' already exists, skipping download"
return $True
}
$DownloadDirectory = Split-Path -ErrorAction Ignore -Path "$Path" -Parent
if (-Not (Test-Path $DownloadDirectory)) {
New-Item -path $DownloadDirectory -force -itemType "Directory" | Out-Null
}
if (Test-Path -IsValid -Path $Uri) {
Write-Verbose "'$Uri' is a file path, copying file to '$Path'"
Copy-Item -Path $Uri -Destination $Path
return $?
}
else {
Write-Verbose "Downloading $Uri"
while($Attempt -Lt $DownloadRetries)
{
try {
Invoke-WebRequest -UseBasicParsing -Uri $Uri -OutFile $Path
Write-Verbose "Downloaded to '$Path'"
return $True
}
catch {
$Attempt++
if ($Attempt -Lt $DownloadRetries) {
$AttemptsLeft = $DownloadRetries - $Attempt
Write-Warning "Download failed, $AttemptsLeft attempts remaining, will retry in $RetryWaitTimeInSeconds seconds"
Start-Sleep -Seconds $RetryWaitTimeInSeconds
}
else {
Write-Error $_
Write-Error $_.Exception
}
}
}
}
return $False
}
<#
.SYNOPSIS
Generate a shim for a native tool
.DESCRIPTION
Creates a wrapper script (shim) that passes arguments forward to native tool assembly
.PARAMETER ShimName
The name of the shim
.PARAMETER ShimDirectory
The directory where shims are stored
.PARAMETER ToolFilePath
Path to file that shim forwards to
.PARAMETER Force
Replace shim if already present. Default = False
.NOTES
Returns $True if generating shim succeeds, $False otherwise
#>
function New-ScriptShim {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ShimName,
[Parameter(Mandatory=$True)]
[string] $ShimDirectory,
[Parameter(Mandatory=$True)]
[string] $ToolFilePath,
[Parameter(Mandatory=$True)]
[string] $BaseUri,
[switch] $Force
)
try {
Write-Verbose "Generating '$ShimName' shim"
if (-Not (Test-Path $ToolFilePath)){
Write-Error "Specified tool file path '$ToolFilePath' does not exist"
return $False
}
# WinShimmer is a small .NET Framework program that creates .exe shims to bootstrapped programs
# Many of the checks for installed programs expect a .exe extension for Windows tools, rather
# than a .bat or .cmd file.
# Source: https://github.com/dotnet/arcade/tree/master/src/WinShimmer
if (-Not (Test-Path "$ShimDirectory\WinShimmer\winshimmer.exe")) {
$InstallStatus = DownloadAndExtract -Uri "$BaseUri/windows/winshimmer/WinShimmer.zip" `
-InstallDirectory $ShimDirectory\WinShimmer `
-Force:$Force `
-DownloadRetries 2 `
-RetryWaitTimeInSeconds 5 `
-Verbose:$Verbose
}
if ((Test-Path (Join-Path $ShimDirectory "$ShimName.exe"))) {
Write-Host "$ShimName.exe already exists; replacing..."
Remove-Item (Join-Path $ShimDirectory "$ShimName.exe")
}
Invoke-Expression "$ShimDirectory\WinShimmer\winshimmer.exe $ShimName $ToolFilePath $ShimDirectory"
return $True
}
catch {
Write-Host $_
Write-Host $_.Exception
return $False
}
}
<#
.SYNOPSIS
Returns the machine architecture of the host machine
.NOTES
Returns 'x64' on 64 bit machines
Returns 'x86' on 32 bit machines
#>
function Get-MachineArchitecture {
$ProcessorArchitecture = $Env:PROCESSOR_ARCHITECTURE
$ProcessorArchitectureW6432 = $Env:PROCESSOR_ARCHITEW6432
if($ProcessorArchitecture -Eq "X86")
{
if(($ProcessorArchitectureW6432 -Eq "") -Or
($ProcessorArchitectureW6432 -Eq "X86")) {
return "x86"
}
$ProcessorArchitecture = $ProcessorArchitectureW6432
}
if (($ProcessorArchitecture -Eq "AMD64") -Or
($ProcessorArchitecture -Eq "IA64") -Or
($ProcessorArchitecture -Eq "ARM64")) {
return "x64"
}
return "x86"
}
<#
.SYNOPSIS
Get the name of a temporary folder under the native install directory
#>
function Get-TempDirectory {
return Join-Path (Get-NativeInstallDirectory) "temp/"
}
function Get-TempPathFilename {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $Path
)
$TempDir = CommonLibrary\Get-TempDirectory
$TempFilename = Split-Path $Path -leaf
$TempPath = Join-Path $TempDir $TempFilename
return $TempPath
}
<#
.SYNOPSIS
Returns the base directory to use for native tool installation
.NOTES
Returns the value of the NETCOREENG_INSTALL_DIRECTORY if that environment variable
is set, or otherwise returns an install directory under the %USERPROFILE%
#>
function Get-NativeInstallDirectory {
$InstallDir = $Env:NETCOREENG_INSTALL_DIRECTORY
if (!$InstallDir) {
$InstallDir = Join-Path $Env:USERPROFILE ".netcoreeng/native/"
}
return $InstallDir
}
<#
.SYNOPSIS
Unzip an archive
.DESCRIPTION
Powershell module to unzip an archive to a specified directory
.PARAMETER ZipPath (Required)
Path to archive to unzip
.PARAMETER OutputDirectory (Required)
Output directory for archive contents
.PARAMETER Force
Overwrite output directory contents if they already exist
.NOTES
- Returns True and does not perform an extraction if output directory already exists but Overwrite is not True.
- Returns True if unzip operation is successful
- Returns False if Overwrite is True and it is unable to remove contents of OutputDirectory
- Returns False if unable to extract zip archive
#>
function Expand-Zip {
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ZipPath,
[Parameter(Mandatory=$True)]
[string] $OutputDirectory,
[switch] $Force
)
Write-Verbose "Extracting '$ZipPath' to '$OutputDirectory'"
try {
if ((Test-Path $OutputDirectory) -And (-Not $Force)) {
Write-Host "Directory '$OutputDirectory' already exists, skipping extract"
return $True
}
if (Test-Path $OutputDirectory) {
Write-Verbose "'Force' is 'True', but '$OutputDirectory' exists, removing directory"
Remove-Item $OutputDirectory -Force -Recurse
if ($? -Eq $False) {
Write-Error "Unable to remove '$OutputDirectory'"
return $False
}
}
if (-Not (Test-Path $OutputDirectory)) {
New-Item -path $OutputDirectory -Force -itemType "Directory" | Out-Null
}
Add-Type -assembly "system.io.compression.filesystem"
[io.compression.zipfile]::ExtractToDirectory("$ZipPath", "$OutputDirectory")
if ($? -Eq $False) {
Write-Error "Unable to extract '$ZipPath'"
return $False
}
}
catch {
Write-Host $_
Write-Host $_.Exception
return $False
}
return $True
}
export-modulemember -function DownloadAndExtract
export-modulemember -function Expand-Zip
export-modulemember -function Get-File
export-modulemember -function Get-MachineArchitecture
export-modulemember -function Get-NativeInstallDirectory
export-modulemember -function Get-TempDirectory
export-modulemember -function Get-TempPathFilename
export-modulemember -function New-ScriptShim
-168
View File
@@ -1,168 +0,0 @@
#!/usr/bin/env bash
function GetNativeInstallDirectory {
local install_dir
if [[ -z $NETCOREENG_INSTALL_DIRECTORY ]]; then
install_dir=$HOME/.netcoreeng/native/
else
install_dir=$NETCOREENG_INSTALL_DIRECTORY
fi
echo $install_dir
return 0
}
function GetTempDirectory {
echo $(GetNativeInstallDirectory)temp/
return 0
}
function ExpandZip {
local zip_path=$1
local output_directory=$2
local force=${3:-false}
echo "Extracting $zip_path to $output_directory"
if [[ -d $output_directory ]] && [[ $force = false ]]; then
echo "Directory '$output_directory' already exists, skipping extract"
return 0
fi
if [[ -d $output_directory ]]; then
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
echo Unable to remove '$output_directory'>&2
return 1
fi
fi
echo "Creating directory: '$output_directory'"
mkdir -p $output_directory
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
echo "Unable to extract '$zip_path'" >&2
return 1
fi
return 0
}
function GetCurrentOS {
local unameOut="$(uname -s)"
case $unameOut in
Linux*) echo "Linux";;
Darwin*) echo "MacOS";;
esac
return 0
}
function GetFile {
local uri=$1
local path=$2
local force=${3:-false}
local download_retries=${4:-5}
local retry_wait_time_seconds=${5:-30}
if [[ -f $path ]]; then
if [[ $force = false ]]; then
echo "File '$path' already exists. Skipping download"
return 0
else
rm -rf $path
fi
fi
if [[ -f $uri ]]; then
echo "'$uri' is a file path, copying file to '$path'"
cp $uri $path
return $?
fi
echo "Downloading $uri"
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$uri" -sSL --retry $download_retries --retry-delay $retry_wait_time_seconds --create-dirs -o "$path" --fail
else
wget -q -O "$path" "$uri" --tries="$download_retries"
fi
return $?
}
function GetTempPathFileName {
local path=$1
local temp_dir=$(GetTempDirectory)
local temp_file_name=$(basename $path)
echo $temp_dir$temp_file_name
return 0
}
function DownloadAndExtract {
local uri=$1
local installDir=$2
local force=${3:-false}
local download_retries=${4:-5}
local retry_wait_time_seconds=${5:-30}
local temp_tool_path=$(GetTempPathFileName $uri)
echo "downloading to: $temp_tool_path"
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
return 1
fi
# Extract File
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
return 1
fi
return 0
}
function NewScriptShim {
local shimpath=$1
local tool_file_path=$2
local force=${3:-false}
echo "Generating '$shimpath' shim"
if [[ -f $shimpath ]]; then
if [[ $force = false ]]; then
echo "File '$shimpath' already exists." >&2
return 1
else
rm -rf $shimpath
fi
fi
if [[ ! -f $tool_file_path ]]; then
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
return 1
fi
local shim_contents=$'#!/usr/bin/env bash\n'
shim_contents+="SHIMARGS="$'$1\n'
shim_contents+="$tool_file_path"$' $SHIMARGS\n'
# Write shim file
echo "$shim_contents" > $shimpath
chmod +x $shimpath
echo "Finished generating shim '$shimpath'"
return $?
}
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/cmake/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
exit 1
fi
exit 0
-130
View File
@@ -1,130 +0,0 @@
<#
.SYNOPSIS
Install native tool
.DESCRIPTION
Install cmake native tool from Azure blob storage
.PARAMETER InstallPath
Base directory to install native tool to
.PARAMETER BaseUri
Base file directory or Url from which to acquire tool archives
.PARAMETER CommonLibraryDirectory
Path to folder containing common library modules
.PARAMETER Force
Force install of tools even if they previously exist
.PARAMETER Clean
Don't install the tool, just clean up the current install of the tool
.PARAMETER DownloadRetries
Total number of retry attempts
.PARAMETER RetryWaitTimeInSeconds
Wait time between retry attempts in seconds
.NOTES
Returns 0 if install succeeds, 1 otherwise
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[Parameter(Mandatory=$True)]
[string] $ToolName,
[Parameter(Mandatory=$True)]
[string] $InstallPath,
[Parameter(Mandatory=$True)]
[string] $BaseUri,
[Parameter(Mandatory=$True)]
[string] $Version,
[string] $CommonLibraryDirectory = $PSScriptRoot,
[switch] $Force = $False,
[switch] $Clean = $False,
[int] $DownloadRetries = 5,
[int] $RetryWaitTimeInSeconds = 30
)
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$Arch = CommonLibrary\Get-MachineArchitecture
$ToolOs = "win64"
if($Arch -Eq "x32") {
$ToolOs = "win32"
}
$ToolNameMoniker = "$ToolName-$Version-$ToolOs-$Arch"
$ToolInstallDirectory = Join-Path $InstallPath "$ToolName\$Version\"
$Uri = "$BaseUri/windows/$ToolName/$ToolNameMoniker.zip"
$ShimPath = Join-Path $InstallPath "$ToolName.exe"
if ($Clean) {
Write-Host "Cleaning $ToolInstallDirectory"
if (Test-Path $ToolInstallDirectory) {
Remove-Item $ToolInstallDirectory -Force -Recurse
}
Write-Host "Cleaning $ShimPath"
if (Test-Path $ShimPath) {
Remove-Item $ShimPath -Force
}
$ToolTempPath = CommonLibrary\Get-TempPathFilename -Path $Uri
Write-Host "Cleaning $ToolTempPath"
if (Test-Path $ToolTempPath) {
Remove-Item $ToolTempPath -Force
}
exit 0
}
# Install tool
if ((Test-Path $ToolInstallDirectory) -And (-Not $Force)) {
Write-Verbose "$ToolName ($Version) already exists, skipping install"
}
else {
$InstallStatus = CommonLibrary\DownloadAndExtract -Uri $Uri `
-InstallDirectory $ToolInstallDirectory `
-Force:$Force `
-DownloadRetries $DownloadRetries `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-Error "Installation failed"
exit 1
}
}
$ToolFilePath = Get-ChildItem $ToolInstallDirectory -Recurse -Filter "$ToolName.exe" | % { $_.FullName }
if (@($ToolFilePath).Length -Gt 1) {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Error "$ToolName was not found in $ToolFilePath."
exit 1
}
# Generate shim
# Always rewrite shims so that we are referencing the expected version
$GenerateShimStatus = CommonLibrary\New-ScriptShim -ShimName $ToolName `
-ShimDirectory $InstallPath `
-ToolFilePath "$ToolFilePath" `
-BaseUri $BaseUri `
-Force:$Force `
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-Error "Generate shim failed"
return 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
}
-79
View File
@@ -1,79 +0,0 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $configuration = "Debug",
[string] $task,
[string] $verbosity = "minimal",
[string] $msbuildEngine = $null,
[switch] $restore,
[switch] $prepareMachine,
[switch] $help,
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
$ci = $true
$binaryLog = $true
$warnAsError = $true
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -task <value> Name of Arcade task (name of a project in SdkTasks directory of the Arcade SDK package)"
Write-Host " -restore Restore dependencies"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -prepareMachine Prepare machine for CI run"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
}
function Build([string]$target) {
$logSuffix = if ($target -eq "Execute") { "" } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
MSBuild $taskProject `
/bl:$log `
/t:$target `
/p:Configuration=$configuration `
/p:RepoRoot=$RepoRoot `
/p:BaseIntermediateOutputPath=$outputPath `
@properties
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
Print-Usage
exit 0
}
if ($task -eq "") {
Write-Host "Missing required parameter '-task <value>'" -ForegroundColor Red
Print-Usage
ExitWithExitCode 1
}
$taskProject = GetSdkTaskProject $task
if (!(Test-Path $taskProject)) {
Write-Host "Unknown task: $task" -ForegroundColor Red
ExitWithExitCode 1
}
if ($restore) {
Build "Restore"
}
Build "Execute"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
ExitWithExitCode 1
}
ExitWithExitCode 0
@@ -1,48 +0,0 @@
parameters:
# Optional: dependencies of the job
dependsOn: ''
# Optional: A defined YAML pool - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#pool
pool: {}
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
jobs:
- job: Generate_Graph_Files
dependsOn: ${{ parameters.dependsOn }}
displayName: Generate Graph Files
pool: ${{ parameters.pool }}
variables:
# Publish-Build-Assets provides: MaestroAccessToken, BotAccount-dotnet-maestro-bot-PAT
# DotNet-AllOrgs-Darc-Pats provides: dn-bot-devdiv-dnceng-rw-code-pat
- group: Publish-Build-Assets
- group: DotNet-AllOrgs-Darc-Pats
- name: _GraphArguments
value: -gitHubPat $(BotAccount-dotnet-maestro-bot-PAT)
-azdoPat $(dn-bot-devdiv-dnceng-rw-code-pat)
-barToken $(MaestroAccessToken)
-outputFolder '$(Build.StagingDirectory)/GraphFiles/'
- ${{ if ne(parameters.includeToolset, 'false') }}:
- name: _GraphArguments
value: ${{ variables._GraphArguments }} -includeToolset
steps:
- task: PowerShell@2
displayName: Generate Graph Files
inputs:
filePath: eng\common\generate-graph-files.ps1
arguments: $(_GraphArguments)
continueOnError: true
- task: PublishBuildArtifacts@1
displayName: Publish Graph to Artifacts
inputs:
PathtoPublish: '$(Build.StagingDirectory)/GraphFiles'
PublishLocation: Container
ArtifactName: GraphFiles
continueOnError: true
condition: always()
-205
View File
@@ -1,205 +0,0 @@
parameters:
# Job schema parameters - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
cancelTimeoutInMinutes: ''
condition: ''
continueOnError: false
container: ''
dependsOn: ''
displayName: ''
steps: []
pool: ''
strategy: ''
timeoutInMinutes: ''
variables: []
workspace: ''
# Job base template specific parameters
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
enableTelemetry: false
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: define the helix type for telemetry (example: 'build/product/')
helixType: ''
# Required: name of the job
name: ''
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- job: ${{ parameters.name }}
${{ if ne(parameters.cancelTimeoutInMinutes, '') }}:
cancelTimeoutInMinutes: ${{ parameters.cancelTimeoutInMinutes }}
${{ if ne(parameters.condition, '') }}:
condition: ${{ parameters.condition }}
${{ if ne(parameters.container, '') }}:
container: ${{ parameters.container }}
${{ if ne(parameters.continueOnError, '') }}:
continueOnError: ${{ parameters.continueOnError }}
${{ if ne(parameters.dependsOn, '') }}:
dependsOn: ${{ parameters.dependsOn }}
${{ if ne(parameters.displayName, '') }}:
displayName: ${{ parameters.displayName }}
${{ if ne(parameters.pool, '') }}:
pool: ${{ parameters.pool }}
${{ if ne(parameters.strategy, '') }}:
strategy: ${{ parameters.strategy }}
${{ if ne(parameters.timeoutInMinutes, '') }}:
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- ${{ each variable in parameters.variables }}:
# handle name-value variable syntax
# example:
# - name: [key]
# value: [value]
- ${{ if ne(variable.name, '') }}:
- name: ${{ variable.name }}
value: ${{ variable.value }}
# handle variable groups
- ${{ if ne(variable.group, '') }}:
- group: ${{ variable.group }}
# handle key-value variable syntax.
# example:
# - [key]: [value]
- ${{ if and(eq(variable.name, ''), eq(variable.group, '')) }}:
- ${{ each pair in variable }}:
- name: ${{ pair.key }}
value: ${{ pair.value }}
# DotNet-HelixApi-Access provides 'HelixApiAccessToken' for internal builds
- ${{ if and(eq(parameters.enableTelemetry, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- group: DotNet-HelixApi-Access
${{ if ne(parameters.workspace, '') }}:
workspace: ${{ parameters.workspace }}
steps:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendStartTelemetry@0
displayName: 'Send Helix Start Telemetry'
inputs:
helixRepo: ${{ parameters.helixRepo }}
${{ if ne(parameters.helixType, '') }}:
helixType: ${{ parameters.helixType }}
buildConfig: $(_BuildConfig)
runAsPublic: ${{ parameters.runAsPublic }}
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
signType: $(_SignType)
zipSources: false
feedSource: https://dnceng.pkgs.visualstudio.com/_packaging/MicroBuildToolset/nuget/v3/index.json
env:
TeamName: $(_TeamName)
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- ${{ each step in parameters.steps }}:
- ${{ step }}
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildCleanup@1
displayName: Execute Microbuild cleanup tasks
condition: and(always(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
env:
TeamName: $(_TeamName)
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendEndTelemetry@0
displayName: 'Send Helix End Telemetry'
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_$(Agent.JobName)
continueOnError: true
condition: always()
- ${{ if eq(parameters.enablePublishTestResults, 'true') }}:
- task: PublishTestResults@2
displayName: Publish Test Results
inputs:
testResultsFormat: 'xUnit'
testResultsFiles: '*.xml'
searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults/$(_BuildConfig)'
continueOnError: true
condition: always()
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(variables['_PublishUsingPipelines'], 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.StagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
@@ -1,70 +0,0 @@
parameters:
configuration: 'Debug'
# Optional: condition for the job to run
condition: ''
# Optional: 'true' if future jobs should run even if this job fails
continueOnError: false
# Optional: dependencies of the job
dependsOn: ''
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: A defined YAML pool - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#pool
pool: {}
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
# Optional: whether the build's artifacts will be published using release pipelines or direct feed publishing
publishUsingPipelines: false
jobs:
- job: Asset_Registry_Publish
dependsOn: ${{ parameters.dependsOn }}
displayName: Publish to Build Asset Registry
pool: ${{ parameters.pool }}
variables:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- name: _BuildConfig
value: ${{ parameters.configuration }}
- group: Publish-Build-Assets
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: DownloadBuildArtifacts@0
displayName: Download artifact
inputs:
artifactName: AssetManifests
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishBuildAssets -restore -msbuildEngine dotnet
/p:ManifestsPath='$(Build.StagingDirectory)/Download/AssetManifests'
/p:BuildAssetRegistryToken=$(MaestroAccessToken)
/p:MaestroApiEndpoint=https://maestro-prod.westus2.cloudapp.azure.com
/p:PublishUsingPipelines=${{ parameters.publishUsingPipelines }}
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_PublishBuildAssets
continueOnError: true
condition: always()
-90
View File
@@ -1,90 +0,0 @@
parameters:
# Optional: 'true' if failures in job.yml job should not fail the job
continueOnError: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Enable publishing using release pipelines
enablePublishUsingPipelines: false
graphFileGeneration:
# Optional: Enable generating the graph files at the end of the build
enabled: false
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
# if enabled then the 'helixRepo' parameter should also be specified
enableTelemetry: false
# Required: A collection of jobs to run - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
jobs: []
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: Override automatically derived dependsOn value for "publish build assets" job
publishBuildAssetsDependsOn: ''
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- ${{ each job in parameters.jobs }}:
- template: ../job/job.yml
parameters:
# pass along parameters
${{ each parameter in parameters }}:
${{ if ne(parameter.key, 'jobs') }}:
${{ parameter.key }}: ${{ parameter.value }}
# pass along job properties
${{ each property in job }}:
${{ if ne(property.key, 'job') }}:
${{ property.key }}: ${{ property.value }}
name: ${{ job.job }}
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if and(eq(parameters.graphFileGeneration.enabled, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
-130
View File
@@ -1,130 +0,0 @@
parameters:
# Optional: Clean sources before building
clean: true
# Optional: Git fetch depth
fetchDepth: ''
# Optional: name of the phase (not specifying phase name may cause name collisions)
name: ''
# Optional: display name of the phase
displayName: ''
# Optional: condition for the job to run
condition: ''
# Optional: dependencies of the phase
dependsOn: ''
# Required: A defined YAML queue
queue: {}
# Required: build steps
steps: []
# Optional: variables
variables: {}
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
runAsPublic: false
## Telemetry variables
# Optional: enable sending telemetry
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _HelixBuildConfig - differentiate between Debug, Release, other
# _HelixSource - Example: build/product
# _HelixType - Example: official/dotnet/arcade/$(Build.SourceBranch)
enableTelemetry: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
phases:
- phase: ${{ parameters.name }}
${{ if ne(parameters.displayName, '') }}:
displayName: ${{ parameters.displayName }}
${{ if ne(parameters.condition, '') }}:
condition: ${{ parameters.condition }}
${{ if ne(parameters.dependsOn, '') }}:
dependsOn: ${{ parameters.dependsOn }}
queue: ${{ parameters.queue }}
${{ if ne(parameters.variables, '') }}:
variables:
${{ insert }}: ${{ parameters.variables }}
steps:
- checkout: self
clean: ${{ parameters.clean }}
${{ if ne(parameters.fetchDepth, '') }}:
fetchDepth: ${{ parameters.fetchDepth }}
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- template: /eng/common/templates/steps/telemetry-start.yml
parameters:
buildConfig: $(_HelixBuildConfig)
helixSource: $(_HelixSource)
helixType: $(_HelixType)
runAsPublic: ${{ parameters.runAsPublic }}
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
# Internal only resource, and Microbuild signing shouldn't be applied to PRs.
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
signType: $(_SignType)
zipSources: false
feedSource: https://dnceng.pkgs.visualstudio.com/_packaging/MicroBuildToolset/nuget/v3/index.json
env:
TeamName: $(_TeamName)
continueOnError: false
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
# Run provided build steps
- ${{ parameters.steps }}
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
# Internal only resources
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildCleanup@1
displayName: Execute Microbuild cleanup tasks
condition: and(always(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
env:
TeamName: $(_TeamName)
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- template: /eng/common/templates/steps/telemetry-end.yml
parameters:
helixSource: $(_HelixSource)
helixType: $(_HelixType)
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: false
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.StagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: false
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
@@ -1,51 +0,0 @@
parameters:
dependsOn: ''
queue: {}
configuration: 'Debug'
condition: succeeded()
continueOnError: false
runAsPublic: false
publishUsingPipelines: false
phases:
- phase: Asset_Registry_Publish
displayName: Publish to Build Asset Registry
dependsOn: ${{ parameters.dependsOn }}
queue: ${{ parameters.queue }}
variables:
_BuildConfig: ${{ parameters.configuration }}
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: DownloadBuildArtifacts@0
displayName: Download artifact
inputs:
artifactName: AssetManifests
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: AzureKeyVault@1
inputs:
azureSubscription: 'DotNet-Engineering-Services_KeyVault'
KeyVaultName: EngKeyVault
SecretsFilter: 'MaestroAccessToken'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishBuildAssets -restore -msbuildEngine dotnet
/p:ManifestsPath='$(Build.StagingDirectory)/Download/AssetManifests'
/p:BuildAssetRegistryToken=$(MaestroAccessToken)
/p:MaestroApiEndpoint=https://maestro-prod.westus2.cloudapp.azure.com
/p:PublishUsingPipelines=${{ parameters.publishUsingPipelines }}
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_Asset_Registry_Publish
continueOnError: true
condition: always()
@@ -1,12 +0,0 @@
# build-reason.yml
# Description: runs steps if build.reason condition is valid. conditions is a string of valid build reasons
# to include steps (',' separated).
parameters:
conditions: ''
steps: []
steps:
- ${{ if and( not(startsWith(parameters.conditions, 'not')), contains(parameters.conditions, variables['build.reason'])) }}:
- ${{ parameters.steps }}
- ${{ if and( startsWith(parameters.conditions, 'not'), not(contains(parameters.conditions, variables['build.reason']))) }}:
- ${{ parameters.steps }}
@@ -1,7 +0,0 @@
parameters:
agentOs: ''
steps: []
steps:
- ${{ if ne(parameters.agentOs, 'Windows_NT') }}:
- ${{ parameters.steps }}
@@ -1,7 +0,0 @@
parameters:
agentOs: ''
steps: []
steps:
- ${{ if eq(parameters.agentOs, 'Windows_NT') }}:
- ${{ parameters.steps }}
@@ -1,33 +0,0 @@
parameters:
# if parameter1 equals parameter 2, run 'ifScript' command, else run 'elsescript' command
parameter1: ''
parameter2: ''
ifScript: ''
elseScript: ''
# name of script step
name: Script
# display name of script step
displayName: If-Equal-Else Script
# environment
env: {}
# conditional expression for step execution
condition: ''
steps:
- ${{ if and(ne(parameters.ifScript, ''), eq(parameters.parameter1, parameters.parameter2)) }}:
- script: ${{ parameters.ifScript }}
name: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
env: ${{ parameters.env }}
condition: ${{ parameters.condition }}
- ${{ if and(ne(parameters.elseScript, ''), ne(parameters.parameter1, parameters.parameter2)) }}:
- script: ${{ parameters.elseScript }}
name: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
env: ${{ parameters.env }}
condition: ${{ parameters.condition }}
@@ -1,91 +0,0 @@
# Please remember to update the documentation if you make changes to these parameters!
parameters:
HelixSource: 'pr/default' # required -- sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'tests/default/' # required -- Helix telemetry which identifies what type of data this is; should include "test" for clarity and must end in '/'
HelixBuild: $(Build.BuildNumber) # required -- the build number Helix will use to identify this -- automatically set to the AzDO build number
HelixTargetQueues: '' # required -- semicolon delimited list of Helix queues to test on; see https://helix.dot.net/ for a list of queues
HelixAccessToken: '' # required -- access token to make Helix API requests; should be provided by the appropriate variable group
HelixConfiguration: '' # optional -- additional property attached to a job
HelixPreCommands: '' # optional -- commands to run before Helix work item execution
HelixPostCommands: '' # optional -- commands to run after Helix work item execution
WorkItemDirectory: '' # optional -- a payload directory to zip up and send to Helix; requires WorkItemCommand; incompatible with XUnitProjects
WorkItemCommand: '' # optional -- a command to execute on the payload; requires WorkItemDirectory; incompatible with XUnitProjects
WorkItemTimeout: '' # optional -- a timeout in seconds for the work item command; requires WorkItemDirectory; incompatible with XUnitProjects
CorrelationPayloadDirectory: '' # optional -- a directory to zip up and send to Helix as a correlation payload
XUnitProjects: '' # optional -- semicolon delimited list of XUnitProjects to parse and send to Helix; requires XUnitRuntimeTargetFramework, XUnitPublishTargetFramework, XUnitRunnerVersion, and IncludeDotNetCli=true
XUnitWorkItemTimeout: '' # optional -- the workitem timeout in seconds for all workitems created from the xUnit projects specified by XUnitProjects
XUnitPublishTargetFramework: '' # optional -- framework to use to publish your xUnit projects
XUnitRuntimeTargetFramework: '' # optional -- framework to use for the xUnit console runner
XUnitRunnerVersion: '' # optional -- version of the xUnit nuget package you wish to use on Helix; required for XUnitProjects
IncludeDotNetCli: false # optional -- true will download a version of the .NET CLI onto the Helix machine as a correlation payload; requires DotNetCliPackageType and DotNetCliVersion
DotNetCliPackageType: '' # optional -- either 'sdk' or 'runtime'; determines whether the sdk or runtime will be sent to Helix; see https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
DotNetCliVersion: '' # optional -- version of the CLI to send to Helix; based on this: https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
continueOnError: false # optional -- determines whether to continue the build if the step errors; defaults to false
steps:
- powershell: 'powershell "$env:BUILD_SOURCESDIRECTORY\eng\common\msbuild.ps1 $env:BUILD_SOURCESDIRECTORY\eng\common\helixpublish.proj /restore /t:Test /bl:$env:BUILD_SOURCESDIRECTORY\artifacts\log\$env:BuildConfig\SendToHelix.binlog"'
displayName: ${{ parameters.DisplayNamePrefix }} (Windows)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixConfiguration: ${{ parameters.HelixConfiguration }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
WorkItemCommand: ${{ parameters.WorkItemCommand }}
WorkItemTimeout: ${{ parameters.WorkItemTimeout }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
XUnitProjects: ${{ parameters.XUnitProjects }}
XUnitWorkItemTimeout: ${{ parameters.XUnitWorkItemTimeout }}
XUnitPublishTargetFramework: ${{ parameters.XUnitPublishTargetFramework }}
XUnitRuntimeTargetFramework: ${{ parameters.XUnitRuntimeTargetFramework }}
XUnitRunnerVersion: ${{ parameters.XUnitRunnerVersion }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $BUILD_SOURCESDIRECTORY/eng/common/msbuild.sh $BUILD_SOURCESDIRECTORY/eng/common/helixpublish.proj /restore /t:Test /bl:$BUILD_SOURCESDIRECTORY/artifacts/log/$BuildConfig/SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Unix)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixConfiguration: ${{ parameters.HelixConfiguration }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
WorkItemCommand: ${{ parameters.WorkItemCommand }}
WorkItemTimeout: ${{ parameters.WorkItemTimeout }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
XUnitProjects: ${{ parameters.XUnitProjects }}
XUnitWorkItemTimeout: ${{ parameters.XUnitWorkItemTimeout }}
XUnitPublishTargetFramework: ${{ parameters.XUnitPublishTargetFramework }}
XUnitRuntimeTargetFramework: ${{ parameters.XUnitRuntimeTargetFramework }}
XUnitRunnerVersion: ${{ parameters.XUnitRunnerVersion }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
@@ -1,102 +0,0 @@
parameters:
maxRetries: 5
retryDelay: 10 # in seconds
steps:
- bash: |
if [ "$AGENT_JOBSTATUS" = "Succeeded" ] || [ "$AGENT_JOBSTATUS" = "PartiallySucceeded" ]; then
errorCount=0
else
errorCount=1
fi
warningCount=0
curlStatus=1
retryCount=0
# retry loop to harden against spotty telemetry connections
# we don't retry successes and 4xx client errors
until [[ $curlStatus -eq 0 || ( $curlStatus -ge 400 && $curlStatus -le 499 ) || $retryCount -ge $MaxRetries ]]
do
if [ $retryCount -gt 0 ]; then
echo "Failed to send telemetry to Helix; waiting $RetryDelay seconds before retrying..."
sleep $RetryDelay
fi
# create a temporary file for curl output
res=`mktemp`
curlResult=`
curl --verbose --output $res --write-out "%{http_code}"\
-H 'Content-Type: application/json' \
-H "X-Helix-Job-Token: $Helix_JobToken" \
-H 'Content-Length: 0' \
-X POST -G "https://helix.dot.net/api/2018-03-14/telemetry/job/build/$Helix_WorkItemId/finish" \
--data-urlencode "errorCount=$errorCount" \
--data-urlencode "warningCount=$warningCount"`
curlStatus=$?
if [ $curlStatus -eq 0 ]; then
if [ $curlResult -gt 299 ] || [ $curlResult -lt 200 ]; then
curlStatus=$curlResult
fi
fi
let retryCount++
done
if [ $curlStatus -ne 0 ]; then
echo "Failed to Send Build Finish information after $retryCount retries"
vstsLogOutput="vso[task.logissue type=error;sourcepath=templates/steps/telemetry-end.yml;code=1;]Failed to Send Build Finish information: $curlStatus"
echo "##$vstsLogOutput"
exit 1
fi
displayName: Send Unix Build End Telemetry
env:
# defined via VSTS variables in start-job.sh
Helix_JobToken: $(Helix_JobToken)
Helix_WorkItemId: $(Helix_WorkItemId)
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(), ne(variables['Agent.Os'], 'Windows_NT'))
- powershell: |
if (($env:Agent_JobStatus -eq 'Succeeded') -or ($env:Agent_JobStatus -eq 'PartiallySucceeded')) {
$ErrorCount = 0
} else {
$ErrorCount = 1
}
$WarningCount = 0
# Basic retry loop to harden against server flakiness
$retryCount = 0
while ($retryCount -lt $env:MaxRetries) {
try {
Invoke-RestMethod -Uri "https://helix.dot.net/api/2018-03-14/telemetry/job/build/$env:Helix_WorkItemId/finish?errorCount=$ErrorCount&warningCount=$WarningCount" -Method Post -ContentType "application/json" -Body "" `
-Headers @{ 'X-Helix-Job-Token'=$env:Helix_JobToken }
break
}
catch {
$statusCode = $_.Exception.Response.StatusCode.value__
if ($statusCode -ge 400 -and $statusCode -le 499) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix (status code $statusCode); not retrying (4xx client error)"
Write-Host "##vso[task.logissue]error ", $_.Exception.GetType().FullName, $_.Exception.Message
exit 1
}
Write-Host "Failed to send telemetry to Helix (status code $statusCode); waiting $env:RetryDelay seconds before retrying..."
$retryCount++
sleep $env:RetryDelay
continue
}
}
if ($retryCount -ge $env:MaxRetries) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix after $retryCount retries."
exit 1
}
displayName: Send Windows Build End Telemetry
env:
# defined via VSTS variables in start-job.ps1
Helix_JobToken: $(Helix_JobToken)
Helix_WorkItemId: $(Helix_WorkItemId)
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(),eq(variables['Agent.Os'], 'Windows_NT'))
@@ -1,241 +0,0 @@
parameters:
helixSource: 'undefined_defaulted_in_telemetry.yml'
helixType: 'undefined_defaulted_in_telemetry.yml'
buildConfig: ''
runAsPublic: false
maxRetries: 5
retryDelay: 10 # in seconds
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), not(eq(variables['System.TeamProject'], 'public'))) }}:
- task: AzureKeyVault@1
inputs:
azureSubscription: 'HelixProd_KeyVault'
KeyVaultName: HelixProdKV
SecretsFilter: 'HelixApiAccessToken'
condition: always()
- bash: |
# create a temporary file
jobInfo=`mktemp`
# write job info content to temporary file
cat > $jobInfo <<JobListStuff
{
"QueueId": "$QueueId",
"Source": "$Source",
"Type": "$Type",
"Build": "$Build",
"Attempt": "$Attempt",
"Properties": {
"operatingSystem": "$OperatingSystem",
"configuration": "$Configuration"
}
}
JobListStuff
cat $jobInfo
# create a temporary file for curl output
res=`mktemp`
accessTokenParameter="?access_token=$HelixApiAccessToken"
curlStatus=1
retryCount=0
# retry loop to harden against spotty telemetry connections
# we don't retry successes and 4xx client errors
until [[ $curlStatus -eq 0 || ( $curlStatus -ge 400 && $curlStatus -le 499 ) || $retryCount -ge $MaxRetries ]]
do
if [ $retryCount -gt 0 ]; then
echo "Failed to send telemetry to Helix; waiting $RetryDelay seconds before retrying..."
sleep $RetryDelay
fi
curlResult=`
cat $jobInfo |\
curl --trace - --verbose --output $res --write-out "%{http_code}" \
-H 'Content-Type: application/json' \
-X POST "https://helix.dot.net/api/2018-03-14/telemetry/job$accessTokenParameter" -d @-`
curlStatus=$?
if [ $curlStatus -eq 0 ]; then
if [ $curlResult -gt 299 ] || [ $curlResult -lt 200 ]; then
curlStatus=$curlResult
fi
fi
let retryCount++
done
curlResult=`cat $res`
# validate status of curl command
if [ $curlStatus -ne 0 ]; then
echo "Failed To Send Job Start information after $retryCount retries"
# We have to append the ## vso prefix or vso will pick up the command when it dumps the inline script into the shell
vstsLogOutput="vso[task.logissue type=error;sourcepath=telemetry/start-job.sh;code=1;]Failed to Send Job Start information: $curlStatus"
echo "##$vstsLogOutput"
exit 1
fi
# Set the Helix_JobToken variable
export Helix_JobToken=`echo $curlResult | xargs echo` # Strip Quotes
echo "##vso[task.setvariable variable=Helix_JobToken;issecret=true;]$Helix_JobToken"
displayName: Send Unix Job Start Telemetry
env:
HelixApiAccessToken: $(HelixApiAccessToken)
Source: ${{ parameters.helixSource }}
Type: ${{ parameters.helixType }}
Build: $(Build.BuildNumber)
QueueId: $(Agent.Os)
Attempt: 1
OperatingSystem: $(Agent.Os)
Configuration: ${{ parameters.buildConfig }}
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(), ne(variables['Agent.Os'], 'Windows_NT'))
- bash: |
curlStatus=1
retryCount=0
# retry loop to harden against spotty telemetry connections
# we don't retry successes and 4xx client errors
until [[ $curlStatus -eq 0 || ( $curlStatus -ge 400 && $curlStatus -le 499 ) || $retryCount -ge $MaxRetries ]]
do
if [ $retryCount -gt 0 ]; then
echo "Failed to send telemetry to Helix; waiting $RetryDelay seconds before retrying..."
sleep $RetryDelay
fi
res=`mktemp`
curlResult=`
curl --verbose --output $res --write-out "%{http_code}"\
-H 'Content-Type: application/json' \
-H "X-Helix-Job-Token: $Helix_JobToken" \
-H 'Content-Length: 0' \
-X POST -G "https://helix.dot.net/api/2018-03-14/telemetry/job/build" \
--data-urlencode "buildUri=$BuildUri"`
curlStatus=$?
if [ $curlStatus -eq 0 ]; then
if [ $curlResult -gt 299 ] || [ $curlResult -lt 200 ]; then
curlStatus=$curlResult
fi
fi
curlResult=`cat $res`
let retryCount++
done
# validate status of curl command
if [ $curlStatus -ne 0 ]; then
echo "Failed to Send Build Start information after $retryCount retries"
vstsLogOutput="vso[task.logissue type=error;sourcepath=telemetry/build/start.sh;code=1;]Failed to Send Build Start information: $curlStatus"
echo "##$vstsLogOutput"
exit 1
fi
export Helix_WorkItemId=`echo $curlResult | xargs echo` # Strip Quotes
echo "##vso[task.setvariable variable=Helix_WorkItemId]$Helix_WorkItemId"
displayName: Send Unix Build Start Telemetry
env:
BuildUri: $(System.TaskDefinitionsUri)$(System.TeamProject)/_build/index?buildId=$(Build.BuildId)&_a=summary
Helix_JobToken: $(Helix_JobToken)
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(), ne(variables['Agent.Os'], 'Windows_NT'))
- powershell: |
$jobInfo = [pscustomobject]@{
QueueId=$env:QueueId;
Source=$env:Source;
Type=$env:Type;
Build=$env:Build;
Attempt=$env:Attempt;
Properties=[pscustomobject]@{ operatingSystem=$env:OperatingSystem; configuration=$env:Configuration };
}
$jobInfoJson = $jobInfo | ConvertTo-Json
if ($env:HelixApiAccessToken) {
$accessTokenParameter="?access_token=$($env:HelixApiAccessToken)"
}
Write-Host "Job Info: $jobInfoJson"
# Basic retry loop to harden against server flakiness
$retryCount = 0
while ($retryCount -lt $env:MaxRetries) {
try {
$jobToken = Invoke-RestMethod -Uri "https://helix.dot.net/api/2018-03-14/telemetry/job$($accessTokenParameter)" -Method Post -ContentType "application/json" -Body $jobInfoJson
break
}
catch {
$statusCode = $_.Exception.Response.StatusCode.value__
if ($statusCode -ge 400 -and $statusCode -le 499) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix (status code $statusCode); not retrying (4xx client error)"
Write-Host "##vso[task.logissue]error ", $_.Exception.GetType().FullName, $_.Exception.Message
exit 1
}
Write-Host "Failed to send telemetry to Helix (status code $statusCode); waiting $env:RetryDelay seconds before retrying..."
$retryCount++
sleep $env:RetryDelay
continue
}
}
if ($retryCount -ge $env:MaxRetries) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix after $retryCount retries."
exit 1
}
$env:Helix_JobToken = $jobToken
Write-Host "##vso[task.setvariable variable=Helix_JobToken;issecret=true;]$env:Helix_JobToken"
env:
HelixApiAccessToken: $(HelixApiAccessToken)
Source: ${{ parameters.helixSource }}
Type: ${{ parameters.helixType }}
Build: $(Build.BuildNumber)
QueueId: $(Agent.Os)
Attempt: 1
OperatingSystem: $(Agent.Os)
Configuration: ${{ parameters.buildConfig }}
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(), eq(variables['Agent.Os'], 'Windows_NT'))
displayName: Send Windows Job Start Telemetry
- powershell: |
# Basic retry loop to harden against server flakiness
$retryCount = 0
while ($retryCount -lt $env:MaxRetries) {
try {
$workItemId = Invoke-RestMethod -Uri "https://helix.dot.net/api/2018-03-14/telemetry/job/build?buildUri=$([Net.WebUtility]::UrlEncode($env:BuildUri))" -Method Post -ContentType "application/json" -Body "" `
-Headers @{ 'X-Helix-Job-Token'=$env:Helix_JobToken }
break
}
catch {
$statusCode = $_.Exception.Response.StatusCode.value__
if ($statusCode -ge 400 -and $statusCode -le 499) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix (status code $statusCode); not retrying (4xx client error)"
Write-Host "##vso[task.logissue]error ", $_.Exception.GetType().FullName, $_.Exception.Message
exit 1
}
Write-Host "Failed to send telemetry to Helix (status code $statusCode); waiting $env:RetryDelay seconds before retrying..."
$retryCount++
sleep $env:RetryDelay
continue
}
}
if ($retryCount -ge $env:MaxRetries) {
Write-Host "##vso[task.logissue]error Failed to send telemetry to Helix after $retryCount retries."
exit 1
}
$env:Helix_WorkItemId = $workItemId
Write-Host "##vso[task.setvariable variable=Helix_WorkItemId]$env:Helix_WorkItemId"
displayName: Send Windows Build Start Telemetry
env:
BuildUri: $(System.TaskDefinitionsUri)$(System.TeamProject)/_build/index?buildId=$(Build.BuildId)&_a=summary
Helix_JobToken: $(Helix_JobToken)
MaxRetries: ${{ parameters.maxRetries }}
RetryDelay: ${{ parameters.retryDelay }}
condition: and(always(), eq(variables['Agent.Os'], 'Windows_NT'))
-579
View File
@@ -1,579 +0,0 @@
# Initialize variables if they aren't already defined.
# These may be defined as parameters of the importing script, or set after importing this script.
# CI mode - set to true on CI server for PR validation build or official build.
[bool]$ci = if (Test-Path variable:ci) { $ci } else { $false }
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { "Debug" }
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
[bool]$binaryLog = if (Test-Path variable:binaryLog) { $binaryLog } else { $ci }
# Set to true to use the pipelines logger which will enable Azure logging output.
# https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md
# This flag is meant as a temporary opt-opt for the feature while validate it across
# our consumers. It will be deleted in the future.
[bool]$pipelinesLog = if (Test-Path variable:pipelinesLog) { $pipelinesLog } else { $ci }
# Turns on machine preparation/clean up code that changes the machine state (e.g. kills build processes).
[bool]$prepareMachine = if (Test-Path variable:prepareMachine) { $prepareMachine } else { $false }
# True to restore toolsets and dependencies.
[bool]$restore = if (Test-Path variable:restore) { $restore } else { $true }
# Adjusts msbuild verbosity level.
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { "minimal" }
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
[bool]$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { !$ci }
# Configures warning treatment in msbuild.
[bool]$warnAsError = if (Test-Path variable:warnAsError) { $warnAsError } else { $true }
# Specifies which msbuild engine to use for build: 'vs', 'dotnet' or unspecified (determined based on presence of tools.vs in global.json).
[string]$msbuildEngine = if (Test-Path variable:msbuildEngine) { $msbuildEngine } else { $null }
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
set-strictmode -version 2.0
$ErrorActionPreference = "Stop"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
function Create-Directory([string[]] $path) {
if (!(Test-Path $path)) {
New-Item -path $path -force -itemType "Directory" | Out-Null
}
}
function Unzip([string]$zipfile, [string]$outpath) {
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
}
# This will exec a process using the console and return it's exit code.
# This will not throw when the process fails.
# Returns process exit code.
function Exec-Process([string]$command, [string]$commandArgs) {
$startInfo = New-Object System.Diagnostics.ProcessStartInfo
$startInfo.FileName = $command
$startInfo.Arguments = $commandArgs
$startInfo.UseShellExecute = $false
$startInfo.WorkingDirectory = Get-Location
$process = New-Object System.Diagnostics.Process
$process.StartInfo = $startInfo
$process.Start() | Out-Null
$finished = $false
try {
while (-not $process.WaitForExit(100)) {
# Non-blocking loop done to allow ctr-c interrupts
}
$finished = $true
return $global:LASTEXITCODE = $process.ExitCode
}
finally {
# If we didn't finish then an error occured or the user hit ctrl-c. Either
# way kill the process
if (-not $finished) {
$process.Kill()
}
}
}
function InitializeDotNetCli([bool]$install) {
if (Test-Path variable:global:_DotNetInstallDir) {
return $global:_DotNetInstallDir
}
# Don't resolve runtime, shared framework, or SDK from other locations to ensure build determinism
$env:DOTNET_MULTILEVEL_LOOKUP=0
# Disable first run since we do not need all ASP.NET packages restored.
$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
# Disable telemetry on CI.
if ($ci) {
$env:DOTNET_CLI_TELEMETRY_OPTOUT=1
}
# Source Build uses DotNetCoreSdkDir variable
if ($env:DotNetCoreSdkDir -ne $null) {
$env:DOTNET_INSTALL_DIR = $env:DotNetCoreSdkDir
}
# Find the first path on %PATH% that contains the dotnet.exe
if ($useInstalledDotNetCli -and (-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -eq $null)) {
$dotnetCmd = Get-Command "dotnet.exe" -ErrorAction SilentlyContinue
if ($dotnetCmd -ne $null) {
$env:DOTNET_INSTALL_DIR = Split-Path $dotnetCmd.Path -Parent
}
}
$dotnetSdkVersion = $GlobalJson.tools.dotnet
# Use dotnet installation specified in DOTNET_INSTALL_DIR if it contains the required SDK version,
# otherwise install the dotnet CLI and SDK to repo local .dotnet directory to avoid potential permission issues.
if ((-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -ne $null) -and (Test-Path(Join-Path $env:DOTNET_INSTALL_DIR "sdk\$dotnetSdkVersion"))) {
$dotnetRoot = $env:DOTNET_INSTALL_DIR
} else {
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
if (-not (Test-Path(Join-Path $dotnetRoot "sdk\$dotnetSdkVersion"))) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-Host "Unable to find dotnet with SDK version '$dotnetSdkVersion'" -ForegroundColor Red
ExitWithExitCode 1
}
}
$env:DOTNET_INSTALL_DIR = $dotnetRoot
}
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
# It also ensures that VS msbuild will use the downloaded sdk targets.
$env:PATH = "$dotnetRoot;$env:PATH"
if ($ci) {
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
Write-Host "##vso[task.prependpath]$dotnetRoot"
Write-Host "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
Write-Host "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
}
return $global:_DotNetInstallDir = $dotnetRoot
}
function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
Invoke-WebRequest "https://dot.net/v1/dotnet-install.ps1" -OutFile $installScript
}
return $installScript
}
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = "") {
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) { $installScript = GetDotNetInstallScript $dotnetRoot
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
InstallDir = $dotnetRoot
}
if ($architecture) { $installParameters.Architecture = $architecture }
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-Host "Failed to install dotnet cli (exit code '$lastExitCode')." -ForegroundColor Red
ExitWithExitCode $lastExitCode
}
}
#
# Locates Visual Studio MSBuild installation.
# The preference order for MSBuild to use is as follows:
#
# 1. MSBuild from an active VS command prompt
# 2. MSBuild from a compatible VS installation
# 3. MSBuild from the xcopy tool package
#
# Returns full path to msbuild.exe.
# Throws on failure.
#
function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements = $null) {
if (Test-Path variable:global:_MSBuildExe) {
return $global:_MSBuildExe
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
$msbuildCmd = Get-Command "msbuild.exe" -ErrorAction SilentlyContinue
if ($msbuildCmd -ne $null) {
# Workaround for https://github.com/dotnet/roslyn/issues/35793
# Due to this issue $msbuildCmd.Version returns 0.0.0.0 for msbuild.exe 16.2+
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split(@('-', '+'))[0])
if ($msbuildVersion -ge $vsMinVersion) {
return $global:_MSBuildExe = $msbuildCmd.Path
}
# Report error - the developer environment is initialized with incompatible VS version.
throw "Developer Command Prompt for VS $($env:VisualStudioVersion) is not recent enough. Please upgrade to $vsMinVersionStr or build from a plain CMD window"
}
}
# Locate Visual Studio installation or download x-copy msbuild.
$vsInfo = LocateVisualStudio $vsRequirements
if ($vsInfo -ne $null) {
$vsInstallDir = $vsInfo.installationPath
$vsMajorVersion = $vsInfo.installationVersion.Split('.')[0]
InitializeVisualStudioEnvironmentVariables $vsInstallDir $vsMajorVersion
} else {
if (Get-Member -InputObject $GlobalJson.tools -Name "xcopy-msbuild") {
$xcopyMSBuildVersion = $GlobalJson.tools.'xcopy-msbuild'
$vsMajorVersion = $xcopyMSBuildVersion.Split('.')[0]
} else {
$vsMajorVersion = $vsMinVersion.Major
$xcopyMSBuildVersion = "$vsMajorVersion.$($vsMinVersion.Minor).0-alpha"
}
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
if ($vsInstallDir -eq $null) {
throw "Unable to find Visual Studio that has required version and components installed"
}
}
$msbuildVersionDir = if ([int]$vsMajorVersion -lt 16) { "$vsMajorVersion.0" } else { "Current" }
return $global:_MSBuildExe = Join-Path $vsInstallDir "MSBuild\$msbuildVersionDir\Bin\msbuild.exe"
}
function InitializeVisualStudioEnvironmentVariables([string] $vsInstallDir, [string] $vsMajorVersion) {
$env:VSINSTALLDIR = $vsInstallDir
Set-Item "env:VS$($vsMajorVersion)0COMNTOOLS" (Join-Path $vsInstallDir "Common7\Tools\")
$vsSdkInstallDir = Join-Path $vsInstallDir "VSSDK\"
if (Test-Path $vsSdkInstallDir) {
Set-Item "env:VSSDK$($vsMajorVersion)0Install" $vsSdkInstallDir
$env:VSSDKInstall = $vsSdkInstallDir
}
}
function InstallXCopyMSBuild([string]$packageVersion) {
return InitializeXCopyMSBuild $packageVersion -install $true
}
function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
$packageName = "RoslynTools.MSBuild"
$packageDir = Join-Path $ToolsDir "msbuild\$packageVersion"
$packagePath = Join-Path $packageDir "$packageName.$packageVersion.nupkg"
if (!(Test-Path $packageDir)) {
if (!$install) {
return $null
}
Create-Directory $packageDir
Write-Host "Downloading $packageName $packageVersion"
Invoke-WebRequest "https://dotnet.myget.org/F/roslyn-tools/api/v2/package/$packageName/$packageVersion/" -OutFile $packagePath
Unzip $packagePath $packageDir
}
return Join-Path $packageDir "tools"
}
#
# Locates Visual Studio instance that meets the minimal requirements specified by tools.vs object in global.json.
#
# The following properties of tools.vs are recognized:
# "version": "{major}.{minor}"
# Two part minimal VS version, e.g. "15.9", "16.0", etc.
# "components": ["componentId1", "componentId2", ...]
# Array of ids of workload components that must be available in the VS instance.
# See e.g. https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-enterprise?view=vs-2017
#
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
if (Get-Member -InputObject $GlobalJson.tools -Name "vswhere") {
$vswhereVersion = $GlobalJson.tools.vswhere
} else {
$vswhereVersion = "2.5.2"
}
$vsWhereDir = Join-Path $ToolsDir "vswhere\$vswhereVersion"
$vsWhereExe = Join-Path $vsWhereDir "vswhere.exe"
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host "Downloading vswhere"
Invoke-WebRequest "https://github.com/Microsoft/vswhere/releases/download/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild")
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
$args += $vsRequirements.version
}
if (Get-Member -InputObject $vsRequirements -Name "components") {
foreach ($component in $vsRequirements.components) {
$args += "-requires"
$args += $component
}
}
$vsInfo =& $vsWhereExe $args | ConvertFrom-Json
if ($lastExitCode -ne 0) {
return $null
}
# use first matching instance
return $vsInfo[0]
}
function InitializeBuildTool() {
if (Test-Path variable:global:_BuildTool) {
return $global:_BuildTool
}
if (-not $msbuildEngine) {
$msbuildEngine = GetDefaultMSBuildEngine
}
# Initialize dotnet cli if listed in 'tools'
$dotnetRoot = $null
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
$dotnetRoot = InitializeDotNetCli -install:$restore
}
if ($msbuildEngine -eq "dotnet") {
if (!$dotnetRoot) {
Write-Host "/global.json must specify 'tools.dotnet'." -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-Host $_ -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-Host "Unexpected value of -msbuildEngine: '$msbuildEngine'." -ForegroundColor Red
ExitWithExitCode 1
}
return $global:_BuildTool = $buildTool
}
function GetDefaultMSBuildEngine() {
# Presence of tools.vs indicates the repo needs to build using VS msbuild on Windows.
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
}
Write-Host "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'." -ForegroundColor Red
ExitWithExitCode 1
}
function GetNuGetPackageCachePath() {
if ($env:NUGET_PACKAGES -eq $null) {
# Use local cache on CI to ensure deterministic build,
# use global cache in dev builds to avoid cost of downloading packages.
if ($useGlobalNuGetCache) {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile ".nuget\packages"
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot ".packages"
}
}
return $env:NUGET_PACKAGES
}
# Returns a full path to an Arcade SDK task project file.
function GetSdkTaskProject([string]$taskName) {
return Join-Path (Split-Path (InitializeToolset) -Parent) "SdkTasks\$taskName.proj"
}
function InitializeNativeTools() {
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
$nativeArgs=""
if ($ci) {
$nativeArgs = "-InstallDirectory $ToolsDir"
}
Invoke-Expression "& `"$PSScriptRoot/init-tools-native.ps1`" $nativeArgs"
}
}
function InitializeToolset() {
if (Test-Path variable:global:_ToolsetBuildProj) {
return $global:_ToolsetBuildProj
}
$nugetCache = GetNuGetPackageCachePath
$toolsetVersion = $GlobalJson.'msbuild-sdks'.'Microsoft.DotNet.Arcade.Sdk'
$toolsetLocationFile = Join-Path $ToolsetDir "$toolsetVersion.txt"
if (Test-Path $toolsetLocationFile) {
$path = Get-Content $toolsetLocationFile -TotalCount 1
if (Test-Path $path) {
return $global:_ToolsetBuildProj = $path
}
}
if (-not $restore) {
Write-Host "Toolset version $toolsetVersion has not been restored." -ForegroundColor Red
ExitWithExitCode 1
}
$buildTool = InitializeBuildTool
$proj = Join-Path $ToolsetDir "restore.proj"
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "ToolsetRestore.binlog") } else { "" }
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Set-Content $proj
MSBuild-Core $proj $bl /t:__WriteToolsetLocation /clp:ErrorsOnly`;NoSummary /p:__ToolsetLocationOutputFile=$toolsetLocationFile
$path = Get-Content $toolsetLocationFile -TotalCount 1
if (!(Test-Path $path)) {
throw "Invalid toolset path: $path"
}
return $global:_ToolsetBuildProj = $path
}
function ExitWithExitCode([int] $exitCode) {
if ($ci -and $prepareMachine) {
Stop-Processes
}
exit $exitCode
}
function Stop-Processes() {
Write-Host "Killing running build processes..."
foreach ($processName in $processesToStopOnExit) {
Get-Process -Name $processName -ErrorAction SilentlyContinue | Stop-Process
}
}
#
# Executes msbuild (or 'dotnet msbuild') with arguments passed to the function.
# The arguments are automatically quoted.
# Terminates the script if the build fails.
#
function MSBuild() {
if ($pipelinesLog) {
$buildTool = InitializeBuildTool
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
$args += "/logger:$path"
}
MSBuild-Core @args
}
#
# Executes msbuild (or 'dotnet msbuild') with arguments passed to the function.
# The arguments are automatically quoted.
# Terminates the script if the build fails.
#
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
throw "Binary log must be enabled in CI build."
}
if ($nodeReuse) {
throw "Node reuse must be disabled in CI build."
}
}
$buildTool = InitializeBuildTool
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
}
foreach ($arg in $args) {
if ($arg -ne $null -and $arg.Trim() -ne "") {
$cmdArgs += " `"$arg`""
}
}
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-Host "Build failed." -ForegroundColor Red
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
}
ExitWithExitCode $exitCode
}
}
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
}
}
}
return $null
}
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$ToolsetDir = Join-Path $ArtifactsDir "toolset"
$ToolsDir = Join-Path $RepoRoot ".tools"
$LogDir = Join-Path (Join-Path $ArtifactsDir "log") $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir "tmp") $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot "global.json") | ConvertFrom-Json
# true if global.json contains a "runtimes" section
$globalJsonHasRuntimes = if ($GlobalJson.tools.PSObject.Properties.Name -Match 'runtimes') { $true } else { $false }
Create-Directory $ToolsetDir
Create-Directory $TempDir
Create-Directory $LogDir
if ($ci) {
Write-Host "##vso[task.setvariable variable=Artifacts]$ArtifactsDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Toolset]$ToolsetDir"
Write-Host "##vso[task.setvariable variable=Artifacts.Log]$LogDir"
$env:TEMP = $TempDir
$env:TMP = $TempDir
}
-463
View File
@@ -1,463 +0,0 @@
#!/usr/bin/env bash
# Initialize variables if they aren't already defined.
# CI mode - set to true on CI server for PR validation build or official build.
ci=${ci:-false}
# Set to true to use the pipelines logger which will enable Azure logging output.
# https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md
# This flag is meant as a temporary opt-opt for the feature while validate it across
# our consumers. It will be deleted in the future.
if [[ "$ci" == true ]]; then
pipelines_log=${pipelines_log:-true}
else
pipelines_log=${pipelines_log:-false}
fi
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
configuration=${configuration:-'Debug'}
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
binary_log=${binary_log:-$ci}
# Turns on machine preparation/clean up code that changes the machine state (e.g. kills build processes).
prepare_machine=${prepare_machine:-false}
# True to restore toolsets and dependencies.
restore=${restore:-true}
# Adjusts msbuild verbosity level.
verbosity=${verbosity:-'minimal'}
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
if [[ "$ci" == true ]]; then
node_reuse=${node_reuse:-false}
else
node_reuse=${node_reuse:-true}
fi
# Configures warning treatment in msbuild.
warn_as_error=${warn_as_error:-true}
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
if [[ "$ci" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
else
use_global_nuget_cache=${use_global_nuget_cache:-true}
fi
function EmitError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
message_type="error"
sourcepath=''
linenumber=''
columnnumber=''
error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-l)
linenumber=$2
shift
;;
-columnnumber|-col)
columnnumber=$2
shift
;;
-code|-c)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
message='##vso[task.logissue'
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
else
message="$message;sourcepath=${BASH_SOURCE[1]}"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
else
message="$message;linenumber=${BASH_LINENO[0]}"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
# Resolve any symlinks in the given path.
function ResolvePath {
local path=$1
while [[ -h $path ]]; do
local dir="$( cd -P "$( dirname "$path" )" && pwd )"
path="$(readlink "$path")"
# if $path was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $path != /* ]] && path="$dir/$path"
done
# return value
_ResolvePath="$path"
}
# ReadVersionFromJson [json key]
function ReadGlobalVersion {
local key=$1
local line=`grep -m 1 "$key" "$global_json_file"`
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
EmitError "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
# return value
_ReadGlobalVersion=${BASH_REMATCH[1]}
}
function InitializeDotNetCli {
if [[ -n "${_InitializeDotNetCli:-}" ]]; then
return
fi
local install=$1
# Don't resolve runtime, shared framework, or SDK from other locations to ensure build determinism
export DOTNET_MULTILEVEL_LOOKUP=0
# Disable first run since we want to control all package sources
export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
# Disable telemetry on CI
if [[ $ci == true ]]; then
export DOTNET_CLI_TELEMETRY_OPTOUT=1
fi
# LTTNG is the logging infrastructure used by Core CLR. Need this variable set
# so it doesn't output warnings to the console.
export LTTNG_HOME="$HOME"
# Source Build uses DotNetCoreSdkDir variable
if [[ -n "${DotNetCoreSdkDir:-}" ]]; then
export DOTNET_INSTALL_DIR="$DotNetCoreSdkDir"
fi
# Find the first path on $PATH that contains the dotnet.exe
if [[ "$use_installed_dotnet_cli" == true && $global_json_has_runtimes == false && -z "${DOTNET_INSTALL_DIR:-}" ]]; then
local dotnet_path=`command -v dotnet`
if [[ -n "$dotnet_path" ]]; then
ResolvePath "$dotnet_path"
export DOTNET_INSTALL_DIR=`dirname "$_ResolvePath"`
fi
fi
ReadGlobalVersion "dotnet"
local dotnet_sdk_version=$_ReadGlobalVersion
local dotnet_root=""
# Use dotnet installation specified in DOTNET_INSTALL_DIR if it contains the required SDK version,
# otherwise install the dotnet CLI and SDK to repo local .dotnet directory to avoid potential permission issues.
if [[ $global_json_has_runtimes == false && -n "${DOTNET_INSTALL_DIR:-}" && -d "$DOTNET_INSTALL_DIR/sdk/$dotnet_sdk_version" ]]; then
dotnet_root="$DOTNET_INSTALL_DIR"
else
dotnet_root="$repo_root/.dotnet"
export DOTNET_INSTALL_DIR="$dotnet_root"
if [[ ! -d "$DOTNET_INSTALL_DIR/sdk/$dotnet_sdk_version" ]]; then
if [[ "$install" == true ]]; then
InstallDotNetSdk "$dotnet_root" "$dotnet_sdk_version"
else
EmitError "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
ExitWithExitCode 1
fi
fi
fi
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
export PATH="$dotnet_root:$PATH"
if [[ $ci == true ]]; then
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
echo "##vso[task.prependpath]$dotnet_root"
echo "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
echo "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
fi
# return value
_InitializeDotNetCli="$dotnet_root"
}
function InstallDotNetSdk {
local root=$1
local version=$2
local architecture=""
if [[ $# == 3 ]]; then
architecture=$3
fi
InstallDotNet "$root" "$version" $architecture
}
function InstallDotNet {
local root=$1
local version=$2
GetDotNetInstallScript "$root"
local install_script=$_GetDotNetInstallScript
local archArg=''
if [[ -n "${3:-}" ]]; then
archArg="--architecture $3"
fi
local runtimeArg=''
if [[ -n "${4:-}" ]]; then
runtimeArg="--runtime $4"
fi
local skipNonVersionedFilesArg=""
if [[ "$#" -ge "5" ]]; then
skipNonVersionedFilesArg="--skip-non-versioned-files"
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
EmitError "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
local install_script_url="https://dot.net/v1/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
echo "Downloading '$install_script_url'"
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script"
else
wget -q -O "$install_script" "$install_script_url"
fi
fi
# return value
_GetDotNetInstallScript="$install_script"
}
function InitializeBuildTool {
if [[ -n "${_InitializeBuildTool:-}" ]]; then
return
fi
InitializeDotNetCli $restore
# return values
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildToolCommand="msbuild"
_InitializeBuildToolFramework="netcoreapp2.1"
}
function GetNuGetPackageCachePath {
if [[ -z ${NUGET_PACKAGES:-} ]]; then
if [[ "$use_global_nuget_cache" == true ]]; then
export NUGET_PACKAGES="$HOME/.nuget/packages"
else
export NUGET_PACKAGES="$repo_root/.packages"
fi
fi
# return value
_GetNuGetPackageCachePath=$NUGET_PACKAGES
}
function InitializeNativeTools() {
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
if [[ "$ci" == true ]]; then
nativeArgs="-InstallDirectory $tools_dir"
fi
"$_script_dir/init-tools-native.sh" $nativeArgs
fi
}
function InitializeToolset {
if [[ -n "${_InitializeToolset:-}" ]]; then
return
fi
GetNuGetPackageCachePath
ReadGlobalVersion "Microsoft.DotNet.Arcade.Sdk"
local toolset_version=$_ReadGlobalVersion
local toolset_location_file="$toolset_dir/$toolset_version.txt"
if [[ -a "$toolset_location_file" ]]; then
local path=`cat "$toolset_location_file"`
if [[ -a "$path" ]]; then
# return value
_InitializeToolset="$path"
return
fi
fi
if [[ "$restore" != true ]]; then
EmitError "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 2
fi
local proj="$toolset_dir/restore.proj"
local bl=""
if [[ "$binary_log" == true ]]; then
bl="/bl:$log_dir/ToolsetRestore.binlog"
fi
echo '<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' > "$proj"
MSBuild-Core "$proj" $bl /t:__WriteToolsetLocation /clp:ErrorsOnly\;NoSummary /p:__ToolsetLocationOutputFile="$toolset_location_file"
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
EmitError "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
# return value
_InitializeToolset="$toolset_build_proj"
}
function ExitWithExitCode {
if [[ "$ci" == true && "$prepare_machine" == true ]]; then
StopProcesses
fi
exit $1
}
function StopProcesses {
echo "Killing running build processes..."
pkill -9 "dotnet" || true
pkill -9 "vbcscompiler" || true
return 0
}
function MSBuild {
local args=$@
if [[ "$pipelines_log" == true ]]; then
InitializeBuildTool
InitializeToolset
local toolset_dir="${_InitializeToolset%/*}"
local logger_path="$toolset_dir/$_InitializeBuildToolFramework/Microsoft.DotNet.Arcade.Sdk.dll"
args=( "${args[@]}" "-logger:$logger_path" )
fi
MSBuild-Core ${args[@]}
}
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
EmitError "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
EmitError "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
InitializeBuildTool
local warnaserror_switch=""
if [[ $warn_as_error == true ]]; then
warnaserror_switch="/warnaserror"
fi
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
EmitError "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
ResolvePath "${BASH_SOURCE[0]}"
_script_dir=`dirname "$_ResolvePath"`
eng_root=`cd -P "$_script_dir/.." && pwd`
repo_root=`cd -P "$_script_dir/../.." && pwd`
artifacts_dir="$repo_root/artifacts"
toolset_dir="$artifacts_dir/toolset"
tools_dir="$repo_root/.tools"
log_dir="$artifacts_dir/log/$configuration"
temp_dir="$artifacts_dir/tmp/$configuration"
global_json_file="$repo_root/global.json"
# determine if global.json contains a "runtimes" entry
global_json_has_runtimes=false
dotnetlocal_key=`grep -m 1 "runtimes" "$global_json_file"` || true
if [[ -n "$dotnetlocal_key" ]]; then
global_json_has_runtimes=true
fi
# HOME may not be defined in some scenarios, but it is required by NuGet
if [[ -z $HOME ]]; then
export HOME="$repo_root/artifacts/.home/"
mkdir -p "$HOME"
fi
mkdir -p "$toolset_dir"
mkdir -p "$temp_dir"
mkdir -p "$log_dir"
if [[ $ci == true ]]; then
export TEMP="$temp_dir"
export TMP="$temp_dir"
fi
-15
View File
@@ -1,15 +0,0 @@
<Project Sdk="Microsoft.NET.Sdk">
<!-- This project exists purely to work around Arcade
behavior. Arcade knows how to restore tasks needed for
publishing to blob feeds, but it requires a project file that
imports nuget restore targets to be specified for this to
work. See details in
https://github.com/dotnet/arcade/commit/f657be5cb7cd4920334dd9162173b131211a1e17#r31728598.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.0</TargetFramework>
</PropertyGroup>
</Project>
-51
View File
@@ -1,51 +0,0 @@
<Project DefaultTargets="PublishPackages">
<!-- TODO: move properties imported from here into a common props file -->
<!-- <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dir.props))\dir.props" /> -->
<!-- Use an explicit SDK import so that arcade uses the build
configuration computed in dir.common.props. -->
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
<Import Project="$(MSBuildThisFileDirectory)\..\.packages\microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<PropertyGroup>
<!-- Set the TargetFramework just to make the SDK happy -->
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<Target Name="PublishPackages">
<ItemGroup>
<PackagesToPublish Include="$(MSBuildThisFileDirectory)\..\artifacts\*.nupkg" />
</ItemGroup>
<ItemGroup>
<ItemsToPushToBlobFeed Include="@(PackagesToPublish)" />
</ItemGroup>
<Error Condition=" '$(AzureFeedUrl)' == '' " Text="AzureFeedUrl must be set" />
<Error Condition=" '$(AzureAccountKey)' == '' " Text="AzureAccountKey must be set" />
<Error Condition=" '$(BUILD_REPOSITORY_URI)' == '' " Text="BUILD_REPOSITORY_URI must be set" />
<Error Condition=" '$(BUILD_SOURCEBRANCH)' == '' " Text="BUILD_SOURCEBRANCH must be set" />
<Error Condition=" '$(BUILD_BUILDNUMBER)' == '' " Text="BUILD_BUILDNUMBER must be set" />
<Error Condition=" '$(BUILD_SOURCEVERSION)' == '' " Text="BUILD_SOURCEVERSION must be set" />
<PropertyGroup>
<AssetManifestFilePath>$(ArtifactsLogDir)AssetManifest\$(AssetManifest)</AssetManifestFilePath>
</PropertyGroup>
<Message Importance="Low" Text="BUILD_REPOSITORY_URI: $(BUILD_REPOSITORY_URI)" />
<PushToBlobFeed ExpectedFeedUrl="$(AzureFeedUrl)"
AccountKey="$(AzureAccountKey)"
ItemsToPush="@(ItemsToPushToBlobFeed)"
ManifestBuildData="Location=$(AzureFeedUrl)"
ManifestRepoUri="$(BUILD_REPOSITORY_URI)"
ManifestBranch="$(BUILD_SOURCEBRANCH)"
ManifestBuildId="$(BUILD_BUILDNUMBER)"
ManifestCommit="$(BUILD_SOURCEVERSION)"
AssetManifestPath="$(AssetManifestFilePath)" />
</Target>
</Project>
+1 -1
+1 -1
+1 -1
-51
View File
@@ -1,51 +0,0 @@
How to bump roslyn
1. Pick a revision of roslyn to use. Ideally, use a revision that Visual Studio is using, but any revision should do.
2. Make a new folder in roslyn-binaries for this revision. Copy over the contents of the *previous folder* into this folder.
3. Make a temporary commit.
4. Find that revision in the 'roslyn compilers' nuget feed: https://dotnet.myget.org/feed/roslyn/package/vsix/7922692f-f018-45e7-8f3f-d3b7c0262841
5. Download the vsix file and unpack it (it's a ZIP file).
6. Inside the vsix you will find lots of metadata, along with executables for this build of roslyn. We want to copy most of these into a new folder in roslyn-binaries, so...
7. Copy all the dll, exe and rsp files from the vsix into our new folder, overwriting the ones from the previous version. It's time to look at the diff.
8. The vsix contains DLLs we don't need to ship. Ideally we will not need to add any new files, but roslyn sometimes picks up new dependencies. Figuring out what to keep will require looking at what facades we ship (note that this is profile-specific - check Facades/subdirs.make) and which of these DLLs are actually used by roslyn instead of Visual Studio.
9. Remove any DLLs from the new folder that we don't need. In practice your commit should be adding 1-2 DLLs at most or something very bad happened.
10. Examine the diffs for the .rsp files. We ship our own versions of the .rsps, so we don't want to carry over the vsix versions, but we *do* want to make note of anything new they have added to the rsp files and add it to ours if necessary.
11. Add and commit your new folder. Push it to a branch and make a PR against roslyn-binaries, then get it merged.
----
11. Okay, we have a new roslyn-binaries. The next step is to prepare a mono bump. Do this by updating the roslyn-binaries submodule to point to the new revision of roslyn-binaries. You can temporarily point it to your own branch for testing and run CI builds against that, just don't merge that into master.
12. Once you commit the submodule bump to your local repository, you can do test builds. **If you do test builds without committing the bump, the build process will erase the bump.**
13. Now we need to change the roslyn paths used by the build process - we're still using the older roslyn folder. We do this by changing the CSC_LOCATION and VBCS_LOCATION variables in configure.ac. They should be something like ```Microsoft.Net.Compilers/3.1.0/csc.exe```, change the version number in those paths, save, and make a local commit. Then do the full autogen, make, make install routine and do some basic tests. This should work.
14. Next, make sure the bump didn't break csi. If you managed the .rsp files correctly in the last step, it should work. This has test coverage, so just pay attention to the automated tests for it.
15. Ensure the 'Linux x64 - prefix sanity checks' lane passed on CI, even if it's currently disabled or not required. This is required for a bump. If it isn't visible on the PR statuses, manually trigger the lane using Jenkins.
16. Ensure that 'make install' output has working csc. Install to a prefix is fine.
17. Verify that the compiler server works, even if it is currently off by default. You can do this by editing your ```mcs/build/config.make``` and changing ENABLE_COMPILER_SERVER to ?=1, doing make clean, and then make -j 1. Use 'top' with the O->COMMAND=mono filter to identify whether the compiler server is being used and working. Also, for safety's sake make sure that all three of the roslyn-binaries paths in there match the expected version.
18. Double-check that you have a clean (points to roslyn-binaries master) PR ready to commit. Now it's msbuild time.
----
19. Prepare a PR against mono/msbuild's xplat-master branch:
20. Modify the CompilerToolsVersion and MicrosoftNetCompilersVersion properties in ```eng/Packages.props``` to point to the same new roslyn packages. The version strings have weird textual info in them so you'll have to figure out the new version string.
21. Verify that a build of this new msbuild succeeds.
22. Verify that you can manually run the MSBuild.dll from this new build using a new source build of mono (i.e. with new roslyn installed to a prefix)
23. Get your PR merged.
----
24. Now that msbuild is updated, we make a final change to the mono PR:
25. Modify ```packaging/MacSDK/msbuild.py```, updating the ```revision =``` line's commit hash to be the commit hash of your post-merge xplat-master commit.
26. Get your PR merged.
----
27. Now we need to update the linux packaging and deploy new packages (this involves changing the https://github.com/mono/linux-packaging-msbuild repository)
28. import upstream xplat-master changes by giving the CI package job a URL to a tarball containing the contents of the latest xplat-master revision.
29. trigger CI lane builds to generate a new package
30. identify necessary changes to the debian/rules file for the build to succeed, if step 29 failed
31. identify any new patches required to make latest xplat-master compatible with debian, including build script changes (see above)
32. eventually the lane succeeds. you should have updated linux packages for debian and ubuntu now.
----
33. Follow up with the VSfM team to make sure they know that roslyn is being bumped, it may break them on the next integration. Ensure the rest of the team knows roslyn has been bumped. Keep an eye out for any new bug reports about issues with csc or msbuild.
+1 -1
+1 -1
+1 -1
-9
View File
@@ -1,9 +0,0 @@
{
"tools": {
"dotnet": "3.0.100-preview5-011568"
},
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "1.0.0-beta.19279.5",
"Microsoft.DotNet.Helix.Sdk": "2.0.0-beta.19279.5"
}
}
+1 -2
View File
@@ -12,8 +12,7 @@ LLVM_DOWNLOAD_LOCATION = "http://xamjenkinsartifact.blob.core.windows.net/build-
CMAKE := $(or $(CMAKE),$(shell which cmake))
NINJA := $(shell which ninja)
EXTRA_LLVM_ARGS = $(if $(filter $(LLVM_TARGET),wasm32), -DLLVM_BUILD_32_BITS=On -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="WebAssembly",) \
$(if $(STATIC_GCC_LIBS),-DCMAKE_EXE_LINKER_FLAGS="-static")
EXTRA_LLVM_ARGS = $(if $(filter $(LLVM_TARGET),wasm32), -DLLVM_BUILD_32_BITS=On -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="WebAssembly",)
# -DLLVM_ENABLE_LIBXML2=Off is needed because xml2 is not used and it breaks 32-bit builds on 64-bit Linux hosts
$(LLVM_BUILD)/$(if $(NINJA),build.ninja,Makefile): $(abs_top_srcdir)/external/llvm/CMakeLists.txt
-94
View File
@@ -1,94 +0,0 @@
:: Set up build environment and build LLVM using MSVC and msbuild targets in mono.sln.
:: Arguments:
:: -------------------------------------------------------
:: %1 Visual Studio target, build|clean, default build
:: %2 Host CPU architecture, x86_64|i686, default x86_64
:: %3 Visual Studio configuration, debug|release, default release
:: %4 Mono MSVC source folder.
:: %5 LLVM build directory.
:: %6 LLVM install directory.
:: %7 Additional arguments passed to msbuild, needs to be quoted if multiple.
:: -------------------------------------------------------
@echo off
setlocal
set BUILD_RESULT=1
:: Get path for current running script.
set RUN_BUILD_LLVM_MSBUILD_SCRIPT_PATH=%~dp0
:: Configure all known build arguments.
set VS_BUILD_ARGS=
set VS_TARGET=build
if /i "%~1" == "clean" (
set VS_TARGET="clean"
)
shift
set VS_PLATFORM=x64
if /i "%~1" == "i686" (
set VS_PLATFORM="Win32"
)
if /i "%~1" == "win32" (
set VS_PLATFORM="Win32"
)
shift
set VS_CONFIGURATION=Release
if /i "%~1" == "debug" (
set VS_CONFIGURATION="Debug"
)
shift
set MONO_MSVC_SOURCE_DIR=%RUN_BUILD_LLVM_MSBUILD_SCRIPT_PATH%..\msvc\
if not "%~1" == "" (
set MONO_MSVC_SOURCE_DIR=%~1
)
shift
set MONO_LLVM_BUILD_DIR=
if not "%~1" == "" (
set MONO_LLVM_BUILD_DIR=%~1
)
shift
set MONO_LLVM_INSTALL_DIR=
if not "%~1" == "" (
set MONO_LLVM_INSTALL_DIR=%~1
)
shift
set "VS_ADDITIONAL_ARGUMENTS=/p:PlatformToolset=v140 /p:MONO_TARGET_GC=sgen"
if not "%~1" == "" (
set VS_ADDITIONAL_ARGUMENTS=%~1
)
:: Setup Windows environment.
call %MONO_MSVC_SOURCE_DIR%setup-windows-env.bat
:: Setup VS msbuild environment.
call %MONO_MSVC_SOURCE_DIR%setup-vs-msbuild-env.bat
if not "%MONO_LLVM_BUILD_DIR%" == "" (
set VS_BUILD_ARGS=/p:_LLVMBuildDir="%MONO_LLVM_BUILD_DIR%"
)
if not "%MONO_LLVM_INSTALL_DIR%" == "" (
set VS_BUILD_ARGS=%VS_BUILD_ARGS% /p:_LLVMInstallDir="%MONO_LLVM_INSTALL_DIR%"
)
set VS_BUILD_ARGS=%VS_BUILD_ARGS% /p:MONO_ENABLE_LLVM=true /p:Configuration=%VS_CONFIGURATION% /p:Platform=%VS_PLATFORM% %VS_ADDITIONAL_ARGUMENTS% /t:%VS_TARGET%
call msbuild.exe %VS_BUILD_ARGS% "%MONO_MSVC_SOURCE_DIR%build-external-llvm.vcxproj" && (
set BUILD_RESULT=0
) || (
set BUILD_RESULT=1
if not %ERRORLEVEL% == 0 (
set BUILD_RESULT=%ERRORLEVEL%
)
)
exit /b %BUILD_RESULT%
@echo on
-52
View File
@@ -1,52 +0,0 @@
#!/bin/bash
# Set up build environment and build LLVM using MSVC and msbuild targets in mono.sln.
# Arguments:
# -------------------------------------------------------
# $1 Visual Studio target, build|clean, default build
# $2 Host CPU architecture, x86_64|i686, default x86_64
# $3 Visual Studio configuration, debug|release, default release
# $4 Mono MSVC source folder.
# $5 LLVM build directory.
# $6 LLVM install directory.
# $7 Additional arguments passed to msbuild, needs to be quoted if multiple.
# -------------------------------------------------------
function win32_format_path {
local formatted_path=$1
local host_win32_wsl=0
local host_win32_cygwin=0
host_uname="$(uname -a)"
case "$host_uname" in
*Microsoft*)
host_win32_wsl=1
;;
CYGWIN*)
host_win32_cygwin=1
;;
esac
if [[ $host_win32_wsl = 1 ]] && [[ $1 == "/mnt/"* ]]; then
formatted_path="$(wslpath -a -w "$1")"
elif [[ $host_win32_cygwin = 1 ]] && [[ $1 == "/cygdrive/"* ]]; then
formatted_path="$(cygpath -a -w "$1")"
fi
echo "$formatted_path"
}
BUILD_LLVM_MSBUILD_SCRIPT_PATH=$(cd "$(dirname "$0")"; pwd)
BUILD_LLVM_MSBUILD_SCRIPT_PATH=$(win32_format_path "$BUILD_LLVM_MSBUILD_SCRIPT_PATH/build_llvm_msbuild.bat")
MONO_MSVC_SOURCE_DIR=$(win32_format_path "$4")
MONO_LLVM_BUILD_DIR=$(win32_format_path "$5")
MONO_LLVM_INSTALL_DIR=$(win32_format_path "$6")
WINDOWS_CMD=$(which cmd.exe)
if [ ! -f $WINDOWS_CMD ]; then
WINDOWS_CMD=$WINDIR/System32/cmd.exe
fi
"$WINDOWS_CMD" /c "$BUILD_LLVM_MSBUILD_SCRIPT_PATH" "$1" "$2" "$3" "$MONO_MSVC_SOURCE_DIR" "$MONO_LLVM_BUILD_DIR" "$MONO_LLVM_INSTALL_DIR" "$7"
+17 -26
View File
@@ -267,9 +267,6 @@ Print the architecture the AOT in this copy of Mono targets and quit.
Generates all required wrappers, so that it is possible to run --interpreter without
any code generation at runtime. This option only makes sense with \fBmscorlib.dll\fR.
Embedders can set
.TP
.I depfile=[filename]
Outputs a gcc -M style dependency file.
.nf
mono_jit_set_aot_mode (MONO_AOT_MODE_INTERP);
@@ -659,9 +656,6 @@ mscorlib and assemblies located on the global assembly cache.
\fB-V\fR, \fB--version\fR
Prints JIT version information (system configuration, release number
and branch names if available).
.TP
\fB--version=number\fR
Print version number only.
.SH DEVELOPMENT OPTIONS
@@ -1179,26 +1173,23 @@ If set, tells mono to attempt using native asynchronous I/O services. If not
set, a default select/poll implementation is used. Currently epoll and kqueue
are supported.
.TP
\fBMONO_THREADS_SUSPEND\fR Selects a mechanism that Mono will use to suspend
threads. May be set to "preemptive", "coop", or "hybrid". Threads may need to
be suspended by the debugger, or using some .NET threading APIs, and most
commonly when the SGen garbage collector needs to stop all threads during a
critical phase of garbage collection. Preemptive mode is the mode that Mono
has used historically, going back to the Boehm days, where the garbage
collector would run at any point and suspend execution of all threads as
required to perform a garbage collection. The cooperative mode on the other
hand requires the cooperation of all threads to stop at a safe point. This
makes for an easier to debug garbage collector and it improves the stability of
the runtime because threads are not suspended when accessing critical
resources. In scenarios where Mono is embedded in another application,
cooperative suspend requires the embedder code to follow coding guidelines in
order to cooperate with the garbage collector. Cooperative suspend in embedded
Mono is currently experimental. Hybrid mode is a combination of the two that
retains better compatability with scenarios where Mono is embedded in another
application: threads that are running managed code or code that comprises the
Mono runtime will be cooperatively suspended, while threads running embedder
code will be preemptively suspended. Hybrid suspend is the default on some
desktop platforms.
\fBMONO_THREADS_SUSPEND\fR
Selects a mechanism that Mono will use to suspend threads. May be set to
"preemptive", "coop", or "hybrid". Threads may need to be suspended by the
debugger, or using some .NET threading APIs, and most commonly when the SGen
garbage collector needs to stop all threads during a critical phase of garbage
collection. Preemptive mode is the mode that Mono has used historically, going
back to the Boehm days, where the garbage collector would run at any point and
suspend execution of all threads as required to perform a garbage collection.
The cooperative mode on the other hand requires the cooperation of all threads
to stop at a safe point. This makes for an easier to debug garbage collector.
As of Mono 4.3.0 it is a work in progress, and while it works, it has not been
used extensively. This option enables the feature and allows us to find spots
that need to be tuned for this mode of operation. Hybrid mode is a combination
of the two that retains better compatability with scenarios where Mono is
embedded in another application: threads that are running managed code or code
that comprises the Mono runtime will be cooperatively suspended, while threads
running embedder code will be preemptively suspended.
Alternatively, coop and hybrid mode can be enabled at compile time by using the
--enable-cooperative-suspend or --enable-hybrid-suspend flags, respectively,
+2 -2
View File
@@ -41,9 +41,9 @@ class X
Version min_mono_version;
#if __MonoCS__
min_mono_version = new Version (6, 2);
min_mono_version = new Version (5, 19);
#else
min_mono_version = new Version (6, 2);
min_mono_version = new Version (5, 10);
#endif
if (version < min_mono_version)
-2
View File
@@ -30,5 +30,3 @@ hidden_suffix = .tmp
platform-check:
@:
PLATFORM_WIN32=1
Loaded 100 of 1977 files, more files were not shown because too many files have changed in this diff. Show more