Compare commits

...
Author SHA1 Message Date
Ankit Jain 86b78b7d19 [3.2-wasm][wasm][debugger] Relocate tests (#19787)
* [wasm][debugger][tests] Move the test files to `tests/debugger`

- Also, rename `class Math` to `class SampleMath` in `sample.cs`,
to allow building sample.dll, with all the tests added.

* [wasm][debugger][tests] Add back missing dependency

* [wasm][debugger][tests] move *datetime-test.cs to `tests/debugger`

* [wasm][debugger][tests] Move CFO test to debugger-cfo-test.cs
2020-05-15 14:48:26 -04:00
Tammy Qiu d46a972c3b Merge pull request #19734 from tqiu8/locale-test
[wasm][debugger][tests] Add locale tests for datetime
2020-05-11 15:29:50 -04:00
monojenkinsandLarry Ewing 57dcba253c [3.2-wasm] [wasm][debugger] Only send resolved events on load (#19739)
* [wasm][debugger] Only send resolved events on load

* Add the request to the context after potentially loading the store

* improve the resolution logic

Co-authored-by: Larry Ewing <[email protected]>
2020-05-11 10:44:31 -05:00
tqiu8 e4acd81888 incorporate PR feedback & reference issue number for disabled locales 2020-05-08 12:00:47 -04:00
tqiu8 c6bd1ac7fa [wasm][debugger][tests] Add locale tests for datetime 2020-05-07 14:16:06 -04:00
Tammy Qiu 1a6e64a938 Merge pull request #19718 from tqiu8/breakpoint-tests
[wasm][debugger][tests] Add tests for multiple breakpoints inline
2020-05-05 18:14:54 -04:00
tqiu8 feb9163b26 [wasm][debugger][tests] Combine two tests in one using [Theory] 2020-05-05 18:14:25 -04:00
tqiu8 1b697209f5 [wasm][debugger][tests] More descriptive test name 2020-05-05 17:57:42 -04:00
tqiu8 40ca9b2f37 [wasm][debugger][tests] Update CheckBreakpoint with more descriptive name and use CheckLocation 2020-05-05 17:51:55 -04:00
tqiu8 0d532de801 [wasm][debugger][tests] add tests for multiple breakpoints 2020-05-05 17:28:25 -04:00
monojenkinsandLarry Ewing 7569ee7b36 [wasm][debugger] Add tests for floating common types (#19717)
Co-authored-by: Larry Ewing <[email protected]>
2020-05-05 13:56:00 -05:00
Larry Ewing fbc81c503e [3.2-wasm][wasm][debugger] Backport of #19688 (#19716)
* [wasm][debugger] Only choose the best match when mutiple locations are found

* OrderBy column just to be certain

* Add tests for the JS breakpoint behavior that we are duplicating

* remove accidentally disabled code
2020-05-05 13:55:15 -05:00
monojenkinsandBrzVlad 8bcfbc80f3 [interp] Move label outside of main loop (#19708)
Wasm has control flow limitations which don't go well with gotos. Move label outside of main loop because it translates to very poor wasm code. Other gotos don't seem to negatively impact instruction dispatch code, so leave them alone, for now ..

Co-authored-by: BrzVlad <[email protected]>
2020-05-05 12:36:08 -05:00
monojenkinsandAnkit Jain 5dc5126323 [3.2-wasm] [wasm][debugger][tests] Split tests into separate files (#19715)
* [wasm][debugger][tests] Move helper methods to base class

* [wasm][debugger][tests] Extract EvaluateOnCallFrame tests to new file

* [wasm][debugger][tests] Extract Array tests to a new file

* [wasm][debugger][tests] Extract CallFunctionOn tests to a new file

* [wasm][debugger][tests] Extract Delegate tests to a new file

* [wasm][debugger] Don't inherit from another test class

.. as that ends up running all the tests of the base class too.

* [wasm][debugger][tests] Improve retries in TestHarnessStartup

Sometimes `/json/list` seems to return an empty array (`[ ]`). Treat
that also as a case for retry.

* [wasm][debugger][tests] Bump timeout from 2s to 5s

* [wasm][debugger][tests] Launch chrome in incognito mode

* debugging

* [wasm][debugger] TestHarnessStartup - don't share `ProcessStartInfo`

.. instances. Instead, create one for every use.
We seem to run into cases where the `get_Environment` is modifying it's
`_environmentVariables` field, concurrently from multiple threads,
causing exceptions like:

```
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Diagnostics.Process.CreateEnvp(ProcessStartInfo psi)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at WebAssembly.Net.Debugging.TestHarnessStartup.LaunchAndServe(ProcessStartInfo psi, HttpContext context, Func`2 extract_conn_url) in /Users/radical/dev/w1/sdks/wasm/ProxyDriver/TestHarnessStartup.cs:line 82
   at WebAssembly.Net.Debugging.TestHarnessStartup.<>c__DisplayClass10_0.<<Configure>b__1>d.MoveNext() in /Users/radical/dev/w1/sdks/wasm/ProxyDriver/TestHarnessStartup.cs:line 167
```

* [wasm][debugger] Fix regression in debug.html

Fixes https://github.com/mono/mono/issues/19653 .

Co-authored-by: Ankit Jain <[email protected]>
2020-05-05 12:24:46 -05:00
monojenkinsandLarry Ewing f6e60534ff [wasm][debugger] Fix type argument (#19710)
Co-authored-by: Larry Ewing <[email protected]>
2020-05-05 11:36:13 -05:00
Alexander Köplinger 6fd32d873a [3.2-wasm][ci] Run only wasm CI legs (#19711) 2020-05-05 17:57:24 +02:00
monojenkinsandAnkit Jain cc6d4bf6cf [3.2-wasm] [wasm][debugger] Fix evaluateOnCallFrame for valuetype locals (#19644)
* [wasm][debugger] Fix `evaluateOnCallFrame` for valuetype locals

.. including generic, and async methods.

* [wasm][debugger] Remove some debug spew

(cherry picked from commit 2e042925b95a2e3eba693525a5c1e495cca47f2f)

Co-authored-by: Ankit Jain <[email protected]>
2020-04-28 11:06:08 -05:00
Larry EwingandZoltan Varga c5f62a7e7d [3.2-wasm] Backport wasm index fix (#19636)
* [corlib] Fix String.Replace (..., StringComparison.InvariantCultureIgnoreCase).

Fixes https://github.com/mono/mono/issues/18978.

* Disable the test here

Co-authored-by: Zoltan Varga <[email protected]>
2020-04-22 17:32:04 -05:00
monojenkinsandZoltan Varga af98855016 [3.2-wasm] [wasm] Reenable the managed collation code, link it out by default, add a packager option to enable it. (#19608)
* [wasm] Reenable the managed collation code, link it out by default, add a packager option to enable it.

* Disable some xunit tests which fail because the test runner can't handle the [SkipOnTargetFramework(TargetFrameworkMonikers.Mono)] attribute.

* [csproj] Update project files

Co-authored-by: Zoltan Varga <[email protected]>
2020-04-22 14:20:34 -05:00
monojenkinsandMaxim Lipnin a0a972cabf [3.2-wasm] [wasm] Add throwing PNSE to where currently disabled tests throw something different (#19633)
* Add throwing PNSE to where currently disabled tests throw something different

* Add try-catch + PNSE to public SslStream ctor in case of WASM

Co-authored-by: Maxim Lipnin <[email protected]>
2020-04-22 13:47:27 -05:00
monojenkinsandAnkit Jain fe5a9ecec8 [3.2-wasm] [wasm][debugger] Fix enumerating for this on valuetypes (#19630)
* [wasm][debugger] `mono_wasm_call_function_on` - remove unncessary

.. parameter, since it is already present in the `args` object.

* [wasm] MonoProxy - convert cfo calls to MonoCommands

* [wasm][debugger] Fix enumerating `this` properties for a valuetype

* [wasm][debugger] `evaluateOnCallFrame` - fix `this` for valuetypes

* [wasm][debugger] Remove MonoCommands no longer used

* [wasm][debugger] Fix showing the name for generic methods

- We were using `method.Name` from Cecil, but that we want to the name
with the type arguments for the generic instance.

Co-authored-by: Ankit Jain <[email protected]>
2020-04-22 13:38:16 -05:00
monojenkinsandZoltan Varga 1bb2a7d171 Bump linker to the tip of mono-2020-02. (#19607)
Co-authored-by: Zoltan Varga <[email protected]>
2020-04-22 09:36:52 -05:00
Larry Ewing 70ff920a89 [wasm-3.2] Fix webdriver initialization (#19617) 2020-04-21 13:31:17 -05:00
monojenkinsandLarry Ewing 0122b3e0eb [3.2-wasm] [wasm][debugger] Fix command response handling (#19615)
* [wasm][debugger] send failed evaluations to the browser

Make sure that we allways either send a single result or return
false from Accept Command to ensure that we don't leave
messages spinning.

Co-authored-by: Larry Ewing <[email protected]>
2020-04-21 11:56:01 -05:00
7f581f1398 [3.2-wasm] [wasm][debugger] Support array expansion in VS via callFunctionOn (#19606)
* [wasm][debugger] Move `getProperties` implementation to `library_mono.js`,

.. from `MonoProxy.cs`. MonoProxy essentially just forwards the
request to `library_mono.js` now.

* [wasm][debugger] Add `length` info array details

- update tests to track the change

* [wasm][debugger][tests] Simplify some of the helper methods

* [wasm][debuggger] Implement `Runtime.callFunctionOn`

- This was prompted by vscode/vs requiring this for array expansion.
- It runs a given js function string, on a given object id.
- We handle the case for dotnet object ids.

Implementation:

- We build a proxy object that reflects the details of the real dotnet object/array
- And run the js function on that proxy object

- Then if `returnByValue` was requested, the result is returned as-is
- Else we cache that object, with a new special object id (`dotnet:cfo_res:..`),
  and return that id in the response.

- Subsequently, we would get `Runtime.getProperties` request on this new object id
- And a `Runtime.releaseObject` request, to free that function result.

- All this is handled currently

Tests:

- A few tests were added which invoke `callFunctionOn`, on JS objects, and checks the result
  - And the same checks are performed on equivalent dotnet objects, with the same js function
  - This should help stay in sync with what we think `callFunctionOn` should work as

  - These tests have an additional parameter - `roundtrip`, which calls a simple function

	`function () { return this; }`

    .. on a dotnet object, and gets the result object id. If `roundtrip == true`, then it
    runs that same function on the result object, and gets a new result object id. And the
    tests run their checks on that.

    - this helps to check that outside of the proxy, the result object behaves same as any
      other dotnet object.

- Also, most of the tests were modified to support an additional mode (`use_cfo`).
  - With this, whenever a test wants to run `Runtime.getProperties on an object, we pass that
    object through the earlier "identity" function, and then run `getProperties` on that
    resultant object.

  - This helps to take advantage most of the existing tests, and use them for `callFunctionOn`
    testing too, by ensuring that the results of CFO returns results same as any other object.

Fixes https://github.com/mono/mono/issues/19229, https://github.com/mono/mono/issues/19531

* Apply suggestions from code review

Applying suggestions to start CI

* Update sdks/wasm/src/library_mono.js

Co-authored-by: Ankit Jain <[email protected]>
Co-authored-by: Larry Ewing <[email protected]>
2020-04-21 11:32:04 -05:00
monojenkinsandLarry Ewing ae8e4b5b04 [3.2-wasm] [wasm][debugger] Add an option to clear navigator.webdriver on attach (#19592)
* [wasm][debugger] Add an option to clear navigator.webdriver on attach

* Pass the flag when it is useful

Co-authored-by: Larry Ewing <[email protected]>
2020-04-20 13:04:15 -05:00
monojenkinsandKenneth Pouncey c8cbcaad0d [wasm][sdk][nuget] Bump version. (#19585)
Co-authored-by: Kenneth Pouncey <[email protected]>
2020-04-18 00:50:30 -05:00
monojenkinsandLarry Ewing 91a8312033 [wasm][debugger] Add comment about embedded pdbs and fix old regression (#19582)
With embedded pdbs we have to try to load symbols even when the
pdb is missing then handle ArgumentException if there is no embedded
pdb.  This restores the intended error path and added an explanation
while still logging unexpected errors

Co-authored-by: Larry Ewing <[email protected]>
2020-04-17 17:42:57 -05:00
Larry EwingandAnkit Jain d76afcc147 [wasm][debugger] Show pointer type+address (#19537) (#19581)
* [wasm][debugger] Simplify the interface between debugger.c and

.. library_mono.js .

* [wasm][debugger] Show pointer type+address

* [wasm][debugger] Address review feedback

* [wasm][debugger] Small cleanups, from review feedback

* [wasm][debugger] declare mono_wasm_add_typed_value as extern

Co-authored-by: Ankit Jain <[email protected]>
2020-04-17 16:51:24 -05:00
Larry EwingandAnkit Jain fdf009e4d1 [wasm][debugger] Show signature for delegate, or target, if available… (#19535) (#19579)
* [wasm][debugger] Show signature for delegate, or target, if available (#19505)

* [wasm][debugger] Show signature for delegate, or target, if available

- As object properties, we return a `Target` which has the signature of
the delegate target.

Fixes https://github.com/mono/mono/issues/19382

* [wasm][debugger] Some tidying up

* [wasm][debugger] Remove unused `sig_desc`

* [wasm][debugger] Simplify code, avoid extra allocations

.. as suggested by @lewing

(cherry picked from commit 4eca12e794)

* [wasm][debugger] Fix tests broken because unused vars are not null

.. anymore, which is because we disabled optimizations in the
interpreter when debugging.

* [wasm][debugger][tests] Add tests to also check delegates in previous

.. frames.

* [wasm][debugger] Disable a test that tried to look at previous async

.. frames. This just happened to work. Currently, the trace has details
of the async machinery, instead of one reflecting what it might have
looked like for a non-async case. So, we can't reliably find the
previous method to inspect it's locals.

* [wasm][debugger] Fix test

Co-authored-by: Ankit Jain <[email protected]>
2020-04-17 16:31:08 -05:00
monojenkinsandLarry Ewing e630444dc6 [wasm][debugger] Always send the remove breakpoint command to the browser too (#19578)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-17 16:09:18 -05:00
Larry Ewing f585a341d0 [wasm][debugger] Handle multi-doc methods and multiple results for breakpoints (#19449) (#19577)
* [wasm][debugger] Rework source loading

Add logic to support embedded sources and
add all the documents from multi-doc methods

Add logic to check source hashes against
pdb hash.

* [wasm][debugger] Fix breakpoints on delegates

Only add methods to a source once by RID

When column is != 0 specified the fronted only wants results
that match that column exactly.

* Allow checking pdb hash and remove source cache

* Fixup typesByName merge
2020-04-17 16:04:58 -05:00
Larry EwingandZoltan Varga 56ba392b3b [wasm] Debugger test improvements. (#19499) (#19573)
* Disable the Loaded: messages, they are not very useful.

* Add a Dotnet-test.setBreakpointByMethod protocol extensions to the proxy to set breakpoints in specific methods during testing.

* Add a new sample debugger test using the setBreakpointByMethod API.

* Remove some unrelated changes.

* Restore logging.

* Disable a log message which is printed when a pdb file is missing.

Co-authored-by: Zoltan Varga <[email protected]>
2020-04-17 00:49:36 -05:00
monojenkinsandLarry Ewing 1a6d1100c4 [wasm][debugger] Fix typo and silence output (#19567)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-16 18:58:17 -05:00
monojenkinsandLarry Ewing 41a4f60bee Change OSPlatform for wasm to "BROWSER" rather than "WEBASSEMBLY" (#19568)
Following the guidance in https://github.com/dotnet/runtime/issues/33328
Change OSPlatform to BROWSER for wasm runtimeArchitecture.

Co-authored-by: Larry Ewing <[email protected]>
2020-04-16 18:58:01 -05:00
monojenkinsandvargaz c6fbfa6726 [wasm] Disable the AOT cache check on wasm as well. (#19553)
Co-authored-by: vargaz <[email protected]>
2020-04-16 14:27:34 -05:00
monojenkinsandLarry Ewing 84d12a33a0 [wasm] force set a local when running debugger tests (#19552)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-15 14:45:24 -05:00
monojenkinsandLarry Ewing 4dba91e7f5 [wasm] Use browser style lang (#19551)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-15 14:45:12 -05:00
monojenkinsandAnkit Jain c66e702ba5 [wasm][debugger] Show signature for delegate, or target, if available (#19505) (#19536)
* [wasm][debugger] Show signature for delegate, or target, if available

- As object properties, we return a `Target` which has the signature of
the delegate target.

Fixes https://github.com/mono/mono/issues/19382

* [wasm][debugger] Some tidying up

* [wasm][debugger] Remove unused `sig_desc`

* [wasm][debugger] Simplify code, avoid extra allocations

.. as suggested by @lewing

(cherry picked from commit 4eca12e794)

Co-authored-by: Ankit Jain <[email protected]>
2020-04-14 02:26:17 -04:00
monojenkinsandtqiu8 07d1f6452e [wasm] Add readme to package update scripts (#19533)
Co-authored-by: tqiu8 <[email protected]>
2020-04-13 19:59:42 -05:00
monojenkinsandAnkit Jain 3926eae609 [3.2-wasm] [wasm][debugger] Improve some valuetype visualization (#19532)
* [wasm][debugger] Fix up `value.description` for local vars too

- Local vars, when being enumerated in `mono_wasm_get_variables` don't
have an associated `name`, and so `_fixup_name_value_objects`
*incorrectly* ends up not fixing up the `value.description` field for
them.

* [wasm][debugger] For some types, use `ToString()` for their ..

.. `description`.

- `System.DateTime`
- `System.DateTimeOffset`
- `System.Decimal`
- `System.TimeSpan`
- `System.Guid`

* [wasm][debugger] Invoke all property getters, for some types

- `System.DateTime`
- `System.DateTimeOffset`
- `System.TimeSpan`

* [wasm][debugger] Show char symbol along with the code

Fixes https://github.com/mono/mono/issues/19381

* [wasm][debugger] Fix DateTime tests, don't use hardcoded strings for

.. description. Instead, use a `DateTime` object in the test to create
the expected string for the test. This should ensure that the expected
and actual were created in the same environment.

* [wasm][debugger] Special case `System.Guid` to use

.. `mono_guid_to_string`, instead of calling `ToString()` on it.

* [wasm][debugger] Don't box valuetypes for calling `ToString()`

Co-authored by @vargaz .

* [wasm][debugger] `System.Guid` is special-cased, so remove it from the

.. allowed list.

* [wasm][debugger] simple cleanup

Co-authored-by: Ankit Jain <[email protected]>
2020-04-13 19:59:15 -05:00
monojenkinsandLarry Ewing 4bfd29e820 [wasm][debugger] explicitly pass --lang=en_US for debugger tests (#19530)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-13 17:37:49 -05:00
monojenkinsandMartin Baulig f2b7b0000d Bump CoreFX to pickup https://github.com/mono/corefx/pull/397. (#19526)
Co-authored-by: Martin Baulig <[email protected]>
2020-04-13 20:15:21 +02:00
monojenkinsandKenneth Pouncey 4f156f6e32 [wasm][browser][tests] Generate sdk version in global.json file automatically. (#19527)
Co-authored-by: Kenneth Pouncey <[email protected]>
2020-04-13 12:29:56 -05:00
monojenkinsandZoltan Varga 8b290302b2 [wasm] Remove .stamp-build-debugger-test-app, the app can always be built. (#19524)
Co-authored-by: Zoltan Varga <[email protected]>
2020-04-13 11:51:11 -05:00
monojenkinsandZoltan Varga 3b761deb75 [3.2-wasm] [wasm] Disable interpreter optimizations when debugging. (#19516)
* [wasm] Disable interpreter optimizations when debugging.

* Fix tests.

Co-authored-by: Zoltan Varga <[email protected]>
2020-04-11 03:10:31 -05:00
monojenkinsandtqiu8 37221f2c32 add powershell and bash scripts for replacing default webassembly nuget packages from link or path (#19515)
Co-authored-by: tqiu8 <[email protected]>
2020-04-11 03:05:11 -05:00
monojenkinsandKenneth Pouncey 31c66244bd [3.2-wasm] [wasm][bcl] Handle correctly backslash in file name (#19514)
* [wasm][bcl] Handle correctly backslash in file name

- Modify IO file to assign backslash to alternate directory separator by default
- Add tests for issue https://github.com/mono/mono/issues/18933

* [wasm][tests] Skip two tests for wasm

- After internal discussions decided to skip these two tests on wasm

   1. GetDirectories_Backslash
   1. GetFiles_Backslash

* [wasm][tests] PathIsRooted tests failing

- skip subtest #9 if WASM
- To fix issue https://github.com/mono/mono/issues/18933 we set the AltDirectorySeparatorChar as well.  This causes the `Assert.IsTrue (!Path.IsPathRooted ("\\"), "IsPathRooted #09");` to fail. Path.IsPathFullyQualified Method handles paths that use both the DirectorySeparatorChar and the AltDirectorySeparatorChar characters.

* [wasm][tests] PathIsRooted tests failing

- skip if WASM

Co-authored-by: Kenneth Pouncey <[email protected]>
2020-04-11 02:58:37 -05:00
monojenkinsandLarry Ewing f9771388ab [3.2-wasm] [wasm][debugger] Wasm debug log level (#19512)
* [wasm][debugging] pass enable_debugging as debug engine log level

enable_debugging was already an int, with this change log_level
will remain at 1 by default but passing -1 will disable the log
spew

* Change the default debug level of 'Failed to lookup sequence point'

Co-authored-by: Larry Ewing <[email protected]>
2020-04-10 18:13:56 -05:00
monojenkinsandKenneth Pouncey 2925dcdc83 [wasm][bcl][zoneinfo] Fix local zone info marshaling (#19511)
- Correctly marshal Intl.DateTimeFormat().resolvedOptions().timeZone information

Co-authored-by: Kenneth Pouncey <[email protected]>
2020-04-10 18:13:40 -05:00
Larry Ewing d351fa1cfe [3.2-wasm][ci] Use path not name (#19513) 2020-04-10 16:59:57 -05:00
Larry Ewing 7055ff9b78 [3.2-wasm][ci] Fix typo in groovy 2020-04-10 15:36:59 -05:00
monojenkinsandLarry Ewing 08481dc20b [wasm][ci] Always build and package wasm (#19509)
Co-authored-by: Larry Ewing <[email protected]>
2020-04-10 14:53:47 -05:00
Larry Ewing 72ac1d024d Upload the zip in sdks/wasm (#19506) 2020-04-10 12:30:26 -05:00
monojenkinsandZoltan Varga a475a78a71 [runtime] Fix Type.GetTypeCode () for generic enums. (#19471)
Fixes https://github.com/mono/mono/issues/17735.

Co-authored-by: Zoltan Varga <[email protected]>
2020-04-08 12:01:16 -05:00
monojenkinsandPranav K 92e0af1500 [3.2-wasm] Produce System.Net.Http.WebAssemblyHttpHandler (#19474)
* Produce System.Net.Http.WebAssemblyMessageHandler

* Branding changes for WebAssembly.Net.Http -> System.Net.Http.WebAssemblyMessageHandler

* Change how fetch options are configured

* Fixup tests

* More fixup

* Update submodule

Co-authored-by: Pranav K <[email protected]>
2020-04-08 10:58:30 -05:00
Larry EwingandThays Grazia 4598104765 [wasm][debugger] Improving evaluate expression implementation (#19444) (#19469)
* - Adding tests to evaluate expression
- Adding a cache to local vars and object properties
- Fixing evaluate expressions like "this.a"
- Fixing evaluate expressions like "d + d"

Co-authored-by: Thays Grazia <[email protected]>
2020-04-07 17:25:02 -05:00
Larry Ewing c68c7cebe3 [sdks] Disable all sdk builds for this 3.2-wasm release branch
Disable the other sdks and enable the wask sdk for this branch
2020-04-07 15:13:29 -05:00
Thays Grazia 3fba2c8166 Some frames were skipped in MonoProxy but we don't skip the frame_id, doing this the call_stack has a different position than it's in the real stack on mono side. (#19447)
I changed the frame_id, so we will follow the frame_id from callstack even if we don't add all of them into the CallStack list in MonoProxy.
I also added a test.

Fixes #19379
2020-04-05 13:16:28 -05:00
Zoltan Varga 6ec38cc080 [llvm] Replace the LLVM got array with a set of scalar variables which are initialized by a callback called by the runtime. (#19267)
Long term, this will make the generated code depends less on got offsets etc., so it might be possible to
avoid recompiling it if the offsets change.
2020-04-05 10:21:10 -04:00
Thays Grazia 8ba4a2bca1 As suggested by @jbevain we should bump the protocol to implement this. (#19437) 2020-04-04 20:58:01 -03:00
Larry Ewing 2c5a02755d [wasm][debugger] Don't surface evaluation errors to the browser (#19439)
Instead of surfacing the error to the browser log it return
an empty result.
2020-04-04 14:24:08 -05:00
Kenneth Pouncey eb468076a7 [wasm] Wasm TimeZoneInfo implementation VFS (#17760)
* [wasm][bcl] TimeZoneInfo implementation for WebAssembly to read from WASM VFS `/zoneinfo`.

- Remove `TimeZoneInfo.WebAssembly.cs` file
- Remove source dependencies on `TimeZoneInfo.WebAssembly.cs`.
- Modify TimeZoneInfo for specific WASM functionality
   - Default root directory is `zoneinfo`
   -  Add icall `mono_timezone_get_local_name` for WASM in CreateLocal
- Add icall implementation
   - ves_icall_System_TimeZoneInfo_mono_timezone_get_local_name

* [wasm][build] Add `-s FORCE_FILESYSTEM=1` to mono wasm runtime build

- Required so that it includes support for loading pre preload packages

Message when generating the zoneinfo data.
   > Remember to build the main file with  -s FORCE_FILESYSTEM=1  so that it includes support for loading this file package

* [wasm] zoneinfo data and support files.

- Three files for zoneinfo VFS support.
   - `mono-webassembly-zoneinfo-fs-smd.js.metadata` - This is the separate metadata output by emscripten file-packager.  Used to parse the zoneinfo data in non browser environments.  See `runtime-tests.js`
   - `mono-webassembly-zoneinfo-fs.js` - Output by emscripten file-packager that will be referenced by browser environments to load the zoneinfo data into the VFS at `/zoneinfo`
   - `zoneinfo.data` - The binary file output by emscripten file-packager.  The file contains the actual zoneinfo data loaded into the VFS and parsed by `TimeZoneInfo`.

* [wasm][driver] Add backing support for icall `mono_timezone_get_local_name`

* Add TimeZone info test.

* [wasm][packger] Add `-s FORCE_FILESYSTEM=1`

- Required so that it includes support for loading preload packages

Message when generating the zoneinfo data.
   > Remember to build the main file with  -s FORCE_FILESYSTEM=1  so that it includes support for loading this file package

* Fix AOT tzd test

* Remove commented code lines for `TimeZoneInfo.WebAssembly.cs` module

* Regenerated zone info without --no-heap-copy as it causes problems.  This will create a warning in browser for now.

* Add zoneinfo support files to package

* Fix TimeZoneInfo tests to use the `--enable-fs` packager parameter

* Add back icall implementation after rebase
   - ves_icall_System_TimeZoneInfo_mono_timezone_get_local_name

* Add back backing support for icall `mono_timezone_get_local_name` after rebase

* Fix build error

* Add back zoneinfo tests to Makefile after rebase

* Fix packager zoneinfo sample builds.

* Address review comment about volatile not needed

* Address review comment about volatile not needed

* Address review comment about volatile not needed

* Address review comment volatile not needed
2020-04-04 13:37:21 +02:00
Ankit Jain b3210bf1e5 [wasm][debugger] Improve array visualization (#19424)
* [wasm][debugger][tests] Refactor test support methods

.. which will help in the upcoming changes.

* [wasm][debugger][tests] Add tests for arrays

- These don't include the broken cases which are fixed in the next
commits

* [wasm][debugger][tests] Fix enumerating locals in async method on a

.. struct.

* [wasm][debugger] Support ValueTypes in arrays

- ValueTypes in array elements are not expanded by default. Instead,
they get an object id like `dotnet:array:{arrayObjectId}:{index}`
- And these get expanded on request

* [wasm][debugger] Fix calls to `mono_wasm_add_null_var` from js

- when being called from `library_mono.js`, with a native string like -

    `MONO.mono_wasm_add_null_var ("string");`

.. which when passed to `Module.UTF8ToString (className)`, returns
`null`. So, if the result of that call is null, then use the `className`
argument as-is.

* [wasm][debugger][tests] Add test for local structs in an async method

* [wasm][debugger][tests] Add `TEST_ARGS` and `TEST_FILTER` arguments to

.. control `dotnet test` arguments.

Example use:

`make run-debugger-tests TEST_ARGS="--filter FullyQualifiedName~ArraysTests"`
or
`make run-debugger-tests TEST_FILTER="ArraysTests"`

* [wasm][debugger][tests] Make `label` the last argument, consistently.

* [wasm][debugger] Address review comments

- Return a `Result` response, instead of throwing an exception

* dummy commit to kick the pr
2020-04-03 20:36:44 -04:00
Thays GraziaandLarry Ewing 3122edcb7c [debugger][wasm] Implement evaluate in wasm debugger. (#19432)
* [debugger][wasm] Implement evaluate in wasm debugger.
-> evaluate expressions with constant values and variable values are working
-> evaluate expressions with methods not working yet
-> evaluate expressions with assignments not working yet

* removing extra ;

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Update sdks/wasm/Mono.WebAssembly.DebuggerProxy/EvaluateExpression.cs

Co-Authored-By: Larry Ewing <[email protected]>

* Fix what was suggested by @radical.

* Changing what @radical suggested

* Changing what @radical suggested

Co-authored-by: Larry Ewing <[email protected]>
2020-04-03 19:14:04 -03:00
Александр Хухлаев f713aef67b [WinForms] Show ToolTips for PropertyTab`s buttons (#19430)
for custom tabs.

when the mouse is on the native tab buttons, tooltips appear, and when on the custom tab buttons, then no
2020-04-03 22:04:46 +02:00
Thays Grazia fcc6731c83 [debugger] Fix set_value and get_value after invoke a method (#19378)
* When we run an invoke in a thread and after this we do a set_value and then a get_value, when we call set_value we consider restore_state, but when we call get_value we are not considering the restore_state.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1077790
2020-04-03 16:14:41 -03:00
Mitchell HwangandMitchell Hwang c6ff50ed49 Bump corefx to get https://github.com/mono/corefx/pull/395 (#19376)
Co-authored-by: Mitchell Hwang <[email protected]>
2020-04-03 19:39:46 +02:00
Zoltan Varga 798ced842a [wasm] Avoid including config.make into wasm/Makefile, its generated by one of the builds triggered by this Makefile. (#19427) 2020-04-03 13:18:08 -04:00
monojenkinsandBrzVlad 4988c9eeca [sgen] Implement api for allocating pinned arrays (#19407)
https://github.com/dotnet/runtime/issues/33583

Co-authored-by: BrzVlad <[email protected]>
2020-04-03 15:49:23 +03:00
monojenkinsandalexischr 5fc01701ad [mono][sre] Add an expected attribute on underlying fields of enum cl… (#19349)
…asses

Follow CoreCLR's example and add the RTSpecialName type attribute to any class field with the 'value__' name when , used as the value store for enumeration class instances.

Second part of the "RTSpecialName fix" that started with https://github.com/dotnet/runtime/pull/33389 - before, our behavior was to add the RTSpecialName immediately, as soon as the EnumBuilder was constructed.
Contributes to https://github.com/dotnet/runtime/issues/2389

Co-authored-by: alexischr <[email protected]>
2020-04-03 08:48:25 -04:00
Alexander Köplinger c55dee223f Fix iOS CI device test build
MONO_ENTER_GC_UNSAFE/MONO_EXIT_GC_UNSAFE aren't in a public header yet: https://github.com/mono/mono/issues/19421
2020-04-03 14:25:32 +02:00
Steve Sanderson bd2ca880f9 Fix Mono WebAssembly regression on Firefox (#19420)
Since 3 days ago, Mono WebAssembly no longer works on Firefox. The error is `ReferenceError: SharedArrayBuffer is not defined`.

This PR adds a trivial fix.
2020-04-03 01:09:25 +02:00
monojenkinsandCoffeeFlux edf3480ade [meta] Add mono_type_get_name_full to public API (#19414)
This is needed by Xamarin.Android to be able to round-trip with names fetched via reflection, as the only public API for this (`mono_type_get_name`) uses the IL format instead.

Fixes https://github.com/mono/mono/issues/19377

Co-authored-by: CoffeeFlux <[email protected]>
2020-04-02 12:45:51 -04:00
54e6ca20b4 [loader] Add append option to loader hook installation functions (#19371)
* [loader] Add append option to loader hook installation functions

Fixes https://github.com/dotnet/runtime/issues/34273

While this specific issue only affects the preload hook, I've added the append option to all the installation functions since this seems like a generally useful capability, both for internal use and embedders.

@BrzVlad tested locally and confirmed this patch solves the problem.

* Update callers in pedump and monodis

Co-authored-by: CoffeeFlux <[email protected]>
Co-authored-by: Ryan Lucia <[email protected]>
2020-04-02 12:39:10 -04:00
Larry Ewing 82dc6e8a5e [wasm][debugger] don't try to process commands sessions without context (#19394) 2020-04-02 08:35:34 -05:00
Kenneth Pouncey 2c783ab80b [wasm][README] WASM Dynamic Runtime doc (#19416)
* update sample config

* Add dynamic runtime config

* update dynamic runtime docs
2020-04-02 08:35:11 -05:00
Ankit Jain 2709c812c0 [wasm][debugger] Add support for visualizing valuetypes (#19232)
* [wasm][debugger] Move some of the variable name/value pairs fix up to

.. library_mono.js from MonoProxy.

* [wasm][debugger] Handle invalid scope id

- this would throw NRE, and the request would fail, and so the test
passed.

* [wasm][debugger][tests] Check functionName at breakpoints

.. and fix up tests.

* [wasm][debugger][tests] Remove unnecessary `ctx` arg

* [wasm][debugger] Add support for visualizing valuetypes

- Valuetypes are expanded into json, and cached. That gets used to
answer any `getProperties` request for them.

- When a valuetype is a local var, it's whole value expansion is
returned in `members` json field. This gets cached by MonoProxy, and
used to answer any `getProperties` requests for it's members.

- When a valuetype is part of an object, it is *not* expanded. Instead,
it gets an objectId of the form `dotnet:valuetype:containingObjectId:fieldOffset`.
  - Later, when the debuggers asks for that special id, we expand *all*
  the valuetypes in that containing object, and cache it.

  - For properties, the property "number" as they are enumerated, is
  used instead of the fieldOffset.

- this cache gets cleared whenever we step/resume the debugger.

* [wasm][debugger] Add support for visualizing enums

TODO: Flags

* [wasm][debugger] Don't invoke property getters when enumerating object

.. members.

* [wasm][debugger] Override `ToString` for `Result` class
2020-04-02 09:04:42 -04:00
sharkcz 2a531e04b3 update perf jitdump for ppc64/ppc64le and s390x (#19405)
Add missing constants to handle ppc64/ppc64le and s390x platforms.

Related: https://github.com/mono/mono/commit/68a8b9290958ee5e9b21d8ff6b73a707e2504d39
Fixes: https://github.com/mono/mono/issues/19402
2020-04-01 22:07:22 +02:00
Alex Thibodeau b1d4be3bab Ensure generic parameter constraint type is included when building image (#19395)
sets.

Making associated change to type_in_image to also check the constrained type for a match. Re-adding asserts now they they no longer trigger.

fixup, accidentally used old function

adjusting coding convention to K&R
2020-04-01 16:01:01 -04:00
Zoltan Varga bd635ac8b4 [wasm] Throw an exception when trying to start a thread instead of asserting. (#19396) 2020-04-01 15:13:20 -04:00
Alexis Christoforides 5733960ac4 [ios][sdks] Fix finalizer thread init in runner (#19412)
Starting with https://github.com/mono/mono/pull/16907 , the runtime ends in GC-Safe state (mode) after mono_jit_init_version() is called. mono_gc_init_finalizer_thread() expects the GC to not already be in safe mode.
2020-04-01 21:05:55 +02:00
Alexander Köplinger 9485a66d20 Fix Windows .msi build using newer xar
The old version didn't compile anymore with OpenSSL 1.1
2020-04-01 16:31:27 +02:00
monojenkinsandEgorBo a92528508e [mono] Fix setMArch, getArchName() doesn't work as expected (#19403)
Turns out `llvm::Triple(llvm::sys::getDefaultTargetTriple()).getArchName()` doesn't work as expected on all targets so the only option is to set arch via defines.

NOTE: it accepts "x86-64", not "x86_64

Co-authored-by: EgorBo <[email protected]>
2020-04-01 14:23:31 +02:00
monojenkinsandEgorBo ad0ee22923 [mono] Fix LLVM JIT on Arm64 (#19397)
`EB.selectTarget ();` used to return `NULL` on arm64 (and hit an assert later)
Turns out it needs MArch on arm (e.g. `"aarch64"`) set. No idea why it doesn't use host arch by default on arm (but works on x86).

Co-authored-by: EgorBo <[email protected]>
2020-04-01 12:43:06 +02:00
Alexander Köplinger e10990125d Fix Windows .msi build with recent cygwin updates
Cygwin now ships libssl 1.1 which removed OpenSSL_add_all_ciphers so the xar configure fails.
Replace the configure check with another existing function to fix it.
2020-04-01 12:36:26 +02:00
monojenkinsandpi1024e 49ea50d5b4 Remove duplicated .gitignore patterns (#19370)
Co-authored-by: pi1024e <[email protected]>
2020-04-01 11:46:54 +02:00
monojenkinsandEgorBo f301481bc1 [mono] fix SIMD instructions' availability (#19387)
Fixes: https://github.com/dotnet/runtime/issues/34304

@tannergooding @davidwrighton
Now it fully matches clang/LLVM behavior, e.g. even `-mattr=+avx2` (or `-mavx2` in clang) doesn't enable `pcmul` and `aes`. However, `-mattr=+sse4.2` (`-msse4.2`) enables `popcnt` (see https://godbolt.org/z/RWAp_D). The intel manual says

![image](https://user-images.githubusercontent.com/523221/78015652-59d70e00-7352-11ea-9f88-64772e76f842.png)

So is there a CPU with AVX set but without `popcnt`? The instruction is pretty useful (it is used in BCL) users might forget to define it when they use `mattr=avx` (or `mattr=avx2`).

However in C++ world programmers use named groups e.g. `mcpu=haswell` rather than listing these sets via attributes.

Co-authored-by: EgorBo <[email protected]>
2020-04-01 11:41:50 +02:00
Thays Grazia a17ab75dea Adding static to ss_req_acquire and ss_req_count (#19390) 2020-03-31 15:37:04 -03:00
Fan Yang 68a8b92909 Initial perf jitdump implementation (#19273)
* Initial perf jitdump implementation

* Incorporate PR feedback

* Incorporate feedback round 2

* Incorporate feedback round 3

* Minor change

* Incorporate feedback round 4
2020-03-31 09:03:43 -04:00
Zoltan Varga b9f52f3681 [runtime] Fix a problem with PR #19361. (#19374) 2020-03-31 05:06:12 -04:00
Kenneth Pouncey d525e3d5a3 Document threading setup for webassembly build (#19341) 2020-03-31 06:33:30 +02:00
Kenneth Pouncey 62e6a2e0c6 Reactivate the browser threads tests. (#19365) 2020-03-31 06:32:38 +02:00
monojenkinsandimhameed 722e58cc15 [mono][llvm] Improve LLVM 9 compatibility. (#19340)
Improve LLVM 9 compatibility.

Several intrinsics were removed in LLVM 7 and above; replace them with
IR sequences that generate the desired instructions.

Fix some type mismatch problems with the generated IR (in
Sse2.StoreAlignedNonTemporal, Vector128.CreateScalarUnsafe, and
Sse2.MaskMove); these were caught by running the coreclr hardware
intrinsic JIT tests.

Support the two-argument forms of Sse.SqrtScalar, Sse.ReciprocalScalar,
and Sse.ReciprocalSqrtScalar.

llvm::linkCoreCLRGC(), in llvm/CodeGen/GCs.h, was removed and replaced with
llvm::linkAllBuiltinGCs(), in llvm/CodeGen/BuiltinGCs.h.

In mono_llvm_dump_value and mono_llvm_dump_module, flush the c library's
stdout stream before using (and flushing) LLVM's own buffered output
functions.

Co-authored-by: imhameed <[email protected]>
2020-03-30 14:01:45 -07:00
Zoltan Varga b745e20e84 [wasm] Add a try-catch around GetCallingAssembly () calls in ResourceManager, it can fail on wasm aot. (#19358)
Fixes https://github.com/mono/mono/issues/19314.
2020-03-30 13:57:26 -04:00
Jérôme Laban 5bf237a2a4 [Wasm] Fix string_decoder when running with threading enabled (#19355)
* [Wasm] Fix string_decoder when running with threading enabled

Avoids the following error message:

```
Failed to execute 'decode' on 'TextDecoder': The provided ArrayBufferView value must not be shared.
```

* [Wasm] Cache text decoder
2020-03-30 10:13:55 -05:00
Zoltan Varga 4f6b3f30f0 [wasm] Throw an exception in some blocking operations in non-threaded mode. (#19363)
* [wasm] Throw an exception from Monitor.Wait () in non-threaded mode.

* [wasm] Throw an exception when waiting for unsignalled events in non-threaded mode.
2020-03-30 08:36:22 -04:00
monojenkinsandBrzVlad 771fbddefa [interp] Optimize Span<byte>.Fill (#19351)
Co-authored-by: BrzVlad <[email protected]>
2020-03-30 11:56:02 +03:00
Zoltan Varga a30be514e2 [llvmonly] Avoid marking the rgctx/vtable variable indirect, its not needed. (#19287)
In normal llvm mode, these variables are marked as indirect and stored to a stack slot so the EH code can access them.
2020-03-30 04:54:00 -04:00
Zoltan Varga 07febe2bb0 [runtime] Allocate the memory for gshared gparams from image sets. (#19361)
Fixes https://github.com/mono/mono/issues/18127.
2020-03-30 04:43:38 -04:00
Larry Ewing cf7d5d5f45 [wasm][sdk] fix output files (#19360) 2020-03-29 16:36:48 -04:00
Zoltan Varga fbc3a9757a [wasm] Fix threads support in the packager. (#19359) 2020-03-29 16:17:07 -04:00
Zoltan Varga 6e9e5b8ef2 Bump ikvm to pick up 26a033ed9de36f462946232a9b1b7984618c5e1f. 2020-03-29 13:21:00 +02:00
Zoltan Varga f2878d45d8 [wasm] Bump emscripten to 1.39.11. (#19342)
* [wasm] Bump emscripten to 1.39.11.

* [wasm] Build the debug runtime using -g, -g4 disables dwarf.

* Add runtime debugging docs.

* Avoid copying the dotnet.wasm.map file its not generated anymore with dwarf.
2020-03-29 04:21:53 -04:00
Zoltan Varga d336d6be30 [llvm] Handle this arguments as inputs to phi nodes, i.e. for IL code doing starg.0. (#19323) 2020-03-28 03:59:37 -04:00
Vlad Brezae 450a6d6ac7 [interp] Constant fold also checked conversion opcodes (#19294)
* [interp] Constant fold also checked conversion opcodes

* [interp] Remove duplicated opcode
2020-03-27 23:06:25 +02:00
Alex Thibodeau e5be81d921 Need to convert from uint instead of cast here to maintain consistency with mono_gchandle_new_weakref_internal. (#19333)
Looks like this was missed in: https://github.com/Unity-Technologies/mono/commit/941a335ea0f20c22a02a7947945f53787a56b2d3
2020-03-27 20:58:32 +01:00
monojenkinsandBrzVlad 09f5ce3588 [interp] Fix issues around not yet supported System.Runtime.Intrinsics (#19344)
Special cases around infinite recursiveness were not handled by the interpreter.

Co-authored-by: BrzVlad <[email protected]>
2020-03-27 17:03:17 +02:00
Kenneth Pouncey 42c9cfb1fd [wasm][tests] Cleanup browser tests (#19325)
* fix clean target

* Upgrade project to netstandard2.1 and change to not append framework when publish

* Move projects to netstandard2.1

* Remove code referencing framework paths
2020-03-27 06:05:55 +01:00
Kenneth Pouncey 48871c1ff5 [wasm][sdk] Remove Required attribute from Bindings Directory. (#19326)
- Causes build error if sdk path is not being overridden.
2020-03-27 06:04:49 +01:00
Zoltan Varga 36f41ac8df [wasm] Build aot-hello using --nobinding. (#19331) 2020-03-26 16:56:15 -04:00
Thays Grazia 282320c35f [debugger] Bump protocol for this commit https://github.com/mono/mono/pull/19248 (#19318)
* Bump protocol for this commit https://github.com/mono/mono/pull/19248 as suggested by @vargaz.

* Fix coding convention.
2020-03-26 16:51:45 -03:00
monojenkinsandCoffeeFlux 1add5cbc06 [meta] Make monovm.h public (#19305)
Co-authored-by: CoffeeFlux <[email protected]>
2020-03-26 11:42:21 -04:00
monojenkinsandCoffeeFlux 4c36b361c4 [loader] Initial work to support ALC collectability (#19304)
This should enable creation of a collectible ALC from managed, though it will not actually unload properly when Unload is called. This should be enough to unblock Xamarin porting work.

If you think the `mono_alc_cleanup` work is just noise I can remove that, but I thought it helpful to check in to make future diffs clearer as I split it up with the LoaderAllocator work.

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-26 11:17:57 -04:00
Thays Grazia 78fe8c2969 [debugger] Fix suspend_policy that will be send to debugger-libs (#19317)
- Fix suspend_policy that will be sent to debugger-libs, because on debugger-libs we want to resume the VM only when it's suspended, but we were sending the wrong suspend_policy in the case of EVENT_KIND_VM_START and mono is started with suspend=y.
  With this fix we can do this PR again: https://github.com/mono/debugger-libs/pull/264/

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/999375/
2020-03-26 10:15:11 -03:00
Martin Baulig c9171a63be [wasm][debugger]: Correctly handle null sessionId. (#19327)
When the sessionId is null, don't send "null" but omit the field;
Canary does not permit "sessionId: null".
2020-03-26 14:07:14 +01:00
Zoltan Varga 151ba57d0f [llvm] Disable running the llvm verifier by default, it was enabled by mistake, and it takes a long time. (#19319) 2020-03-26 08:10:30 -04:00
Martin Baulig b0e3196b9f [wasm][debugger]: Move some of the internal helper classes into a separate file. (#19324) 2020-03-26 11:52:38 +01:00
monojenkinsandlateralusX c5a9510329 Add stack guarantee on Mono Windows to handle stack overflow. (#19296)
When hitting stackoverflow on Windows, the amount of stack available to exception handler is too small. Windows doesn't support alt stack as other platforms, but there is a way to guarantee a certain amount of available stack during stack overflow exceptions using
SetThreadStackGuarantee.

Fix add support on Windows platforms including SetThreadStackGuarantee API to reserve space needed to handle stack overflow exceptions. On Windows debug build the size needs to be larger since we might end up calling JIT that could have quite deep callstack that, on unoptimized
builds, will need more stack.

Co-authored-by: lateralusX <[email protected]>
2020-03-26 10:29:52 +01:00
Kenneth Pouncey eb9cb9b302 [wasm] Add make target to clean the framework directory (#19316) 2020-03-26 05:46:51 +01:00
d9089e914b [wasm][bindings] Remove typedarray length validation (#19315)
* [wasm][bindings] Remove typedarray length validation

fixes https://github.com/mono/mono/issues/19288

* Update sdks/wasm/framework/src/WebAssembly.Bindings/Core/TypedArray.cs

Co-Authored-By: campersau <[email protected]>

Co-authored-by: Larry Ewing <[email protected]>
Co-authored-by: campersau <[email protected]>
2020-03-26 05:46:00 +01:00
Matt Mitchell 437ca1d81c Feed cleanup (#19321)
Switch away from blob feeds
2020-03-25 19:52:08 +01:00
Alex ThibodeauandJonathan Chambers b1a1f06110 Do not access unloading domains in debugger (case 1013579) (#19299)
There was a race where a domain was being unloaded while debugger would access it. A domain was only removed from the 'appdomains_list' as a very last step. The domain was already invalid to access at this point (locks freed for example).

Worse, images would be unloaded if only referenced by that domain. All places the debugger iterates domains hold the loader lock. The loader lock is acquired by the domain unloading process, so as long as a domain is not unloading when we access it inside of the loader lock we are safe.

include domain unloading check in collect_domain_bp

Co-authored-by: Jonathan Chambers <[email protected]>
2020-03-25 14:15:08 -03:00
monojenkinsandCoffeeFlux 2881514b3d [loader] Fix some log masks in native-library (#19280)
Copy-paste error

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 16:43:41 +01:00
CzBiX 9967b065b2 Refine UnixEndPoint to support Windows Unix socket (#19274)
On windows, remote end point client will not return file name, but socket address size still isn't `0x02`.
So check empty file name after bytes readed.
2020-03-25 16:25:57 +01:00
monojenkinsandCoffeeFlux 45694f7c99 [loader] Clear filename when debugging assembly unloading (#19303)
Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 11:09:35 -04:00
monojenkinsandCoffeeFlux a89b6d3217 [metadata] Skip symbol name mangling on non-Windows for netcore (#19281)
Fixes dotnet/runtime#33246

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-25 15:23:00 +01:00
Kenneth Pouncey 61830e01f6 [wasm][sdk] Add functionality to override the bindings framework location (#19310)
* [wasm][sdk] Add functionality to override the bindings framework location

- Facilitate testing of intree build during the browser tests.

* white space
2020-03-25 12:02:46 +01:00
Vlad Brezae 8ec501a34f [jit] Initialize correct class for tls fields (#19300)
Initialize the klass containing the field to be accessed, rather than the class of the method from where we are accessing.
2020-03-25 10:06:27 +02:00
Zoltan Varga 8543df76a9 [llvmonly] Throw an ExecutionEngine exception with a helpful message when we try to execute a method which failed AOT compilation. (#19302) 2020-03-25 00:40:09 -04:00
dotnet-maestro[bot]anddotnet-maestro[bot] 4eb25bc4b6 Update dependencies from https://github.com/dotnet/arcade build 20200321.1 (#19289)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20171.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20171.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-25 00:40:51 +01:00
monojenkinsandCoffeeFlux 9f390a7d8a [metadata] Fix leak in custom attribute lookup (#19279)
https://github.com/dotnet/runtime/commit/9dec6ae2ea81d46884bf49d2df8b26cc1e28eb12#r37604565

Co-authored-by: CoffeeFlux <[email protected]>
2020-03-24 19:40:34 -04:00
a9bf876bb7 [debugger] Implementing step through multithreaded code. (#19103)
* Implementing step through multithreaded code.

* Fixing concurrency in the case that more than one thread will run the same code, then when the jit is finished, the thread which jitted the code calls jit_done and the others that were waiting don't call, in this case, the other threads can run before the thread which is calling the jit_done set the breakpoint, so the other threads will not stop in the breakpoint.
As I understood we don't need to check if the try_suspend_runtime is returning false or true when we  hit a breakpoint from context, so removed the call.

Fixes #14456

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
Co-authored-by: monojenkins <[email protected]>
2020-03-24 18:24:33 -03:00
Alexander Köplinger 4cbcc9195d Reenable a bunch of disabled iOS Mono SDKs test suites (#19301)
* Reenable a bunch of disabled iOS Mono SDKs test suites

System, System.Net.Http, System.Web.Services, Mono.CSharp and Mono.Data.Sqlite test suites are running now.

Also tweaks the test UI so you can query it via Xcode's UITest in an automated way.

* Switch to APIs from public headers added in 1425b4d4d0
2020-03-24 20:41:38 +01:00
monojenkinsandlambdageek 1425b4d4d0 [mono] Make some API functions public and private (#19259)
See https://github.com/dotnet/runtime/pull/33633 and https://github.com/xamarin/xamarin-macios/blob/master/runtime/exports.t4

* `mono_install_load_aot_data_hook` was already a `MONO_API`, but it was not in a public header.
  It is added to `mono/jit/mono-private-unstable.h`
* `mono_gc_init_finalizer_thread` was [used by Xamarin.iOS](https://github.com/xamarin/xamarin-macios/blob/63ab48e679716264b33af00f86c24426da1d9f97/runtime/mono-runtime.h.t4#L278) for a long time without being in any public header.  It's one of the reasons we have to compile Mono for XI with `--disable-visibility-hidden`.  Make the function a proper `MONO_API` unconditionally - but make it do nothing if the runtime is compiled without `--with-lazy-thread-creation` (the default).  It is now a public Mono API function.
* `mono_trace_init` was a `MONO_API` function in a non-public header that was already used by embedders that need to set up logger hooks.  It is now not necessary to call this before calling the logger functions to set up logger hooks such as `mono_trace_set_log_handler`

Co-authored-by: lambdageek <[email protected]>
2020-03-24 09:29:34 -04:00
Vlad Brezae 77bcd7497d [interp] Replace ldloca + ldfld/stfld pairs with ldloc/stloc (#19286)
* [interp] Move INDIRECT flag to a separat counter variable

We will optimize away some ldlocas, so a local can stop being indirect at a future cprop iteration.

* [interp] Replace ldloca + ldfld/stfld pairs with ldloc/stloc

If loaded/stored field is the only field of the valuetype. This is meant to mainly optimize IntPtr accesses.

* [interp] Add counter for removed LDLOCA
2020-03-24 13:59:03 +02:00
Alexander Köplinger 81939f45c4 Bump nunitlite to get https://github.com/mono/NUnitLite/pull/26 2020-03-24 11:17:32 +01:00
monojenkinsandlateralusX db7c5aa72e Remove Mono runtime vcruntime140.dll release build dependency. (#19270)
CoreCLR static link vcruntime140.dll, but dynamic link rest of C-runtime using ucrt.lib in release build. Commit align Mono runtime to follow same pattern, removing dependency on vcruntime140.dll.

Co-authored-by: lateralusX <[email protected]>
2020-03-24 08:24:01 +01:00
monojenkinsandlateralusX 7d33bd36f1 Implement icall used by RuntimeHelpers.EnsureSufficientExecutionStack on netcore Mono. (#19271)
Current implementation always returned true for Windows/Linux/Android. There are however use of this API in Task library to make sure recursive tasks won't hit stackoverflow. There are several tests in System.Threading.Tasks that currently hit this, at least on Windows, causing test failures.

Fix use stack limits already setup in register_thread and used by GC when doing conservative stack scan. If the limits have not been setup or is not supported on platform, we will use old defaults, always assume there is enough stack space available. Heuristics around size of minimum execution
stack needed are picked from corresponding CoreCLR implementation.

With this fix all tests run in System.Threading.Tasks.Tests pass on Windows using Mono runtime:

```
=== TEST EXECUTION SUMMARY ===
     System.Threading.Tasks.Tests  Total: 682, Errors: 0, Failed: 0, Skipped: 3, Time: 11,419s
----- end 2020-03-20 13:43:35,72 ----- exit code 0 ----------------------------------------------------------
```

Before fix the following tests hit stackoverflow:

LongContinuationChain_ContinueWith_DoesNotStackOverflow
LongContinuationChain_Unwrap_DoesNotStackOverflow
LongContinuationChain_Await_DoesNotStackOverflow
RunStackGuardTests

Co-authored-by: lateralusX <[email protected]>
2020-03-24 08:22:32 +01:00
Johan Lorensson fa0f2ce8ab Move unbox tramp method address table to data segment, MONO_ARCH_CODE_EXEC_ONLY. (#19292)
unbox tramp method addresses are emitted as a table of calls and read
at runtime by get_call_table_entry. This is problematic on platforms
that doesn't allow data to be read from instruction stream.

Fix follow same pattern as we method address table emitted as function
pointers into table in data segment when using MONO_ARCH_CODE_EXEC_ONLY.
2020-03-24 00:28:15 -04:00
monojenkinsandlambdageek 9504df9ad2 [mono] retry mono_threads_pthread_kill if result == EAGAIN on Linux (#19291)
Try to address https://github.com/dotnet/runtime/issues/32377
(signal queue overflow) by sleeping and retrying a few times.

This is kind of a hack, but it would be good to see if it will address the issue

Co-authored-by: lambdageek <[email protected]>
2020-03-23 23:14:00 -04:00
monojenkinsandlambdageek 0102498d01 [mono] Add headers for unstable APIs (#19272)
We have functions that are de-facto part of the Mono embedding API that are
used by Xamarin.iOS or Xamarin.Android that are not in a state where we want to
commit to supporting them forever in their current form.

Nonetheless, we would like to make sure that the symbols for these functions
are visible even if we otherwise compile the runtime with
`-fvisisibility=hidden`, and we would like to discourage declaring the
functions in the embedder's own headers.

The functions that go into the
`mono/{utils,metadata,jit}/mono-private-unstable.h` headers will all be marked
with `MONO_API MONO_RT_EXTERNAL_ONLY` but we will not guarantee that the
functions will be there from one release to the next or that they will not
change their signatures or their behaviors.

Co-authored-by: lambdageek <[email protected]>
2020-03-23 19:54:25 -04:00
Zoltan Varga bf75db1199 [jit] Perform inlining earlier during call processing to avoid loading vtable/rgctx args which are not needed. (#19283)
* [jit] Perform inlining earlier during call processing to avoid loading vtable/rgctx args which are not needed.

* Avoid inlining calls marked with .tail.
2020-03-23 15:16:56 -04:00
Jay Krell 93457ec8c3 [debug_interp] Do not use uninitialized local. (#19275) 2020-03-23 12:38:37 +01:00
CoffeeFlux 47d542a1b8 [loader] Fix mistake in netcore_load_reference comment
The code here was right, the comment wrong. Also updated to mention that we now check APP_PATHS.
2020-03-23 07:11:05 +01:00
Zoltan Varga 3a08f494a9 [llvm] Unify the emission of GOT access. NFC. (#19250)
* [llvm] Use the _typed version of get_aotconst () everywhere. Lowercase aotconst variable names.

* [llvm] Unify the emission of GOT access. NFC.

* [llvm] Unify the mini_llvm_init_method icalls, use only 1 icall.
2020-03-21 21:17:58 -04:00
Vlad Brezae e0ea5d5de5 [interp] Remove MINT_TYPE_P (#19269)
MINT_TYPE_P is equivalent to the integer versions. By removing it we reduce the number of interpreter types and instructions, making it also easier to apply optimizations on instructions targeting these types.
2020-03-20 17:08:38 +02:00
campersau 12055c887b [wasm] WasmHttpMessageHandler abort connection when response or response content is disposed (#18765) 2020-03-20 16:07:52 +01:00
Aleksey Kliger (λgeek) a416864867 [coop] Make MONO_ENTER_GC_UNSAFE fatal if no_safepoints is set (#18992)
Previously it would print a warning, only, because otherwise we would get a
second assert in the same place while the crash machinery would run from the
current thread from the abort in `mono_fatal_with_history`

The solution is to turn off the "no safepoints" bit since we know we're
definitely going to assert here.

Also record the current state in the checked mode state transition history to
make it a bit easier to diagnose where the problem surfaced.

Also fixed up a pair of other debugging/tracing messages so the message strings
accurately reflect the state.
2020-03-20 10:58:41 -04:00
41d607b629 Updating the Copyright line display when running 'mono --version' (#19060)
Removes the copyright year.

```
$ mono --version
Mono JIT compiler version 6.6.0.166 (2019-08/d9001b5ae70 Wed Feb 12 19:20:16 EST 2020)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          yes(610)
	Suspend:       hybrid
	GC:            sgen (concurrent by default)
```

Co-authored-by: Alfonso, Ivan <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
2020-03-20 13:22:08 +01:00
Thays Grazia 871bfad41d [debugger] Fix NOT_IMPLEMENTED while debugging. (#19248)
- Changed the behavior on debugger-agent, if we can't parse the new behavior is to return invalid_argument and not assert and stop debugging
- Changed the mono_domain_set_fast before return from assembly_commands.
- Add error message when return INVALID_ARGUMENT

Fixes #19146
2020-03-20 09:21:26 -03:00
Jay Krell ca1e5c3214 Remove dead code from bug-10127.cs. (#18762) 2020-03-20 13:08:06 +01:00
Phil Jaenke aa688393ba [FreeBSD] Fix for libinotify support and compile (#19234)
Resolves #18528 in for both detection and compiling. This should backport cleanly to next release.

Tested on 12.1-RELEASE with libinotify; configure detects features correctly, compiles successfully, resolves some observed issues during run.

```
#define HAVE_SYS_INOTIFY_H 1
#define HAVE_LIBINOTIFY 1
#define HAVE_INOTIFY_INIT 1
#define HAVE_INOTIFY_ADD_WATCH 1
#define HAVE_INOTIFY_RM_WATCH 1
#define HAVE_INOTIFY 1
```
2020-03-20 13:00:13 +01:00
dotnet-maestro[bot]anddotnet-maestro[bot] 4a22f285e5 Update dependencies from https://github.com/dotnet/arcade build 20200312.3 (#19236)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20162.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20162.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-20 12:49:55 +01:00
Alexander Köplinger 0763fb8b13 Bump boringssl submodule 2020-03-20 11:54:54 +01:00
monojenkinsandlambdageek 2a11ec6687 [mono] Enter GC Unsafe mode in some string allocation functions (#19261)
All these functions are MONO_RT_EXTERNAL_ONLY and they all allocate managed
objects.

Fixes a coop state machine assertion if an embedder calls the function and it needs to take the GC
lock to do a collection.

Co-authored-by: lambdageek <[email protected]>
2020-03-19 19:00:26 -04:00
monojenkinsandEgorBo d1362b026e [mono] return false for AdvSimd.IsSupported and friends (#19263)
Fixes StackOverflow in https://github.com/dotnet/runtime/pull/33749#issuecomment-601271356
Intrinsify all `get_IsSupported` under `System.Runtime.Intrinsics*` to just `false` (except the sets we support, see mono_emit_simd_intrinsics).

Co-authored-by: EgorBo <[email protected]>
2020-03-20 01:30:02 +03:00
Zoltan Varga 8ca4d3e6a2 [runtime] Fix the mapping of 'int' in jit icall signatures, it should be int32 not native int. (#19249)
* [runtime] Fix the mapping of 'int' in jit icall signatures, it should be int32 not native int.

* Add a sizet type to icall signatures.
2020-03-19 06:38:19 -04:00
monojenkinsandlateralusX 729bd0a88d Ignore register unwinder callback for empty patch_info data table on Windows Mono Runtime. (#19254)
Running System.Runtime.Serialization.Xml test suite triggers calls to mono_code_manager_reserve with 0 size code block. On Windows, allocating code regions from code manager will also add callback for code region into unwinder function table, but since region could be 0 length, it will be causing a crash in RtlInstallFunctionTableCallback OS call.

Fix hardens code to not try to register callbacks for code regions of size 0, since that is considered a critical error by Windows OS API.

Co-authored-by: lateralusX <[email protected]>
2020-03-19 11:02:03 +01:00
Vlad Brezae 4359b8ff6f [interp] Optimize out intptr ctor (#19252)
* [interp] Extract nint/nfloat intrisifying code into separate method

* [interp] Optimize out intptr ctor

We avoid the newobj_vt opcode and the ctor's contents by emitting a conversion opcode directly.
2020-03-19 10:37:21 +02:00
monojenkinsandlateralusX 468ef7b362 Mono DISABLE_SIMD still emits OP codes depending on SIMD support. (#19251)
Fix makes sure runtime checks also report that SIMD has been disabled. Returning FALSE from is_simd_supported will turn of MONO_OPT_SIMD and disable all SSE/AVX cpu features will prevent code checking cpu capabilities to emit SIMD instructions.

Co-authored-by: lateralusX <[email protected]>
2020-03-18 23:31:47 -04:00
monojenkinsandlateralusX d43e869bc8 Fix warning building Mono using -Wint-conversion. (#19253)
Co-authored-by: lateralusX <[email protected]>
2020-03-18 23:29:25 -04:00
Vlad Brezae 001d4b3338 [interp] Inline also constructors of valuetypes (#19238)
* [interp] Inline also valuetype constructors

* [interp] Unify vt and non-vt inlining code

* [interp] Fix c++
2020-03-18 23:47:08 +02:00
Vlad Brezae 0d9d923e94 [interp] Do cfold for a few more unary operators (#19200)
That are accessing the stack value at level -2.
2020-03-18 23:11:30 +02:00
Johan Lorensson 56088a653f Fix additional constants embedded in instruction stream on AMD64. (#19246)
A couple of more op codes embedded constant values in instruction stream.
When using MONO_ARCH_CODE_EXEC_ONLY these needs to be moved into GOT slots.
2020-03-18 13:51:00 +01:00
Jay Krell acfa161730 Remove useless ip += from MINT_JMP, it is assigned later. (#19217) 2020-03-18 02:42:56 -04:00
Kenneth Pouncey c1b0b95549 [wasm] cleanup npm/node install. (#19244)
browser tests were failing with node command not being found if the other tests were not run before.  This makes sure that the  NPM command can be found by installing it if not there.

This also separates out the npm version that is being installed.
2020-03-17 11:11:53 +01:00
Johan Lorensson 696ec4da7e Fix usage of JIT_FLAG_CODE_EXEC_ONLY flag. (#19237)
Current implementation didn't convert the JIT flags into options on cfg
meaning that code currently checked JIT_FLAG_CODE_EXEC_ONLY never
executed. Fix handles the new JIT flag in same way as other JIT flags,
converted into bit field on cfg.
2020-03-17 00:51:10 -04:00
Alexis ChristoforidesandAleksey Kliger edf6c286a1 [merp] Add breadcrumb for StackHash (#19240)
* [merp] Add breadcrumb for StackHash

Contributes to https://github.com/mono/mono/issues/18715

* Update mono/utils/mono-state.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>

* Update mono/metadata/threads.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
2020-03-16 22:23:21 -04:00
Zoltan Varga 05880086f5 [jit] Transition the SIMD code to use the MONO_CPU_... set of flags instead of the SIMD_VERSION_... flags. (#19139)
* [jit] Transition the SIMD code to use the MONO_CPU_... set of flags instead of the SIMD_VERSION_... flags.

* Enable SIMD version checking when targeting llvm.
* AOT test assemblies using -mcpu=native.

* Remove unused mono_arch_cpu_enumerate_simd_versions (). Move the SIMD_VERSION_ constants to simd-intrinsics.c.

* Use -mcpu=native for aot testing.
2020-03-16 21:02:32 -04:00
Mitchell HwangandMitchell Hwang ab893a369a [Loader] Change mono_trace level from info to debug (#19110)
* [metadata] Change mono_trace levels from info to debug

* [mini] Change mono_trace level from info to debug

* [sgen] Change mono_trace levels from info to debug

* Amended metadata changes

* Amended mini changes

* Amend assembly.c no special case

* Amend sgen-mono.c elevate to warning level

Co-authored-by: Mitchell Hwang <[email protected]>
2020-03-16 08:27:24 -04:00
monojenkinsandEgorBo 2e73a3a25a [mono] Implement all Ssse3 and Sse42 intrinsics (#19225)
Implement all `Ssse3` and `Sse42` intrinsics for LLVM back-end.
`Sse41` is left over (that's why `Sse42.IsSupported` still returns false always outside of corelib)

Godbolt: https://godbolt.org/z/7cd7q5

Co-authored-by: EgorBo <[email protected]>
2020-03-14 23:20:24 -04:00
Zoltan Varga 4ac244261c [wasm] Add a sample for collecting coverage data. (#19223) 2020-03-14 15:16:43 -04:00
monojenkinsandEgorBo 8248c90656 [mono] Implement missing SSE2 intrinsics (#19222)
https://github.com/dotnet/runtime/pull/33465 implemented 99% of them but a few were missing. This PR implements the missing pieces and unlocks `Sse2.IsSupported`.

@vargaz I hope I didn't step on your toe (feel free to close if you already have it locally).

Co-authored-by: EgorBo <[email protected]>
2020-03-13 23:36:01 -04:00
Zoltan Varga d147c60e31 [jit] Add support for emitting r4/r8 constants outside of the text segment. (#19219) 2020-03-13 18:26:29 -04:00
Zoltan Varga 786fdf6a38 [runtime] Coverage profiler improvements. (#19213)
* [interp] Add support for the coverage profiler.

* [runtime] Add 'write-at-method' and 'send-to-method' options to the coverage profiler, they work similarly to options in the AOT profiler.

* Fix c++-isms.
2020-03-13 18:01:00 -04:00
monojenkinsandlambdageek ed2df9c816 [threads] If shutting_down, don't create new thread, kill current thread (#19082)
If the runtime is shutting down, instead of creating a new thread, or throwing
an exception on the current thread, just shut down the current thread.

This takes advantage of `mono_threads_set_shutting_down`'s behavior that if you
call it after shutdown is already started, it will suspend, detach and exit
from the current thread.

Fixes https://github.com/dotnet/runtime/issues/31842

Co-authored-by: lambdageek <[email protected]>
2020-03-13 10:45:14 -04:00
monojenkinsandjkotas 873067cb27 Fix line endings under src/mono (#19215)
Fixes dotnet/runtime#2222

Co-authored-by: jkotas <[email protected]>
2020-03-13 09:54:51 -04:00
Thays Grazia b777471fca If there is a crash and the debugger is attached, we call mini_get_dbg_callbacks ()->send_crash(). (#19191)
That call ends up in mono_debugger_agent_send_crash which does all kinds of questionable stuff like trying to take the loader lock. Eventually it ends up calling socket_transport_send which tries to do a transition to GC Safe mode.
I inserted a MONO_ENTER_GC_UNSAFE in mono_debugger_agent_send_crash function and this fixes the assert.

Fixes #18794
2020-03-13 00:46:06 -03:00
Thays Grazia 0c2f59e016 [wasm][debugger] Continue SS after leaving managed code (#19140)
* Continue stepping over/into/out of managed code to native code, without this fix the behavior was resume when a SS leaves the managed code.
2020-03-13 00:44:49 -03:00
Larry Ewing 6e7146041a [wasm][tests] be slightly more careful looking for the connection string (#19211)
be slightly more strict about connection strings from chrome
2020-03-12 21:42:31 -05:00
monojenkinsandvargaz cb39826fc6 [jit] Add more SSE2 opcodes. Enable SSE2. (#19187)
Co-authored-by: vargaz <[email protected]>
2020-03-12 21:47:42 -04:00
Larry Ewing 150ffb7af3 [wasm][bindings] rework argument marshalling in call_method (#19150)
store a converter object for each type string passed in
use that object to simplify the marshalling step. do one allocation
for arguments and exceptions.
2020-03-12 20:43:43 -05:00
Larry Ewing 35415b28a7 [wasm][bindings] clean up symbol resolution (#19149)
Instead of hard coding a switch of available types we do a one type lookup for each type by name and store that result.
2020-03-12 20:37:30 -05:00
Larry Ewing 54f64a9d08 [wasm][websocket] Buffer writes when endOfMessage == false (#19185)
* [wasm][websocket] Buffer writes when endOfMessage == false

* [wasm][websocket] Set binaryType to arrayBuffer and handle that case
2020-03-12 20:32:24 -05:00
Aleksey Kliger (λgeek) f72f47b90e [corlib] Capture the ExceptionDispatchInfo when rethrowing from TaskContinuation (#19168)
* [corlib] Capture the ExceptionDispatchInfo when rethrowing from TaskContinuation

In `task.GetAwaiter().OnCompleted(action)` if `action` throws, the
`TaskContinuation.RunOrScheduleAction method will catch the exception and then
call `RuntimeAugments.ReportUnhandledException`.  We must not simply throw the
exception again - in that case we will lose all the frames leading up to the
`catch` in `RunOrScheduleAction`.  Instead use ExceptionDispatchInfo to
caputure the original throw.

Addresses https://github.com/mono/mono/issues/19166

* [test] Add regression test

For annoying reasons we can't run this as a simple NUnit test because the
exception is unhandled on some arbitrary threadpool worker thread, and those
don't show up within an NUnit setup.

Additionally because the unhandled exception in the root domain would normally
terminate the application, we need to run this in a new appdomain.
2020-03-12 20:07:08 -04:00
Ankit Jain e0ec8de4b2 [master] Bump msbuild to track xplat-master (#19204) 2020-03-12 23:38:30 +01:00
Alexander Köplinger dcdec02b84 [metadata] Disable three icalls on netcore monodroid (#19198)
They aren't used there.
2020-03-12 19:17:36 +01:00
Johan Lorensson b58d7489be Disable Windows as default build host for WSL sdk builds. (#19196)
SDK builds done under WSL was done like Cygwin in order to be a
Cygwin replacement going forward. This has caused issues when building
sdk's under WSL when Linux is the assumed build host.

Adding an additional define where WSL build host can be overridden to
Windows, but default will now be Linux. This can in future be used
from configure scripts, setting it in sdks/Make.config when needed,
but will be an opt in feature to override Linux as default build host
for WSL.
2020-03-12 13:12:33 -04:00
Zoltan Varga 8fe10ad8e3 [wasm] Add some makefile dependencies. (#19193) 2020-03-12 11:25:08 -05:00
monojenkinsandlateralusX 70141f4959 Share managed EventSource/EventPipe implementation between CoreCLR and Mono. (#19173)
First set of changes needed in order to start sharing managed EventSource/EventPipe code between CoreCLR and Mono runtime. Shareable code has been moved into S.P.C library project and split into runtime specific source files when needed, kept within each runtime specific S.P.C project files.

Mono runtime has been extended with a set of icalls needed by EventPipeInternal, just to make sure EventPipe can be initialized, current Mono runtime implementation returns dummy values.

Current change also enables ETW provider support on Mono Windows runtime.

NOTE, this is just initial changes needed in order to share managed EventSource/EventPipe code between runtimes. Future changes will incrementally add EventPipe native code into Mono runtime.

Co-authored-by: lateralusX <[email protected]>
2020-03-12 13:36:24 +01:00
Alexis Christoforides 662f744a93 [merp] Remove unnecessary call to msync in mono_state_free_mem (#19167)
Closes https://github.com/mono/mono/issues/19136
2020-03-12 06:51:36 -04:00
Alexander Köplinger aeda6f0b26 Fix mistake in previous commit 2020-03-12 01:19:38 +01:00
Alexander Köplinger 306ebdfd30 Don't disable dllmap for mobile on netcore 2020-03-12 01:13:27 +01:00
Alexander Köplinger 0cc83ec613 Add assert when dllmap is disabled and fix support build in netcore mode
Ported from https://github.com/dotnet/runtime/commit/a54d391d9daebf7b53e7f1efc1690c1c35177570
2020-03-12 01:09:46 +01:00
Vlad Brezae b99f79fa94 [interp] Add more Math intrinsics (#19174)
* [interp] Add missing intrinsics for double Math operations

* [interp] Add math intrinsics for float operations

* [interp] Add more math intrinsics

With different type of signature

* [interp] Remove intrinsic for Round for now

Which has different semantics from the libc's round
2020-03-12 00:14:41 +02:00
Larry Ewing 691665884b [wasm][benchmarks] Add --wasm-package to bench runner to consume package builds (#19190)
This adds an option the standalone-benchmark runner to extract the package zips created with the package target and available from jenkins for commits to master.
2020-03-11 15:20:33 -05:00
Zoltan Varga 19fa221a33 Avoid passing /features:peverify-compat to csc, it generates very inefficient code in some cases. (#19144)
See https://github.com/mono/mono/issues/18572#issuecomment-595924767.

Fixes https://github.com/mono/mono/issues/18572.

Disable the IL verifier for now, it can't handle code compiled without /features:peverify-compat.
2020-03-11 19:43:19 +01:00
Vlad Brezae 94ff330b60 [interp] Optimize delegate invokes (#19065)
Add a specialized MINT_CALL_DELEGATE invoke, which is faster than executing a separate generic managed wrapper method.
2020-03-11 20:09:55 +02:00
monojenkinsandEgorBo 7d623e5e62 [mono] Optimize boxing in Pattern Matching idioms (#18812)
This PR optimizes boxing for pattern matching idioms. Basically, it copies existing optimizations from CoreCLR to mono, see [impBoxPatternMatch](https://github.com/dotnet/runtime/blob/084dd1a5aae0fbe6ed4f9164ac4ee1c5f0b2398f/src/coreclr/src/jit/importer.cpp#L5856-L6032) (one of them was recently added by me for coreclr, see https://github.com/dotnet/runtime/pull/1817)
#### 1) `box + isinst + brtrue/brfalse`  -->  `ldc.i4.0/1 + brtrue/brfalse`
```csharp
public static int Case1<T>(T t)
{
    if (t is int)
        return 42;
    return 0;
}
/*
    IL_0000: ldarg.0
    IL_0001: box !!T
    IL_0006: isinst [System.Private.CoreLib]System.Int32
    IL_000b: brfalse.s IL_0010
    IL_000d: ldc.i4.s 42
    IL_000f: ret
    IL_0010: ldc.i4.0
    IL_0011: ret
*/
```
#### 2) `box + isinst + ldnull + ceq/cgt.un`  -->  `ldc.i4.0/1`
```csharp
public static bool Case2<T>(T t)
{
    return t is int;
}
/*
    IL_0000: ldarg.0
    IL_0001: box !!T
    IL_0006: isinst [System.Private.CoreLib]System.Int32
    IL_000b: ldnull
    IL_000c: cgt.un
    IL_000e: ret
*/
```
#### 3) `box + isinst + unbox.any`  -->  `nop`
```csharp
public static int Case3_1<T>(T o)
{
    if (o is int x)
        return x;
    return 0;
}

// or

public static int Case3_2<T>(T o) => o is int n ? n : 42;

// or

public static int Case3_3<T>(T o)
{
    return o switch
    {
        int n => n,
        string str => str.Length,
        _ => 0
    };
}
```
Tests can be found here: https://github.com/dotnet/runtime/tree/master/src/coreclr/tests/src/JIT/Generics/Conversions/Boxing

[Sharplab.io link](https://sharplab.io/#v2:EYLgtghgzgLgpgJwDQxASwDYB8CwAoAAQAYACAZQAsIEAHAGQmADoAlAVwDsY0w4BufPgIBmEgQCMANjEAmEgGF8Ab3wk1Y0ROlouC6HHEAeACoA+ABTGSMAJSr1KvOuck0AMxLmYrqK652nFyCCAHYSABYZAUDgsKJo5wBfezUUjTEpEmAAe2yMPSg4GRMLK1s0xyC1UOsfPxgE9WSYkjSRDO1deX1hAH0jM0sSbIDnSqr3T2y6nW8AD1Gq5xq5xtiSeLTmts1M2YK4PuLBqxGSAF5TYZndDhIAfhI7kAiowRb2rXqDvuESoZGFTSyzC0ygAHc0DAAMYUYFVcZLIL7O6XJ5IeFIjKkWAIC5XXFMOhwDgAcxgFAxLSxal6+I2mJciTWJGaznwiSAA===) to observe IL code for the samples above.

#### Codegen diff example:
```csharp
static bool Test(int n) => Validate(n);

static bool Validate<T>(T t) => t is int;
```
Before (for `Test`):
```asm
0000000000000000	pushq	%r14
0000000000000002	pushq	%rbx
0000000000000003	pushq	%rax
0000000000000004	movl	%edi, %ebx
0000000000000006	movabsq	$0x7fdf9c5090c0, %rax
0000000000000010	movabsq	$0x7fdfad0268e8, %r14
000000000000001a	leaq	0x1be800(%r14), %rdi
0000000000000021	movl	$0x14, %esi
0000000000000026	callq	*(%rax)
0000000000000028	movl	%ebx, 0x10(%rax)
000000000000002b	movq	(%rax), %rax
000000000000002e	cmpq	%r14, (%rax)
0000000000000031	sete	%al
0000000000000034	addq	$0x8, %rsp
0000000000000038	popq	%rbx
0000000000000039	popq	%r14
000000000000003b	retq
```
After:
```asm
0000000000000000	movb	$0x1, %al
0000000000000002	retq
```

Co-authored-by: EgorBo <[email protected]>
2020-03-11 19:45:48 +03:00
imhameed b74bbe9269 [arm] Fix branch island disassembly for arm64 and thumb. (#19188)
Followup to https://github.com/mono/mono/pull/19126 and
https://github.com/mono/mono/pull/19169.

Bring arm64 `mono_arch_get_call_target` in line with other platforms by
returning NULL on failure instead of asserting. Bits 30-26 in an arm64
unconditional branch are exactly 0b00101, so 0x1f (0b11111) is used as a
mask to determine if the branch uses an immediate target--0x7 (0b111)
would incorrectly match against branches using register targets. The
extracted immediate is now sign extended in a branch-free way.

Avoid following blx entries in the method address table; ld64 can emit 4
different instruction sequences for islands involving thumb, and I'd
rather not implement support for them all now because 32-bit iOS is old
and large unlinked assemblies seem rare.
2020-03-11 13:43:08 +01:00
monojenkinsandpi1024e 30f5d34400 No need to have a "while (TRUE)" statement if we can check at the end of the loop or at the beginning. (#19111)
Whenever there must be a loop that goes indefinitely, there are either return statements or break statements in the code.

However, in this codebase, there are "infinite loops" that appear as such until the next line is checked, which is an if statement saying to break if a condition is true.

This unneeded complexity can be resolved by inverting the condition and making that the while-loop condition.

Co-authored-by: pi1024e <[email protected]>
2020-03-11 13:41:15 +01:00
Jay Krell 2746d98c1e [interp][32bit] Fix warning about precedence/parentheses. (#19184)
/s/mono2/mono/mini/interp/transform.c:3166:25: note: place parentheses around
      the '&&' expression to silence this warning
        if (mt >= MINT_TYPE_I1 && mt <= MINT_TYPE_I4
            ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
2020-03-11 10:20:40 +01:00
Jay Krell 60a192b98d [tools/sgen] Fix warnings about adding integers to string pointers. (#19180)
Kind of a silly warning.

mono/sgen/sgen-protocol-def.h:102:1: warning: adding 'unsigned long' to
      a string does not append to the string [-Wstring-plus-int]

BEGIN_PROTOCOL_ENTRY_HEAVY2 (binary_protocol_block_alloc, TYPE_POINTER, addr, TYPE_SIZE, size)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tools/sgen/sgen-grep-binprot.c:394:2: note: expanded from macro
      'BEGIN_PROTOCOL_ENTRY_HEAVY2'
        BEGIN_PROTOCOL_ENTRY2 (method,t1,f1,t2,f2)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tools/sgen/sgen-grep-binprot.c:291:26:
  note: BEGIN_PROTOCOL_ENTRY3 (binary_protocol_world_stopping,
  TYPE_INT, generation, TYPE_LONGLONG, timestamp, TYPE_POINTER, thread)exp

mono/sgen/sgen-protocol-def.h:102:1: note: use array indexing to
  silence this warning
2020-03-11 10:19:26 +01:00
Jay Krell 731b9cac54 [mini] Fix warning about unused variables. (#19182)
/s/mono2/mono/mini/mini-generic-sharing.c:1415:9: warning: unused variable
      'param_names' [-Wunused-variable]
        char **param_names;
2020-03-11 10:17:53 +01:00
Jay Krell 0ef53bae78 [wasm] Fix warnings about unused variables. (#19179)
/s/mono2/mono/metadata/sgen-mono.c:2883:16: warning: unused variable
      'last_major_gc_warned' [-Wunused-variable]
        static gint32 last_major_gc_warned = -1;
                      ^
/s/mono2/mono/metadata/sgen-mono.c:2884:16: warning: unused variable
      'num_degraded' [-Wunused-variable]
        static gint32 num_degraded = 0;
                      ^
/s/mono2/mono/metadata/sgen-mono.c:2886:9: warning: unused variable
      'major_gc_count' [-Wunused-variable]
        gint32 major_gc_count = mono_atomic_load_i32 (&mono_gc_stats.maj...
               ^
2020-03-11 10:13:36 +01:00
Jay Krell be9ced1a0f Fix warnings about misleading indentation. (#19186)
This is subtle and not actually misleading,
because it tends to be space tab or space tab, not wrong
number of tabs or significant wrong number of spaces.

/s/mono2/mono/metadata/class.c:4010:6: warning: misleading indentation;
      statement is not part of the previous 'if' [-Wmisleading-indentation]
                                 if (mono_class_implement_interface_slo...

/s/mono2/mono/metadata/class.c:4039:3: warning: misleading indentation;
      statement is not part of the previous 'if' [-Wmisleading-indentation]
        if (m_class_is_delegate (target) && mono_class_has_variant_gener...

/s/mono2/mono/mini/driver.c:1780:4: warning: misleading indentation; statement
      is not part of the previous 'if' [-Wmisleading-indentation]
                if (strncmp (argv [i], "--debugger-agent=", 17) == 0) {
2020-03-11 10:11:39 +01:00
Jay Krell fe5078961f [wasm] Fix warnings about unused variables. (#19181)
/s/mono2/mono/mini/mini-runtime.c:2284:15: warning: unused variable 'ji'
      [-Wunused-variable]
        MonoJitInfo *ji;
2020-03-11 10:08:41 +01:00
Larry Ewing 10cf3041d6 [wasm][packager] add option to pull from a different runtime directory (#19178) 2020-03-10 18:53:53 -05:00
Larry Ewing 488da04761 [wasm][standalone-benchmarks] Support mac (#19175)
This just handles systems (mac) without realpath installed and with a version of bash
that doesn't support &>>. It also trims path information from the benchmark name
2020-03-10 15:03:07 -05:00
monojenkinsandmdh1418 9ce7798c41 [mono] Include filename in Invalid image error messages (#19112)
Fixes dotnet/runtime#31649
Fixes dotnet/runtime#31650

**Testing**:
Reproduced Error via
`<Runtime Repo Root>/build.sh /p:RuntimeFlavor=mono`
Removing `[ActiveIssue("https://github.com/dotnet/runtime/issues/31650", TestRuntimes.Mono)]` in AssemblyTests.cs
Running `dotnet msbuild /t:RebuildAndTest /p:XunitMethodName=System.Reflection.Tests.AssemblyTests.LoadFile_NoSuchPath_ThrowsFileNotFoundException`

Fixing issue resulted in no errors.

Co-authored-by: mdh1418 <[email protected]>
2020-03-10 10:48:43 -04:00
monojenkinsandEgorBo 84bc62d113 [mono] Disable SIMD support for Windows target (#19172)
A temp fix for https://github.com/dotnet/runtime/issues/1933

Co-authored-by: EgorBo <[email protected]>
2020-03-10 14:45:55 +01:00
Jay Krell 5a065f628f [interp] Add EXCEPTION_CHECKPOINT to MINT_JMP consistent with other method tranform paths. (#19157) 2020-03-10 13:00:59 +01:00
Johan Lorensson fd61732c83 Convert PLT table and call site to execute only on AMD64. (#19114)
Current implementation embedded critical runtime information directly
into PLT slot. It also depends on finding call site in generic trampoline
reading call target from instruction stream in order to locate PLT slot
in use and then read GOT offset as well as PLT info offset from PLT slot
instruction stream.

This is problematic on platforms where code is execute only. Fix changes
how the metadata needed in order to correctly patch PLT is discovered.
Instead of depending on reading instruction stream, it is loaded into
RAX (free to be used when calling through PLT on AMD64) in PLT slot
before jmp takes place that moves control over to generic trampoline.

The PLT slot is the only place where we have access to both GOT index
(used in jmp), PLT info offset (currently embedded after jmp instruction)
and emitted PLT slot index. Since PLT slot index can be used to recover
GOT offset used by PLT slot as well as PLT info offset at runtime,
PLT slot index will be emitted into instruction stream and loaded into
RAX (prepared to be configurable to other reg if needed) before doing the
jump over to generic trampoline. Size of emitted imm constant is optimized
based on number of total PLT slots used in image, meaning that 1, 2, or 4
bytes could be used to store PLT slot index as an imm constant in
instruction stream. The additional jmp should have minimal to no
performance overhead since it should complete within 1 cycle, reading
imm constant from instruction stream that shouldn’t incur additional
cache misses and sine there is no data dependency between mov and jmp,
there should be options to pipeline both instructions. Since mov has
smaller latency than indirect jmp, mov should be complete once control
gets into the generic tramp (if pipelined).

In order to resolve plt info offset at runtime, needed information is
now emitted as part of got_info_offsets, increase table with 4 bytes/plt
slot, same size currently emitted into the PLT slot instruction stream.

Code size of PLT slot is currently 10 bytes (6 bytes jmp and 4 byte PLT
info offset). Since PLT info offset has been moved into got_info_offset
table, size of PLT slot will be 2, 4, 5 byte mov (depending on needed imm
size) and 6 bytes jmp instruction. As an example, mscorlib can emit all
its PLT slots using 2 byte imm constant, meaning that the PLT slot will
still be 10 bytes, but since PLT info offset of 4 bytes is moved into
got_info_offset, total image increase will be 4 bytes/PLT slot. This is
however still cheaper than alternatives that would burn 1 trampoline/PLT
slot (at least 10 additional bytes/slot) or setup lookup tables in image
or calculate more info at runtime, all-consuming more memory in total.

Note, using this approach is optional and runtime needs to be built using
MONO_ARCH_CODE_EXEC_ONLY in order to enable it since it’s only an opt in
features on platforms that can't read from instructions stream.

Current implementation is AMD64 only, but same pattern could be applied
to other architectures when needed.
2020-03-10 12:19:52 +01:00
monojenkinsandEgorBo 1ec2c9f7fd [Mono] Intrinsify Interlocked.And and Interlocked.Or using LLVM (#19135)
These APIs were introduced recently in https://github.com/dotnet/runtime/pull/33042 and already used in several places e.g. in `Task`, `InterlockedBitVector32`, `SafeHandle`, `RegexCharClass`

It should emit `%reg = atomicrmw and (or)` in LLVM IR. (and unlike `Interlocked.Add` these API return old value, see https://github.com/dotnet/runtime/pull/33102

Addresses https://github.com/dotnet/runtime/issues/32239 for Mono.

Co-authored-by: EgorBo <[email protected]>
2020-03-10 11:49:09 +03:00
monojenkinsandvargaz 5b644ae5a9 [jit] Implement support for all Sse1 intrinsics for netcore. (#19155)
* Add generic OP_XOP opcodes for opcodes which the JIT doesn't care about.
  Add a SimdOp enum to list the operations performed by these opcodes.
* Add a SimdIntrinsic struct so the mapping between the .net methods
  and the JIT opcodes can be specified declaratively.
* Add all intrinsics from the Sse class.

Co-authored-by: vargaz <[email protected]>
2020-03-09 22:48:05 -04:00
monojenkinsandpi1024e 8b447b4ccf No need to force to bool, when C does that automatically in the if statement. (#19095)
In C, the if statement checks if the value is 0 or not 0. !!, or forcing to 1 or 0, is unnecessary here.

Co-authored-by: pi1024e <[email protected]>
2020-03-09 20:28:36 +01:00
imhameed 90430277a1 [arm][ios] Follow branch islands when determining method entry addresses. (#19126)
Fixes https://github.com/mono/mono/issues/18612; `aot_code_low_addr` and
`aot_code_high_addr` were set to a range that included only first-level
branch islands, so `find_aot_module` would fail early when passed a
genuine function address.

This change only affects monotouch for now.

Why not use relative offsets stored in a read-only section? One reason
is that llvm-as cannot handle a `.long` directive containing a
subtraction expression involving externally-defined symbols. The AOT
method address table and AOT LLVM-generated code are currently emitted
in separate object files, so the method address table would include
expressions with undefined symbols.

Related: https://xamarin.github.io/bugzilla-archives/70/707/bug.html
2020-03-09 12:28:13 -07:00
Larry Ewing 9057498794 [wasm] Convert MonoString to string without utf8 intermediate (#19156)
* [wasm] Convert MonoString to string without utf8

Previously when converting strings from mono objects to js we used a utf8 intermediate buffer, this change builds the js string from the mono string without an intermediate.
2020-03-09 13:42:21 -05:00
Larry Ewing 66545a2abd [wasm] remove m_strdup (#19158) 2020-03-09 11:58:11 -04:00
Rares95andRares Sebastian Popa a25fe91245 Remove redundant mono_gc_collect during domain unload (#19130)
* Remove redundant mono_gc_collect during domain unload

Ifdef out mono_gc_collect when Boehm GC is present as it doesn't use remsets

* Fix trailing whitespace

Co-authored-by: Rares Sebastian Popa <[email protected]>
2020-03-09 16:47:45 +01:00
Aleksey Kliger (λgeek) 621a2219ef [corlib] Suppress finalization of underlying console FileStreams (#19137)
* [corlib] Suppress finalization of underlying console FileStreams

We already suppress the finalization of the CStreamReader and CStreamWriter,
but during shutdown, the underlying FileStream may be finalized which
invalidates its access field which can lead to a crash.

Addresses https://github.com/mono/mono/issues/19005
2020-03-09 10:01:16 -04:00
Marek Safar 1d575d8fe7 Bump linker (#19151) 2020-03-09 13:29:21 +01:00
dotnet-maestro[bot]anddotnet-maestro[bot] ebe382d49d Update dependencies from https://github.com/dotnet/arcade build 20200308.2 (#19163)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20158.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20158.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2020-03-09 13:27:40 +01:00
monojenkins 1713304e55 [bcl] Update BCL Linked Size 2020-03-09 07:14:17 +01:00
Marek Safar c6a3143c1e Freeze linker on mono-2020-02 branch 2020-03-08 10:35:42 +01:00
Zoltan Varga 8daffc3e4c Bump emscripten to 1.39.9. (#19147) 2020-03-07 23:16:18 -06:00
Zoltan Varga d083e2a828 [llvm] Move intrinsics definitions into a template header file llvm-intrinsics.h. (#19133)
* [llvm] Move intrinsics definitions into a template header file llvm-intrinsics.h.

* [llvm] Add a call_intrins () helper function.

* [llvm] Add globals for the SSE vector types to make code more compact.

* [llvm] Refactor the registration and lookup of intrinsics.

* Use the Intrinsics class from LLVM to register intrinsics.
* Identify intrinsics using an id instead of their name.

* Disable the wasm intrinsics on older llvm versions.

* Simplify mono_llvm_register_overloaded_intrinsic ().
2020-03-07 00:49:17 -05:00
monojenkinsandpi1024e 1690cadee2 Changed signedness of iterator to fix warnings. (#19014)
numOperands is an unsigned integer. In addition, getArgOperand has unsigned int arguments. As a result, I have no idea why i is an int, but to fix compiler warnings, I made it an unsigned int.

Co-authored-by: pi1024e <[email protected]>
2020-03-06 14:19:11 +01:00
Zoltan Varga be52633175 [jit] Use OP_EXTRACT_R4 in simd-intrinsics.c when using LLVM. (#19129)
* [jit] Use OP_EXTRACT_R4 in simd-intrinsics.c when using LLVM.

* [jit] Check for sse41 support before using the sse41.dpps intrinsic.
2020-03-06 07:32:47 -05:00
Larry Ewing 9ab10fd987 [wasm][debugger] Hash messages and sessions directly (#19128)
* [wasm][debugger] Hash messages and sessions directly
2020-03-05 16:07:19 -06:00
Larry Ewing f95d1413d3 [wasm][debugger] Check runtime state before attempting to resolve breakpoints (#19134)
Handle the case where the breakpoint request comes in very early before the runtime initialized by deferring resolution until normal runtime init.
2020-03-05 13:35:41 -06:00
Thays Grazia 93e7ef2b0f [debugger] NRE when 2 threads try to call GetThreads at same time (#19122)
As analysed by @jstedfast and me the only way to get this exception with this callstack below is 2 threads call GetThreads at same time, so while one of them is filling the items in the array, the other one is already trying to use the array with null items.

```
System.NullReferenceException: Object reference not set to an instance of an object
  at Mono.Debugging.Soft.SoftDebuggerSession.GetId (Mono.Debugger.Soft.ThreadMirror thread) [0x00000] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:3227
  at Mono.Debugging.Soft.SoftDebuggerSession.OnGetThreads (System.Int64 processId) [0x0002c] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:879
  at Mono.Debugging.Soft.SoftDebuggerSession.GetThread (Mono.Debugging.Client.ProcessInfo process, Mono.Debugger.Soft.ThreadMirror thread) [0x00008] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:902
  at Mono.Debugging.Soft.SoftDebuggerSession.HandleBreakEventSet (Mono.Debugger.Soft.Event[] es, System.Boolean dequeuing) [0x00501] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:2038
  at Mono.Debugging.Soft.SoftDebuggerSession.HandleEventSet (Mono.Debugger.Soft.EventSet es) [0x0005b] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:1687
  at Mono.Debugging.Soft.SoftDebuggerSession.EventHandler () [0x0005d] in /Users/runner/runners/2.164.6/work/1/s/monodevelop/main/external/debugger-libs/Mono.Debugging.Soft/SoftDebuggerSession.cs:1625

```
 PR on mono/debugger-libs -> https://github.com/mono/debugger-libs/pull/300

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1070384/
2020-03-05 13:22:34 -03:00
Sam Patel 4c348d6567 [llvm] Don't set the LLVM JIT TLS variable in AOT. (#19127)
Should fix https://github.com/mono/mono/issues/19115. This bug was
introduced in https://github.com/mono/mono/pull/18824, which
conditionally initalized an LLVM JIT-related TLS slot while preserving
unconditonal stores to that same slot.

The value stored in this TLS slot should only be used during JIT
compilation, so set/reset it in `mono_llvm_compile_method` instead of
during IR construction in `mono_llvm_emit_method`. No non-JIT code
should depend on the existence or presence of this TLS variable, so
remove `mono_llvm_jit_set_tls_cfg`.
2020-03-05 08:29:01 -05:00
Aleksey Kliger (λgeek) a374dbb6f1 [merp] Add a test for Mono.Runtime.SendExceptionToTelemetry() (#18781)
Contributes to https://github.com/mono/mono/issues/18078
2020-03-04 21:25:57 -05:00
Larry Ewing a497dc7c5d [wasm][debugger] Rework breakpointRequest handling across sessions (#19113)
Try to follow CDP ordering constraints by not responding Debugger.enable until
we've fired scriptParsed events for any already loaded assemblies.

Send Debugger.scriptParsed asynchronously as we load assemblies instead of
waiting for them all to load first and fix some of the sequence point processing
to speed up debugger initialization time.

Rework breakpoint request handling by always passing the request to the browser
then storing both the request and the resultant id for use in when any new scripts
are loaded.

Pass any previous breakpoint requests to the new execution context on page reload.
2020-03-04 18:39:15 -06:00
Ryan LuciaandRyan Lucia a5f8905afa [loader] Clean up locking around adding assemblies to domain/ALC (#18967)
This cleans up three components of assembly loading that could potentially cause races/crashes.

First, both `add_assemblies_to_domain` and `add_assembly_to_alc` now ignore resolved references on netcore. We were seeing a crash on CI related to an assertion in here that resulted from two threads using the same MonoImage simultaneously as one is being loaded in with LoadFile, and this should solve that. Fixes dotnet/runtime#2305

Second, we take both the domain and ALC loaded assemblies locks simultaneously around the assembly being added to both lists, ensuring they stay in sync.

Finally, MonoImage now takes the image lock when writing to references instead of using the global `assemblies_mutex` in assembly.c for all images. This potentially helps reduce contention. Work here is not complete, but for this PR it's good enough. See dotnet/runtime#32889

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-04 17:09:08 -05:00
Stefan e1d3b7ba64 Minor formatting of the manuals (#19124) 2020-03-04 17:51:28 +01:00
Thays Grazia c40f1ba79f [dim] Class overriding interface method that was already override by another interface (#18962)
The problem was a parent class overriding an interface that was already override by another interface.
Refactor class-init.c
2020-03-04 12:48:25 -03:00
dotnet-maestro[bot] c1b392f02b Update dependencies from https://github.com/dotnet/arcade build 20200301.1 (#19093)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20151.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20151.1
2020-03-04 13:13:34 +01:00
Jay Krell 35cdbcb4a8 [cxx] Remove static initializers. (#19089)
In a current shipping Visual C++, 19.21.27702.2,
there is a bug involving const member data in structs, where for example:

static const struct a { // const on this line in either case
  const int b; // const on this line
} c = {1};

will have a static initializer, run usually reliably before main, but:

static const struct a { // const on this line in either case
  int b; // no const on this line
} c = {1};

will not.
The semantic difference is minuscule, so much so, that the first form
is relatively rare, and the bug shipped.

Just always use the second form.

The difference is that if you have a non-const a, b is still const, I guess.
If you have a const a, then no difference.

You can find these easily:

cd ...
for %a in (*.exe *.dll) do link /dump /disasm %a > %a.txt
findstr /c:"dynamic initializer" *.txt

Which could be automated and tested in CI.

This probably not an actual problem, but it is needlessly inefficient,
and potentially a problem. The timing/ordering of the initializers
is a bit not deterministic. You get writable data where you expect
read only.

It is in fact a compiler bug, fixed in later releases.
But it has been shipping for a while, I first noticed it over two years
ago, in code of mine that it caused to not work (a delayload implementation
which runs before main).
2020-03-04 12:54:13 +01:00
Aleksey Kliger (λgeek) 0bcc388ce8 [threads] Add a global "runtime startup is finished" flag (#18994)
This contributes to https://github.com/mono/mono/issues/18916
(to make coop locking not bother with state transitions if the runtime isn't
ready yet).
2020-03-03 18:29:04 -05:00
Steve Pfister c483eaf070 Changed Max and MinLengthAttribute validation (#19084)
It should now validate against ICollection or any other type that has a Count property for that matter.

We're still relying on the reference source implementation because corefx strips some types out.

Fixes https://github.com/mono/mono/issues/19022
2020-03-03 15:26:46 -05:00
monojenkinsandEgor Bogatov ede05b997b [Mono] Fix Sse2.ConvertScalarToVector128 and add signed overloads (#19108)
@vargaz noticed I used an incorrect opcode for ConvertScalarToVector128*
I will add a sort of fuzzy tests later (which use all kind of SRI APIs and compare results with CoreCLR) something like this: https://github.com/mono/mono/blob/master/netcore/tests/HwIntrinsics/Program.cs
Also, add signed version of ConvertScalarToVector128*.

Co-authored-by: Egor Bogatov <[email protected]>
2020-03-03 23:14:59 +03:00
Alexis Christoforides 0ec7957d7c [merp] Create a 'reason' breadcrumb for the crash dump process (#19072)
For example, the system name of a signal ('segv') or 'MANAGED_EXCEPTION' are typical arguments.
2020-03-03 11:47:19 -05:00
Johan Lorensson b5a9fe49e1 Fix Mono Tool MSVC Debug build. (#19075)
Changes done in object-offsets.h adding offsets for TARGET_AMD64 exposed
a build configuration error in the Mono Tool MSVC debug build due to
incorrectly setting both X86 and AMD64 defines causing compile errors
when including object-offset.h. Release configuration didn't have this
configuration error, meaning that CI builds wouldn't trigger build error.

Fix makes sure all MSVC build configurations for Mono Tools
(genmdesc, monodis, pedump) matches the build configuration for other
binaries and that debug build configuration is inline with release build
configuration.
2020-03-03 16:11:30 +01:00
Zoltan Varga 4622ad0635 [WIP] [aot] Add support for emitting the method address table as a table of normal pointers in the data segment. (#19086)
* [aot] Rename acfg->opts to acfg->jit_opts.

* [aot] Add support for emitting the method address table as a table of normal pointers in the data segment.
2020-03-03 07:43:11 -05:00
Zoltan Varga b91580bafa [amd64] Obtain the trampoline argument in the generic_trampoline in the AOT case without decoding the call instruction. (#19091)
* [amd64] Obtain the trampoline argument in the generic_trampoline in the AOT case without decoding the call instruction.

* [amd64] Unify the aot specific trampolines so they have the same size/layout, so its easier to compute the address before the call.
2020-03-02 20:16:53 -05:00
Ankit Jain 3aab3884f6 [master] Bump msbuild to track xplat-master (#19077) 2020-03-02 17:49:08 -05:00
monojenkinsandEgor Bogatov 0f0c284a41 [Mono] Implement missing Sse2 intrinsics (#19101)
For now we only implement intrinsics used by corlib internally to optimize code.

Fixes https://github.com/dotnet/runtime/issues/32510 crash.

`ConvertScalarToVector128UInt32/64` is expanded into LLVM's `insertelement`
`ConvertToUInt32/64` is expanded into LLVM's `extractelement`

Co-authored-by: Egor Bogatov <[email protected]>
2020-03-03 01:09:46 +03:00
monojenkinsandRyan Lucia 0e38adf329 Use GArray instead of GList when getting custom attributes from an image (#19083)
Xamarin.Forms calls this _constantly_ during startup, to the point where the malloc calls were actually showing up in the profile. This should help with that, and gives us some ms back during startup.

There are also a bunch of calls that return empty arrays, which in theory could be created in managed for a perf boost. However, it seems the Xamarin.Android team has mostly gotten rid of those calls and that optimization would complicate the code and require changes to our managed CustomAttribute implemented for both legacy and netcore, so I've opted to not check it in for now. If the empty calls continue to show up down the line, I can dig the code back up from my local branch and PR it separately.

If this is producing huge arrays and malloc continues to show up in the profile, consider https://github.com/dotnet/runtime/issues/32972. Even without that, this is a substantial improvement.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 16:46:09 -05:00
Zoltan Varga 89e8a484e1 [wasm] Allow SIMD only in netcore mode. (#19087) 2020-03-02 12:56:43 -05:00
monojenkinsandRyan Lucia f7e1a23dd8 [loader] Free some leaked strings in NativeLibrary resolution (#19085)
Oops.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 17:46:09 +01:00
monojenkinsandRyan Lucia 80ff8817d0 [threading] Switch domain assemblies lock from OS to Coop (#18918)
I don't think you can create a root domain without reaching the thread info attach call, so this should be safe to change outright.

Fixes dotnet/runtime#32177

We should complete https://github.com/mono/mono/issues/18915 and then convert over other locks as needed, but this fixes the immediate CI hang.

Co-authored-by: Ryan Lucia <[email protected]>
2020-03-02 17:45:57 +01:00
Zoltan Varga 89902e8d22 [wasm] Pass --exclude-feature deserialization to the linker. (#19031)
* [wasm] Pass --exclude-feature deserialization to the linker.

* Add a --linker-exclude-deserialization packager option.
2020-02-28 23:41:03 -05:00
Alexander Köplinger 1a6eb5a597 Update mono-extensions 2020-02-28 19:58:22 +01:00
Jay Krell ec9db26150 Remove non-functional spawn code.
Subject to configuration, spawn has three widely divergent behaviors:

succeed but do nothing
print and exit
do what it sounds like
Rather than mislead the callers, provide only option 3 and let the caller decide what to do otherwise.
2020-02-28 13:19:55 -05:00
Maxim Lipninandmonojenkins 68470ddfb0 [wasm] Use PNSE version of System.Net.NetworkInformation.NetworkInterface (#19074)
PNSE coverage of wasm bcl has shown that only three public members of `System.Net.NetworkInformation.NetworkInterface` class throw `PlatformNotSupportedException`.

Now all public members throw PNSE on WASM.

Relates to https://github.com/mono/mono/issues/18735


Co-authored-by: monojenkins <[email protected]>
2020-02-28 19:17:40 +01:00
Alexander Köplinger 716b73221e Update mono-extensions 2020-02-28 19:05:43 +01:00
Alexander Köplinger 00b19731c6 Bump mono-extensions 2020-02-28 15:25:03 +01:00
monojenkinsandRyan Lucia 8951940260 [loader] Remove unnecessary newline in loader mono_trace call (#19069)
Co-authored-by: Ryan Lucia <[email protected]>
2020-02-28 11:37:18 +01:00
Vlad Brezae 941a335ea0 [runtime] Use pointer size gchandles (#19039)
* [runtime] Use pointer size gchandles

This would enable in the future to use CoreCLR's GC with mono. This doesn't change the current embedding api, but, in the future, we will end up aligning with coreclr on an embedding api that uses pointer sized gchandles.

* [gc] Change format of gchandles to be netcore compatible

On netcore, the GCHandle class assumes the lowest bit to be usable, for perf improvements. Move the responsability to have this bit available down to the GC implementation, since for CoreCLR GC we shouldn't do any shifting above the gc interface.
2020-02-28 09:05:55 +02:00
Larry Ewing d11eeadb66 [wasm][debugger] use console.debug to signal runtime init (#19057)
* [wasm][debug] use console.trace instead to init runtime

Using a `debugger` breakpoint in Mono.mono_wasm_runtime_ready has
unfortunate side effects, this changes the logic to use console.trace
and checks the stackTrace when the event is arrives.

* add a console.trace stub to runtime-test.js

* Use console.debug with a magic number instead
2020-02-27 12:07:55 -06:00
Johan Lorensson 188ac1b0a7 Clean up libmono-dynamic.vcxproj dependency for netcore. (#19040) 2020-02-27 18:08:34 +01:00
Larry Ewing 76f5086687 [wasm][debugger] allow stepping out of c# (#19059) 2020-02-27 07:50:59 -06:00
Johan Lorensson d2e6bd2aff Detect build config mismatch for MSVC Windows LLVM build. (#19044)
If LLVM build is done using different build configuration, Mono build
will fail to static link LLVM libraries due to incompatibility between
release and debug C++ runtime library, generating a lot of linker errors
like this:

error LNK2038: mismatch detected for 'RuntimeLibrary':
value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug'
in mini-llvm-cpp.obj

error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL':
value '0' doesn't match value '2' in mini-llvm-cpp.obj

This is normally not a problem when doing local builds of LLVM but
when using an external provided build, like netcore Mono build,
this problem is more likely to occur.

This fix adds an additional check to make sure build configuration reported
by llvm-config.exe --build-mode matches current Mono build configuration
and if not give a more specific msbuild error:

error : LLVM build Configuration=Release doesn't match current Mono
build Configuration=Debug
2020-02-27 13:28:07 +01:00
Nikita Voronchev 4b42804803 [WinForms] Colorize PropertyGrid's Categories properly (#19045)
Assume, one have colorized `PropertyGrid propertyGrid` control:
```
propertyGrid.CategoryForeColor = Color.Red;
propertyGrid.LineColor = Color.Blue;
```

Before changes proposed by the current PR we get the following image:
![before](https://user-images.githubusercontent.com/12382656/75364102-3391fe80-58cc-11ea-9136-347bcd567731.png)
After:
![after](https://user-images.githubusercontent.com/12382656/75364115-3987df80-58cc-11ea-95d1-20691a4a8467.png)
(item labeled "_Child 2_" is selected by user)
2020-02-27 11:25:14 +01:00
imhameed 5e5b1ed425 [aot] Avoid mono_defaults when creating trampolines. (#19055)
Fixes https://github.com/mono/mono/issues/19009.

https://github.com/mono/mono/pull/18505 made PLT initialization occur at
image load time. When initializing the PLT for mscorlib,
`mono_defaults.corlib` is not yet initialized.

This change works around this by using `mscorlib_aot_module`--which is
initialized relatively early--instead of `mono_defaults.corlib`. PLT
initialization is also moved slightly later: it now happens after
`use_page_trampolines` has been initialized. The `USE_PAGE_TRAMPOLINES`
macro has also been changed to dereference `mscorlib_aot_module`.

This commit also removes an unused parameter from
`mono_aot_create_specific_trampoline`.
2020-02-27 08:56:59 +01:00
Larry Ewing 7f6a9bb783 [wasm][debugger] Always pass failed breakpoint requests to target (#19053)
* [wasm][debugger] Always pass failed breakpoint requests to target

Originally we had a simple way to detect that breakpoints were
meant for the runtime but that is no longer true.  Now if we
can't location a breakpoint we need to pass the request on to the
target so it can make an attempt.

* Fix test byUrl is a logical breakpoint and should not fail
2020-02-26 23:07:57 -06:00
Ankit Jain de54af770c [wasm][debugger] Show the typename for valuetypes (#19046)
- no details of the valuetype are accessible

Fixes #18232
2020-02-26 20:32:48 -06:00
Larry Ewing 9793a1e5be [wasm][debugger] clean up scriptid parsing (#19052) 2020-02-26 20:26:56 -06:00
Larry Ewing ac5240640c [wasm][debugger] Since System.Url normalizes the path sep also try to match on the doc.Url (#19051)
Match against the pdb document url as well as the Url we calculated in case the urls have been
processed incompatibly
2020-02-26 17:39:32 -06:00
Steve Sanderson 3b0e35d1bc Debug proxy updates (#19026)
Currently, the Mono WebAssembly debug proxy logic is treated as "shared source" with ASP.NET Core. Each time we update by copying the sources from this repo into the ASP.NET Core repo, we need to make some updates to avoid problems. The intention of this PR is to make the updates here at source, so we can simply copy the sources as-is and don't need to update them each time.

The changes are:

Making everything internal, since this is not intended as public API surface for ASP.NET Core developers. It's an internal implementation detail of the framework.
To retain convenience for the ProxyDriver and Mono test code, this PR also adds a simple public entrypoint called DebuggerProxy. This exposes nothing except the ability to connect a debugger proxy instance to a websocket. In the ASP.NET Core code, we'll simply omit this file since it's not needed, but even if we did get it in the future when the proxy feature is published as a package, it would still be fine since it doesn't expose any public API that would be expensive to support and maintain.
Replacing all the Console.WriteLine calls with use of ILogger from the standard Microsoft.Extensions.Logging package, as used by most .NET Core-based application frameworks including ASP.NET Core.
This allows external code to control what happens with log output, what severity levels are filtered in/out, etc. In ProxyDriver and Mono test code, I've configured it to behave just like before: it logs to console, and doesn't filter out anything.
This has the benefit that various bits of commented-out logging code no longer need to be commented out. Instead, when calling the logger.Log* methods, each call site can specify a severity level and then (for example) the filter level could be set to Debug so that all Trace items are filtered out by default.
2020-02-26 12:52:36 -06:00
Zoltan Varga 6c5a18ba50 [runtime] Remove MONO_LLVM_INTERNAL. (#19032) 2020-02-26 17:48:29 +01:00
Ankit Jain 546833b534 Merge pull request #19035 from radical/async-local-fixes
[wasm][debugger] Fixes to get async method locals
2020-02-26 11:18:28 -05:00
monojenkinsandJohan Lorensson 907d59798c Fix Mono netcore preload hook to work with Windows CoreRun.exe. (#19027)
Current hook compares TPA list file names using case sensitive compare. On Windows, the TPA list, when passed from CoreRun.exe makes all file paths lower case, meaning that our checks fails to find assemblies in TPA list.

Since CoreRun.exe is used for both CoreCLR and Mono, removing the tolower on file paths added to the TPA list in CoreRun.exe could be an option, but that might introduce regressions in CoreCLR, so this fix isolate the change to Mono's netcore preload hook.

Co-authored-by: Johan Lorensson <[email protected]>
2020-02-26 12:06:40 +01:00
Zoltan Varga 91e026de9d [wasm] Improve error handling in the packager. (#19037) 2020-02-25 23:56:18 -05:00
Katelyn Gadd 5c8da9e284 Add basic set of standalone benchmarks for wasm (#18977)
Adds a basic set of standalone benchmarks from multiple sources, along with simple infrastructure for running them in mono, netcore and node.js wasm. A couple of them also have original JS versions you can run from a JS shell.
2020-02-25 18:24:10 -08:00
monojenkinsandpi1024e 3cba804069 Braces around subobject of initialization warning fix (#18866)
When compiling for Mono, Apple's compiler gives a warning saying to put braces around the 0 as it is a subobject. To silence the compiler, I put braces around the 0. Simple fix.

Co-authored-by: pi1024e <[email protected]>
2020-02-25 16:42:42 -08:00
Jay Krell 79e6ce6423 Change macro CLOSE_PIPE to function mono_close_pipe. (#19030)
Extracted from https://github.com/mono/mono/pull/19025
2020-02-25 19:54:38 +01:00
Jay Krell 1bd3b7a614 Fix warnings: GPid vs. pid_t function signature mixup in glib test code. (#19028)
Extracted from https://github.com/mono/mono/pull/19025
2020-02-25 19:46:10 +01:00
Ankit Jain 5c0b1e1c83 [wasm][debugger] Fix getting async method locals, and this
- async local field names don't start with `<`, surface those
- `this` for the actual async method is in a field named like
`<>4__this`, surface it as the `this` object
2020-02-25 13:36:38 -05:00
Ankit Jain f6bef135d5 [wasm][debugger][tests] Update tests to track test api changes 2020-02-25 11:38:37 -05:00
Ankit Jain c2f8fd8bc7 [wasm][debugger][tests] Ensure to always check location when sending
.. commands.
2020-02-25 10:24:50 -05:00
Ankit Jain 5f85de2b56 [wasm][debugger][tests] extract method to set breakpoints 2020-02-25 10:23:39 -05:00
716718a4bc Bump linker. (#19006)
* Bump linker.

* Bump linker tools framework version

* [csproj] Update project files

Co-authored-by: Marek Safar <[email protected]>
Co-authored-by: monojenkins <[email protected]>
2020-02-25 12:48:18 +01:00
Matthew Baird 6b3a7986ef Ensure ListView ItemCheck event can update the checked state (#18926) (#18937)
Fixes #18926
2020-02-25 10:19:01 +01:00
Vlad Brezae 49deb4bcea [interp] Enable more interp tests (#19012) 2020-02-25 09:12:01 +02:00
Zoltan Varga 8da9361ab2 [llvm] Remove loaded llvm support, the startup perf penalty doesn't seem to exist any more. (#18868)
* [llvm] Remove loaded llvm support, the startup perf penalty doesn't seem to exist any more.

* Fix the windows build.

* Remove more mini-llvm-loaded.c references.

* Remove more --enable-loadedllvm references.

* Make --enable-loadedllvm the same as --enable-llvm.

* Fix run-jenkins.sh.

* Address review comments.
2020-02-24 20:35:00 -05:00
Larry Ewing ed012b6e89 [wasm][debugger] revert DotNetUrl changes (#19021) 2020-02-24 18:31:52 -06:00
Jay Krell 9fa3eb26e1 [interp] Reorder InterpMethod to reduce padding (#18982)
for alignment. Sizes goes from 192 to 184.
2020-02-25 01:30:26 +01:00
Jay Krell bb40a1a2d9 Whittle away at warnings: (#18874)
Return address of local (deliberate).
 Pointer cast to different size.
 Function parameter and declaration badly mismatch (pid_t vs. void*).
 Unused locals.
 Top level volatile is ignored.
 Negative shift undefined.
 Call to undeclared function (getpid).

Some are fixed, some are disabled temporarily.
2020-02-24 18:41:06 -05:00
Larry Ewing fafa41f448 Use AsLocation when creating location (#19016) 2020-02-24 14:55:48 -06:00
monojenkinsandpi1024e 655dd296bf Fix compilation warning for converting negative integer constant to unsigned. (#19013)
When compiling, Visual Studio gives two compiler warnings in the sysmath.c file about FPILOGB0/-2147483648 being converted into an unsigned integer.

The standard library math.h states that FPILOGB0 be INT_MIN, so I replaced -2147483648 with the INT_MIN macro so the behavior is much more clear as well as stifling the compiler warnings.

Changes tested manually and automatically.

Co-authored-by: pi1024e <[email protected]>
2020-02-24 13:51:14 -05:00
Ankit Jain 27c5660659 [wasm][debugger] Don't mark the runtime ready before clearing the (#19015)
.. breakpoints. Chrome tries set the breakpoints from previous sessions
when the debugger starts up. And we set that in mono, if the runtime is
ready.

But in `MonoProxy.RuntimeReady`, we mark the runtime ready *before*
clearing the breakpoints. So, we end up setting the breakpoints, as
asked by chrome debugger, but then subsequently clearing them!

This manifests as previously set breakpoints in managed code not getting
hit, but if we reset the breakpoint manually then it gets hit.
2020-02-24 13:50:10 -05:00
Ankit Jain ace2e34aae Merge pull request #19003 from radical/improve-locals-access
[wasm][debugger] Improve accessing locals
2020-02-24 13:49:45 -05:00
Alexis Christoforides 1dffbd0200 [merp] Produce hashes for unmanaged thread stacks also (#18943)
Before, this was only done for managed stacks.

Addresses https://github.com/mono/mono/issues/18941
2020-02-24 13:02:17 -05:00
Alexis Christoforides 981da38038 [merp] Increase buffer size for state dump (#18798)
Addresses https://github.com/mono/mono/issues/18796
2020-02-24 10:21:34 -05:00
Ankit Jain 881dfd770f [master] Bump msbuild to track xplat-master (#18987) 2020-02-24 15:53:08 +01:00
dotnet-maestro[bot] 9f2297bcfd Update dependencies from https://github.com/dotnet/arcade build 20200223.1 (#19011)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20123.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20123.1
2020-02-24 15:50:09 +01:00
Jay Krell 9fb0820ca5 [interp] Use alloca for InterpFrame. (#19007)
This is like https://github.com/mono/mono/pull/19002 but next_free is initialized upon allocation instead of `pop_frame` which is a slight optimization.
2020-02-24 10:30:46 +01:00
Jay Krell ae1c7c010c [interp] Decruft PowerPC. (#18981) 2020-02-24 09:12:56 +01:00
monojenkins e4cf488054 [bcl] Update BCL Linked Size 2020-02-24 08:24:19 +01:00
Jay Krell 5941141433 [interp] Remove some unused paths in newobj and cleanup a little. (#18998) 2020-02-23 15:15:14 +02:00
Jay Krell e9886db7e4 [interp] Pop leave and calli_nat frames. (#19000) 2020-02-23 15:11:14 +02:00
Ankit Jain 9049967106 [wasm][debugger] mono_wasm_get_var_info - don't repeatedly walk the
.. stack, instead find the target frame, and read in one go.
2020-02-22 18:07:06 -05:00
Ankit Jain 01e089bc46 [wasm][debugger] Fix nested class names for this 2020-02-22 17:58:37 -05:00
Ankit Jain 547e4180df [wasm][debugger] Fix issue in getting locals for previous frames
`mono_wasm_get_var_info` didn't correctly reset the `data.cur_frame`
required for `mono_walk_stack_with_ctx`, as the stack is walked
repeatedly to get the variables.
2020-02-22 17:49:56 -05:00
Zoltan Varga cc8a5b5513 [corlib] Link out more SR/SRE types (#18770)
* [corlib] Invoke TypeBuilderInstantiation.MakeGenericType through a delegate created in the TypeBuilder ctor to break a linker dependency between RuntimeType and TypeBuilderInstantiation.

* [corlib] Update linker descriptor.

* UnmanagedMarshal is not used from native code at all.
* AssemblyName is not used from native code at all.
2020-02-22 17:34:00 -05:00
monojenkins 008a10c43a [bcl] Update BCL Linked Size 2020-02-22 22:48:39 +01:00
Thays Grazia a8d34b2a03 [wasm][debugger] Enable embedded ppdb on wasm (#18966)
* Enabling PPDB on Wasm.
2020-02-22 00:35:59 -06:00
Ankit Jain d02bbe4c32 [wasm][debugger] Handle exceptions from JS
- correctly mark `Result` with the error
2020-02-21 19:43:11 -05:00
Ankit Jain cbc722ddaf [wasm][debugger] SetException on all the inspector waiters for uncaught exceptions 2020-02-21 19:43:07 -05:00
Fan Yang 041fbe0900 Revert linker change of 5a65a97 (#18991)
This is needed because  the required='false' field in
```
<type fullname="System.Reflection.Emit.TypeBuilder" required='false' preserve="fields" feature="sre">
			<method name="SetCharSet" feature="sre" />
			<!-- reflection.c mono_reflection_call_is_assignable_to () -->
			<method name="IsAssignableTo" feature="sre" />
```
Fails to preserve the IsAssignableTo method.
2020-02-22 01:27:09 +01:00
Aleksey Kliger (λgeek) 7707eda633 [metadata] Mark mono_signature_get_return_type external only (#18990)
Also mark mono_signature_get_params external only.

Runtime should use mono_signature_get_return_type_internal
and mono_signature_get_params_internal
2020-02-21 17:35:55 -05:00
Larry Ewing dce53cc9f3 [wasm][debugger] Store more per session state and force some event ordering (#18747)
* [wasm][debugger] Store more per session state and force some event ordering

Register a DebugStore and ExecutionContext per session and force some commands
to block on source loading so that scripts are loaded before attempting to use
them. This should populate the sources in vscode-js-debug properly.
2020-02-21 15:06:45 -06:00
monojenkinsandNathan Ricci 840ba017ef Added some parenthesis and a cast to control order of operations. (#18972)
This fixes https://github.com/mono/mono/issues/17140; intermittent incorrect results from GetTotalBytesAllocated for current thread. uint64_t and a pointer difference are not the same type/width on all architectures, so incorrect results can happen with out the parenthesis.

Co-authored-by: Nathan Ricci <[email protected]>
2020-02-21 14:17:39 -05:00
monojenkinsandNathan Ricci ac7a77f245 Correct values for HighMemoryLoadThresholdBytes, MemoryLoadBytes. (#18913)
Correct values fort HighMemoryLoadThresholdBytes, MemoryLoadBytes. I looked in the code for CoreCLR and spoke with Maoni Stephens and Jan Kotas about the values used in CoreCLR, so I am confident they are all correct now.

HighMemoryThresholdBytes is a system wide memory usage threshold at which CoreCLR changes its compaction strategy. There is no direct correspondance to anything in mono-sgen. But since it used by software caches to determine when memory in the system is "nearly" exhauste, mono-sgent will use .9 * total physical memory as an acceptable value.

Co-authored-by: Nathan Ricci <[email protected]>
2020-02-21 19:36:20 +01:00
Larry Ewing a3e03430ea [wasm][tests] clean up a couple of the Contains assertions 2020-02-21 10:52:41 -06:00
Aleksey Kliger (λgeek) ed017481e8 [monodis] Escape names that start with a digit (#18759)
Code like this:

```
using System;
class Program
{
    static byte[] bla = new byte[] {1,2,3,4,5};
    static void Main()
    {
    }
}
```

Compiles to a private field declaration for the array with a compiler generated
name, that name should be quoted if it starts with a digit.  For example:

```
	IL_0007:  ldtoken field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=5' '<PrivateImplementationDetails>'::'74F81FE167D99B4CB41D6D0CCDA82278CAEE9F3E2F25D5E5A3936FF3DCEC60D0'
```

Fixes https://github.com/mono/mono/issues/18750
2020-02-21 11:43:27 -05:00
Cody RussellandAlexander Köplinger 7a3c321b42 Bump bockbuild for Pango nil check fix. (#18936)
https://github.com/mono/bockbuild/pull/153

Co-authored-by: Alexander Köplinger <[email protected]>
2020-02-21 17:24:43 +01:00
monojenkinsandBernhard Urban-Forster 802d57a741 [mono/metadata] add definitions for ENCLog/ENCMap (#18976)
Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-21 14:42:33 +01:00
Jay Krell f4d5c11610 [interp] Cleanup: Replace retval-> with result. and retval with &result. (#18951) 2020-02-21 11:24:38 +01:00
Jay Krell 3e3499e89a [interp] Allow historical inlining due tO: (#18948)
1. This wasn't generally conserving the important stack.
2. Larger mechanisms are in place now.

Granted, tuning out rare paths could still be valuable.
2020-02-21 10:43:40 +02:00
Jay Krell 40187209f8 [interp] Non-recursive newobj. (#18931)
A few weird here, esp. for string, but CI passes.
Acknowledged but skipped for now, a request to also refactor the opcode -- it doesn't seem like much throwaway work to go in either order, even if the code is dead (which I don't know).

Also amend recent nonrecursive PR to simulate stack +2 instead of +1.
2020-02-20 18:19:27 -05:00
Ankit Jain 91b125af92 [wasm] Improve locals visualization (#18919)
* [wasm] Improve locals visualization in the debugger

- improves handling of generic types, null objects, delegates
- improves getting the name for nested types, arrays, generic types

top todos from this:
- Remove arity in "foo`2<int, int>" from the type name
- Convert nested type names from `foo/bar` to `foo.bar`.

* [wasm] Fix scoping for `mono_wasm_add_null_var`

- which caused runtime uncaught exceptions

* [wasm] Fix possible lost exception in GetScopeProperties

- An exception, eg., because of an invalid scope id just ends up
uncaught instead of showing up as a failed Result.

* [wasm] Address review feedback

- Replace `stringify_type` with `mono_type_full_name` which does exactly
the same thing!

* [wasm][debugger] Fixup nested class, and generic type names

```
	// Fix up generic names like Foo`2<int, string> to Foo<int, string>
	// and nested class names like Foo/Bar to Foo.Bar
```
2020-02-20 14:23:02 -05:00
sharkcz 74f90c824b Add struct offset definitions to s390x (#18958)
* [mini-s390x] standardize on using MONO_STRUCT_OFFSET

* [mono-s390x] add struct offset definitions to s390x

Add s390x specific struct offset definitions to object-offsets.h and
update the array member accesses in mini-s390x.c.

Related: https://github.com/mono/mono/commit/79135c6ea57b516addf187a08cb01826bb0197c6
Fixes: https://github.com/mono/mono/issues/18930
2020-02-20 15:05:26 +01:00
Thays Grazia dd5926cc79 [debugger] Enable reading embedded ppdb (#18895)
To fix the issue of enabling embedded pdb everywhere the problem was that when the roslyn tests are compiling sources it creates embedded pdb with source file "", and before enable reading the embedded pdb we don't read this information, and now we read and our StackTrace file written in our corlib doesn't expect a source file "", it's okay if it's NULL but not "".
I could fix in corlib or in mono, I prefer to fix on mono because I don't know if there is any other place that doesn't expect source file "".
2020-02-20 09:59:40 -03:00
Egor Bogatov e441aa4d21 Remove TestEnvVarSwitchForInnerHttpHandler test as it should be in XA/LinkerTests.cs instead (#18952)
The test will be moved to XA's `src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs` 

It only works if 
```xml
<AndroidHttpClientHandlerType>System.Net.Http.MonoWebRequestHandler, System.Net.Http</AndroidHttpClientHandlerType>
```
is set in `Xamarin.Android.Bcl-Tests.csproj` and we don't want it (other tests expect SocketsHttpHandler to be default one). As an alternative we can add it to the `Xamarin.Android.Bcl-Tests.csproj`'s Linker exceptions but `/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs` is a better place for it.
2020-02-20 11:10:58 +01:00
Jay Krell 90c66725d3 Remove LOCK_DECLARE and LOCK_INIT -- unused and incorrect comments (not idempotent). (#18945) 2020-02-20 10:51:29 +01:00
monojenkinsandBernhard Urban-Forster 1d92b01cf4 [metadata] use descriptive constants for method table columns (#18944)
Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-20 09:30:38 +01:00
Jay Krell a03dee09f9 [interp] inline newobj. (#18903)
The real goal is to make it not recursive.
This is just a step along the way.
Note that the actual not-recursive change has *not* yet been developed,
so this *might* amount to nothing useful.
2020-02-19 23:32:15 +01:00
Vlad Brezae 5d87716ea4 [sgen] Disable managed allocator when using nursery-canaries (#18932)
The managed allocator included in a full aot image does not support canaries, leading to inconsistent object layout.

Fixes https://github.com/mono/mono/issues/18925
2020-02-20 00:10:49 +02:00
Vlad Brezae 47687bdb47 [interp] Fix incorrect propagation of indirect local (#18924)
We never don't track the value of locals who have their address taken in a method, since their value can change without the pass detecting it, leading to propagation of incorrect values. We failed to check for this when doing a stloc.np optimization.

Fixes System.Reflection.Metadata.Tests.ImmutableByteArrayInteropTest.DangerousCreateFromUnderlyingArray

Enable more interp tests, which don't seem to fail on my machine.
2020-02-20 00:09:16 +02:00
monojenkinsandRyan Lucia 7d38e12a1e [loader] Add check to mono_domain_ensure_entry_assembly for AOT (#18938)
Fixes a crash with AOT because domain->setup is NULL here

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-19 22:48:50 +01:00
Zoltan Varga 5fe94fd99f [wasm] Avoid spilling refs to the stack in gsharedvt wrappers, those functions don't need safepoints. (#18820) 2020-02-19 14:30:08 -05:00
Mitchell Hwang 30c1012ed5 [bcl] Default XmlSerializer stream serialize to UTF8 Encoding (#18836)
* [mcs] Default XmlSerializer stream serialize to UTF8 Encoding

* [bcl] Add XmlSerialize Test to preserve UTF chars in stream
2020-02-19 14:11:07 -05:00
monojenkinsandRyan Lucia 45cb6fbba6 [loader] Parse in more netcore hosting properties (#18850)
This includes some other minor cleanup to the netcore hosting functions.

Setting `assemblies_path` with `APP_PATHS` despite it also being set by `MONO_PATH` doesn't scare me too much because:
1) Anyone using `MONO_PATH` is probably not going through the hosting API and is using `mono-sgen` directly
2) The default host doesn't set `APP_PATHS` (and we've worked fine without it being set at all up until now
3) This is a temporary measure; we will have to refactor our preload hook when we want to properly support the other loader properties anyway

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-19 14:10:33 -05:00
Zoltan Varga b4337dd4e8 [llvm] Remove the disable-self-init debug option, it doesn't work any more. (#18929) 2020-02-19 11:43:20 -05:00
Egor Bogatov 6ef5dd4523 Move HttpClientHandlerTests.Android.cs to nunit (#18883) 2020-02-19 16:34:37 +03:00
Alexander Köplinger 885226b1f0 [netcore] Disable ARM64 CI runs (#18922)
They were moved over to dotnet/runtime: https://github.com/dotnet/runtime/pull/32303
2020-02-19 02:36:38 +01:00
Jay Krell d1603aaca1 Remove some #if __cplusplus wrt inlining. (#18891)
The code is fine, and we should put it back at some point,
but I want things to be clearer regarding https://github.com/mono/mono/issues/18827.

i.e. That this does not cause it.
I don't know what does cause it, granted.
Maybe a preexisting race condition/deadlock, I don't know.

Repro is run a loop like:
```
type \s\1.cmd
for /l %%a in (1 1 999) do echo %%a && C:\s\mono2\msvc\build\sgen\x64\bin\Release\mono-sgen.exe \s\mono2\mono\tests\bug-10127.exe
```
2020-02-19 01:26:46 +01:00
Jay Krell 6d0895ad3b Make MONO_API_DATA clearer. (#18897)
Make MONO_API_DATA clearer, its old form -- MONO_API extern.
extern "C" { } should be added around it not within it, for clarity.
I've always found extern "C" without braces unclear, esp. for data.

i.e.

```
 extern "C" {
1   extern void function(); // redundant but clear
                            // same as void function();
2   extern int data; // not redundant, and clear
}
```

vs.
```
3   extern "C" void function(); // I guess clear.
4   extern "C" int data; // unclear -- is it extern or extern "C" or both?
```

This PR in particular turns 4 into 2.
2 is clearly a declaration, not a definition.
4 is unclear as to if it is a declaration or definition.

I was looking into https://github.com/mono/mono/issues/18827.

This does not actually change anything but makes things clearer.

This is similar/related in spirit to https://github.com/mono/mono/pull/18891.
It should not make a difference.
In this case, `#if __cplusplus` remains, but the meaning of it is clearer.
It becomes *only* about the common `extern "C" { }` and not even the less clear `extern "C"`.
2020-02-19 01:20:07 +01:00
Jay Krell 90f4013483 [interp] Make newobj_vt[st]_fast non-recursive. (#18902)
* [interp] Make newobj_vt_fast non-recursive.

* PR feedback: Attempt to provide similer primitive increase_max_stack_height
that is not so roundabout as move_stack (n) + move_stack (-n).

* [interp] Make newobj_vtst_fast not recursive.

This is builds upon https://github.com/mono/mono/pull/18875
and is easiest reviewed after it is merged.

Or, perhaps, this replaces it.

* comment-smithing
2020-02-19 01:17:14 +01:00
Alexis Christoforides 39ea4f4bbd [mono] Remove SIGTRAP handler, as it interferes with debugging (#18851)
* [mono] Remove SIGTRAP handler, as it interferes with debugging

See https://github.com/mono/mono/pull/18243#issuecomment-586013427

* Fix lack of cleanup for SIGSYS
2020-02-18 16:33:44 -05:00
Aleksey Kliger (λgeek) 972896b2f1 [merp] Capture Environment.FailFast message in crash report (#18863)
* [merp] Capture Environment.FailFast message in crash report

* [test] merp-crash-test refactor crasher data; add optional validator

Switch from a Tuple<T1,T2> to a struct for the crash items.  Add a third
optional validator field which is an action to run on the json blob that we get
back.

Not updating existing tests to do any json validation, yet.

* [test] Add new MERP crash test for Environment.FailFast

Check that the resulting json blob includes the failfast message.
2020-02-18 15:59:14 -05:00
Vlad Brezae 4b96fefa7a [interp] Unwrap RuntimeWrappedException if needed, when throwing non exception objects (#18905)
Enable the entire System.Linq.Expressions suite with interpreter
2020-02-18 19:59:18 +01:00
Marius Ungureanu 104c201053 Bump bockbuild to bring in Gtk# regression fix (#18909)
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1055130
2020-02-18 19:46:24 +01:00
Alexander Köplinger e25d05be8d [netcore] Disable a flaky test on interpreter (#18904)
See https://github.com/mono/mono/issues/18695, it's most likely fixed in dotnet/runtime already but we can't easily get updated binaries into mono/mono.
2020-02-18 15:40:01 +01:00
Jay Krell 4649b5784f Remove #if __cplusplus from MagicTypeInfo. (#18898)
This wastes about 24 static const bytes which I think is ok.
6 bytes per struct, plus 2 padding, and there are 3 of them.

I was looking into https://github.com/mono/mono/issues/18827
and wanting to rule out #if __cplusplus.
2020-02-18 10:53:17 +01:00
Zoltan Varga 7038b1a426 [wasm] Implement generation of empty assemblies in wasm-tuner. (#18880)
The linker can completely link away some assemblies, but the rest of the ninja
build still requires them. Previously they were generated by running csc, but
thats too slow, so generate them using Cecil from wasm-tuner.
2020-02-18 02:33:45 -05:00
Jay Krell 15d61562ca Make non-recursive mutexes SRWLOCK instead of CRITICAL_SECTION. (#18847) 2020-02-17 22:34:44 +01:00
Jay Krell 1288a22890 [interp] Consolidate non-recursive call sites. (#18825)
Consolidate native_stack_addr as a result (resolving a FIXME).
 Builds on:
  https://github.com/mono/mono/pull/18857
  https://github.com/mono/mono/pull/18858
  https://github.com/mono/mono/pull/18869
2020-02-17 17:02:24 +01:00
Jay Krell 38256821ed Fix THREADS_SUSPEND_DEBUG to compile (id vs. mono_thread_info_get_tid). (#18871) 2020-02-17 15:59:48 +01:00
dotnet-maestro[bot] 14ae61555b Update dependencies from https://github.com/dotnet/arcade build 20200216.1 (#18887)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20116.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20116.1
2020-02-17 15:55:48 +01:00
imhameed 78fcd4b9f4 [llvm] Disable X86 call frame size optimization. (#18645)
* [llvm] Disable X86 call frame size optimization.

Fixes https://github.com/mono/mono/issues/17285.

For calls that require parameters passed in stack memory,
X86CallFrameOptimization can sometimes replace a sequence of movs to
prologue-reserved stack memory with a sequence of pushes and a compensating sub
rsp after the call. LLVM accumulates an OpAdjustCfaOffset MCCFIInstruction for
each push and and records the ephemeral parameter space consumed by the call
using an OpGnuArgsSize MCCFIInstruction (i.e. DW_CFA_GNU_args_size).

MonoException doesn't support encoding OpGnuArgsSize MCCFIInstructions, and
attempting to encode this instruction can cause unpredictable behavior, due to
our use of llvm_unreachable with assertions disabled [1]. Moreover, even if
MonoException happens to emit valid call frame instructions despite the UB
induced by llvm_unreachable, Mono's exception handler will restore the stack
pointer to the value it contained at the time right before the protected call
instruction was made, without compensating for any ephemeral parameter space.

The stack pointer could be fixed before the jump to the EH handler by
supporting DW_CFA_GNU_args_size or by executing call frame instructions for the
EH handler's frame all the way to the end of the protected instruction address
range--which will include a sub rsp with an appropriate immediate--to determine
an appropriate delta. But it's easier to just disable this optimization. We
already disable it for AOT [2]. It's sometimes disabled by LLVM on Darwin [3].

Also, this commit adds a test (which is a nearly verbatim copy of laszk's
reproduction sample in #17285) for EH failures caused by the interaction of our
EH code and X86CallFrameOptimization.

See also [4], [5], and [6].

[1] https://github.com/dotnet/llvm-project/blob/adeaa08e7bbc9aba5d67cb16c2b348be12deb000/llvm/lib/CodeGen/AsmPrinter/MonoException.cpp#L209-L210
[2] https://github.com/mono/mono/blob/1317cf06da06682419f8f4b0c9810ad5d5d3ac3a/mono/mini/aot-compiler.c#L9872-L9875
[3] https://github.com/dotnet/llvm-project/blob/adeaa08e7bbc9aba5d67cb16c2b348be12deb000/llvm/lib/Target/X86/X86CallFrameOptimization.cpp#L146-L152
[4] https://bugs.llvm.org/show_bug.cgi?id=24792
[5] https://reviews.llvm.org/D13132
[6] http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151005/304074.html

* Address feedback: add an explanatory comment.
2020-02-17 15:41:50 +01:00
Alexander Köplinger e9981608c4 Bump linker submodule (#18884)
Contains fix for -out argument.
2020-02-17 14:54:44 +01:00
Zoltan Varga 3e1f029eb8 [wasm] Make cil-strip quiet. (#18881) 2020-02-17 14:22:39 +01:00
Bernhard Urban-Forster f6dfce1aba [cross] check against correct alignment value (#18712)
It's essentially dead code in any of our configurations.

`MONO_ALIGN_gint8` is define here:
https://github.com/mono/mono/blob/ab2321c1dc1bf0293dadd7e52436b2bd87f2b7c3/mono/metadata/abi-details.h#L26-L35

And the correct value is set here:
https://github.com/mono/mono/blob/ab2321c1dc1bf0293dadd7e52436b2bd87f2b7c3/mono/metadata/abi.c#L71-L75
2020-02-17 14:22:10 +01:00
Zoltan Varga 79ab49333e [jit] Run static cctors for inlined methods, the cctor might not have ran in AOT mode. (#18706)
* [jit] Run static cctors for inlined methods, the cctor might not have ran in AOT mode.

* Emit the class init code later since it contains a branch.

* Emit the class init in the first code bblock.
2020-02-17 08:21:49 -05:00
Bernhard Urban-Forster 79135c6ea5 [abi-details] add explaining comment for macro magic (#18713)
* [abi-details] add explaining comment for macro magic

I think the fallback was added because "let's get it back to work quickly", back in the dark times without CI: https://github.com/mono/mono/commit/b655fb35c61fd3422003de01eed8e634d46b4186

* add constant version. to be used when offset is needed for static array sizes

* missing offset for windows

* windows define fix
2020-02-17 14:21:13 +01:00
Jay Krell 9a147baef1 Remove unnecessary ifdef before undef. (#18877)
* Remove unnecessary ifdef before undef.

* PR feedback: Remove educational comment.
2020-02-17 14:01:34 +01:00
Jay Krell 9adb352348 Remove duplicated Windows header content. (#18846) 2020-02-17 13:48:53 +01:00
Jay Krell 9a6b9c8a2a Fix/quash some warnings. (#18873)
mini\mini.c(1253,59): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3100,43): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3273,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3281,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3480,51): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3490,49): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3626,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3664,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3722,48): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini.c(3944,47): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\mini-runtime.c(549,64): warning C4390:  ';': empty controlled statement found; is this the intent?
mini\method-to-ir.c(11838,2): warning C4065:  switch statement contains 'default' but no 'case ' labels
metadata\sre.c(2286,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2179,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2204,11): warning C4309:  '=': truncation of constant value
metadata\sre.c(2232,11): warning C4309:  '=': truncation of constant value
mini\method-to-ir.c(11838,2): warning C4065:  switch statement contains 'default' but no 'case
mini\type-checking.c(167,3): warning C4334:  '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
2020-02-17 12:48:56 +01:00
Jay Krell e8601e7b12 Remove unnecesssary include. (#18872)
Less to build if you edit the file.
2020-02-17 12:47:20 +01:00
Jay Krell 68c9084466 [interp] Use method_entry at interp entry instead of duplicating the code. (#18869)
tracing was a mix of:
 - sometimes ifdef out, sometimes not; make it always ifdefed
 - sometimes int, sometimes gint, sometimes gboolean; make it always int

Much of this PR will end up deleted when the multiple nonrecursive calls are
consolidated, but two will remain -- entry and call: label.

https://github.com/mono/mono/pull/18825 is where this is going but it was deemed
too large and requested to be split up. This is part of it.
2020-02-17 12:46:22 +01:00
Jay Krell 2d0ddf0abd [interp] Work toward consolidating multiple nonrecursive call sites to just one. (#18858)
https://github.com/mono/mono/pull/18825 was deemed too large so this breaks
it down into smaller pieces.
To consolidate the multiple nonrecursive call sites requires they share the smae
retval variable. Go ahead and share them all.
2020-02-17 12:43:34 +01:00
Jay Krell d98b7bf9f9 [interp] Fix incorrect comment -- stackval has no size. (#18876) 2020-02-17 12:42:08 +01:00
monojenkinsandEgor Bogatov 79bb3e2031 [mono] Fix msvc mono build and add CopyMonoRuntimeFilesFromArtifactsToDestination target (#18797)
Fixes
```
fatal error C1083: Cannot open include file: '../../support/libm/complex.c': No such file or directory
```
when `mono-netcore.sln` is opened in vs2019

Also, add `CopyMonoRuntimeFilesFromArtifactsToDestination` target to mono.proj to be able to copy mono runtime bits to any folder (needed for the performance team, cc @fanyang-mono)

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-17 12:02:18 +01:00
Jay Krell fed64a7019 [interp] Step toward consolidatation of non-recursive call sites. (#18857)
https://github.com/mono/mono/pull/18825 was too large.
This starts by replacing imethod with cmethod which is a necessary
part of the consolidation of multiple non-recursive call sites to just one.
2020-02-17 09:12:52 +01:00
Zoltan Varga 56f664f2d3 [llvmonly] Allow more direct calls. (#18819)
* [runtime] Add a few m_method_is_... inline functions.

* [llvmonly] Allow direct calls to tls accessors, its not a problem in llvmonly mode.

* [llvmonly] Allow direct calls to icalls/pinvokes/generic instances in the same module.

* Fix the loaded llvm build.
2020-02-15 01:52:28 -05:00
Zoltan Varga 12c13fdf37 [llvm] Remove unused llvm.mono.load/store intrinsics. (#18853) 2020-02-15 00:57:02 -05:00
Zoltan Varga e3ad74614a [llvmonly] Avoid adding the aot init wrappers to the method table, its not needed. (#18848) 2020-02-14 20:33:01 -05:00
imhameed 013e3c07e3 [llvm] Fix LLVM JIT when used with multiple AppDomains. (#18824)
We currently reuse LLVM's command line parsing system to configure and
enable passes. When using the default code generator pass configuration,
the `ImplicitNullChecks` and `X86CallFrameOptimization` passes can only
be enabled or disabled via the command line argument parser: the
`cl::opt`s controlling these (and many other) passes are private to
their translation units, and `TargetPassConfig`/`X86PassConfig` read
from these `cl::opt`s with no other means to override these values.

Unfortunately, some LLVM command line options have restrictions on the
number of times they may be set, and running the LLVM command line
argument parser more than once can fail, because the 'number of
occurrences' counter is stored in each global `cl::opt`. This causes
several tests to fail--one such test is unhandled-exception-7.cs.

This change:
- removes the lazy LLVM JIT initialization logic and instead
  runs this initialization once during `mini_init`,

- moves some stray JIT-only code from mini-llvm.c to llvm-jit.cpp,

- wraps the declarations in domain-internals.h with a
  `G_BEGIN_DECLS`/`G_END_DECLS` pair, so that they avoid name
  mangling when included in C++ source, and

- removes `exception_cb` and `emitted_cb`: neither were actually used
  by live code.
2020-02-14 17:29:45 -08:00
Vlad Brezae c9ba14796a [interp] Fix filter clauses (#18809)
When executing a filter we create a new interp frame, that is a duplicate of the original frame that contains the filter. Once the execution of the filter is finished, we were copying the stack contents back to the base frame. The problem with this is that when the filter is run, the stack is not completely unwinded, meaning that later we can still run a finally block in a frame called by the base frame. This finally block can access the stack arguments, which would reside on the stack of base frame. This means that we must not override the stack of the base frame, only the locals.

Fixes some tests from System.Linq.Expressions suite.
2020-02-14 14:38:25 -08:00
Zoltan Varga 948d7e3b3e [llvmonly] Emit stubs for methods which failed compilation. (#18802)
Emit a stub for methods which failed llvm compilation. Currently the stub will
throw a MissingMethodException, later it can fall back to the interpreter etc.
This is required for cross-assembly direct calls, since the caller doesn't
know that the callee has failed llvm compilation leading to missing
symbols during linking.
2020-02-14 10:59:41 -08:00
dotnet-maestro[bot] 5b775170f2 Update dependencies from https://github.com/dotnet/arcade build 20200214.1 (#18856)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20114.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20114.1
2020-02-14 18:21:46 +01:00
monojenkinsandLevi Broderick 2448e5685f Fix broken MSDN blogs links (#18852)
Fixes MSDN blogs links throughout our docs and source code comments that were broken during the great MSDN migration of 2019.

Note: I am changing the contents of the _davbr Blog Archives_ folder by fixing the broken links. If we need to keep these files exactly as they were when they were copied from the blog, I can revert.

Co-authored-by: Levi Broderick <[email protected]>
2020-02-14 18:15:42 +01:00
Egor Bogatov b5df4be189 Make MonoWebRequestHandler linker friendly (#18832)
Create it using reflection so it will be always linked out. However if it's set via `XA_HTTP_CLIENT_HANDLER_TYPE` linker will preserve it in a special substep: https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/PreserveHttpAndroidClientHandler.cs#L72-L80

I had to make its ctor **public** because Linker instantiate it via `Activator.CreateInstance(string)` ([here](https://github.com/xamarin/xamarin-android/blob/60363ef45cd6994a322adac1b73d4271b0658c1e/src/Mono.Android/Android.Runtime/AndroidEnvironment.cs#L369)). But the whole type remains internal so it's not a real API change.

Fixes issue from https://github.com/mono/mono/pull/18818#issuecomment-585611647
2020-02-14 13:04:46 +01:00
Fan YangandAlexander Köplinger 549a38cd4d Bump linker submodule (#18844)
* Bump linker submodule

* Use -o instead of -out for linker output

The arguments changed in linker.

Co-authored-by: Alexander Köplinger <[email protected]>
2020-02-14 12:20:19 +01:00
Bernhard Urban-Forster 26a5d63384 [sdks] add anroid-interpreter-{arm64-v8a,armeabi-v7a,x86,x86_64} targets (#18814)
The Xamarin.Android team prefers that we ship two different `libmonosgen-2.0.so` for each target, e.g. for arm64 we will have with this PR:

* `./sdks/out/android-arm64-v8a-release/lib/libmonosgen-2.0.so`
* `./sdks/out/android-interpreter-arm64-v8a-release/lib/libmonosgen-2.0.so`

As the names indicate one has the interpreter enabled.

This is contrary to what we do for Xamarin.iOS. There, `--enable-minimal=interpreter` has the side-effect that the build will produce a `libmono-ee-interp.dylib` that can be linked next to `libmonosgen-2.0.dylib`. We _could_ do the same for Android, but that would require us to

* Add `--disable-visibility-hidden` to the Android Mono runtime builds. There are concerns however that this will slow down `dlopen (libmonosgen-2.0.so)` on the Xamarin.Android side.
* Do some changes to the interpreter infrastructure, because `--disable-minimal=interpreter` is designed to be used with a cross compiler in mind. For example, we would need a different solution for this: https://github.com/mono/mono/blob/d7e6fcbda82639fd5afe106b0e7a65a2a72a4f63/mono/mini/tramp-arm64.c#L631-L634
2020-02-14 08:26:58 +01:00
Thays Grazia fa0af37014 Removing PPDB. 2020-02-13 22:39:27 -06:00
Thays Grazia 2d0337ee31 Enabling PPDB on Wasm. 2020-02-13 22:39:27 -06:00
Zoltan Varga 2146f1f418 [runtime] Disable assertion messages for g_assertf as well if DISABLE_ASSERT_MESSAGES is used. (#18774) 2020-02-13 13:20:21 -08:00
monojenkinsandRyan Lucia b89e6208b3 [mini] Move netcore hosting functionality into separate file (#18799)
Made minor fixes to the preload hook in the process.

The runtime will still crash if you don't specify the TPA paths and include corlib in that list, but it will at least hit an assert that makes clear it can't find corlib instead of attempting to dereference a null pointer in the preload hook.

The monovm header functions are exported for testing purposes, but the file is intentionally not made public as of now.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-13 15:36:46 -05:00
Zoltan VargaandBernhard Urban-Forster 362cb3084f [wasm] Avoid compiling release configuration with -s ASSERTIONS=1 in AOT mode, it generates a lot of extra code. (#18703)
* [wasm] Avoid compiling release configuration with -s ASSERTIONS=1 in AOT mode, it generates a lot of extra code.

* Update sdks/wasm/packager.cs

Co-Authored-By: Bernhard Urban-Forster <[email protected]>

Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-02-13 10:25:05 -08:00
Alexander Köplinger c03b6244e0 [tests] Disable bug-10127.exe runtime test on Windows x64 (#18828)
It hangs: https://github.com/mono/mono/issues/18827
2020-02-13 13:50:00 +01:00
Vlad Brezae e8f392412b [interp] Always init locals (#18811)
Storing uninitialized locals in a managed object shouldn't crash the runtime, so they should be zeroed. This doesn't seem to regress performance on wasm since the initlocals flag is always set, currently. We should be able to remove it most of the time, once we add some multi basic block support to the cprop phase. This will enable us to further reduce call overhead.

Fixes https://github.com/mono/mono/issues/18527
2020-02-13 13:00:33 +01:00
Vlad Brezae 0db68c3f27 [interp] Add missing conversion opcodes from r4 (#18813)
Fixes tests from System.Linq.Expressions suite
2020-02-13 12:58:28 +01:00
Zoltan Varga 3129d4ca8b [runtime] Add --enable-minimal=processes,config options to disable support for processes and .config files. (#18822) 2020-02-13 12:55:02 +01:00
Vincent Povirk a645a2d3c7 [System.Drawing] Release the Graphics HDC on Dispose. (#18801)
Libgdiplus will fail when attempting to delete a Graphics object
that has an unreleased HDC. This allows Dispose to succeed in
that case.

This more closely resembles the behavior of corefx.
2020-02-13 10:40:07 +01:00
Jay Krell 19bc9a8dd3 [interp][wasm] Make newobj_fast not recursive, cleanup, fix EXCEPTION_CHECKPOINT. (#18734)
* [interp[wasm] Make newobj_fast not recursive.
This contributes to https://github.com/mono/mono/issues/18646,
and against master might even fix it.
Though much recursion remains in the interpreter/transform.

* [interp] Cleanup redundant code and child_frame vs. frame.
Only the recursive paths need child_frame separate from frame.

Callers of method_entry tended to duplicate parts of it.
Notice it is marked MONO_ALWAYS_INLINE.

And if that isn't right, it should be split into fast and slow forms,
i.e. regarding frame->imethod->transformed.

* Need to EXCEPTION_CHECKPOINT more often -- when transform occurs but does not raise an exception.

* Preserve prior faster paths.

* Making call_vararg added the following to all calls:
  storing is_void in a larger scoped local
  save is_void
  restore is_void
  check is_void, along with preexisting check of retval == null

Because retval must be set even for void call_varargs,
because the arglist is found from it.

This removes the check of retval == null, it should be redundant with is_void.

* A little cleaner -- unlikely is in method_entry.

* Comma to make comment easier to read.
2020-02-13 00:13:34 -08:00
monojenkinsandJo Shields e901d4fe1a Enable Mono ARM & ARM64 Linux builds (#18810)
Traditionally, Mono is build with native compilers (i.e. "build for this architecture" running in an ARM or ARM64 chroot on an ARM64 server), but to align with CoreCLR, let's go the cross-compile way.

This enables cross-compiler support, by passing a whole bunch of extra info to `./configure`, some of which is set in the common YAML (like the `--sysroot` value)

Co-authored-by: Jo Shields <[email protected]>
2020-02-13 01:15:38 -05:00
Zoltan Varga 0a4c3069be [wasm] Update WASM-REQUESTS.md. (#18816) 2020-02-12 18:04:49 -08:00
Zoltan Varga c76bbb23f9 Revert "[wasm] Pass direct-pinvoke to the AOT compiler. (#18775)" (#18807)
This reverts commit eb85a108a3.
2020-02-12 16:21:51 -08:00
Zoltan Varga 3776b416ad [llvm] Remove a wasm limitation which no longer exists. (#18803) 2020-02-12 16:19:21 -08:00
monojenkinsandRyan Lucia cd46f5c77c [WIP][threading] Switch to a native implementation of LowLevelLifoSemaphore (#18559)
This change is motivated on two main fronts. The first is maintainability - the current managed implementation is difficult to understand and I worry diagnosing any potential issues would be a massive time sink. The second is performance - the current implementation appears to suffer from significant lock contention when running the TechEmpower plaintext benchmark. My hope is that the simpler, cleaner native implementation here will avoid both problems, but I don't want to merge it until we have benchmarking data. However, even if the numbers are similar, I still think it's worth merging just from a maintainability perspective.

The native LifoSemaphore implementation has only ever been tested on posix-like platforms, so Windows behavior is unknown. Currently the Windows implementation of LowLevelLifoSemaphore is very different, so unless we have need for the LifoSemaphore elsewhere in the runtime this isn't a concern.

Many thanks to @filipnavara for the initial implementation.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-12 17:14:30 -05:00
Jo Shields d7e6fcbda8 Fix native mkbundle (#18739)
* Use clearer Bash conditionals, since we force Bash anyway

* Add libmono-native.so to bundle

* Add logic to add a etc/mono/config without $mono_libdir to bundle

Mostly resolves: #16991
2020-02-12 15:57:54 -05:00
Zoltan Varga 3cbc14f6ff [wasm] Remove chakra and xs from the jsvu update list. (#18804)
Chakra is no longer getting updated and we don't use XS.
2020-02-12 10:05:06 -08:00
Zoltan Varga a50f0b1197 [wasm] Switch to the lts version of node. (#18805)
* [wasm] Switch to the lts version of node.

* Fix a typo.
2020-02-12 10:02:55 -08:00
Ankit Jain 8437ecf9d1 [master] Bump msbuild to track xplat-master (#18790)
* [master] Bump msbuild to track xplat-master

* Bump roslyn to 3.5.0-beta3-20111-03 .

* Update nuget.exe to match xbuild

- the closest version available right now, to `5.5.0-rtm.6422` is
`5.5.0-preview2`, so using that for now.
2020-02-12 17:10:31 +01:00
Egor Bogatov 76e577c00f Allow users to switch to MonoWebRequestHandler on Android via UI (#18764)
* Bring back old HttpHandler for Android
2020-02-12 16:25:46 +03:00
Zoltan Varga e2c0b5ef27 [runtime] Call mono_stack_mark_record_size () only in a checked build, its slow, and the __FUNCTION__ argument creates a C string for every function name. (#18773) 2020-02-12 10:43:35 +01:00
monojenkinsandRyan Lucia 07dc4b88e7 [utils] Add unused attribute to fix warning in checked-build.c (#18800)
At least on my machine, this is the only warning we have when building netcore in the default configuration.

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-11 22:57:15 -08:00
Zoltan Varga 1c7295cfdf [jit] Add an #ifdef around MONO_CPU_WASM_SIMD. (#18771) 2020-02-11 22:56:34 -08:00
monojenkinsandEgor Bogatov fc75795ed2 [mono] Implement Sse2.AddSaturate using LLVM (#18793)
Mono currently supports a limited subset of **Sse1**-**Sse42** intrinsics used only by CoreLib internally. So once a new API is used to optimize things there we also have to implement it in Mono. So it recently happened in https://github.com/dotnet/runtime/pull/32036 (`Sse2.AddSaturate` was used for the first time).

So this PR implements it for mono (with all Sse2 overloads) using named LLVM intrinsics. It turns out it's different between LLVM6 (we currently based on) and LLVM9 (we plan to migrate to soon).

for `Vector128<byte>` overload it emits
```llvm
%result = call <8 x i16> @llvm.x86.sse2.paddus.b(<16 x i8> %left, <16 x i8> %right)
```
which is then emitted as `vpaddusw`

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-12 00:55:25 +01:00
Zoltan Varga eb85a108a3 [wasm] Pass direct-pinvoke to the AOT compiler. (#18775) 2020-02-11 12:08:49 -08:00
Zoltan Varga b260b327fe [runtime] Simplify access to runtime TLS variables when DISABLE_THREADS is defined, they become simple globals. (#18772) 2020-02-11 11:49:24 -08:00
Alexander Köplinger e8527db725 [mini] Wrap check for debug info enabled in ENABLE_NETCORE in all places (#18789)
See if this fixes the weird bug-10127.exe test failure that only occurs on Windows C++ builds.
2020-02-11 16:50:14 +01:00
Alexander Köplinger 071b7259d5 Move offsets-tool into mono/tools (#18783)
This allows us to consume it in the dotnet/runtime build.
2020-02-11 12:18:45 +01:00
Bernhard Urban-Forster 742c414bde [interp] reclaim stack fragments when not used. (#18777)
* [interp] reclaim stack fragments when not used.

When a requested frame size doesn't fit in an already allocated stack fragment, we instead allocate another, larger fragment.
When doing this we missed to free the existing stack fragment, and in combination with a second problem (`current->next` not being cleared when unused) it can accumulate to a lot of unused stack fragments and in worst-case to OOM, as it happened on Linux corlib xunit tests.

Thanks to Jay for tracking it down.

* be cautious for GC

* add barrier
2020-02-11 10:59:49 +01:00
Bernhard Urban-Forster 4cd05a7be4 [cross] support arm-linux-gnueabihf cross compiling with netcore (#18714)
* [cross] support arm-linux-gnueabihf cross compiling with netcore

Some notes on usage:

```console
$ sudo apt install g{++,cc}-gnu-linux-gnueabihf python3-pip

$ ./autogen.sh --disable-boehm --target=arm-linux-gnueabihf \
    --with-cross-offsets=arm-linux-gnueabihf-cross-offsets.h \
    --with-core=only \
    CFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer" \
    CXXFLAGS="-O0 -ggdb3 -fno-omit-frame-pointer"

$ make -C tools/offsets-tool-py/

$ python3 tools/offsets-tool-py/offsets-tool.py \
    --targetdir=/home/lewurm/work/mono \
    --abi=arm-linux-gnueabihf \
    --monodir=/home/lewurm/work/mono \
    --outfile=arm-linux-gnueabihf-cross-offsets.h \
    --libclang=/usr/lib/x86_64-linux-gnu/libclang-7.so.1 \
    --sysroot=/usr/arm-linux-gnueabihf \
    --netcore
```

* better include path guessing
2020-02-11 09:49:21 +01:00
Jay Krell b0c434280d Add volatile for cross thread communication. (#18760) 2020-02-10 21:54:41 -05:00
Alexis Christoforides 72fe53a2ee [MacSDK] Get more verbose output from fc-cache operation (#18782) 2020-02-10 21:53:00 -05:00
dotnet-maestro[bot] 900e77c868 [master] Update dependencies from dotnet/arcade (#18674)
* Update dependencies from https://github.com/dotnet/arcade build 20200201.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20101.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20101.2

* Update dependencies from https://github.com/dotnet/arcade build 20200209.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20109.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20109.1
2020-02-10 17:03:10 +01:00
Jay Krell 8082e87c4e [cxx] Fix C++ build -- int vs. enum. (#18727) 2020-02-09 11:26:52 +01:00
monojenkinsandRyan Lucia 1635c21e59 [loader] Set domain->entry_assembly earlier in startup (#18720)
When locating AppContext.BaseDirectory on netcore, if APP_CONTEXT_BASE_DIRECTORY is not set we call into the runtime for a fallback location, which checks domain->entry_assembly. On legacy this is fine because we set that property in `prepare_thread_to_exec_main`, but on netcore the new managed assembly loading algorithm will call into managed and check the property before that function is run, which will result in AppContext.BaseDirectory returning an empty string. The solution is to set entry_assembly a bit earlier in initialization. This is the appropriate location because it's what embedding API users will call to run an assembly.

In the process, I got rid of some code that would bypass the usual lookup algorithm unnecessarily.

Fixes dotnet/runtime#31800

Co-authored-by: Ryan Lucia <[email protected]>
2020-02-07 17:18:15 +01:00
Alexis Christoforides 96e64ad4e6 [MacSDK] Keep logged post-install script output from previous installs (#18441)
This is helpful for issues such as https://devdiv.visualstudio.com/DevDiv/_workitems/edit/596552 OC, where the user is asked to both provide these logs for debugging, and also reinstall the SDK to resolve the issue
2020-02-07 11:15:48 -05:00
Alex Soto 0c200c5125 [iOS] Replace removed dsymutil -t switch with -num-threads (#18741)
Apple removed `-t` switch from dsymutil in favor of `-num-threads`
2020-02-07 11:13:31 +01:00
Alexis Christoforides d9e090e867 [merp] Add an exception type for managed exceptions (#18685)
Addresses https://github.com/mono/mono/issues/18681
2020-02-06 23:33:27 -05:00
Alexis Christoforides 081498393a [merp] Add tests for crashing via POSIX signal (#18532) 2020-02-06 23:30:11 -05:00
Alexander Köplinger a40e1ae681 Skip WebAssembly build in sdks archive build on non-master branches
WebAssembly doesn't use our usual release branches yet.
2020-02-06 22:36:38 +01:00
monojenkinsandSteve Pfister 88c44e26f6 [mono] Enable debugging for netcore configurations by default. (#18686)
What this means is:

1. --debug is no longer needed since the default is TRUE
2. --debug=ignore is a new switch to go back to the old mono default

Fixes https://github.com/dotnet/runtime/issues/31662

Co-authored-by: Steve Pfister <[email protected]>
2020-02-06 22:19:04 +01:00
Manuel de la Pena b2ad5f34e1 [NUnitLite] Bump nunitlite submodule. (#18731)
Update the nunitlite submodule to pick up some fixes in the NUnit v3
outputwriter that are needed by xamarin-macios.
2020-02-06 15:58:38 +01:00
mdh1418 7936510ec3 [corlib] Silence undocumented exceptions in ReadXdgUserDir (#18684) 2020-02-06 08:49:28 -05:00
Johan Lorensson 6fe343db8c Reduce number of mono header files copied into SDK include folder. (#18702) 2020-02-06 09:11:55 +01:00
Larry Ewing 4cb6ca5599 [wasm][debugger] Use System.Text.Json in the driver 2020-02-05 20:37:46 -06:00
Larry Ewing 8ce81f5113 [wasm][debugger] Let chrome pick an open port by default 2020-02-05 20:37:46 -06:00
Larry Ewing dd151fea35 [wasm][debugger] Silence some logging and extract the null case 2020-02-05 20:37:46 -06:00
Larry Ewing 32826eb418 [wasm][debugger] Make command formatting slightly more sane and reduce c&p code 2020-02-05 20:37:46 -06:00
campersau a2d1aec5d2 [wasm] Close WebSocket on Dispose (#18628) 2020-02-05 18:58:05 -06:00
Alexis Christoforides 31c12db374 [runtime] Improve handling crashing signals (#18243)
The default handlers for SIGTRAP, SIGSYS on macOS and Linux crash the process, so we now have Mono report this.

This patch also changes how the name of the signal is acquired for passing on to the crash reporting functions, allowing code reuse on crashing signals.
2020-02-05 19:04:57 -05:00
Jay Krell 0e49716ff8 [interp] Remove recursion from call_vararg. (#18697)
[interp] Make call_vararg non-recursive.

[interp] Remove recursion from call_vararg.

This contributes to https://github.com/mono/mono/issues/18646 but is not enough to fix it. The interpreter still recurses a lot.
2020-02-05 17:45:53 -05:00
Johan Lorensson b8369e630c Stack corruption running full AOT interpreter using ValueTuple. (#18710)
Calling a method compiled using full AOT from interpreter can lead to
stack corruption if signature include a ValueTuple without fields on
compilers where MONO_ZERO_LEN_ARRAY is not 0 (msvc, clang). This is due
to doing memset on memory allocated using MONO_SIZEOF_GENERIC_INST but
then memset using sizeof (MonoGenericInst). Fix makes sure we allocate
and memset the same size of memory since otherwise it will lead to an
overwrite of allocated memory when type_argc is 0.
2020-02-05 17:35:13 -05:00
Zoltan Varga 64eacfa900 [wasm] Pass the binding assemblies to the linker normally using -d framework_dir instead of explicitly. The latter makes the linker not link them. (#18704) 2020-02-05 11:04:49 -05:00
Zoltan Varga 2c8371c221 [wasm] Rename aot-dummy to aot-instances. (#18692) 2020-02-05 11:04:31 -05:00
monojenkins ab2321c1dc [bcl] Update BCL Linked Size 2020-02-05 08:18:03 +01:00
Zoltan Varga e9830fac8d [jit] Avoid calling mono_aot_direct_icalls_enabled_for_method () in non-aot mode, it might be a stub if aot is disabled. (#18707) 2020-02-05 00:00:27 -05:00
Alexander Köplinger 8027746aea Remove netcore/corerun/Makefile from mono-output.m4
It was removed from DIST_SUBDIRS with https://github.com/mono/mono/commit/2017e493a970a79aa32df2f05e5f1fd877cb3fdc
2020-02-04 23:36:51 +01:00
Bernhard Urban-Forster 71ad5a0771 [mini] iltests.exe depends on TestDriver.dll (#18638) 2020-02-04 23:28:37 +01:00
Matt Sylvia 2f74ab393f Update deprecated query parameter to header (#18705)
Use of `access_token` is now deprecated, switching to HTTP header

```
Please use the Authorization HTTP header instead, as using the `access_token` query parameter is deprecated.
```
2020-02-04 22:57:46 +01:00
Jay Krell b5fb8dbe6a [interp]Factor out common code. (#18672) 2020-02-04 19:53:48 +01:00
Alexander Köplinger 44fe895ab0 Treat 2020-02 as a valid release branch name too
This should give us breathing room until 2100-02 :)
2020-02-04 19:28:24 +01:00
monojenkinsandEgor Bogatov 2017e493a9 Add RunSample target to mono.proj (#18635)
1) Delete `src/mono/netcore/corerun/` folder, we can build/use coreclr's corerun (it has `--clr-path`)
2) Add `RunSample` to build & run HelloWorld with mono runtime.

Co-authored-by: Egor Bogatov <[email protected]>
2020-02-04 19:15:56 +01:00
Zoltan Varga a592b4b871 [wasm] Add linker substitutions for fields. (#18700) 2020-02-04 12:41:31 -05:00
Alexander Köplinger 303122e94a Bump bockbuild 2020-02-04 17:41:44 +01:00
Alexander Köplinger 088692ff3a Bump xsp to get correct version number 2020-02-04 17:41:44 +01:00
Jo Shields 8d00b7cd8b Bump version to 6.13 (#18699) 2020-02-04 17:41:01 +01:00
Jay Krell cdb7944e7b Add --profiling-funcs to release wasm build. (#18663)
[wasm] Put function names in call stacks.

Add --profiling-funcs to release wasm build.
This is a weaker for of -g2 or -g3 or -g4 or --profiling.
It sounds like the weakest form that preserves names.
Documentation is around here: https://emscripten.org/docs/tools_reference/emcc.html#emcc-g1

Now when reproducing for example https://github.com/mono/mono/issues/18646, you get:
```
 e.stack

<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[interp_transform_call]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[interp_inline_method]@[wasm code]
<?>.wasm-function[generate_code]@[wasm code]
<?>.wasm-function[generate]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_class_init_full]@[wasm code]
<?>.wasm-function[mono_interp_transform_method]@[wasm code]
<?>.wasm-function[do_transform_method]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke_checked]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke_array]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke]@[wasm code]
<?>.wasm-function[ves_icall_InternalInvoke_raw]@[wasm code]
<?>.wasm-function[do_icall]@[wasm code]
<?>.wasm-function[do_icall_wrapper]@[wasm code]
<?>.wasm-function[interp_exec_method_full]@[wasm code]
<?>.wasm-function[interp_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_jit_runtime_invoke]@[wasm code]
<?>.wasm-function[do_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_try_invoke]@[wasm code]
<?>.wasm-function[mono_runtime_invoke]@[wasm code]
<?>.wasm-function[mono_wasm_invoke_method]@[wasm code]
wasm-stub@[wasm code]
Db@[native code]
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:189579
call_method@http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:153837
http://192.168.0.20:8000/_framework/wasm/dotnet.js:1:156177
callEntryPoint@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:37229
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:33889
http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:32396
i@http://192.168.0.20:8000/_framework/blazor.webassembly.js:1:31247
promiseReactionJob@[native code]
```
2020-02-04 11:36:22 -05:00
Jay Krell edb18739cb [interp] Remove goto related to code_type being unknown | interp | compiled. (#18678)
[interp] Remove goto for code_type unknown | interp | compiled.

[interp] Remove goto for code_type unknown | interp | compiled.

The goto kinda implies a loop, but it is really just handling one case
before the other, but the source misorders the checks, perhaps for perf,
which maybe, can be done equivalently with G_UNLIKELY, and also remove
some possible extra memory reads.
2020-02-04 09:15:58 -05:00
Jay Krell f85a820677 When the interpreter does recurse, use a different function (#18664)
[interp][wasm]Use per-call recursive functions in interpreter to aid debugging.

 When the interpreter does recurse, use a different function
for each call site, so you can tell from a WebAssembly stack
which case it is. i.e. with function names but no line numbers.

With this, we can see that call_varargs contributes to the problem
here: https://github.com/mono/mono/issues/18646

Also, do not inline the interpreter, it probably/hopefully never was.
2020-02-04 05:56:00 -05:00
monojenkinsandZoltan Varga 1a81bef2bd [runtime] Fix the --enable-minimal=aot build. (#18688)
Fixes https://github.com/mono/mono/issues/18675.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-04 09:51:42 +01:00
Zoltan Vargaandmonojenkins 08f4ef6d0c Bump linker. (#18691)
* Bump linker.

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-02-04 09:26:09 +01:00
Jay Krell 8c0f6db26f [interp]Fix the type of clause_args. (#18671)
[interp]Fix the type of clause_args.
2020-02-04 00:25:57 -05:00
Jérôme Laban f3d2cc2176 [Wasm] Update to emscripten 1.39.7 (#18687) 2020-02-03 22:23:47 -05:00
monojenkinsandZoltan Varga 7deefe779f [aot] Avoid inflating gparams with byreflike types during generic sharing. (#18680)
Fixes https://github.com/mono/mono/issues/18676.

Co-authored-by: Zoltan Varga <[email protected]>
2020-02-03 21:35:20 -05:00
Zoltan Varga 62a57c687c [wasm] Set the 'top-runtime-invoke-unhandled' option so unhandled exceptions calls AppDomain.UnhandledException. (#18642) 2020-02-03 18:42:49 -05:00
d44a5d3a0b [mini] Add method-triggered printing for runtime stats (#18621)
* [mini] Add optional method name to --stats (#18387)

* Implement parsing for optional method name to --stats command line argument.

* Implement parsing for optional method name to --stats command line argument.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Respond to PR feedback.

* Refactored away helper functions.

* Refactored counter setup into separate function. Moved stats optional method into its own condition.
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* Renamed mono_setup_counters->mono_enable_counters. Restore whitespace to 0a51d89c42
Co-authored-by: Kyle Markland <[email protected]>
Co-authored-by: Rene Garcia <[email protected]>

* [mini] Print header for stats. (#18442)

* Add an enum for dumping stats at points in time. Print header before dumping stats in mini_cleanup.

* Make print_stats_header private.

* Remove unused variable.

* Refactored print_stats_header to better accomodate optional methods for stats

* Change printf calls to g_printf in response to feedback.

* Undo changes in response to @CoffeeFlux 's feedback. Add g_printf calls to mini_cleanup

* Avoid always printing stats at shutdown.

* Condition on `mono_stats` instead of `mono_jit_stats`.

* [mini] Global JIT Stats Printing (#18483)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Rename mono_print_jit_stats to mono_runtime_print_stats and make it more generic.

* Rename runtime_cleanup_stats back to jit_stats_cleanup.

* Fix a line that was indented with spaces.

* [mini] parse fully qualified method name using mono_method_desc_new (#18499)

* Make print_jit_stats global to enable interp and JIT-triggered stat dumps.

* Rename some functions for consistency.

* Add a comment to clarify the necessary locking in different contexts.

* Use MonoMethodDesc struct and associated helpers to parse from command
line args. Remove unused stats_method_name and setter function.

Squashed commit of the following:

commit de7410696dab10723c76f545076522daa17e85d1
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 23:01:23 2020 -0500

    Utilize new stats printing function.

commit 1e5038adfec06c670dbaced771a8ff9344502d17
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 22:37:53 2020 -0500

    Remove unused stats_method_name and setter function. Use mono_method_desc_full_match instead of mono_method_desc_match.

commit 04f86c66c139e064bc35738f955f960232eb6cbb
Author: Rene Garcia <[email protected]>
Date:   Thu Jan 16 18:11:37 2020 -0500

    Modify mono_method_desc_match to actually match namespace and class. Add global MonoMethodDesc, and parse it from command line args. Leave TODO print statement in mono_jit_compile_method_inner.

* Move mono_method_desc to mini-runtime header. Fix whitespace.

* Cherry pick @KyleMarkland 's changes, and resolve merge conflicts.

* Fix leak. Give better error message when parsing empty method name.

* Move freeing of mono_stats_method_desc.

Co-authored-by: KyleMarkland <[email protected]>

* Add a conditional to transform.c to enable stat dumping from the interpreter. (#18540)

* [mini] Use g_printerr instead of g_printf before exiting (#18574)

* Rename g_printf -> g_print

* Use g_printerr on errors instead of g_printf.

* Pass NULL to mono_counters_dump. Revert g_printf to g_print.

* Update manpage

* Feedback

Co-authored-by: rgarc <[email protected]>
Co-authored-by: KyleMarkland <[email protected]>
2020-02-03 16:05:06 -05:00
Vlad Brezae 01b2e15c06 [tests] Reenable enum equals test on interpreter (#18673)
Fixed by https://github.com/mono/mono/commit/9ea350ed3c963c77f0f4f77ec590cc3c84773424
2020-02-03 07:54:10 -08:00
Ryan Lucia 0f47ea5d80 [build] Check for Python 3 in configure.ac (#18625)
I fear this causing packaging problems, but as-is we require Python 3 for most of our Python files but don't actually check for that in configure. We should be consistent about this. Some projects (notably wasm) also use the Python version detected here, so bumping this will also bump the version used in those cases (a good thing, since there are some known bugs when using the scripts with Python 2).
2020-02-03 08:45:49 -05:00
monojenkinsandVlad Brezae 9ea350ed3c [interp] Fix enum constrained calls on netcore (#18659)
We were failing to dereference a managed pointer to the enum, due to incorrect checks, when doing a constrained call.

Fixes https://github.com/mono/mono/issues/18530

Co-authored-by: Vlad Brezae <[email protected]>
2020-02-03 03:08:15 -08:00
Phil Jaenke 20a641e4a0 Fix ppc64 build by fixing -mminimal-toc more correctly. (#18665)
The original fix back at #18554 did not work quite right, which was missed for lack of CI.
Reworked this check to instead test for `-mminimal-toc` on all powerpc hosts, and to only do the ELF version check on FreeBSD.

Fixes #18554 #18578 current CI build failures on AIX
2020-02-03 11:16:33 +01:00
Alex Thibodeau 050340ef1a Adjust mono_class_init_internal to no longer bypass cleanup when a failure is detected. (#18649)
This fixes: https://issuetracker.unity3d.com/issues/unity-physics-collisions-do-not-work-and-errors-are-thrown-when-entering-play-mode
2020-02-02 11:29:57 -05:00
imhameed 4bed68d073 [llvm] Enable jump threading with LLVM JIT. (#18655)
See https://github.com/mono/mono/issues/16243. This will allow elision
of redundant bounds checks. This pass is already used during AOT (as
part of opt -O2).
2020-02-02 01:18:31 -05:00
Fan Yang 3a93ebd6fe Abort cmov optimization, when one of the out blocks is try block (#18626) 2020-01-31 16:40:47 -05:00
mdh1418 9ea6b6a58a [wam] Rename other .stamp-build-debug-sample to debug_sample/dotnet.js (#18653) 2020-01-31 16:34:12 -05:00
Zoltan Varga 2a92593837 [wasm] Bump emscripten to 1.39.6. (#18641) 2020-01-31 13:18:11 -05:00
Zoltan Varga a475b84a3e [aot] Add support for making direct cross-assembly calls in llvm mode. (#18505)
* [aot] Handle '|' during symbol mangling.

* [aot] Clean up the construction of the linker command line.

* [aot] Pass -Wl,-install_name to the linker on osx, otherwise the shared object will be named based on the output file name i.e. <assembly>.dylib.tmp.

* [llvm] Pass the MonoAotFileInfo pointer to the init functions, not used yet.

* [jit] Add support for direct calls between methods in different AOT images in LLVM mode.

* Add a 'direct-extern-calls' AOT option.
* Initialize the PLT/GOT during method initialization,
  this normally happens in load_method () but that is
  not called when a method is directly called.
* Add self initialization support for JITted methods, they can be
  called directly since they are fallbacks if LLVM fails.

* Fix the build.

* Avoid calling BeginInvoke etc. directly, its implemented by the runtime.

* Unify some cases.

* Init PLT at image load time since directly called methods without initializers might make calls through it.

* Fix a startup problem.

* Make the mono_is_corlib_image () check for during startup.

* Check for is_inited in the init icall wrappers, since they might be called from JITted code which doesn't check it.

* Address review comments.
2020-01-31 11:41:49 -05:00
Zoltan Varga 4ef40a91ed [jit] Avoid generating wbarriers for vtype stores to the stack. (#18556)
The existing heuristic doesn't work with llvm.
2020-01-31 09:54:32 -05:00
Zoltan Varga a6707bfeb9 [wasm] Remove .stamp-build-debug-sample, use the output file instead. (#18644) 2020-01-31 01:21:21 -05:00
Zoltan Varga 745c8ff20b [wasm] Avoid overwriting emcc flags when SIMD is enabled. (#18643) 2020-01-31 01:13:17 -05:00
Johan Lorensson 1a2c433b7c Move generic_class_init out of line to assist branch prediction. (#18497)
When calling a generic class we need to make sure it has been inited.
This is done by calling generic_class_init. Since this call happens
frequently, but will only run once, it should be put out of line, assisting
branch predictor, preventing misspredictions (in all places but the first).

Currently this is not the case, since we will generate code like this:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: jne         000000000000048F
000000000000048A: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
000000000000048F: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]

Where we will always (except for first call) take the forward branch.

Instead we should mark the call to the method as out of line, at least
giving the code generator the information so it can move it out of line
and preventing a jmp, unless when it needs to init, the slow path. Doing
that will generate the following code:

0000000000000484: cmp         byte ptr [rcx+2Dh],0
0000000000000488: je          00000000000004C1
000000000000048A: mov         rax,qword ptr [mono_aot_EmptyGame_llvm_got+0C8h]
...
00000000000004C1: call        p_3_plt_EmptyGame__jit_icall_mono_generic_class_init_llvm
00000000000004C6: jmp         000000000000048A
2020-01-31 00:39:37 -05:00
monojenkinsandZoltan Varga 1317cf06da [aot] Fix some races if GOT entries are already set. (#18636)
decode_patch_info () would not decode patches if their corresponding GOT entry
was already set, causing half initialized MonoJumpInfo entries to be returned
to callers. Set the type of the MonoJumpInfo entry to MONO_PATCH_INFO_NONE
in these cases, and have the callers handle it.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 17:12:54 -05:00
monojenkinsandZoltan Varga 04c01efd49 [jit] Add a 'top-runtime-invoke-unhandled' option to treat exceptions which reach the top invoke frame as unhandled when embedding. (#18610)
This is not really a MONO_DEBUG option, but easier to implement it this way.

Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 15:59:30 -05:00
mdh1418 109a61227c [mono] Prevent direct MonoClass field access (#18634)
* [wasm] Add --enable-checked-build=private_types configure option to disable direct access of MonoClass fields.

* [mono] Indirectly access MonoClass fields
2020-01-30 15:57:55 -05:00
monojenkinsandZoltan Varga 4e90bb7e69 [wasm] Fix mono_vfree (). (#18632)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-30 13:28:51 -05:00
Steve Pfister e0fcb6093b [wasm][debugger] Cancel Existing Single Step Request When Creating New (#18608)
When stepping out of a c# breakpoint into the runtime (or JS... Not sure if that's accurate),
multiple single step requests can be created when the breakpoint is hit again.  In order to handle this scenario for wasm,
we just cancel the existing one.

NOTE: The debugger args addition seemed a better way to handle this than outright canceling in debugger-engine. If there
are implications to doing this, I can take it out.
2020-01-30 15:19:31 +01:00
Ryan Lucia e3624d356b [wasm] Finish renaming mono.js to dotnet.js (#18623)
This work mostly happened a while back, but this commit finishes the job.
2020-01-29 23:47:46 -05:00
Ryan Lucia cf5204021b [wasm] README updates (#18624)
Use -j with all example make invocations, mention Ninja dependency, mention raising ulimit, recommend py3 for running the server (when using py2, the port isn't properly released).
2020-01-29 23:47:28 -05:00
Larry Ewing b4cfcd5c5e Merge pull request #18602 from lewing/wasm-debug-inspect
[wasm][debugger] Initial support for proxying flat sessions
2020-01-29 17:05:43 -06:00
Alexander Köplinger 5e3e34220c [corlib] Split corlib xunit tests even more for iOS (#18619)
Follow-up to https://github.com/mono/mono/pull/18228 since two parts wasn't enough to fix the size issue on 32bits devices.
We now use three parts.

Fixes https://github.com/mono/mono/issues/18614
2020-01-29 21:28:06 +01:00
Aleksey Kliger (λgeek) 81f184c2ad [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP (#18567)
* [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP

Only prevent `gdb` or `lldb` from being invoked.

MERP is controlled by separate mechanisms --- configure flags for crash
reporting and an explicit opt-in icall.

The existing code collects crashed process data in-process, but it writes the
data out (using mono_merp_invoke) in the forked child process.  This is a bit
surprising, but in the interest of not disturbing the code too much, this PR
doesn't change that.  We should revisit `dump_native_stacktrace` again in the
future and rationalize these decisions.

Addresses https://github.com/mono/mono/issues/18565
2020-01-29 10:03:25 -05:00
Vlad Brezae 60b28b3b56 [interp] Fix virtual calls in mixed mode (#18526)
* [interp] Enable virtual calls to compiled code

If we have an aot-ed version of a method we should always go through it, because the IL for that method could be stripped.

* [interp] Disable jit call for wrappers

Some generic wrappers don't seem to work in mixed mode, probably due to the fact that the transition already reuses a gshared wrapper.
2020-01-29 00:01:58 -08:00
Larry Ewing c75a1364c5 [wasm][debugger] use PdbReaderProvider helper to try to load more pdb forms 2020-01-28 22:32:53 -06:00
Larry Ewing c6e9369de2 Merge remote-tracking branch 'upstream/master' into checkpoint 2020-01-28 22:17:07 -06:00
Larry Ewing edfe20ffc7 [wasm][debugger] start reworking logging logic 2020-01-28 18:53:36 -06:00
Larry Ewing 0fe46d8d35 [wasm] simplify EndOfMessage case 2020-01-28 18:53:36 -06:00
Larry Ewing a6ebd5cb38 Rename Ws* to something more reasonable 2020-01-28 18:53:27 -06:00
Egor Bogatov 883162e861 Use memmove in Buffer.MemoryCopy (#18547)
* Use memmove in Buffer.MemoryCopy

* add test case
2020-01-28 20:00:30 +01:00
Martin Baulig 7d2924adcb [wasm][debugger-proxy]: If we cannot load a PDB file, send a log entry. (#18601) 2020-01-28 12:31:50 -05:00
Nikita Voronchev e2b632e4a7 [WinForms] Incorrect Tree Nodes drawing at the OwnerDrawText mode (#18605)
Assume we have the following form `MyForm`:
```
using System;
using System.Drawing;
using System.Windows.Forms;

namespace test_TreeView
{
    public class MyForm : Form
    {
        TreeView treeView = new TreeView()

        public MyForm()
        {
            treeView.Nodes.Add("root node");
            treeView.Nodes[0].Nodes.Add("first child node");

            treeView.BackColor = Color.Gray;
            treeView.LineColor = Color.Yellow;

            treeView.DrawMode = TreeViewDrawMode.OwnerDrawText;
            treeView.DrawNode += treeView_DrawNode;

            Controls.Add(treeView);
        }

        private void treeView_DrawNode(object sender, DrawTreeNodeEventArgs e)
        {
            var brush = new SolidBrush(Color.FromArgb(100, 255, 0, 0));
            e.Graphics.FillRectangle(brush, e.Bounds);
        }
    }
}
```
The displayed result is
![image](https://user-images.githubusercontent.com/12382656/73263417-92872980-41e1-11ea-8f7c-7cecbb376b60.png)
One can see couple drawing issues:
* White rectangles under each `TreeNode`.
* The area of custom drawing (red rectangles) is shifted with respect to this white rectangles.

Current PR proposes the fix of this issues.
Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:38:27 +01:00
Karl 1bfd317362 [Winforms] Form fixes (#18427)
* Fix Return handling. Ensure correct ClientSize.

When pressing Return / Enter, ensure any target button is both visible
and enabled.

In CreateHandle(), ensure the ClientSize is correct if set. If not,
update it to make it right. Suspend layout while changing it so that
child controls with anchors other than Top Left are not moved.

* Select active control. PerformLayout after updating for client size.

Select the active control with SendControlFocus to avoid upsetting what
is currently selected.

* Improve Form window decoration adjustment handling

Reverts the incorrect attempted fix contained in previous commits.

Basically, when a property is changed that affects window decorations,
ensure the client size stays the same.

* Use UpdateFormStyles on FormBorderStyle too.
2020-01-28 16:32:51 +01:00
Karl 2fdafcf3a4 Don't select ListBox item if SelectionMode == None (#18492)
When the user types a letter or digit, KeySearch looks for and selects
an item starting with that character. However, it should not if
SelectionMode is None.
2020-01-28 16:31:18 +01:00
Nikita Voronchevandmonojenkins c191dba87d [WinForms] Make ContextMenuStrip behavior closer to the reference one (#18598)
* [WinForms] Make `ContextMenuStrip` behavior closer to the reference one

* Bump API snapshot submodule

Co-authored-by: monojenkins <[email protected]>
2020-01-28 16:30:46 +01:00
Alexander Köplinger 726a9134c8 Merge pull request #18491 from PreferLinux/RTF+Textbox-work
I've implemented a heap of missing RTF and RichTextBox stuff, and also made some improvements or fixes to various aspects of it. I'm sorry this is so big...

To give an overview of the changes, here's a list:
- Removed the RTF Charcode / Charset handling that wasn't actually used, had a lot of missing functionality and was basically reimplementing character encoding handling, and replaced it with ensuring the correct character encoding is being used (changing when needed).
- Improved RTF handling of objects with object results (process the given object result). Sometimes pictures can be like this.
- Improved line wrapping and alignment
- Added or greatly improved support for:
  - paragraph spacing
  - line spacing
  - paragraph indents (left, right, hanging)
  - tabstops (left, centre, right, decimal)
  - text background colour (like highlighting)
  - superscript / subscript
  - character offset (vertical)
  - text visibility
- Significantly improved RichTextBox RTF generation
- Cache DPI, rather than doing `CreateGraphics()` every time it is needed
- A variety of other fixes and improvements, with a few I can think of relating to:
  - caret position
  - replacing text
  - invalidation and drawing
  - when tags are added or deleted
2020-01-28 16:26:36 +01:00
monojenkins b87c271b6f [csproj] Update project files 2020-01-28 15:26:03 +00:00
monojenkins e4d69eb248 Bump API snapshot submodule 2020-01-28 15:24:53 +00:00
Steve Pfister 4b4b297450 [wasm][debugger] Handle When BreakpointRequest Commands Come Out of Order
BreakpointRequest commands can come out of order and before the runtime is ready (RuntimeReady). A null check for the DebugStore was added to guard against this since these requests seem to be for JS breakpoints. These seem to be able to be safely ignored.

Fixes https://github.com/mono/mono/issues/18584
2020-01-28 09:57:40 -05:00
Cody Russell a19aaa31da Bump bockbuild for Pango patch (#18566)
Fixes bug #1048838, correct SF font not loading in VSMac
2020-01-28 13:11:04 +01:00
Larry Ewing 4b620fa84d [wasm][debugger] Make the file loading slightly more async 2020-01-27 19:14:38 -06:00
Larry Ewing e1ec438c44 [wasm][debugger] Track the sessionId with all events and commands 2020-01-27 19:14:30 -06:00
Ryan Lucia a5fe7852de [utils] Default to g_print in mono_counters_dump if the stream is NULL (#18582) 2020-01-27 13:24:53 -05:00
Ryan Lucia f41680e395 [eglib] Use the stdout handler with g_printf (#18581)
This ensures it works properly on Android.
2020-01-27 11:57:11 -05:00
Nikita Voronchev ef05316107 [WinForms] Display caret while click on ComboBox (#18523)
If one click on a `ComboBox` the caret does not appear. This is so because `document.PositionCaret` call hides the caret.
2020-01-27 17:00:33 +01:00
Steffen Kieß 1fd0748362 Fix UnixEncoding.GetBytes() for empty strings (#18520)
Current versions of the mono return a null pointer as address of the first
element for empty arrays, i.e.

    fixed (byte* ptr = new byte[0]) Console.WriteLine (ptr == null);

will print "true". This causes a problem in the UnixEncoding class where it
triggers a check for null pointers. Fix this by allowing null pointers when
the number of elements is zero.
2020-01-27 17:00:16 +01:00
dotnet-maestro[bot] 66ff74a1f3 Update dependencies from https://github.com/dotnet/arcade build 20200126.3 (#18597)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20076.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20076.3
2020-01-27 16:53:25 +01:00
felixdoerre 6358345667 Fix open port for X11 applications (#18583)
Whenever mono opens an X11 window, the application opens a tcp socket for controlling the (non-tcp)-connection to X11. This is bad practice, as a tcp socket is not needed.

This pull request cleans the logic, by replacing the tcp socket with a fifo. That way this loopback connection is not visible from outside the .NET application, and all logic surrounding it should remain unchanged.
2020-01-27 14:32:15 +01:00
Karl ba3b6a169b [Winforms] X11 Fix bounds change from within bounds-related events (#18513)
The real change here is, in `XplatUIX11.SetWindowPos()`, simply moving the `SendMessage` call to the end instead of earlier. The problem with it earlier is that any change in bounds from any of the bounds-related events (OnLocationChanged / OnMove, OnSizeChanged / OnResize, OnLayout, OnClientSizeChanged) will be overridden. When `SendMessage` is called, `UpdateBounds` is run, and that raises those events as relevant. After those events finish, `SetWindowPos` continues with setting the size of the window to the value originally specified – which is now potentially wrong. So send the message later on.

There is also no point that I can see in writing the values to `hwnd` twice, because as far as I can see the only one that can be changed is `hwnd.ClientRect` (set by `PerformNCCalc`).
2020-01-27 13:41:00 +01:00
Adrian 23b8f10cb8 Update locale tables - add Tatar CLDR files tt.xml and tt_RU.xml (Fixes #18524) (#18525)
Fixes #18524

The `tt.xml` and `tt_RU.xml` files were added in CLDR 34: http://cldr.unicode.org/index/downloads/cldr-34

Upgrading to v34 (or the latest v36) introduced a larger number of new cultures.  I went with the smallest change possible to have Tatar added.
2020-01-27 11:46:23 +01:00
Phil Jaenke 58b3ac9ad0 Fix for -mminimal-toc detection, enable BTLS on FreeBSD PowerPC (#18578)
Fixes #18554 by making autoconf actually check if `-mminimal-toc` works instead of assuming on powerpc. Tested with FreeBSD 12.x and 13.x on powerpc64 ELFv2.
While we are in here, also enable BTLS which has passed testing.
2020-01-27 11:20:54 +01:00
Vincent Povirk 21e2ebcb06 Allow passing IntPtr values with UnmanagedType.AsAny. (#18544)
This is used by the SplashScreen class of WindowsBase in .NET Core to pass a NULL pointer.
2020-01-27 11:18:55 +01:00
Pranav K c8516b2848 Remove unused variables (#18579)
These show up as build errors in Blazor when they're copied as sources. Seemed trivial enough to fix.
2020-01-27 11:17:44 +01:00
abrevet-dev ed88c509dd [WinForms] Fix #18506 ActiveTracker, do not propagate message to control when it is not enabled and visible. (#18541)
Fixes #18506
2020-01-27 11:07:58 +01:00
Ankit Jain a6a89a5ef4 [master] Bump msbuild to track xplat-master (#18575) 2020-01-27 11:00:26 +01:00
Jérôme Laban d23da5628e [Wasm] Fix threading support file in packager (#18593) 2020-01-26 22:42:52 -05:00
monojenkinsandZoltan Varga c9a97e5258 [runtime] Disable lldb backtrace display on osx, it hangs on attaching in lldb. (#18590)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-26 14:31:07 -05:00
Thays GraziaandAleksey Kliger f8c34dc56b [debugger] Access invalid memory address using PointerValue Command. (#18537)
* Validate the address that came from IDE using PointerValue. The IDE can send an invalid address and it was crashing mono.

Fixes #18191
Fixes #15612

Co-authored-by: Aleksey Kliger (λgeek) <[email protected]>
2020-01-26 00:00:41 -03:00
Maher Jendoubi 4704c85c9d Fix typos (#18585)
* Contributing: fix typos

* fixed encoding and other typos

* fixed encoding and spotted other typos

* Update TransformationRules.cs
2020-01-25 19:55:22 +01:00
Zoltan Varga 9fc4cae875 [jit] Compute has_references correctly for gshared types whose constraint is a generic valuetype. (#18552)
Also emit write barriers correctly for these types.
2020-01-25 09:20:31 -05:00
Zoltan Varga 9e377af674 [wasm] Fix netcore build. (#18564)
* [wasm] Fix netcore build.

Add a DOTNET_REPO_DIR variable to Make.config which should point to the dotnet/runtime repo. Compile
the netcore runtime and corelib from this repo.

* Disable wasm+netcore on CI.
2020-01-25 06:29:58 -05:00
monojenkinsandZoltan Varga 653db97098 [runtime] Avoid loading ThreadAbortException class on netcore. (#18563)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 20:13:10 -05:00
Phil Jaenke 85b34350c6 Fix for failing compile-runtime-tests (#18569)
compile-runtime-tests is looking for python3, but $(env python3) will return error (no `/usr/local/bin/python3`); if this is the case, create a symlink from 3.6 or 3.7 to /usr/local/bin/python3 so it works. We already have this listed in `SHEBANG_FILES` for the port.
2020-01-24 18:16:44 -05:00
monojenkinsandZoltan Varga 0215d2e331 Initial CMAKE build system support. (#18561)
Co-authored-by: Zoltan Varga <[email protected]>
2020-01-24 17:00:49 -05:00
Radek Doulik 14570f14ac [aprofutil] Update the tool name 2020-01-24 10:04:02 +01:00
mdh1418 ae9335ac21 [mcs] Fix support for digit separators (#18539)
* [mcs] skip "_" when building number

* [mcs] Refactor digit separator feature report

* [mcs] Clean up digit separator code

* [mcs] Throw errors for invalid digit separator usage

* [mcs] Fix hex and bin usage of digit separator tests

* [mcs] Add digit separator usages that throw errors

* [mcs] Update il
2020-01-24 09:18:32 +01:00
Jay Krell 8d69965919 Remove "handles" from System.Threading.Semaphore. (#18512)
Actually, there were no handles.
This reduces MonoError.
2020-01-23 13:58:39 -05:00
monojenkinsandYoussef Victor 05da2da407 Fix typos (#18546)
Found more occurrences of the typos in dotnet/runtime#2061

The work done here is made by simple search & replace.

I'll point to replaces that needs attention in a review.

Co-authored-by: Youssef Victor <[email protected]>
2020-01-23 19:40:58 +01:00
Ankit Jain cc6e927d10 [bockbuild] preinstall: kill any running VBCSCompiler processes (#18488)
- roslyn's shared compiler server hashes it's pipename on the path to
the assembly.
- On macOS, this looks like `/Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/VBCSCompiler.exe`
- When installing a new mono, with that same `x.y.z` version, this
process will effectively be out of sync with the new updated
mono/msbuild/roslyn
- This manifests as:

```
  Using shared compilation with compiler from directory: /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn
  /Library/Frameworks/Mono.framework/Versions/6.8.0/lib/mono/msbuild/Current/bin/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : Roslyn compiler server reports different hash version than build task. [/Users/builder/azdo/_work/3/s/xamarin-android/build-tools/xaprepare/xaprepare/xaprepare.csproj]
```

- See https://github.com/xamarin/xamarin-android/pull/4108

- So, we kill any running roslyn shared compiler servers
2020-01-23 19:34:26 +01:00
Thays Grazia 6b475d5d00 [debugger] Native thread not executing managed code considered as terminated (#18501)
* If a thread was suspended and doesn't have any managed stack, it was considered as terminated, but it wasn't really terminated because it can execute managed code again, and stop in a breakpoint so if the execution arrives in debugger_agent_breakpoint_from_context we reset the flag terminated to FALSE.

* Adding terminated = FALSE in single_step_from_context.
2020-01-23 12:41:10 -03:00
Fan Yang 4d9832a399 Add disable_omit_fp support for LLVM (#18489)
Fixes #18417
2020-01-23 16:31:14 +01:00
Alexander Köplinger 0f1ac3e5bb Disable netcore mono builds that now live in dotnet/runtime (#18549) 2020-01-23 14:45:10 +01:00
Zoltan VargaandVlad Brezae 4586c4f091 [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype. (#18496)
* [jit] Compute the instance size/alignment correctly for gshared types whose constraint is a generic valuetype.

Fixes https://github.com/mono/mono/issues/18455.

* [jit] Simplify code

Fix some line damage

Co-authored-by: Vlad Brezae <[email protected]>
2020-01-23 07:57:36 -05:00
Bernhard Urban-Forster 082662cc8a [interp] context can be uninitialized for get_resume_state callback (#18533)
`jit_tls->interp_context` gets initialized lazily, that is, upon the first interpreter execution on a specific thread (e.g. via interp_runtime_invoke). However, with mixed mode the execution can purely happen in AOT code upon the first interaction with the managed debugger.

Stack trace:

```
  thread #1, name = 'tid_407', queue = 'com.apple.main-thread'
    frame #0: 0x0000000190aedc94 libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x0000000190a0f094 libsystem_pthread.dylib`_pthread_cond_wait$VARIANT$armv81 + 672
    frame #2: 0x000000010431318c reloadcontext.iOS`mono_os_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-os-mutex.h:219:8
    frame #3: 0x0000000104312a68 reloadcontext.iOS`mono_coop_cond_wait(cond=0x0000000104b9ba78, mutex=0x0000000104b9ba30) at mono-coop-mutex.h:91:2
    frame #4: 0x0000000104312858 reloadcontext.iOS`suspend_current at debugger-agent.c:3021:4
    frame #5: 0x000000010431be18 reloadcontext.iOS`process_event(event=EVENT_KIND_BREAKPOINT, arg=0x0000000145d09ae8, il_offset=0, ctx=0x0000000149015c20, events=0x0000000000000000, suspend_policy=2) at debugger-agent.c:4058:3
    frame #6: 0x0000000104310cf4 reloadcontext.iOS`process_breakpoint_events(_evts=0x000000028351a680, method=0x0000000145d09ae8, ctx=0x0000000149015c20, il_offset=0) at debugger-agent.c:4722:3
    frame #7: 0x000000010432f1c8 reloadcontext.iOS`mono_de_process_breakpoint(void_tls=0x0000000149014e00, from_signal=0) at debugger-engine.c:1141:2
    frame #8: 0x000000010430f238 reloadcontext.iOS`debugger_agent_breakpoint_from_context(ctx=0x000000016f656790) at debugger-agent.c:4938:2
    frame #9: 0x00000001011b73a4 reloadcontext.iOS`sdb_breakpoint_trampoline + 148
    frame #10: 0x00000001008511b4 reloadcontext.iOS`reloadcontext_iOS_Application_Main_string__(args=0x000000010703a030) at Main.cs:14
    frame #11: 0x00000001010f9730 reloadcontext.iOS`wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 272
    frame #12: 0x00000001042fd8b8 reloadcontext.iOS`mono_jit_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at mini-runtime.c:3162:3
    frame #13: 0x0000000104411950 reloadcontext.iOS`do_runtime_invoke(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, exc=0x0000000000000000, error=0x000000016f656ff8) at object.c:3052:11
    frame #14: 0x000000010440c4dc reloadcontext.iOS`mono_runtime_invoke_checked(method=0x0000000145d09ae8, obj=0x0000000000000000, params=0x000000016f656f20, error=0x000000016f656ff8) at object.c:3220:9
    frame #15: 0x0000000104415ae0 reloadcontext.iOS`do_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5184:3
    frame #16: 0x00000001044144ac reloadcontext.iOS`mono_runtime_exec_main_checked(method=0x0000000145d09ae8, args=0x000000010703a030, error=0x000000016f656ff8) at object.c:5281:9
    frame #17: 0x0000000104414500 reloadcontext.iOS`mono_runtime_run_main_checked(method=0x0000000145d09ae8, argc=1, argv=0x000000016f6570d0, error=0x000000016f656ff8) at object.c:4734:9
    frame #18: 0x00000001042d3b54 reloadcontext.iOS`mono_jit_exec_internal(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1320:13
    frame #19: 0x00000001042d39a4 reloadcontext.iOS`mono_jit_exec(domain=0x0000000145f00130, assembly=0x0000000281ba2900, argc=1, argv=0x000000016f6570d0) at driver.c:1265:7
    frame #20: 0x0000000104597994 reloadcontext.iOS`::xamarin_main(argc=5, argv=0x000000016f657a80, launch_mode=XamarinLaunchModeApp) at monotouch-main.m:483:8
    frame #21: 0x00000001008510dc reloadcontext.iOS`main(argc=5, argv=0x000000016f657a80) at main.m:104:11
    frame #22: 0x0000000190af8360 libdyld.dylib`start + 4
[...]
* thread #5, name = 'Debugger agent', stop reason = signal SIGABRT
  * frame #0: 0x0000000190aedec4 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x0000000190a0d724 libsystem_pthread.dylib`pthread_kill$VARIANT$armv81 + 216
    frame #2: 0x000000019095d844 libsystem_c.dylib`abort + 100
    frame #3: 0x00000001045871b4 reloadcontext.iOS`log_callback(log_domain=0x0000000000000000, log_level="error", message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", fatal=4, user_data=0x0000000000000000) at runtime.m:1213:3
    frame #4: 0x0000000104544fc8 reloadcontext.iOS`eglib_log_adapter(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, message="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n", user_data=0x0000000000000000) at mono-logger.c:405:2
    frame #5: 0x000000010456093c reloadcontext.iOS`monoeg_g_logstr(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, msg="* Assertion at ../../../../../mono/mini/interp/interp.c:7176, condition `context' not met\n") at goutput.c:134:2
    frame #6: 0x0000000104560598 reloadcontext.iOS`monoeg_g_logv_nofree(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, format="* Assertion at %s:%d, condition `%s' not met\n", args="e\x12z\x04\x01") at goutput.c:149:2
    frame #7: 0x000000010456061c reloadcontext.iOS`monoeg_assertion_message(format="* Assertion at %s:%d, condition `%s' not met\n") at goutput.c:184:22
    frame #8: 0x0000000104560674 reloadcontext.iOS`mono_assertion_message(file="../../../../../mono/mini/interp/interp.c", line=7176, condition="context") at goutput.c:203:2
    frame #9: 0x000000010459b570 reloadcontext.iOS`interp_get_resume_state(jit_tls=0x000000014900d000, has_resume_state=0x000000016fc7a9f4, interp_frame=0x000000016fc7a9e8, handler_ip=0x000000016fc7a9e0) at interp.c:7176:2
    frame #10: 0x0000000104319420 reloadcontext.iOS`compute_frame_info(thread=0x0000000104fe4130, tls=0x0000000149014e00, force_update=1) at debugger-agent.c:3422:3
    frame #11: 0x0000000104320d40 reloadcontext.iOS`thread_commands(command=1, p="", end="", buf=0x000000016fc7acf8) at debugger-agent.c:9048:3
    frame #12: 0x000000010431cca0 reloadcontext.iOS`debugger_thread(arg=0x0000000000000000) at debugger-agent.c:10132:10
    frame #13: 0x000000010447eb04 reloadcontext.iOS`start_wrapper_internal(start_info=0x0000000000000000, stack_ptr=0x000000016fc7b000) at threads.c:1232:3
    frame #14: 0x000000010447e788 reloadcontext.iOS`start_wrapper(data=0x000000028203ef40) at threads.c:1305:8
    frame #15: 0x0000000190a11d8c libsystem_pthread.dylib`_pthread_start + 15
[...]
```

Thanks to @drasticactions for helping me to reproduce.

Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1050615
2020-01-23 13:52:15 +01:00
dotnet-maestro[bot] 4321ba1e3b Update dependencies from https://github.com/dotnet/arcade build 20200118.1 (#18514)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20068.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20068.1
2020-01-23 10:34:41 +01:00
Alexander Köplinger a6f2903d58 configure.ac: remove AC_SEARCH_LIBS for libintl (#18531)
This causes an additional dependency of mono on libintl which we didn't have before.
Reported by a user on gitter, it's problematic on macOS since it makes the mono binary depend on the libintl we ship in the Mono .pkg.

It was introduced in Mono 6.6 by https://github.com/mono/mono/commit/18e0ebfe89be0a175d2f904b9bb1ec6816daa318
2020-01-23 00:29:56 +01:00
Alexander Köplinger b278d88a5f configure.ac: Don't look for X11 in netcore mode 2020-01-22 21:23:54 +01:00
Karl 3d168887a6 Store the current encoding code page, not encoding
The encoding itself could be null, while encoding_code_page
should always be valid.
2020-01-21 10:09:20 +13:00
Karl ff0605fe18 Use correct default encoding. 2020-01-21 10:04:57 +13:00
Karl d826bebd54 Calculate DPI before Recalculate. 2020-01-18 17:11:13 +13:00
Karl f39d652642 Fix NullReferenceException when parsing RTF.
charset_stack was never initialised.
2020-01-18 16:51:32 +13:00
Cody Russell c4aa327737 Bump bockbuild to get GTK NSPasteboardType change. (#18080) 2020-01-17 19:32:26 +01:00
Nikolay Sivov 0a96741f93 Compilation warnings fixes (#18495)
Compilation warnings fixes
2020-01-17 14:16:03 +01:00
Vlad Brezae 92bddddd7b [tests] Re-enable profiler test that used to fail (#18471)
It is not clear if it will fail again or not, but it is not reproducible.

Closes https://github.com/mono/mono/issues/8709
2020-01-17 13:10:59 +01:00
Bernhard Urban-Forster 73eeda130f [prof] remove wrong cast for format string (#18477)
[prof] remove wrong cast for format string

Fixes this warning:
```
  CC       mprof-report.o
mprof-report.c:408:62: warning: format specifies type 'char *' but the argument has type 'guint64' (aka 'unsigned long long') [-Wformat]
                        snprintf (format, sizeof (format), "%s : %%" PRIu64 "\n", (guint64)key_format);
                                                            ~~                    ^~~~~~~~~~~~~~~~~~~
                                                            %llu
/Applications/Xcode112.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:57:62: note: expanded from macro
      'snprintf'
  __builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), __VA_ARGS__)
                                                             ^~~~~~~~~~~
```

mistake in refactoring commit https://github.com/mono/mono/commit/07402062541fa9555bef99dbaac84cfb587926cd#diff-c825627ea9ddb3fa753fa3617eaec7c8R408
2020-01-17 11:05:50 +01:00
Aleksey Kliger (λgeek) 1f0e147cdd [netcore] Mark InitializeCurrentThread NoInlining (#18493)
Follow-up for https://github.com/mono/mono/pull/18480

We might want to replace calls to this with an intrinsic
2020-01-17 10:43:05 +01:00
monojenkins ed6a63bb25 [bcl] Update BCL Linked Size 2020-01-17 10:01:30 +01:00
Bernhard Urban-Forster b4c506c304 [runtime] fix warnings on linux (gcc 8.3.0) (#18479)
* [aot] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
aot-compiler.c: In function ‘emit_trampolines’:
aot-compiler.c:7675:28: warning: ‘_e’ directive writing 2 bytes into a region of size between 1 and 256 [-Wformat-overflow=]
    sprintf (end_symbol, "%s_e", symbol);
                            ^~
aot-compiler.c:7675:4: note: ‘sprintf’ output between 3 and 258 bytes into a destination of size 256
    sprintf (end_symbol, "%s_e", symbol);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At top level:
aot-compiler.c:753:1: warning: ‘link_shared_library’ defined but not used [-Wunused-function]
 link_shared_library (MonoAotCompile *acfg)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:558:1: warning: ‘mangle_symbol_alloc’ defined but not used [-Wunused-function]
 mangle_symbol_alloc (const char * symbol)
 ^~~~~~~~~~~~~~~~~~~
aot-compiler.c:535:1: warning: ‘mangle_symbol’ defined but not used [-Wunused-function]
 mangle_symbol (const char * symbol, char * mangled_symbol, gsize length)
 ^~~~~~~~~~~~~~~~~~~
```

* [amd64] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
tramp-amd64.c: In function ‘mono_arch_create_generic_trampoline’:
tramp-amd64.c:242:65: warning: variable ‘ex_offset’ set but not used [-Wunused-but-set-variable]
  int i, lmf_offset, offset, res_offset, arg_offset, rax_offset, ex_offset, tramp_offset, ctx_offset, saved_regs_offset;
                                                                 ^~~~~~~~~
```

* [lldb] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
lldb.c:243:1: warning: ‘buffer_add_buffer’ defined but not used [-Wunused-function]
 buffer_add_buffer (Buffer *buf, Buffer *data)
 ^~~~~~~~~~~~~~~~~
lldb.c:215:1: warning: ‘buffer_add_id’ defined but not used [-Wunused-function]
 buffer_add_id (Buffer *buf, int id)
 ^~~~~~~~~~~~~
lldb.c:208:1: warning: ‘buffer_add_long’ defined but not used [-Wunused-function]
 buffer_add_long (Buffer *buf, guint64 l)
 ^~~~~~~~~~~~~~~
lldb.c:188:1: warning: ‘buffer_add_short’ defined but not used [-Wunused-function]
 buffer_add_short (Buffer *buf, guint32 val)
 ^~~~~~~~~~~~~~~~

```

* [interp] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
interp/interp.c:3425:1: warning: ‘g_error_xsx’ defined but not used [-Wunused-function]
 g_error_xsx (const char *format, int x1, const char *s, int x2)
 ^~~~~~~~~~~
  CC       interp/libmono_ee_interp_la-mintops.lo
  CC       interp/libmono_ee_interp_la-transform.lo
  CCLD     libmono-ee-interp.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       libmono_dbg_la-debugger-agent.lo
  CC       libmono_dbg_la-debugger-state-machine.lo
  CC       libmono_dbg_la-debugger-engine.lo
  CCLD     libmono-dbg.la
ar: `u' modifier ignored since `D' is the default (see `U')
  CCLD     mono-sgen
  AR       libmain.a
ar: `u' modifier ignored since `D' is the default (see `U')
  CC       interp/libmaintest_a-whitebox.o
In file included from interp/whitebox.c:3:
interp/transform.h:163:12: warning: ‘stack_type’ defined but not used [-Wunused-variable]
 static int stack_type [] = {
            ^~~~~~~~~~
interp/transform.h:155:20: warning: ‘stack_type_string’ defined but not used [-Wunused-variable]
 static const char *stack_type_string [] = { "I4", "I8", "R4", "R8", "O ", "VT", "MP", "F " };
                    ^~~~~~~~~~~~~~~~~

```

* [thread state] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
  CC       libmonoutils_la-mono-threads-state-machine.lo
mono-threads-state-machine.c:33:1: warning: ‘get_thread_no_safepoints’ defined but not used [-Wunused-function]
 get_thread_no_safepoints (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
mono-threads-state-machine.c:26:1: warning: ‘get_thread_suspend_count’ defined but not used [-Wunused-function]
 get_thread_suspend_count (int thread_state)
 ^~~~~~~~~~~~~~~~~~~~~~~~
```

* [domain] fix warnings on linux (gcc 8.3.0)

Fixes these warnings:
```
appdomain.c:2415:1: warning: ‘get_app_context_base_directory’ defined but not used [-Wunused-function]
 get_app_context_base_directory (MonoError *error)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC       libmonoruntimesgen_la-assembly-load-context.lo
  CC       libmonoruntimesgen_la-domain.lo
domain.c:135:1: warning: ‘mono_domain_alcs_lock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_lock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~
domain.c:138:1: warning: ‘mono_domain_alcs_unlock’ declared ‘static’ but never defined [-Wunused-function]
 mono_domain_alcs_unlock (MonoDomain *domain);
 ^~~~~~~~~~~~~~~~~~~~~~~
domain.c:2129:1: warning: ‘mono_alc_free’ defined but not used [-Wunused-function]
 mono_alc_free (MonoAssemblyLoadContext *alc)
 ^~~~~~~~~~~~~
domain.c:2108:1: warning: ‘mono_domain_create_default_alc’ defined but not used [-Wunused-function]
 mono_domain_create_default_alc (MonoDomain *domain)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
domain.c:2084:1: warning: ‘create_alc’ defined but not used [-Wunused-function]
 create_alc (MonoDomain *domain, gboolean is_default)
 ^~~~~~~~~~

```

* [native library] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libmonoruntimesgen_la-native-library.lo
native-library.c:295:1: warning: ‘mono_loader_register_module_locking’ defined but not used [-Wunused-function]
 mono_loader_register_module_locking (const char *name, MonoDl *module)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* [sample] fix warnings on linux (gcc 8.3.0)

Fixes this warning:
```
  CC       libtestlib_la-test-mono-embed.lo
In file included from test-mono-embed.c:7:
../../samples/embed/teste.c:20:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 gimme () {
 ^~~~~

```

* fix for checked builds
2020-01-17 09:36:32 +01:00
Aleksey Kliger (λgeek) c1498b8d4c [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread (#18480)
* [runtime] NOHANDLES for ves_icall_System_Threading_Thread_GetCurrentThread

This function (and its call to mono_handle_new) was showing up in the
flamegraph for some async-heavy benchmarks
2020-01-16 21:57:54 -05:00
Aleksey Kliger (λgeek) 2ba25542e3 [interp] Add missing files to tarball (#18482) 2020-01-16 19:27:23 -05:00
Thays Grazia e2d8f5ee91 Revert "On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)" (#18490)
This reverts commit 61894d4a07.
2020-01-16 17:23:58 -05:00
Alexander Köplinger e4552f54fb Bump xsp to fix issues with removed Mono.Security APIs (#18478)
See https://github.com/mono/xsp/pull/88
2020-01-16 21:49:21 +01:00
Thays Grazia 61894d4a07 On android sometimes there is no valid_info but the ctx is valid and the debug can continue. (#18463)
Fixes #18106
2020-01-16 17:34:22 -03:00
Alexander Köplinger bafdd7865f Bump corefx to disable two tests. (#18464)
HashTableTests.Ctor_Int_Int_GenerateNewPrime and BitConverterTests.ToString_ByteArrayTooLong_Throws allocate a lot of memory and this causes issues on Windows i386 and other platforms.
2020-01-15 23:52:44 +01:00
Bernhard Urban-Forster bdde94b8e2 [arm] account for signed bit when calculating offset to PLT entry (#18454)
[arm] account for signed bit when calculating offset to PLT entry

See `bl` documentation (make sure you look at the thumb2 one):
<img width="1003" alt="Screenshot 2020-01-14 at 22 43 33" src="https://user-images.githubusercontent.com/75403/72385601-2bb53b00-3720-11ea-9858-d73bc2d02a7e.png">


Fixes https://github.com/mono/mono/issues/18247
2020-01-15 18:35:53 +01:00
Aleksey Kliger (λgeek) e7e81a47c9 [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects (#18457)
* [loader] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Match .NET Framework behavior.

Addresses https://github.com/mono/mono/issues/18388

* [tests] ReflectionOnlyLoad(byte[]) should not apply binding redirects

Regression test for https://github.com/mono/mono/issues/18388
2020-01-15 10:47:27 +01:00
Maher Jendoubi f4567e83dd Contributing: fixing some typos (#18453) 2020-01-14 22:30:06 +01:00
Jay Krell 6eea4a18f0 Reduce some stack use and cleanup. (#17809) 2020-01-14 22:17:00 +01:00
Vlad Brezae fef06df537 [utils] Make sure we include WRAP_ATOMIC_INTRINSIC if needed
Currently, this doesn't impact any targets, but do it for consistency.

Fixes https://github.com/mono/mono/issues/10651
2020-01-14 22:03:48 +01:00
Vlad Brezae 7fb011aa5a [utils] Include missing define of MONO_GNUC_VERSION
We were adding the WRAP_ATOMIC_INTRINSIC all the time on arm64, due to missing MONO_GNUC_VERSION.
2020-01-14 22:03:48 +01:00
Jay Krell 0740206254 Fix many printf warnings (#16484)
* Do not `printf` `long long` or `unsigned long long`.

*Do* printf `gint64` or `guint64`.

Do not printf %ll.
Do not printf %z.

 gint64 ==  int64_t, which could be long, or long long, or maybe other
guint64 == uint64_t, ditto

For printing gint64/guint64, use PRId64, PRIx64, etc.
For printing gsize/gssize, use `G_GSIZE_FORMAT` (seems like excess `G`).

%z definitely would warn for every single use, with Mingw.

And %ll might also, since on 64bit platforms, gint64 tends to be
long, not long long, and would merit %l not %ll.
(No matter that %ll is merited on 32bit, and the behavior
would then be correct on both.).

If there becomes a system w/o PRId64 etc., and it is a 64bit system,
then we could use %l under a portable name.
Or %I64. This appears unlikely (unless reach is extended to older Windows runtime).

Note that PRIxPTR for gsize/intptr_t may be a good idea too.
2020-01-14 22:03:22 +01:00
Ryan Lucia 3b19228a66 [netcore] Implement Thread.GetCurrentProcessorId (#18450)
CoreCLR has a managed cache surrounding this, but for the moment the naive implementation will suffice. We can later move their managed cache to shared on top of this PR with minimal trouble.

We can't guarantee the id will be [0..CpuCount) for a variety of reasons, and a followup PR to the dotnet docs will clarify this. I've chosen not to intentionally add an offset, but no one should be relying on that behavior, and if someone thinks adding an offset would drive the point home the perf cost is irrelevant.
2020-01-14 22:01:13 +01:00
Bernhard Urban-Forster 05f2651a12 [interp] initial infrastructure for whitebox testing of optimization passes (#18268) 2020-01-14 21:59:48 +01:00
Vlad Brezae d9db79b4b2 [sgen] Fixes for max-heap-size option (#18446)
* [sgen] Don't subtract twice the nursery size from max_heap_size

We were subtracting the nursery size when initializing max_heap_size as well as adding it to the allocated_heap when allocating the nursery (alloc_nursery).

* [sgen] Trigger collection if failing to allocate los section

If the memory governor says we are exceeding the max_heap_size by allocating a los section (1MB) for a new los object, attempt to clear some space by triggering a major collection, before completely bailing out.
2020-01-14 11:42:14 -08:00
Jay Krell 5237696f65 [cxx] Make the externC of icalls configuration-dependent instead of always. (#17573)
There are pluses and minuses here:

 + Minimize externC.

 + Maximimze typesafe linkage.

 - Minimize configuration-sensitivity.

 - Maximize commonality across all configurations, so that
   success in one configuration maximally but not 100% implies
   success in others (which is why to avoid `#if` in general,
   to contribute toward compilation success).

 + Use higher level macros to explain reason.
   Even if ICALL_EXTERN_C was always externC (presently not),
   using it instead of plain G_EXTERN_C offers a builtin comment.
2020-01-14 19:28:54 +01:00
Egor Bogatov 08bb5660f6 Optimize typeof(T).IsValueType (#18307)
* Optimize Type.IsValueType

* Ignore gsharedvt types

* use mini_is_gsharedvt_variable_klass

* test

* fix build
2020-01-14 09:37:16 -06:00
Alexander Köplinger 55393c2c9b Add normalized .bat files after the .gitattributes EOL change 2020-01-14 15:52:06 +01:00
Jay Krell c32414966c Remove comment about unexplained C++/LLVM problem. (#18447)
The explanations/fixes were:
 Compile consistently as C++:
   https://github.com/mono/mono/commit/893486638cae8efc46323d6c2209fc99febcd05d

 Make inconsistent compilation safe since it was fragile:
   https://github.com/mono/mono/commit/b2817e6404d573e837e1d8c7e047135974a3b311
2020-01-14 14:20:15 +01:00
Jonathan Chambers 480bef7217 Add mono_gc_thread_detach callback. (#18408)
Add mono_gc_thread_detach callback for Boehm

Use under Boehm to detach without threads lock as Boehm will take it's own lock internally.

Note in `on_gc_notification` we take threads lock after Boehm already has it's own lock. For consistency and to prevent deadlocks always take lock ordering of Boehm then threads lock.
2020-01-14 12:56:09 +01:00
Jay Krell dc866c2aad [cxx] Fixes for all lanes to build as C++ but w/o configure.ac change. (#17452)
* [cxx] dlmalloc remove extern "C" and remove NO_MALLINFO.

* [cxx] Move CallInst bitfields to the end to minimize padding for alignment
and fix an unexplained LLVM problem.
2020-01-14 12:32:08 +01:00
Egor Bogatov f5e1bd3cdf Merge pull request #18293 from EgorBo/vt-gettype-boxing
Optimize boxing for var.GetType()
2020-01-14 14:04:21 +03:00
Zoltan Varga 71a1a84c19 [wasm] Pass --used-attrs-only true to the linker. (#18439) 2020-01-14 09:42:10 +01:00
Alexander Köplinger 53ade46065 [netcore] Fix build warning 2020-01-13 21:27:23 +01:00
Aleksey Kliger (λgeek) 6abab691ec [netcore] Require use of MonoClass getters in Debug builds (#18267)
1. Update `build.sh` to use `--enable-checked-build=private_types` which forces the use of getter methods to access `MonoClass` fields.
2. Fix up some places that were accessing MonoClass directly.
3. Get rid of a spurious comma operator usage from an earlier refactoring:
   ```c
      z = a, b, c;
   ```
   is the same as
   ```c
      z = a; b; c;
   ```
   which is the same as just `z = a;` if `b` and `c` have no side-effects.
2020-01-13 21:15:09 +01:00
Alexander Köplinger a5cc4cd9ec Write version.h for netcore and allow preview VS for msvc builds 2020-01-13 20:47:35 +01:00
Karl 8bd438f318 [Winforms] Calculate GroupBox preferred size the same way as Panel (#18429)
GroupBox's GetPreferredSize() is not great. I've simply copied the code from Panel, and used that. It uses LayoutEngine.GetPreferredSize(), which does things a lot better than the code this used.
2020-01-13 17:15:09 +01:00
Karl 4ee4d368c6 [Winforms] Always draw at least one line of a Label. (#18422)
Previously, if the Label wasn't quite high enough nothing would be drawn as the line didn't quite fit. It should draw the line anyway, clipped if necessary.
2020-01-13 17:12:36 +01:00
Karl 42956f554b [Winforms] Improve X11 keyboard handling. (#18428)
- Ignore key events with vkey == 0. They won't match a real key code so don't try. Besides, trying can cause segfaults on ChromeOS Linux mode.
- Check special-case keys (numpad) before looping to get them. No point leaving them for afterwards...
2020-01-13 17:12:10 +01:00
Karl 7a8242ed36 [Winforms] Don't have non-active controls focused (#18426)
- Make the state of Control.CanSelect depend on whether the control is visible and enabled.
- When setting Control.Visible to false, if this control is focused, move the focus to the next control.
2020-01-13 17:11:38 +01:00
Karl d7e84ac6d1 [Winforms] Implement correct Control.Region / ClipRegion support on X11 (#18421)
Fixes https://xamarin.github.io/bugzilla-archives/20/20233/bug.html

Previously, on Linux, setting Control.Region just set the clipping for painting, but didn't influence the visibility of other controls behind the one it was set on. This implements this correctly via X11 Shape extension.
2020-01-13 17:10:57 +01:00
Maxim Lipnin 87cfaf15ae [netcore] Make System.Diagnostics.TraceSourceTests.DefaultTraceListenerClassTests.EntryAssemblyName_Null_NotIncludedInTrace Pass (#18403)
* [netcore] Extract Assembly.GetEntryAssembly method to shared part
2020-01-13 17:06:55 +01:00
Johan Lorensson 407898621e Make sure .bat and .cmd files have crlf in wd. (#18437)
Looks like https://github.com/mono/mono/commit/c9dc20f59ebe7fbed2a439d69c2d03a2d4500577 broke Windows build for some scenarios using .bat files. Fix makes sure we convert .bat and .cmd files to use crlf in wd.
2020-01-13 15:01:41 +01:00
dotnet-maestro[bot] 49c5101200 Update dependencies from https://github.com/dotnet/arcade build 20200112.1 (#18438)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20062.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20062.1
2020-01-13 15:00:54 +01:00
Egor Bogatov 423192ed85 [netcore][llvm] Implement Sse1-4.2 subsets used by corlib (#18103)
* Implement SSE41 subset used by corlib

* Implement Sse2.MoveMask

* only works with llvm

* Implement Sse3 and Ssse3 subsets used by corlib

* Implement a few SSE1 methods

* Address feedback, also, implement Sse.Add/Subtract

* Fix build

* Implement Sse.Multiply, Sse.Store

* Implement Sse.CompareNotEquals

* Implement Sse.MoveScalar

* Finish SSE1 corlib subset

* Implement Sse.LoadVector128, Sse.Shuffle

* Sse.Shuffle cleanup

* Implement Sse2 APIs

* More of SSE2: LoadAlignedVector128, Compare*

* Implement Sse2.Unpack* and Sse2.StoreScalar

* Implement Sse2.PackUnsignedSaturate

* Implement Sse2.ShiftRightLogical

* Implement Sse2.Shuffle

* Implement Vector128<T>.Zero

* Fix CreateScalarUnsafe

* Implement Vector128.As*, Fix Vector128.CreateScalarUnsafe

* Fix failures

* Fix failures

* Fix Sse.MoveMask

* remove redundant null checks

* Fix AOT failures

* fix compilation warrning

* rename create_vector_mask_*

* Fix failures found via tests

* Index in Sse41.Insert has to be a constant

* add local tests for mono

* Update tests (cleanup)

* Code cleanup

* test

* fix typo

* Clean up

* Clean up

* Implement And, AndNot, Or, Xor, Divide for Sse1

* Cleanup

* Fix build

* limit emit_vector128 with LLVM

* enable IsSupported for corlib

* Fix build on wasm (if-defs issue)

* Don't intrinsify Vector256

* Address feedback
2020-01-13 13:51:51 +01:00
Karl e92fe19fa6 [Winforms] X11Keyboard LookupString fixes (#18423)
Two small fixes to X11 keyboard handling here:
- For Xutf8LookupString, use the actual buffer size rather than the starting buffer size. Using the previous fixed size made the buffer overflow handing code immediately after the change pointless.
- For XLookupString, handle similar to Xutf8LookupString, getting a byte array. The old was sometimes causing problems where input on the numeric keypad didn't enter text. I believe using a StringBuilder similar to how it was _should_ work, but for some reason I could not get it to work reliably. So I changed it to this, which seems to work well for me. The only question I have is whether it should be Encoding.ASCII like I used, Encoding.Default, or another specific encoding.
2020-01-13 12:19:36 +01:00
Zoltan Varga 4b718ca1f2 [jit] Fix an assertion in the Thread.VolatileRead () intrinsic. (#18430)
Fixes https://github.com/mono/mono/issues/18418.
2020-01-13 12:09:09 +01:00
Vlad Brezae a13f5ccbb6 [interp] Fix struct copying during pinvoke transitions (#18402)
If we are in pinvoke wrapper, we need to use the marshalled native size, not the managed size.
2020-01-13 12:05:25 +01:00
Zoltan Varga 09a7317e5a [wasm] Disable a test which OOMs. 2020-01-13 10:00:49 +01:00
Karl ff494e6065 Only calculate scrollbars once handle created 2020-01-13 17:54:00 +13:00
Karl b8f747e452 Significantly improve DPI caching.
Particularly, don't call CreateGraphics() in TextBoxBase's constructor.
2020-01-13 17:54:00 +13:00
Karl e9abb44242 Correct code style problems. 2020-01-13 17:54:00 +13:00
Karl c41ee8050e Improve font name and charset handling.
- Use alternative names as a fallback if present.

- Use the untagged name (\fname, name that doesn't
identify charset ) if present.

- Use font codepage or charset correctly to get the correct encoding.

Rip out the old nonfunctional RTF charset handling.
This was not used, and furthermore was fundamentally flawed.
I have replaced it by simply using the correct text Encoding, mentioned
above.
2020-01-13 17:54:00 +13:00
Karl 9b87a23ee6 Replace Document Height / Width Changed events with SizeChanged 2020-01-13 17:54:00 +13:00
Karl 150d5e209d Round font ascent / descent up in LineTag. 2020-01-13 17:54:00 +13:00
Karl 24c2e505a3 Fix updating to cover lines modified while suspended. 2020-01-13 17:54:00 +13:00
Karl e586084ae8 Rearrage slightly, fix kerning. 2020-01-13 17:54:00 +13:00
Karl 3d9e4fa7f4 Fix CharCount, CharIndexToLineTag, recalculate wrapped lines.
CharCount / length was inconsistent about counting line endings. Fixed
to include them all, as that was much easier than the other way.

When shortening a wrapped line, it is necessary to start the recalculate
at the previous line, in case the wrap point has changed.

In CharIndexToLineTag, if there is a line boundary at the specified
index, return the line starting at the index.

Fixed a few other things too.
2020-01-13 17:54:00 +13:00
Karl 11a89a53ab Recalculate after changing canvas. Fix scrolling. 2020-01-13 17:54:00 +13:00
Karl 0b6dd983dd Later RTF para overrides earlier. Emit \nosupersub.
RTF paragraph controls override earlier instructions, rather than
keeping the maximum.
Emit \nosupersub between superscript and subscript text. Missing it out
technically gives a superscript (or subscript) of the subscript (or
superscript), which is not what we want.
rather than keeping the maximum.
2020-01-13 17:54:00 +13:00
Karl df3b5a26b4 Improve indent and kerning handling. 2020-01-13 17:53:59 +13:00
Karl ca7352ad24 Add tabstop support. Use floats for spacings.
Floats are now used to preserve paragraph spacing and indents. The RTF
format uses twips (1/20") for these, which is usually less than 1px.
2020-01-13 17:53:33 +13:00
Karl b35c560df3 Lots of TextControl stuff, plus others.
* RTF.cs: Add support for JPEG and EMF pictures.
  Allow for attribute etc. groups inside the picture group.

* Line.cs: In InsertString, ensure that the tag passed in is currently
  in the line – not necessarily valid when the passed tag is the
  cursor, for instance.

* LineTag.cs: Only break tags for formatting when needed.

* TextControl.cs: Fix invalidating text that isn't left-aligned.
Fix caret stuff:
  Positioning after the line ending no longer happens
  Up and down when not left-aligned
  Tag used when moving left to start of line
  Page up / down
  Remove empty tags when navigating away from them
Change tag background draw location – I think it is a correction!
When inserting at a position without being given the tag, choose an
  empty tag at that position if one exists.
Change ReplaceSelection order back (corrected some formatting and
  similar stuff), and format the new text to match the tag.
Change IncrementLines to transverse the tree (using a new function)
  rather than repeatedly call GetLine.

* RichTextBox.cs: Don't streamline the line in case it removes tags we
  want to keep – we'll skip them ourselves instead.
Emit pictures.

* TextBoxBase.cs: Remove redundant check – a previous check means
  value != acutal_border_style, therefore at least one of them !=
  BorderStyle.Fixed3D, therefore the if condition was always true.
2020-01-13 16:42:44 +13:00
Karl 681b514e88 Use a float for the RTF font size to avoid rounding error when dividing by two to get the point size. 2020-01-13 16:42:44 +13:00
Karl dbf98ffc93 Add line spacing support. Move dpi to TextControl.
Line spacing seems to be working perfectly.
2020-01-13 16:42:44 +13:00
Karl f25ca77bfe Fix ReplaceSelection() again. Line spacing support. Fix drawing non-multiline selection.
It was inserting at the start, and then removing what had been inserted +/- whatever length as needed.
2020-01-13 16:42:44 +13:00
Karl e01e43025d Fix TextControl.ReplaceSelection to insert before removing so that setting SelectedText doesn't lose formatting. 2020-01-13 16:42:44 +13:00
Karl 1821cc8adf RichTextBox: Cache dpi
The dpi isn't likely to change during the lifetime of the object, so is relatively safe to cache.
Creating a Graphics object each time the dpi is wanted when reading RTF is also rather silly performance-wise.
Even worse, when generating RTF it can cause a recalc, which can mean that some lines <= end_line don't exist.
2020-01-13 16:42:44 +13:00
Karl be446f04f3 Three fixes in RichTextBox, one GenerateRTF, two ContentResized.
In GenerateRTF, check tag != null before trying to generate anything based on it. Not sure
why it hasn't been a problem before, but either way I was getting NullReferenceExceptions
because of it today.

Run ContentSizeChanged() when either width or height changes, irrespective of Multiline status.
This fixes various problems with stuff jumping around vertically.

Correct ContentSizeChanged to consider the left and right margins instead of the left margin twice.
Was probably a typo at an earlier stage, otherwise would have been a * 2.
2020-01-13 16:42:43 +13:00
Karl 42cecb6ed2 Fix line calculation for vertical positioning of tags.
* Line.cs: Fix handling of differing tag Ascents and CharOffsets in recalculateLine.

* TextControl.cs: Remove unneeded assignments.
2020-01-13 16:42:43 +13:00
Karl fee6509476 * Line.cs: Finished off indent support, also added tabstops.
* LineTag.cs: Added support for visibility. Corrected handling of tabs
  and added tabstop support.

* RichTextBox.cs: Added support for text visibility, indents,
  tabstops.

* TextControl.cs: Make invisible text work correctly. Fixed a bug with
  tag backcolor. Added support for indents.
2020-01-13 16:42:35 +13:00
Karl f98c1e0087 Fixed so that it sets measuredText to an empty rectange if not
measuring text.
2020-01-13 16:09:02 +13:00
Karl e4a5b4a720 * RichTextBox.cs: Align Justified as Left instead of as Center.
* TextControl.cs: Provide an output on PictureTag.Draw(... out
  Rectangle measuredText ...) if requested.
2020-01-13 16:08:29 +13:00
Karl cc91b61cea * RTF.cs: More work on ReadObjGroup
* RichTextBox.cs: Fixed up the insertion of pictures.
2020-01-13 16:06:41 +13:00
Karl aab3dca9fc All sorts of work, mostly on RichTextBox. 2020-01-13 16:03:11 +13:00
Zoltan Varga 73468eda91 Bump linker. (#18424) 2020-01-12 10:57:28 +01:00
Ankit Jain fc7b254953 [master] Bump msbuild to track xplat-master (#18432) 2020-01-12 10:56:21 +01:00
monojenkins ee3da3821b [bcl] Update BCL Linked Size 2020-01-11 08:58:48 +01:00
Martin Baulig a702e4ff02 [Sockets]: Always reset internal SAEA completion when reattempting connection in MultipleConnectAsync. (#18389)
This should fix #18030.
2020-01-10 19:01:36 +01:00
Zoltan Varga 3563bd90dc Revert "[wasm] Pass --used-attrs-only true to the linker. (#18401)" (#18415)
This reverts commit 6f4fe05209.
2020-01-10 16:49:43 +01:00
Ankit Jain 9666bcfae7 [master] Bump msbuild to track xplat-master (#18412) 2020-01-10 12:45:01 +01:00
Vlad Brezae d33a100961 [interp] Add gc unsafe transition during pinvokes (#18399)
Native methods can be run in gc safe mode, meaning we can be in gc safe mode while setting/getting the arguments for a native transition. Call context computation requires us to be in unsafe mode.

Fixes https://github.com/mono/mono/issues/16969
2020-01-10 09:32:04 +02:00
Zoltan Varga 5a65a97c58 [corlib] Mark the SRE types with required=false so they can be linked out. (#18379)
Currently SRE types are still kept by the reference in RuntimeType::MakeTypeBuilderInstantiation ().
2020-01-10 04:22:45 +01:00
Larry Ewing 439091eddf [wasm] use updated node to run jsvu (#18386)
* [wasm] use updated node to run jsvu

* [wasm] make the browser tests use the node we've installed

* [wasm] Fix browser tests to reference dotnet.js
2020-01-09 22:42:03 +01:00
Zoltan Varga 6f4fe05209 [wasm] Pass --used-attrs-only true to the linker. (#18401) 2020-01-09 19:12:16 +01:00
Alexander Köplinger c9dc20f59e Fix line endings in .bat files
They need to be CRLF otherwise we'll get conflicts when moving into dotnet/runtime which has a .gitattributes setting forcing .bat to CRLF.
2020-01-09 18:06:53 +01:00
Ankit Jain 46f834fda7 [master] Bump msbuild to track xplat-master (#18391) 2020-01-09 17:45:05 +01:00
Maxim Lipnin f5aca4235e [netcore][xunit tests] Re-enable System.Diagnostics.Tests.ProcessTests.ProcessStart_UseShellExecute_ExecuteOrder (#18397)
#14903
2020-01-09 17:01:57 +01:00
monojenkins e55a1a4f62 [bcl] Update BCL Linked Size 2020-01-09 14:21:06 +01:00
Vlad Brezae 345550fb99 [interp] Throw overflow exception when converting NaN (#18384)
Fixes https://github.com/mono/mono/issues/18061
2020-01-09 12:08:15 +02:00
Bernhard Urban-Forster 207cf805e5 [memory barrier] use pagesize determined by helper (#18383)
[memory barrier] use pagesize determined by helper

`limits.h` doesn't always define `PAGESIZE` properly (e.g. iOS), so it would fall back to `0x1000` (4096) but it's really `0x4000` (16384) on iOS. We already have a helper in the runtime to determine the pagesize, so let's use that instead.


Fixes https://github.com/mono/mono/issues/18273
2020-01-09 10:46:03 +01:00
Marek Safarandmonojenkins 9cc751d618 [corlib] Switch to CharUnicodeInfo from corefx (#18305)
* [corlib] Switch to CharUnicodeInfo from corefx

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>
2020-01-09 09:36:55 +01:00
Zoltan Varga 147c7972a8 [wasm] Reenable linker substitutions. (#18381) 2020-01-09 05:18:58 +01:00
Zoltan Varga 46fe5ad3f6 Bump linker. (#18380) 2020-01-09 05:18:17 +01:00
Martin Baulig ba7c00e4a9 Add workaround for #17710. (#18275) 2020-01-08 17:50:55 -05:00
Ryan Lucia 9fd95db929 [reflection] Fix check for Type.FullName to match CoreCLR (#18385)
Updated test to follow in dotnet/runtime
2020-01-08 17:02:25 -05:00
Ryan Lucia 1178ab3a1a [metadata] Propagate error in mono_runtime_try_invoke_array (#18382)
This was prompted by https://github.com/mono/mono/issues/18339, which seems to be failing to locate a native library and then hitting an assertion. We should propagate the error instead, so that if others run into something similar they can see what's actually going wrong instead of just getting a runtime crash.
2020-01-08 15:57:16 -05:00
Egor Bogatov ee5850a2b5 Update FieldInfoTest.cs 2020-01-08 23:21:41 +03:00
dotnet-maestro[bot] 5dbc923cb3 [master] Update dependencies from dotnet/arcade (#18206)
* Update dependencies from https://github.com/dotnet/arcade build 20191215.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19615.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19615.1

* Update dependencies from https://github.com/dotnet/arcade build 20191222.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19622.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19622.1

* Update dependencies from https://github.com/dotnet/arcade build 20191229.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19629.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19629.1

* Update dependencies from https://github.com/dotnet/arcade build 20200105.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20055.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20055.1
2020-01-08 20:47:44 +01:00
Jay Krell e8d38961ee [Coop] Remove handles from Mono.Interop.ComInteropProxy. (#16993) 2020-01-08 13:34:11 -05:00
Alexander Köplinger ba505510f2 FreeBSD: fix provisioning script to force reinstall perl5 at the end
It didn't work otherwise.
2020-01-08 16:46:58 +01:00
Zoltan Varga 34dee0ea4e [runtime] Initialize field info before accessing it in ves_icall_System_RuntimeFieldHandle_SetValueDirect (). (#18378)
Fixes https://github.com/mono/mono/issues/18364.
2020-01-08 16:39:39 +01:00
Ankit Jain d31f6ac4da [master] Bump msbuild to track xplat-master (#18376) 2020-01-08 15:01:03 +01:00
Zoltan Varga e999846088 [jit] Implement fast TLS access in gshared methods. (#18377)
Fixes https://github.com/mono/mono/issues/18370.
2020-01-08 13:32:52 +01:00
Zoltan Varga d0a3fb1b1b [wasm] Rename binaries mono -> dotnet. (#18373) 2020-01-08 03:04:55 +01:00
Ankit Jain aa87d3db6d [master] Bump msbuild to track xplat-master (#18372) 2020-01-07 20:56:41 +01:00
Bernhard Urban-Forster 6481e8f7a3 [interp] run regression with all combinations of available optimizations (#17644)
[interp] run regression with all combinations of available optimizations

Like we do for the JIT.
2020-01-07 19:35:54 +01:00
Zoltan Varga 3d4cd57a52 [wasm] Pass --explicit-reflection to the linker only in aot mode. (#18367) 2020-01-07 18:50:22 +01:00
Phil Jaenke f3f11d1081 FreeBSD: Fix for perl5.26 / perl5-5.30 conflict (#18368)
The base image appears to have perl5.26 which conflicts with perl5-5.30, resulting in the /usr/local/bin/perl binary not being placed by perl5-5.30, which results in failed builds. The `perl5.26` appears to be part of the base image and has no packages actually dependent on it.
2020-01-07 18:14:55 +01:00
Calvin Buckley 524a8298d0 Force Python 3.x from env in shebang lines (#18285)
* Fix byte/string incompatiblity in Python 3
2020-01-07 17:28:06 +01:00
Zoltan Varga ec7f4c68f9 [netcore] Remap String ctors to the String:Ctor () methods. (#18357) 2020-01-07 17:12:34 +01:00
Ankit Jain 71070d34c8 [master] Bump msbuild to track xplat-master (#18297)
This enables parallel build support with msbuild!
2020-01-07 16:21:51 +01:00
Calvin Buckley b3ce5bfa88 More disambiguation of Python in makefiles (#18284) 2020-01-07 15:44:03 +01:00
Vlad Brezae 376c46bcf3 [sgen] Fix xref computation with tarjan bridge (#18239)
* [sgen] Include also derived classes as bridges

When using `MONO_GC_DEBUG=bridge=` debug option

* [sgen] Fix xref computation with tarjan bridge

Between C# and java (on android) there are objects that live on both worlds. This means that there exists a C# object with a corresponding java object. The relationship between them is strong, meaning if C# object is alive then java object must stay alive, and vice-versa. We keep java bridge objects always alive through a GCHandle (on the java gc). When doing a C# collection we select all bridge objects that appear to be dead on the C# side. These objects are candidates for collection, assuming the java side has nothing against it. Before triggering a collection on the java side (following the mono gc) we switch all the strong gchandles to the java objects to be weak (for these objects that are candidate to be collected) and we recreate the reference graph from the C# side to the java side (if C# Bridge1 can reference C# Bridge2 then, on the java side, we add a reference from Java Bridge1 to Java Bridge2; this is done by adding to an array of references inside Java Bridge1). In order to minimize the amount of work that needs to be done on the java side, we compute the minimal amount of references that need to be added, by computing the strongly connected components of the object graph.

An optimized way to construct the SCCs and the xrefs is by using the tarjan algorithm (https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm). Our algorithm is non recursive (scan_stack emulates the recursive order of traversal in a dfs algorithm, while loop_stack is the stack used by the algorithm). A color represents an SCC. We have some optimizations in place where we might merge colors if they don't contain bridges, since the client only cares about SCCs containing bridge objects and the links between them. color_merge_array is used to keep track of all neighbors of a node until we are creating the scc for that node. It is populated when scanning all the refs inside an object (compute_low). All the colors in the color_merge_array will be cross references with that scc.

Before this commit we were only clearing the color_merge_array when creating an SCC. This is problematic because we could end up with xrefs inside of an SCC that belong to another SCC. Consider the simple graph of nodes 0,1,2,3 where 0 <=> 2, 0->1, 2->3. Assume we start scanning with node 0. When creating the SCC for node 3 the followed path is 0 -> 2 -> 3, while the loop stack will contain (0,1,2,3). After creating SCC for node 3, we will finish scanning node 2 which would detect the xref to Bridge3, which would have been added to the color_merge_array. Because node 2 is not the root of the SCC it belongs to (its lowlink points towards node 0 which has a lower index), we are not creating an SCC with it, and the link to node 3 remains in color_merge_array. Because the next node from the scan_stack is node 1, which is also the root of the SCC that it belongs to, we will create an SCC for it and wrongly add the node 3 reference from color_merge_array to it. In order to fix this issue, we will always clear the color_merge_array once we finished scanning the xrefs for a node. If the node in question is not the root of an scc, then we will remember them as xrefs pointing out from this object. When we finally reach node 0 (which will be the root of the SCC containing nodes 0 and 2), we will then know that all xrefs for this color are the union of the xrefs of all objects belonging to this color (which represents the objects that we are popping from the loop_stack until we encounter the root node).

Even though this change adds required bookeeping for xrefs, I didn't notice any change in performance on the bridge tests that we have in mono/tests.

* [sgen] Some logging improvements in tarjan bridge

* [sgen] Disable optimization when comparing bridge outputs

When this optimization is enabled, the tarjan bridge will create more SCCs in order to reduce amount of xrefs in the graph. This would render the `bridge-compare-to` debug flag unusable with tarjan bridge.
2020-01-07 16:31:49 +02:00
Jay Krell 2129ac6c28 Set thread pool thread name just based on string pointer check. (#16637)
* Set thread pool thread name just based on string pointer check.

When the generation-based solution went in, there were not
yet constant thread names, so a pointer based approach could fail
due to reuse. We can do slightly simpler now.

* Remove the generation field which is no longer needed.
Remove the duplicated knowledge of how to set a constant and put
it behind a funny sounding but perhaps reasonable flag.
Revise corlib version.
2020-01-07 09:01:22 -05:00
EgorBo 322ba565b6 disable a test 2020-01-07 17:00:23 +03:00
Steve Pfisterandmonojenkins 0ccf503f0a Add SignedCms to the TypeForwards in the System.Security.Cryptography.Pkcs facade (#18325)
* Add SignedCms to the typeforwards in the System.Security.Cryptography.Pkcs facade

* Added additional missing types from System.Security.Cryptography.Pkcs.

* It would be nice if the files I added had namespaces ;-)

* Bump API snapshot submodule

* [csproj] Update project files

Co-authored-by: monojenkins <[email protected]>

Fixes https://github.com/mono/mono/issues/18323
2020-01-07 14:03:15 +01:00
EgorBo e6c0cdae0c Address Zoltan's feedback (enable for AOT) 2020-01-07 15:18:51 +03:00
EgorBo 1c5a295082 Merge branch 'master' of github.com:mono/mono into vt-gettype-boxing 2020-01-07 14:50:16 +03:00
Marek SafarandBernhard Urban-Forster adcd62248d [System.Private.CoreLib] Cleanup intrinsic tracking (#17884)
* [System.Private.CoreLib] Cleanup intrinsic tracking

1. Removed icalls where intrinsic version for the method exists

2. Add [Intrinsic] attribute to methods which have runtime intrinsic
   to make them easier to track and to tell linker not to analyze them

3. Use recursive syntax for any intrinsic which can be called via
   reflection to correctly apply the intrinsic when invoked.

```c#
public static bool IsSupported { get => IsSupported; }
```

When one calls X86.IsSupported, the call gets replaced by an
intrinsics. When one calls it with an Invoke, the body of IsSupported
gets replaced by intrinsics. It's just a hack to get Invoke support for free.

* [interp] implement System.Type::op_Equality

* Undo changes which are not yet supported by interpreter

* [interp] Implement System.Runtime.CompilerServices.RuntimeHelpers::OffsetToStringData

Co-authored-by: Bernhard Urban-Forster <[email protected]>
2020-01-07 11:17:06 +01:00
Zoltan Varga 487dacce79 [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods. (#18334)
* [jit] Avoid passing a vtable argument to DIM methods when making calls out of gsharedvt methods.

Fixes https://github.com/mono/mono/issues/18276.

* Fix compilation with mcs.
2020-01-07 02:57:10 +01:00
Jay Krell 6a8ca2252b Do not depend on gboolean to have multiple bits. (#17825) 2020-01-06 16:11:21 -05:00
Zoltan Varga 384db9e890 [netcore] Optimize RuntimeType::IsSZArray. (#18347) 2020-01-06 16:36:59 +01:00
Zoltan Varga 7236dadd32 [netcore] Mark SRE types with required=false so they are only retained when they are actually used. 2020-01-06 15:44:50 +01:00
Jo Shields 4c9e28b80a Don't run make install in boehm submodule (#17907)
Don't run `make install` in boehm submodule
2020-01-06 09:20:46 -05:00
Zoltan Varga 090e25ec2d [wasm] Rename the interp-sample targets to interp-hello. (#18342) 2020-01-06 07:40:40 +01:00
Zoltan Varga 47424e34ae Bump emscripten to 1.39.5. 2020-01-05 21:45:10 -06:00
monojenkins 59a1eade7c [csproj] Update project files 2020-01-05 13:43:39 +01:00
monojenkins 36e89d52d1 Bump API snapshot submodule 2020-01-05 13:43:39 +01:00
Zoltan Varga 3acab49ce7 Bump linker+cecil. 2020-01-05 13:43:39 +01:00
Aleksey Kliger (λgeek) 07fee8a91d [metadata] Size 0 Blob heap is ok when resolving assembly refs (#18313)
* [metadata] Size 0 Blob heap is ok when resolving assembly refs

Sometimes ILasm can produce images with a Blob heap of size 0.  In cases where
we're loading assembly references from such an image, the hash (which is
optional) can be at index = 0 and the Blob heap size is also 0.

Also: add the hash value to the output of `monodis --assemblyref`

* [metadata] Add a separate assertion for the index == size == 0 case

And a comment explaining how it is likely to be triggered.  If the assembly is
reasonable the caller of mono_metadata_blob_heap should be updated to use
mono_metadata_blob_heap_null_ok instead

* [bcl] Allow Mono's ILASM to produce a size 0 Blob heap

ECMA 335 II.24.2.4 says that the user string and blob heaps should have an
entry at index 0 consisting of the single byte 0.  However .NET Framework (and
.NET Core) ilasm will entirely omit the Blob heap (that is, create a blob heap
entry of size 0) if it is not needed.  This PR changes Mono's ILASM to emit the
initial byte on demand only if one of the MetaDataStream.Add() methods is
called. Otherwise we will also emit a stream of size 0.

This is needed to compile some test cases.

* [tests] Add regression test for loading assemblies with size 0 Blob heap

Depends on ILASM that can emit a size 0 Blob heap
2020-01-03 10:42:44 -05:00
Zoltan Varga 22afdb80e1 [netcore] Add some linker exclusions. 2020-01-03 13:11:39 +01:00
monojenkins 95101b19fc [bcl] Update BCL Linked Size 2020-01-03 09:22:27 +01:00
Larry Ewing 14a3add47f [wasm][debugger] put the testharness base url in one place 2020-01-02 19:28:18 -06:00
Larry Ewing 11bf192d1a [wasm][deubgger] clean up task usage and remove delay 2020-01-02 19:28:18 -06:00
Marek Safar c9036e7555 Cleanup a few dependencies to reduce wasm bcl size 2020-01-02 14:24:44 -06:00
Ryan Lucia 33ca3d33e1 [loader] Add assembly name matching on netcore to fix DefaultContext test (#18272)
* [loader] Rename assembly name check strictness flag

* [loader] Add predicate for assembly name matching on netcore

* [loader] Enable LoadInDefaultContext test

* Make mono_assembly_names_equal external, change flags in domain search

* Move ifdef inside mono_loader_get_strict_assembly_name_check

* Fix pedump

* No reason to make this external

* Feedback
2020-01-02 15:21:45 -05:00
Egor Bogatov 458d80002d Add !align 8 to all managed allocators (calls) (#18283) 2020-01-02 20:45:40 +01:00
Jérôme Laban 52674e238a [Wasm] Use Path.Combine in Packager (#18332)
Improves Windows compatibility
2020-01-02 20:38:41 +01:00
Ryan Lucia ff6294d231 [reflection] Convert internal GetCustomAttributes calls to Attribute[] (#18176)
* [reflection] Avoid creating object[] in GetCustomAttributesBase

* [reflection] Migrate Attribute type checking to CustomAttribute.cs
2020-01-02 11:58:15 -05:00
Ryan Lucia 675f1eb321 [eglib] Fix whitespace in memory.c (#18320) 2020-01-02 11:56:50 -05:00
Thays Grazia e20956305e The behaviour expected of filter exceptions is: (#18309)
-> throw in a async task with a try catch in the caller -> don't stop
-> throw in a new thread with a try catch in the caller -> stop
-> throw in the main thread without any try catch -> stop

Including a new test to test the fix of #17601.

Fixing #16588, create a while 1 to stops the program execution when an exception is thrown and no try catch is found and MONO_DEBUG=suspend-on-unhandled.

Fixes #17601
Fixes #16588
2020-01-02 11:26:14 -03:00
Jonathan Nagy 5470386775 Set inited flag _after_ TermInfoDriver initialisation completes (#18318)
Fixes #18317
2020-01-01 19:02:48 -05:00
Steve Pfister 6f24832646 Pull in DirectoryInfo Invalidate After Create Fix (#18310)
* Bump to mono/corefx@bff1d77
2019-12-30 21:05:14 -05:00
Martin Baulig 400fb9c2cb Socket.BeginMConnect() should not attempt connections on unsupported address families. (#18302)
Fixes #16513.
2019-12-30 13:08:03 -05:00
Aleksey Kliger (λgeek) 61eb2be2e1 [eglib] Don't define _FORTIFY_SOURCE on Cygwin builds (#18308)
See https://github.com/msys2/MINGW-packages/issues/5803

Without -lssp (or -fstack-protector), -D_FORTIFY_SOURCE=2 leads to linker
errors.
2019-12-30 09:54:57 -05:00
imhameed 9719635e58 Use byte-aligned LLVM loads and stores for unaligned mini IR loads and stores. (#18296)
Should fix https://github.com/mono/mono/issues/18221.
2019-12-26 14:34:02 -05:00
EgorBo 28cae98ef0 Optimize box(T)+GetType 2019-12-25 02:32:46 +03:00
Aleksey Kliger (λgeek) ffdffa54f2 [domain] Don't add NULL runtime to runtimes list (#18260)
Also when iterating over the runtimes, actually look at the current element,
not the first element.

Also mention the unsupported runtime version in the warning message when it is
passed to mono_init_internal.

Fixes https://github.com/mono/mono/issues/17916
2019-12-23 10:44:32 -05:00
Fan Yang a1a4bcc49d Fix llvm jit assertion (#18280)
* Fix assertion when Alignment is greater than 32
2019-12-23 08:38:36 -05:00
Fan Yang 7ffcaaf59a Add cast to ((align) - 1) for Alignment macros (#18255)
* Add cast to ((align) - 1) for Alignment macros

* Add test for Alignment macro change
2019-12-23 08:34:37 -05:00
Alexander Köplinger 720767a9ff Fix invalid assetManifestPlatform in pipeline-netcore-runtime.yml
Spaces are note valid there.
2019-12-23 12:24:53 +01:00
Larry Ewing 1587c49f5f [wasm][debugger] Clean up startup and improve comments
Properly await more of the async logic and add retries when we get
an empty response from the "json/list" endpoint.
2019-12-20 21:41:05 -06:00
Larry Ewing 14b736e942 [wasm] resolve the full path of pdbs so that ../ doesn't break inclusion 2019-12-20 21:41:05 -06:00
Larry Ewing 04b9e5cb8f [wasm][debugger] rename the debugger target to run-debugger-tests 2019-12-20 21:41:05 -06:00
Larry Ewing 1669812689 [wasm][debugger] get the debug harness working again 2019-12-20 21:41:05 -06:00
Maxim Lipnin b55d8f9700 [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes (#18223)
* [wasm][xunit tests] Avoid skipping test methods with ITraitAttribute custom attributes

* [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate

* [wasm][xunit tests] Exclude XLinqTests.TreeManipulationTests.IEnumerableOfXNodeRemoveWithEvents test method
2019-12-20 19:18:51 -06:00
Ryan Lucia 0f8aac3c62 [loader] Check AppContext.BaseDirectory on netcore in preload hook 2019-12-20 19:17:01 -06:00
Ryan Lucia 57b6c8775b [loader] Remove unnecessary newlines in tracing 2019-12-20 19:16:11 -06:00
Ryan Lucia 5c87445310 [netcore] Re-enable some BinaryFormatter tests (#18277) 2019-12-20 19:52:53 -05:00
Alexander Köplinger cfba7c550f [netcore] Update dotnet sdk version in global.json (#18264)
Should unbreak the publishing build of netcore-mono.
2019-12-20 17:26:09 +01:00
Larry Ewing 47bca852c1 [wasm] fix the wrapped function signatures to match current documentation 2019-12-20 09:38:18 -06:00
Larry Ewing 5db64e27e9 [wasm] fix the mono_clear_bps condition 2019-12-20 09:38:18 -06:00
Aleksey Kliger (λgeek) d6f3de3c4c [tests] Add test files to tarball (#18261)
Some test sources weren't included.

Fixes https://github.com/mono/mono/issues/18202
2019-12-20 11:11:26 +01:00
Alexander Köplinger 7fd24a40bb Bump corefx to get test fix 2019-12-20 00:37:55 +01:00
Larry Ewing 83a898d25b [wasm] Make sure we're comparing the same date (#18251) 2019-12-19 15:35:05 +01:00
Alexander Köplinger c8e19bec68 [System] Add additional hardening against race in MonoTlsStream (#18250)
See https://github.com/mono/mono/issues/15805#issuecomment-567131297
2019-12-19 02:25:06 +01:00
Bernhard Urban-Forster e7b98d3587 [interp] add guard that a local hasn't been modified for stlocfld fusion (#18129)
[interp] add guard that a local hasn't been modified for stlocfld fusion

Fixes https://github.com/mono/mono/issues/18120
2019-12-19 02:05:57 +01:00
Nikita Voronchev 0beb9611df [WinForms][UIA] Add new internal event to track form's owner change (#18117)
This PR is needed for https://github.com/mono/uia2atk/pull/8
2019-12-18 22:24:06 +01:00
Alexander Köplinger a74416d950 [corlib] Move a few more test cases to part2, turns out part1 was still too large
Follow up to 3891fe97.
2019-12-18 22:18:33 +01:00
Alexander Köplinger 3891fe97d2 [bcl] Split corlib and System.Core xunit test assemblies for monotouch profiles (#18228)
They cannot deal with large test assemblies so we need to split them.

We're doing this by introducing a new `#skip <PROFILE>` directive in the .sources format which defines where to split the assembly.
The logic is very simple: files up until the first `#skip` are common files which get included in each part, subsequent files are only in the respective part assembly.

Fixes https://github.com/mono/mono/issues/18213
2019-12-18 21:11:17 +01:00
Larry Ewing 2ff93e4c76 [wasm] Fix test case spelling (#18242) 2019-12-18 13:51:21 -06:00
Maxim Lipnin 56f397bef3 [wasm][xunit tests] Re-enable System.Tests.MathFTests.Round_Digits and System.Tests.MathFTests.Truncate (#18234) 2019-12-18 06:49:45 -06:00
Larry Ewing 60f081c077 [wasm] handle completed Tasks and exceptions when resolving Task prom… (#18215)
* [wasm] handle completed Tasks and exceptions when resolving Task promises

* Preserve the old void invoke behaviour
2019-12-17 22:47:23 -06:00
Larry Ewing f46d5fda1d [wasm][debugger] fix exception when we can't locate debug assembly (#18230) 2019-12-17 21:37:22 -06:00
Nathan Ricci ed7432d988 Change ICONST to I8CONST when propagating constants through sign extension. (#18222)
Change ICONST to I8CONST when propagating constants through sign extension.
2019-12-17 21:35:56 -05:00
Evgenii Stepanov 06673e723e Relax assert in mono_arch_build_imt_trampoline. (#18195)
This assert fails when the trampoline code size worst case is realized.

Observed on a Android device with HWASan [1] enabled, where heap
addresses have a non-zero tag in the most significant byte, and thus
require 4 instructions to materialize in emit_imm64.

[1] https://source.android.com/devices/tech/debug/hwasan
2019-12-17 01:18:53 +01:00
Maxim Lipnin df3837eb7d [wasm][xunit tests] Re-enable System.Buffers.Text.Tests.ParserTests (#18220) 2019-12-17 01:18:34 +01:00
Zoltan Varga aa9227f7c8 [llvm] Generate correct LLVM IR for OP_XEQUAL. (#18217) 2019-12-17 00:52:02 +01:00
Zoltan Varga 970d695718 [wasm] Disable linker substitutions for now, it generates invalid IL when running make build-aot-hello-netcore. (#18218) 2019-12-17 00:10:04 +01:00
Zoltan Varga 6bf2c17f34 [wasm] Pass --deterministic to the linker in msbuild. (#18214) 2019-12-17 00:09:46 +01:00
Zoltan Varga 9f5e38229e [netcore] Add AssemblyLoadContext:OnAssemblyLoad() to linker descriptor. (#18219) 2019-12-17 00:09:24 +01:00
Jay Krell de359d4930 Remove handles/MonoError from some marshal icalls. (#18036)
Remove handles/MonoError from some marshal icall functions.

This does some small other things.
  In particular, if a function is not static for some platforms, make it not static for all.
  static historically was valuable, because of default linker semantics around visibility.
  I believe it is less important now, since visibility is default constrained.
  Also being consistent across platforms seems good.

Maybe split off a separate PR for that.
2019-12-16 23:06:11 +01:00
Ryan Lucia e19914fe69 [metadata] Check for GTDs in setup_generic_array_ifaces (#18101)
Judging by https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/metadata.c#L3515 the cached_class is sometimes intentionally a GTD, which can get passed all the way down to setup_generic_array_ifaces and cause a crash.
2019-12-16 14:17:44 -05:00
Zoltan Varga ac1a12f997 [wasm] Regen wasm_m2n_invoke.g.h. (#18212) 2019-12-16 18:54:54 +01:00
Marek Safar 47199fede8 Revert "[wasm][ci] Deactivate linker on browser tests (#18152)"
This reverts commit 079acc2fb8.
2019-12-16 17:12:55 +01:00
Marek Safar f8266639ea Bump linker 2019-12-16 17:12:55 +01:00
Maxim Lipnin ba69304049 [wasm][xunit tests] Re-enable System.Tests.MathFTests (#18208) 2019-12-16 09:53:02 -06:00
Jérôme Laban 5562065e73 [Wasm] Enable dynamic runtime tests (#18098)
* [Wasm] Enable dynamic runtime tests

- Add dynamic runtime browser tests in CI
- Change `release-dynamic` to `dynamic-release` to align with other runtimes
- Add `-dynamic-runtime` flag for packager.exe
- Add debug configuration for dynamic runtime
- Fix invalid MAIN_MODULE=2 flag, which forces the emscripten linker to pass even on exported functions.
2019-12-16 07:28:26 -06:00
Calvin Buckley 84221f04b5 Fix locating Python interpreter when it isn't "python" (#18197)
Fix locating Python interpreter when it isn't "python"

The "python" interpreter is specified by PEP-394. In it, an OS
can decide not to ship a "python" executable, and instead only
ship python2/3, providing some ambiguity for scripts that are
version-neutral, like the one Mono uses.

This uses an automake macro to locate a suitable Python, and
changes the makefile to use the located Python, whatever it may be.
2019-12-16 11:06:00 +01:00
Zoltan Varga 61a103f371 [wasm] Set the main thread, some EH code depends on it. (#18203) 2019-12-16 08:41:43 +01:00
Marek Safar afcafb6c5e [monolinker] Fix path to wasm references 2019-12-15 17:05:37 +01:00
Maxim Lipnin c454cb92eb [wasm][xunit] Re-enable System.MemoryTests and System.SpanTests tests (#18179) 2019-12-14 14:59:48 +01:00
Jay Krell e76fb5dbf9 Add full compiler barriers to any fence that lacked a compiler barrier. (#18177)
A compiler barrier generates no code.
It "only" inhibits certain compiler optimizations, which I do not consider
particularly worrisome, as most code is "fully" optimized, and
compiler barriers are relatively rare.

It could be a concern in code paths that greatly benefit
from compiler optimizations and for which barriers greatly
inhibits the compiler.

I would further suggest, though it is not here, that any compiler
half barrier (read, write) be a compiler full barrier instead,
as they sound pretty cheap, so simplify.

Similar cannot be said about CPU barriers.
They do not always generate code, but when they do, it can cost.

This is follow-up from https://github.com/mono/mono/pull/17849#issuecomment-556273801
https://github.com/mono/mono/pull/17849#issuecomment-557661604.
2019-12-13 22:49:15 +01:00
Marek Safar 6dc5295ed0 [wasm] Improve virtualenv installation script 2019-12-13 22:37:08 +01:00
monojenkins b0e7a1e4f9 [bcl] Update BCL Linked Size 2019-12-13 22:24:52 +01:00
Marek Safar 15a8c2a7fa [wasm] Run linker tests with wasm substitution settings 2019-12-13 22:15:59 +01:00
Aleksey Kliger (λgeek) d0f27ce4fe [cominterop] Attach to runtime from IUnknown and IDispatch methods (#18175)
* [cominterop][test] Calling QueryInterface from an unattached thread asserts

If we pass a managed COM object to native code and the native code calls one of
the IUnknown methods, we assert because the thread is not attached but we want
to enter GC Unsafe mode.

* [cominterop] Attach to runtime from IUnkown and IDispatch methods

If the COM objects for a managed object is passed to a thread that hasn't
interacted with Mono before, we need to attach before doing GC thread state
transitions.

Addresses https://github.com/mono/mono/issues/18137

* [test] Call managed method through COM wrapper from unattached thread

This works, but I don't think we had a test previously.  Call a managed method
via a COM interface from a thread that is not attached to the runtime, to
ensure that it is attached.
2019-12-13 15:32:06 -05:00
Ankit Jain ea50ca48ba Bump msbuild to track xplat-master (#18183)
- bump roslyn to 3.5.0-beta1-19606-04, to match msbuild
- bump nuget.exe to 5.5.0.6319 (5.5.0-preview1), to match msbuild
2019-12-13 20:06:16 +01:00
Jay Krell 1310043953 Fix on-demand initialization race conditions [threads.c]. (#18166) 2019-12-13 19:57:33 +01:00
Jay Krell 2eca6a1e37 Fix on-demand initialization race conditions [marshal.c]. (#18169)
* Fix on-demand initialization race conditions [marshal.c].

* PR feedback: fix incorrect merge
2019-12-13 19:57:17 +01:00
Ryan Lucia 7bc25cef6d [netcore] Implement AppDomain.AssemblyLoad (#18143) 2019-12-13 11:36:08 -05:00
Jay Krell d2539babf4 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:30:08 +01:00
Jay Krell b637c1db67 Fix on-demand initialization race conditions [marshal-ilgen.c] (#18160)
* Fix on-demand initialization race conditions [marshal-ilgen.c]

* PR: Feedback more MONO_STATIC_POINTER_INIT instead of manual mono_atomic_store_seq.

* PR feedback: Narrow scope of from_oadate.

* One more MONO_STATIC_POINTER_INIT vs. mono_atomic_store_seq post-review.

* Repair whitespace.

* Alternate formating.
2019-12-13 17:29:30 +01:00
Alexander Köplinger 7e65a73d4d [sdks] Use Xcode 11.3 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-12-13 17:27:24 +01:00
Kenneth Pouncey 96fa4e8c6f [wasm][debug-proxy] Fix crash and send back failed message. (#18146)
* [wasm][debug-proxy] Fix crash and send back failed message.

* Remove debugging error message

* Remove debug Console.WriteLine
2019-12-13 10:22:55 -06:00
Jay Krell 57c6b81b96 Fix on-demand initialize race conditions [cominterop.c] (#18158) 2019-12-13 12:28:34 +01:00
Jay Krell 64ad646066 Fix on-demand initialize race conditions [custom-attrs.c] (#18159) 2019-12-13 12:27:39 +01:00
Steve Pfister 97bf0a1bcc Relax faultcode parsing for Soap11 message errors. (#18154)
* Relax faultcode parsing for Soap11 message errors.

The mono implementation was reading the faultcode content as an XmlQualifiedName, which applied the rule where you could not
start a name with a number. This caused interoperability problems with the .net framework because their parsing just looks
for <namespace>:<name>, where name can be any string.

Example:

<faultcode>s:1</faultcode>

On mono, a soap response that contains a fault code name of 1 will throw an error.  If you run the same code on the .net framework, it will not.

Fixes https://github.com/mono/mono/issues/12995

* [csproj] Update project files
2019-12-13 12:26:53 +01:00
Jay Krell 0dc19600d1 Fix on-demand initialization race conditions [object.c]. (#18163)
Fix on-demand initialization race conditions [object.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 09:25:56 +01:00
Zoltan Varga e391861c01 [wasm] Add linker substitutions. (#18149)
* [wasm] Add linker substitutions in netcore mode.

* Enable non-netcore mode as well.

* Disable conditionals which are not true in threaded mode for now.
2019-12-13 06:46:33 +01:00
Jay Krell 9cefdf59ad Fix on-demand initialization race conditions [sgen]. (#18170)
Fix on-demand initialization race conditions [sgen].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 04:55:58 +01:00
Jay Krell a93e4712ec Fix on-demand initialization race conditions [remoting.c]. (#18168)
Fix on-demand initialization race conditions [remoting.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:46:07 +01:00
Jay Krell 943a2760ec Fix on-demand initialization race conditions [native-library.c]. (#18162)
Fix on-demand initialization race conditions [native-library.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 01:26:07 +01:00
Jay Krell 1fe3d7eda6 Fix on-demand initialization race conditions [reflection.c]. (#18167)
Fix on-demand initialization race conditions [reflection.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:45:36 +01:00
Jay Krell 51bc81919f Fix on-demand initialization race conditions [mini] (#18161)
Fix on-demand initialization race conditions [mini].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-13 00:36:19 +01:00
Jay Krell 620d7cf84d Fix on-demand initialization race conditions [sre.c]. (#18164)
Fix on-demand initialization race conditions [sre.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:55:33 +01:00
Jay Krell f96ba075a1 Fix on-demand initialization race conditions [threadpool.c]. (#18165)
Fix on-demand initialization race conditions [threadpool.c].

Extracted from https://github.com/mono/mono/pull/18150 which reviewer said was too big.
2019-12-12 23:46:06 +01:00
Ryan Lucia 88dde8b763 [netcore] Implement AppDomain.TypeResolve (#18141) 2019-12-12 15:00:28 -05:00
Jay Krell a1e16d3273 Remove handles/MonoError from 12 easy socket icalls. (#18035)
Also put _icall back on ends so the search coincides between managed and native.
2019-12-12 13:32:14 -05:00
Maxim Lipnin 6d9c1359ec [wasm] Disable particular failing System.Net.Security.Tests tests rather than the whole namespace (#18151) 2019-12-12 13:07:44 -05:00
Ryan Lucia 7d88d4d502 [netcore] Enable AppDomainTests.AssemblyResolveInvalidAssemblyName (#18157)
I don't remember why this wasn't enabled originally, but it seems to be passing now. I probably accidentally fixed it back when I made changes to our assembly name parsing.
2019-12-12 12:27:51 -05:00
Zoltan Varga fb67916c0f [wasm] Change mono_wasm_runtime_invoke () to return the exception itself in the out argument instead of a boolean. (#18135) 2019-12-12 17:23:42 +01:00
Kenneth Pouncey 079acc2fb8 [wasm][ci] Deactivate linker on browser tests (#18152)
There are problems on the CI when using the linker building the Browser Tests.

https://github.com/mono/mono/commit/3142396c3c6a62f389420113129783bb156de399

PR https://github.com/mono/mono/pull/18128
2019-12-12 12:53:05 +01:00
Ryan Lucia 3ddcbda8b6 [netcore] Enable AppDomainTests.MonitoringIsEnabled (#18142) 2019-12-11 21:19:45 -05:00
Ryan Lucia acc8c0c696 [netcore] Remove MonoDomain.DoAssemblyResolve (#18139)
This is no longer called in netcore builds as of a while ago, so get rid of it and the corresponding linker exclusion.
2019-12-12 00:16:50 +01:00
Jay Krell 256eda8c7f Make Socket.Socket_icall static, as most icalls are, removing the first, unused parameter. (#18130)
Also partial undo recent and append _icall to end.
This is verbose, but maybe worthwhile to aid search.

This is extracted from https://github.com/mono/mono/pull/17978.
2019-12-11 14:40:27 -05:00
Ryan Lucia 0e05bda235 [utils] Correctly convert MONO_DL_LOCAL to RTLD_LOCAL (#18121)
On Linux, neither flag being specified will result in a default of RTLD_LOCAL. Unfortunately, on MacOS it's the opposite - the default is RTLD_GLOBAL. This means currently there is no way to set RTLD_LOCAL on Macs, so despite the potential for this to break someone it directly contradicts our documentation and needs to be changed.
2019-12-11 10:42:10 -05:00
Marek Safar 3142396c3c Bump linker (#18128)
* Bump linker

* [csproj] Update project files
2019-12-11 16:38:32 +01:00
Alexander Köplinger 8ae9708e87 [corlib] Rename internal LogLevel enum to avoid conflict with Android LogLevel (#18124)
After https://github.com/mono/mono/pull/18040 we now have InternalsVisibleTo for the Mono.Android assembly.
This exposed the internal LogLevel enum in BCLDebug which causes an issue when consumed in XA:

```
Java.Lang/Object.cs(119,18): error CS0104: 'LogLevel' is an ambiguous reference between 'Android.Runtime.LogLevel' and 'System.LogLevel'
```

To fix this we rename the internal enum in corlib since it's not really used.
2019-12-11 13:39:38 +01:00
Jay Krell 97dd6cf04c Sockets: Foo_internal to Foo_icall and ves_icall_Foo_internal to ves_icall_Foo. (#18017)
The first is to avoid requiring the signature matching in icall resolution.
The second is just to shorten names.
2019-12-11 10:40:00 +01:00
Maxim Lipnin 93c18d6fd1 [wasm] Re-enabling a test that was fixed by the corefx bump (#18108)
* [wasm] Clean up rsp-file

* Bump corefx
2019-12-11 10:39:35 +01:00
Kenneth Pouncey d6d74e819f [wasm][debugger] Prevent crash on disconnect of client (#18107) 2019-12-11 06:37:19 +01:00
Bernhard Urban-Forster a65b7af6d5 [interp] relop result is always int (#18114)
This can lead to problems on big endian.

```
result.i = (int) 1;
--> 0x0000_0001_XXXX_XXXX;

result.l = (long) 1;
--> 0x0000_0000_0000_0001;
```

On little endian it's always `0x0000_0000_0000_0001`, so it doesn't matter.
2019-12-10 22:06:48 +01:00
Ryan Lucia a778c46362 [utils] Make mono_dl_open default to RTLD_LOCAL on netcore (#17912)
* [utils] Make mono_dl_open default to RTLD_LOCAL

The exception is for fallback handlers added via the embedding API, in which case we convert the flags over to preserve legacy behavior

* Convert flags properly for embedding API use

* Changes

* Turn off -fvisibility=hidden temporarily

TESTING

* Undo some previous crimes, maybe fix failure

* Leave legacy mono unchanged, defaulting to RTLD_GLOBAL
2019-12-10 14:58:58 -05:00
Bernhard Urban-Forster ef422d9cb7 [interp] fix build when DEBUG_INTERP is enabled (#18112)
broke with c4136ebd94
2019-12-10 18:55:09 +01:00
Maxim Lipnin cd2e0e8f7a [wasm] Clean up rsp-file (#18113) 2019-12-10 18:08:52 +01:00
Bernhard Urban-Forster 80b1e103cb [debugger] skip suspend for unattached threads (#18105)
When the debugger tries to suspend all the VM threads, it can happen with cooperative-suspend that it tries to suspend a thread that has previously been attached, but then did a "light" detach (that only unsets the domain). With the domain set to `NULL`, looking up a JitInfo for a given `ip` will result into a crash, but it isn't even necessarily needed for the purpose of suspending a thread.

More details: Consider the following:
```
(lldb) c
error: Process is running.  Use 'process interrupt' to pause execution.
Process 12832 stopped
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
    frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
   2713         MonoDomain *domain = (MonoDomain *) mono_thread_info_get_suspend_state (info)->unwind_data [MONO_UNWIND_DATA_DOMAIN];
   2714         if (!domain) {
   2715                 /* not attached */
-> 2716                 ji = NULL;
   2717         } else {
   2718                 ji = mono_jit_info_table_find_internal ( domain, MONO_CONTEXT_GET_IP (&mono_thread_info_get_suspend_state (info)->ctx), TRUE, TRUE);
   2719         }
Target 0: (WatchWCSessionAppWatchOSExtension) stopped.
(lldb) bt
* thread #9, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  * frame #0: 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6
    frame #1: 0x004e177b WatchWCSessionAppWatchOSExtension`mono_thread_info_safe_suspend_and_run(id=0xb0767000, interrupt_kernel=0, callback=(WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical at debugger-agent.c:2708), user_data=0xb04dc718) at mono-threads.c:1358:19
    frame #2: 0x00222799 WatchWCSessionAppWatchOSExtension`notify_thread(key=0x03994508, value=0x81378c00, user_data=0x00000000) at debugger-agent.c:2747:2
    frame #3: 0x00355bd8 WatchWCSessionAppWatchOSExtension`mono_g_hash_table_foreach(hash=0x8007b4e0, func=(WatchWCSessionAppWatchOSExtension`notify_thread at debugger-agent.c:2733), user_data=0x00000000) at mono-hash.c:310:4
    frame #4: 0x0021e955 WatchWCSessionAppWatchOSExtension`suspend_vm at debugger-agent.c:2844:3
    frame #5: 0x00225ff0 WatchWCSessionAppWatchOSExtension`process_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0, il_offset=0, ctx=0x00000000, events=0x805b28e0, suspend_policy=2) at debugger-agent.c:4012:3
    frame #6: 0x00227c7c WatchWCSessionAppWatchOSExtension`process_profiler_event(event=EVENT_KIND_THREAD_START, arg=0x039945d0) at debugger-agent.c:4072:2
    frame #7: 0x0021b174 WatchWCSessionAppWatchOSExtension`thread_startup(prof=0x00000000, tid=2957889536) at debugger-agent.c:4149:2
    frame #8: 0x0037912d WatchWCSessionAppWatchOSExtension`mono_profiler_raise_thread_started(tid=2957889536) at profiler-events.h:103:1
    frame #9: 0x003d53da WatchWCSessionAppWatchOSExtension`fire_attach_profiler_events(tid=0xb04dd000) at threads.c:1120:2
    frame #10: 0x003d4d83 WatchWCSessionAppWatchOSExtension`mono_thread_attach(domain=0x801750a0) at threads.c:1547:2
    frame #11: 0x003df1a1 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop_internal(domain=0x801750a0, cookie=0xb04dcc0c, stackdata=0xb04dcba8) at threads.c:6008:3
    frame #12: 0x003df287 WatchWCSessionAppWatchOSExtension`mono_threads_attach_coop(domain=0x00000000, dummy=0xb04dcc0c) at threads.c:6045:9
    frame #13: 0x005034b8 WatchWCSessionAppWatchOSExtension`::xamarin_switch_gchandle(self=0x80762c20, to_weak=false) at runtime.m:1805:2
    frame #14: 0x005065c1 WatchWCSessionAppWatchOSExtension`::xamarin_retain_trampoline(self=0x80762c20, sel="retain") at trampolines.m:693:2
    frame #15: 0x657ea520 libobjc.A.dylib`objc_retain + 64
    frame #16: 0x4b4d9caa WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke + 279
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb26e Foundation`____addOperations_block_invoke_4 + 20
    frame #24: 0x65de007b libdispatch.dylib`_dispatch_call_block_and_release + 15
    frame #25: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #26: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #27: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #28: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #29: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #30: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #31: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```

Going further, `info` is about this thread:
```
(lldb) p/x *(int *)(((char *) info->node.key) + 0xa0)
(int) $2 = 0x01243f93
(lldb) thread list
Process 12832 stopped
  thread #1: tid = 0x1243ee1, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_303', queue = 'com.apple.main-thread'
  thread #2: tid = 0x1243ee6, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10
  thread #3: tid = 0x1243ee7, 0x65f81aea libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'SGen worker'
  thread #4: tid = 0x1243ee9, 0x65f7e3d2 libsystem_kernel.dylib`semaphore_wait_trap + 10, name = 'Finalizer'
  thread #5: tid = 0x1243eea, 0x65f816e2 libsystem_kernel.dylib`__recvfrom + 10, name = 'Debugger agent'
  thread #6: tid = 0x1243f1d, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.uikit.eventfetch-thread'
  thread #8: tid = 0x1243f93, 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
* thread #9: tid = 0x12443a9, 0x00222870 WatchWCSessionAppWatchOSExtension`debugger_interrupt_critical(info=0x81365400, user_data=0xb04dc718) at debugger-agent.c:2716:6, name = 'tid_a31f', queue = 'NSOperationQueue 0x8069b670 (QOS: UTILITY)', stop reason = breakpoint 1.1
  thread #10: tid = 0x1244581, 0x65f7fd32 libsystem_kernel.dylib`__workq_kernreturn + 10
(lldb) thread select 8
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
    frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
libsystem_kernel.dylib`mach_msg_trap:
->  0x65f7e396 <+10>: retl
    0x65f7e397 <+11>: nop

libsystem_kernel.dylib`mach_msg_overwrite_trap:
    0x65f7e398 <+0>:  movl   $0xffffffe0, %eax         ; imm = 0xFFFFFFE0
    0x65f7e39d <+5>:  calll  0x65f85f44                ; _sysenter_trap
(lldb) bt
* thread #8, name = 'tid_6d0f', queue = 'NSOperationQueue 0x8069b300 (QOS: UTILITY)'
  * frame #0: 0x65f7e396 libsystem_kernel.dylib`mach_msg_trap + 10
    frame #1: 0x65f7e8ff libsystem_kernel.dylib`mach_msg + 47
    frame #2: 0x66079679 libxpc.dylib`_xpc_send_serializer + 104
    frame #3: 0x660794da libxpc.dylib`_xpc_pipe_simpleroutine + 80
    frame #4: 0x66079852 libxpc.dylib`xpc_pipe_simpleroutine + 43
    frame #5: 0x66043a8f libsystem_trace.dylib`___os_activity_stream_reflect_block_invoke_2 + 30
    frame #6: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #7: 0x65de3d71 libdispatch.dylib`_dispatch_block_invoke_direct + 257
    frame #8: 0x65de3c62 libdispatch.dylib`dispatch_block_perform + 112
    frame #9: 0x6604349a libsystem_trace.dylib`_os_activity_stream_reflect + 725
    frame #10: 0x6604ef19 libsystem_trace.dylib`_os_log_impl_stream + 468
    frame #11: 0x6604e44d libsystem_trace.dylib`_os_log_impl_flatten_and_send + 6410
    frame #12: 0x6604cb3b libsystem_trace.dylib`_os_log + 137
    frame #13: 0x6604f4aa libsystem_trace.dylib`_os_log_impl + 31
    frame #14: 0x4b4eb4e9 WatchConnectivity`WCSerializePayloadDictionary + 393
    frame #15: 0x4b4d7c4d WatchConnectivity`-[WCSession onqueue_sendResponseDictionary:identifier:] + 195
    frame #16: 0x4b4da435 WatchConnectivity`__66-[WCSession onqueue_handleDictionaryMessageRequest:withPairingID:]_block_invoke.411 + 35
    frame #17: 0x453c7df7 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #18: 0x453c7cf4 Foundation`-[NSBlockOperation main] + 88
    frame #19: 0x453cacee Foundation`__NSOPERATION_IS_INVOKING_MAIN__ + 27
    frame #20: 0x453c6ebd Foundation`-[NSOperation start] + 835
    frame #21: 0x453cb606 Foundation`__NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 27
    frame #22: 0x453cb12e Foundation`__NSOQSchedule_f + 194
    frame #23: 0x453cb067 Foundation`____addOperations_block_invoke_2 + 20
    frame #24: 0x65dedf49 libdispatch.dylib`_dispatch_block_async_invoke2 + 77
    frame #25: 0x65de4461 libdispatch.dylib`_dispatch_block_async_invoke_and_release + 17
    frame #26: 0x65de126f libdispatch.dylib`_dispatch_client_callout + 14
    frame #27: 0x65de3788 libdispatch.dylib`_dispatch_continuation_pop + 421
    frame #28: 0x65de2ee3 libdispatch.dylib`_dispatch_async_redirect_invoke + 818
    frame #29: 0x65df087d libdispatch.dylib`_dispatch_root_queue_drain + 354
    frame #30: 0x65df0ff3 libdispatch.dylib`_dispatch_worker_thread2 + 109
    frame #31: 0x66024fa0 libsystem_pthread.dylib`_pthread_wqthread + 208
    frame #32: 0x66024e44 libsystem_pthread.dylib`start_wqthread + 36
```
which is a thread in a "light" detach state (aka. coop detach), where we only unset the domain:
https://github.com/mono/mono/blob/4cefdcb7ce2d939ee78fb45d1b4913eb3bc064fd/mono/metadata/threads.c#L6084-L6111

Fixes https://github.com/mono/mono/issues/17926
2019-12-10 12:16:05 +01:00
Aleksey Kliger (λgeek) 7704bd1b92 [mini] Fix delegate trampoline virtual call via delgate Invoke (#18073)
* [mini] Fix delegate trampoline virtual call via delgate Invoke

If we need to jit the Invoke method of a delegate, we get tramp_info with a
NULL method.

Background: normally when we create a delegate around a virtual method,
handle_delegate_ctor will just create a virtual invoke trampoline with
mono_arch_get_delegate_virtual_invoke_impl which doesn't get here.  But if
we're asked to compile the delegate's Invoke method, then compile_special ()
will create a tramp_info with a null method, and return a delegate trampoline.

That's the case here - we had
  var del = SomeDelegate(obj.VirtualMethod);
  var invoke_method = del.GetType().GetMethod ("Invoke");
  invoke_method.Invoke (del, args);
or
  var del = SomeDelegate(obj.VirtualMethod);
  var another_del = OtherDelegate (del.Invoke);
  another_del (args);

in both cases, we end up in mono_delegate_trampoline with tramp_info->method ==
NULL.

in the second case the IL is like this:
   newobj instance void Derived::'.ctor'
   ldvirtftn instance void class Base::VirtualMethod()
   newobj instance void class SomeDelegate::'.ctor'(object, native int)

So delegate->target is a derived instance but delegate->method is some base
class method.

Addresses https://github.com/mono/mono/issues/17718

* [tests] Add tests for compiling delegate's Invoke method

Where the delegate calls a virtual method
2019-12-10 12:12:50 +01:00
Bernhard Urban-Forster fd44435a84 [debugger] remove unused thread suspend count (#18102) 2019-12-10 08:25:31 +01:00
Nikolay Sivov d1fc2ffa84 [support] Remove some dead code (#16674)
Both 'ptv' and 'tv' are unused since select() was replaced with poll() in 57b521e.
2019-12-10 00:21:00 +01:00
Steve Pfister 4cefdcb7ce Bump to mono/corefx@f3815a9 (#18090)
Picks up https://github.com/mono/corefx/pull/376
2019-12-09 21:29:40 +01:00
Jay Krell 5cbb1f4be8 [cxx] Fix mono-boehm to compile as C++. (#18086)
https://jenkins.mono-project.com/job/test-mono-pull-request-i386/32263/consoleText

```
      ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
boehm-gc.c: In function 'void mono_gc_base_init()':
boehm-gc.c:202:42: error: invalid conversion from
'void (*)(size_t) {aka void (*)(unsigned int)}' to
GC_on_heap_resize_proc {aka void (*)(long unsigned int)}' [-fpermissive]
  GC_set_on_heap_resize (on_gc_heap_resize);
                                          ^
In file included from ../../mono/utils/gc_wrapper.h:44:0,
                 from ../../mono/metadata/gc-internals.h:16,
                 from boehm-gc.c:16:
../../external/bdwgc/include/gc.h:132:21: note:
 initializing argument 1 of 'void GC_set_on_heap_resize(GC_on_heap_resize_proc)'
 GC_API void GC_CALL GC_set_on_heap_resize(GC_on_heap_resize_proc);
```
2019-12-09 18:59:10 +01:00
abrevet-dev 3d83c7429e [WinForms] No event should be triggered when calling ComboBox.Items.RemoteAt() (#18069)
* [WinForms] Fix wrong behavior when calling ComboBox.Items.RemoteAt() with index equals to SelectedIndex

* [WinForms] Fix #10643 No event should be triggered when calling ComboBox.Items.RemoteAt()

Fixes #10643
2019-12-09 15:51:28 +01:00
dotnet-maestro[bot] 612f2a8e88 Update dependencies from https://github.com/dotnet/arcade build 20191208.1 (#18089)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19608.1
2019-12-09 15:26:16 +01:00
Maxim Lipnin 17bba9e7dd [wasm] Clean up rsp-file (#18087) 2019-12-09 07:57:00 -06:00
Egor Bogatov 2befdcfa52 [netcore] Enable more tests for interpreter and llvm on CI (#17957)
* Enable more tests

* fix build?

* forward XUNIT_MONO_ENV_OPTIONS

* increase timeout

* ignore some tests for interpreter

* ignore more tests

* move interp specific issues to a separate rsp

* fix CI

* fix CI

* undo build.sh chmod

* revert chmod for xunit-summary-py

* improve xunit summary

* Ignore the whole System.Net.Tests namespace

* Ignore System.Text.Json

* disable the whole BinaryFormatterTests

* Disable System.Net.Http.Functional.Tests
2019-12-09 14:55:44 +01:00
Kenneth Pouncey 1462208991 [wasm][bcl][websockets] Add WebSocket support. (#18062)
Add the ability to initialize a `WebAssembly.Net.WebSockets::ClientWebSocket` from `System.Net.WebSockets.ClientWebSocket`.

`System.Net.WebSockets.ClientWebSocket` will delegate all calls to the WebAssembly ClientWebSocket.  

- Add support for marshaling `Uri` class correctly.  This was a cause of failures.
- Add bindings tests for `Uri` marhaling.
2019-12-09 13:32:17 +01:00
Jay Krell ed6b8024b0 fix warning: threadpool-worker-default.c: In function 'worker_try_create': 'now' may be used uninitialized (#18056)
https://jenkins.mono-project.com/job/build-source-tarball-mono-pullrequest/854/console
2019-12-09 08:17:54 +01:00
Jay Krell a6c4eafee8 Add some missing barriers when dealing with global caches (#18032)
* Add some missing barriers when dealing with global caches
accessed from multiple threads without locks.

This is based on earlier PR https://github.com/mono/mono/pull/17849.

Read-side barriers are removed.
  More analysis is needed to determine if they are needed (which pointer
  read vs. written).
Write-side barriers are compiler barrier + store-release instead of full barrier.

Note all candidates are converted to new form yet.

Cases that do motivate read-wide barriers are skipped.
e.g.
`mono_arch_start_dyn_call`
`get_agent_domain_info`
`init_jit_info_dbg_attrs`

* PR: Remove 'Try one more time..'
2019-12-08 15:30:48 +01:00
Sam Patel 74567a0a34 Update glib-remap.h to include g_module_address (#18076) 2019-12-07 07:11:59 +01:00
abrevet-dev 4be2a9a207 [WinForms] Handle menu for wheel button click (close the contextmenu) (#18049)
Fixes #18019
2019-12-06 18:41:47 +01:00
Ankit Jain d1db937e78 Bump msbuild to track xplat-master (#18050)
.. to get changes from https://github.com/mono/msbuild/pull/162 .

And bump roslyn to 3.4.0-beta4-19569-03 to match msbuild.
2019-12-06 16:55:44 +01:00
Aleksey Kliger (λgeek) 3dabeddfc6 [threads] Add back mono_threads_attach_tools_thread as a public API (#18048)
* [utils] Add back mono_threads_attach_tools_thread

In https://github.com/mono/mono/commit/a5da7b21f4b6dbc5eaa09c2addee91b84dc1dbd5
we got rid of "tools" threads internally to the runtime.

However since the API was previously marked with MONO_API it was an internal
API that some embedders depended on.

This PR adds back an (external-only) limited form of tools thread.

The runtime is aware of the Tools thread in that FOREACH_THREAD_* macros will
iterate over them, and the thread has a coop thread state machine. (That is,
mono_thread_info_current() and GC Safe and GC Unsafe transitions all work.)

However the thread is:
1. Not stopped by the GC
2. Is not interrupted by profiler sampling.
3. Does not have a "current domain"
4. (As a consequence of the above) cannot call managed methods or touch managed
objects.

Such threads are useful for low-level interaction with the runtime such as
querying metadata, the JIT state and other coordination.

mono_threads_attach_tools_thread should be called no more than once.  It should
not be called on a thread that is already attached with mono_thread_atach, and
vice versa.

Addresses https://github.com/mono/mono/issues/18011

* [threads] Make mono_threads_attach_tools_thread into a public API
2019-12-06 10:50:34 -05:00
Bernhard Urban-Forster 121928d29a [ppc32] delete leftover code from pre-monoctx area (#18072)
Also update assert which doesn't make sense since we have moved over to MonoContext

Some context: e7011c780f

Fixes: https://github.com/mono/mono/issues/18064
2019-12-06 11:35:11 +01:00
Jay Krell 12bc4717fd [cxx] Remove use of register keyword. (#18057)
[cxx] Remove use of register keyword.

https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html

```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/utils/mono-md5.c:259:2: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
        register guint32 a, b, c, d;
        ^~~~~~~~~
4 warnings generated.
```
2019-12-06 03:55:56 +01:00
Jay Krell a131e9bbb0 [cxx][netcore] Fix goto around init. (#18058)
https://jenkins.mono-project.com/job/test-mono-pull-request-wasm/19408/parsed_console/log_content.html
```
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
/mnt/jenkins/workspace/test-mono-pull-request-wasm/mono/metadata/icall.c:882:1: error: jump to label 'exit' [-fpermissive]
```
2019-12-05 15:41:09 -06:00
Egor Bogatov 63f155ac46 Let user overwrite opt -Ox level (#18012) 2019-12-05 13:44:05 -06:00
Jay Krell b99ea70e43 Remove handles/MonoError from two socket icalls. (#17977)
Remove handles/MonoError from two socket icalls.

Ongoing removal of handles() icalls.
This is small, to pause and remove `mono_class_get_field_from_name_full` from w32socket.c.
2019-12-05 19:36:11 +01:00
Egor Bogatov 84839ab297 Merge pull request #18053 from EgorBo/fix-fneg-llvm
LLVM: use fneg instead of fsub for `-x`
2019-12-05 21:13:11 +03:00
Jay Krell ae9192f96f [netcore] Remove approx. 10 icalls. (#18018)
* [netcore] Remove ves_icall_System_Runtime_InteropServices_Marshal_PtrToStringUni and _len icalls.
They are implemented in C#.
The C# is equivalent to the C, and could be used for classic runtime, except for the treatment of large/negative lengths.

* Remove a few more icalls.
Instead of ifdef and leaving them uncompiled when compiling netcore,
declare them and put asserts in them.
2019-12-05 13:10:29 -05:00
Alexander Köplinger 269d509e42 Remove DNS lookups of the local hostname in tests (#18059)
This allows us to workaround issues we currently have in the macOS bot network where DNS lookup of the hostname fails.
2019-12-05 18:08:26 +01:00
Zoltan Varga e67673fac8 [llvm] Avoid decomposing OP_STOREV_MEMBASE early for types which contains no references. (#18041) 2019-12-05 16:02:43 +01:00
Martin Baulig cc720b83aa [System.Net.Http]: Add workaround for mono/mono#17897. (#17992)
Use `(UriKind)300` (which is the same as setting `MONO_URI_DOTNETRELATIVEORABSOLUTE=true`)
when parsing Uri headers.
2019-12-05 14:36:07 +01:00
Jay Krell 5a5e092a53 Fix ifndef include guard in icall-decl.h. (#18055)
There was no clash so ok either way, but this is more like intended.
2019-12-05 10:19:24 +01:00
Egor Bogatov a3fbb64493 Convert to 0/1 in OP_XEQUAL (#18038) 2019-12-05 02:38:34 +01:00
abrevet-dev 186f9cd8c4 [WinForms] Fix wrong behavior of RadioButton auto-check (#18044)
Fixes #18020
2019-12-05 00:41:49 +01:00
Alexander Köplinger 4590bf6cac Bump bockbuild to get https://github.com/mono/bockbuild/pull/136 (#18043) 2019-12-05 00:33:08 +01:00
Alexander Köplinger f965e5d3e8 Fix formatting of README.netcore.md 2019-12-05 00:11:14 +01:00
EgorBo 55a3c10e72 remove some items from rsp 2019-12-05 02:09:11 +03:00
EgorBo 92047c453f use fneg instead of fsub 2019-12-05 02:03:24 +03:00
Bernhard Urban-Forster 9d58bc1d12 [amd64] align application stack pointer in signal handler (#18021)
* [amd64] align application stack pointer in signal handler

16byte stack pointer alignment is only required in certain places by the ABI, so the assumption that it holds all the time is wrong (though, it passed CI 😅). We have to setup the frame properly, i.e. we have to ensure the stack pointer is properly aligned. That's what we have done previously too.

Regression of https://github.com/mono/mono/pull/17922
Fixes https://github.com/mono/mono/issues/18006

* add ALIGN_DOWN_TO macro

* remove unnecessary ifndefs
2019-12-04 15:23:19 -06:00
Alexander Köplinger 59554712cc Update submodules before running external/bdwgc/autogen.sh (#18042)
Otherwise we'll run into issues because in configure.ac we update submodules _after_ we try to call external/bdwgc/configure (which won't be there if autogen.sh didn't run).
This would manifest itself as `*** No rule to make target 'all'.` in external/bdwgc when you build as there's no Makefile.
2019-12-04 22:16:21 +01:00
Jonathan Peppers 8511ae20f0 [corlib] add InternalsVisibleTo for Mono.Android (#18040)
* [corlib] add InternalsVisibleTo for Mono.Android

Context: https://github.com/xamarin/xamarin-android/pull/3991

In Xamarin.Android, we have a bit of System.Reflection usage on
startup:

https://github.com/xamarin/xamarin-android/blob/bbbe2ed36e6eeb7cce0905dcbfdaeaa30754070f/src/Mono.Android/Android.Runtime/UncaughtExceptionHandler.cs#L47-L67

These methods are in `mscorlib.dll` and `internal`. If we had
`[assembly: InternalsVisibleTo ("Mono.Android")]` we could avoid the
System.Reflection.

This is already done for `Xamarin.iOS`, `Xamarin.Mac`, etc. so
hopefully it's OK to add one more?

We could then, just call these methods directly:

* `System.Diagnostics.Debugger.Mono_UnhandledException`
* `AppDomain.CurrentDomain.DoUnhandledException`

The results are possibly a 100-250ms improvement to startup:

    Before:
    12-03 14:27:51.450  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s727ms
    12-03 14:27:57.310  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s723ms
    12-03 14:28:03.216  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s703ms
    12-03 14:28:09.054  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s679ms
    12-03 14:28:14.953  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s711ms
    After:
    12-04 09:13:18.248  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s608ms
    12-04 09:13:24.087  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s570ms
    12-04 09:13:29.997  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s566ms
    12-04 09:13:35.805  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s498ms
    12-04 09:13:41.670  1876  1898 I ActivityManager: Displayed Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity: +3s475ms

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Bump API snapshot submodule
2019-12-04 21:01:20 +01:00
Kenneth Pouncey 83cac0306c [wasm][build] Modify to use -s USE_ZLIB=1 instead of zlib support headers (#18031)
* [wasm] Modify to use -s USE_ZLIB=1 instead of zlib support headers

- This will remove the dependency on the mono zlib header files.
- Moves the zlibtest.cs to tests/src directory to help clean up the main directory.

* [wasm][packager] Remove the zlib headers from the zlib build

- use `-s USE_ZLIB=1` instead when building.
2019-12-04 20:11:43 +01:00
Alexander Köplinger 4e8ead7caf Remove mk/ folder (#18039)
The plan to put additional helpers into this directory didn't materialize and we can just inline the one thing we have in common.mk where needed.
2019-12-04 19:55:22 +01:00
Kenneth Pouncey 42c5ccc298 [wasm][reference] Update binary references to include wasm websocket assemblies. (#18029) 2019-12-04 11:52:19 +01:00
Jonathan Peppers b816189b94 [android] don't probe directories for .config files (#18024)
* [android] don't probe directories for .config files

In Xamarin.Android, on startup we see logging like this for every .NET
assembly:

    12-03 14:27:42.104 26842 26842 D Mono    : Prepared to set up assembly 'FormsViewGroup' (/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll)
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly FormsViewGroup[0xebcee9c0] added to domain RootDomain, ref_count=1
    12-03 14:27:42.104 26842 26842 D Mono    : Assembly Loader loaded assembly from location: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/storage/emulated/0/Android/data/Xamarin.Forms_Performance_Integration/files/.__override__/FormsViewGroup.dll.config'.
    12-03 14:27:42.104 26842 26842 D Mono    : Config attempting to parse: '/Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config'.

In most cases, no `.dll.config` file exists and so it probes this
macOS-looking path:

    /Users/builder/jenkins/workspace/archive-mono/2019-10/android/release/sdks/out/android-x86-release/etc/mono/assemblies/FormsViewGroup/FormsViewGroup.config

It looks like this is happening because nothing in Xamarin.Android
calls `mono_set_dirs` or `mono_set_config_dir` and so it falls back to
using `MONO_CFG_DIR`. `MONO_CFG_DIR` is set to a path on the build
machine.

Even if Xamarin.Android *did* call `mono_set_config_dir`, the code
appends `mono/assemblies`:

    cfg = g_build_filename (cfg_dir, "mono", "assemblies", aname, cfg_name, (const char*)NULL);

So it doesn't seem useful to use this fallback at all, there would
never be a file found.

I wrapped the appropriate code in `#ifndef TARGET_ANDROID` blocks.

I was able to see a minor *maybe* ~20ms performance improvement on
startup:

    Before:
    12-03 14:27:48.494 26903 26903 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:601::443900
    12-03 14:27:54.356 26963 26963 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::495400
    12-03 14:28:00.263 27021 27021 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:586::147100
    12-03 14:28:06.135 27074 27074 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:593::639100
    12-03 14:28:11.988 27129 27129 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:571::194900
    After:
    12-03 14:46:50.590 27737 27737 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:503::918700
    12-03 14:46:56.517 27794 27794 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:570::533600
    12-03 14:47:02.368 27850 27850 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:537::435000
    12-03 14:47:08.289 27904 27904 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:587::830900
    12-03 14:47:14.130 27965 27965 I monodroid-timing: Runtime.init: end, total time; elapsed: 0s:555::593400

This was the Xamarin.Forms integration project in xamarin-android.
Running a `Debug` build on a HAXM x86 emulator on Windows.

https://github.com/xamarin/xamarin-android/tree/master/tests/Xamarin.Forms-Performance-Integration

* Add --enable-minimal=cfgdir_config

* This also defines `DISABLE_CFGDIR_CONFIG`
* Used this setting for android
2019-12-04 11:50:59 +01:00
abrevet-dev a79c4d11a4 [WinForms] Editing the DialogResult of the form in the button click event was ignored. (#18025)
Fixes #12022
2019-12-04 11:49:43 +01:00
Jérôme Laban 91d3d63858 Update to emscripten 1.39.4 (#18016)
Fixes for i64 returning values with dynamic runtime configuration.

See https://github.com/emscripten-core/emscripten/issues/9850
2019-12-04 09:11:22 +01:00
monojenkins 37ba834ee1 [bcl] Update BCL Linked Size 2019-12-04 08:29:56 +01:00
Aleksey Kliger (λgeek) 467d46f8ab [configure.ac] Build on FreeBSD/PowerPC64 (#18015)
Community contribution from https://github.com/mono/mono/issues/18009
from @igorzigorz
2019-12-03 19:56:30 -05:00
Egor Bogatov 85826e5c27 Merge pull request #18013 from EgorBo/intrin-string-memcpy
[netcore] Improve jitdiff tool
2019-12-04 02:19:32 +03:00
Maxim Lipnin e7ef2f3d5f [Wasm] Enable system nunit tests on ci (#17716)
* [wasm][nunit tests] Enable nunit tests for System on CI

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Mail.SmtpClientTest tests

* Fix test suit name

* [wasm][nunit tests] Exclude MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest nunit tests

* [wasm][nunit tests] Exclude MonoTests.System.Net.NetworkInformation.NetworkInterfaceTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.EventDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.ComponentModel.AsyncOperationManagerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.ServicePointTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebClientTestAsync nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebRequestTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Timers.TimerTest nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.HttpListenerTest.PropertiesWhenClosedSet3 test

* [wasm][nunit tests] Increase timeout for System nunit tests

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest.Find_FindByIssuerName test

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Mail.SmtpClientTest tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerPrefixCollectionTest nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.HttpListenerTest nunit tests

* Increase CI timeout for System.Core nunit tests

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.Sockets.UdpClientTest nunit tests

* [wasm][nunit tests] Update NetworkInterface class to throw PNSE

* [wasm][nunit tests] Ignore failing MonoTests.System.ComponentModel.PropertyDescriptorCollectionTests nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Collections.Specialized.NameValueCollectionTest nunit tests

* [wasm][nunit tests] Adjust the use of Ignore attribute

* [wasm][nunit tests] Categorize failing MonoTests.System.Collections.Concurrent.ConcurrentBagTests nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.ComponentModel.BackgroundWorkerTest nunit tests

* [wasm][nunit tests] Categorize failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Ignore failing MonoTests.System.Net.NetworkInformation.IPInterfacePropertiesTest nunit tests

* [wasm][nunit tests] Remove the category for passing MonoTests.System.Net.HttpListenerRequestTest nunit tests

* [wasm][nunit tests] Exclude failing part of MonoTests.System.Net.ServicePointTest.DnsRefreshTimeout test

* [wasm][nunit tests] Remove PNSE expectation in failing MonoTests.System.Net.Sockets.TcpListenerTest nunit tests.

* [wasm][nunit tests] Exclude failing MonoTests.System.Net.WebProxyTest nunit tests due to libc issue

* [wasm][nunit tests] Remove PNSE expectation in MonoTests.System.Net.WebRequestTest.DefaultWebProxy test

* [wasm][nunit tests] Exclude failing MonoTests.System.Security.Cryptography.X509Certificates.X509Certificate2CollectionTest nunit tests due to compare info issue
2019-12-03 22:40:09 +01:00
abrevet-dev 174f2f6a70 [WinForms] SetRowSpan and SetColumnSpan does not trigger the good exception. (#18002)
According to .Net documentation:

Exceptions
ArgumentOutOfRangeException

value is less than 1.
2019-12-03 21:38:38 +01:00
EgorBo 5f58bb0959 Improve TryParseFunctionName in jitdiff 2019-12-03 18:13:48 +03:00
EgorBo ea0824cb22 Remove dead docs 2019-12-03 18:11:42 +03:00
Egor Bogatov 8150a38dae Don't throw NRE when len == 0 in Buffer.Memmove (#18004) 2019-12-03 15:21:41 +01:00
Steve Pfister a3c25e7b4f Added Obsolete to AppDomain.CreateDomain for monodroid. (#18000)
* Added Obsolete to AppDomain.CreateDomain for monodroid.  The intent is to be helpful for when .NET 5 comes along

* Changed AssemblyLoadContext link to a better doc

* Bump API snapshot submodule
2019-12-03 14:23:49 +01:00
Jo Shields a2f9fb77a3 Bump version to 6.11 (#17990) 2019-12-02 15:58:38 -05:00
Di Wu 0370372ce3 Fix raw security descriptor parsing (#17482)
Fix security identifier parsing error with the following SDDL:

```
O:S-1-5-21-1356517589-3987021482-2501375073-1001G:S-1-5-21-1356517589-3987021482-2501375073-513D:(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;S-1-5-21-1356517589-3987021482-2501375073-1001)
```
2019-12-02 20:03:49 +01:00
Jay Krell 72aa81e901 Remove handles/MonoError from Mono.RuntimeClassHandle. (#17805) 2019-12-02 19:18:03 +01:00
Bernhard Urban-Forster a9a0a1e5b9 [profiler] disable inlining when coverage profiler is used and fix allocation sizes (#17986)
* [profiler] let compiler decide about the size of an entry

* [profiler] disable inlining when coverage profiler is used
2019-12-02 19:04:13 +01:00
Vlad Brezae 640ffa7a62 [sgen] Don't trigger collections during allocation of thread objects (#17970)
Allocation of thread objects usese the mono_object_new_mature API. These objects are allocated before the thread is finished attaching to the runtime. This means that a collection can happen on an unattached thread as well as all its callbacks, which is a counter-intuitive behavior.

On android this is problematic because at the end of the SGen collection we need to call the java collection, which needs to have the thread attached. This could probably be fixed instead from the Xamarin.Android side, but this approach seems simpler and saner.

Fixes https://github.com/mono/mono/issues/17878
2019-12-02 12:54:21 -05:00
Jay Krell d7f5a5fdfe Remove handles/MonoError from Mono.Security.Cryptography.KeyPairPersistence icalls. (#17967) 2019-12-02 12:51:16 -05:00
dotnet-maestro[bot] 9bd1b2e065 Update dependencies from https://github.com/dotnet/arcade build 20191201.1 (#17985)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19601.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19601.1
2019-12-02 18:03:11 +01:00
abrevet-dev 1454b010ae [WinForms] the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true (#17981)
* [WinForms] Fix #17980 Prevent from crashing when DataGridView  is null.

Reproducer:

DataGridViewLinkColumn links = new DataGridViewLinkColumn();
links.UseColumnTextForLinkValue = true;

* [WinForms] Fix #17980 The field useColumnTextForLinkValue of DataGridViewLinkCell was not cloned.

* [WinForms] Fix #17980 the Text property was not set for every cell in the column when UseColumnTextForLinkValue is true

Fixes #17980
2019-12-02 17:39:43 +01:00
abrevet-dev 865e955840 [WinForms] Fix #9726 imageLayout of DataGridViewImageCell was not cloned. (#17974)
Fixes #9726
2019-12-02 17:38:08 +01:00
Thays Grazia 3834be3d22 [DIM] Fix behavior when there is an override of a method in a generic interface. (#17963)
* Fix behavior when there is an override of a method in a generic interface.
Fixes #17869
2019-12-02 13:20:00 -03:00
Jay Krell 7356aedc77 Refactor Monitor to eliminate near identical duplicated code. (#17964)
The only difference in the copies, was that one had an infinite timeout
so implied TRUE result unless interrupted.

Where the other copy had a timeout and could return FALSE as a result.

They both used the same underlying, with timeout, so it wasn't really an optimization,
just duplicated code.

Also there is subtelty around having a return value or an out parameter.
That is carefully preserved.

Also, coop is fixed, by creating a frame only in a slow path.
Presumably creating a thread is quick and stateless enough such that this
is not too late.
2019-12-02 11:06:11 -05:00
Calvin Buckley 369806e3e6 Disable ConnectFailAsync test on AIX (#17975)
Hangs runtime; prevents CI from completing run. (Another test can
also hang the runtime, but is hard to reproduce and inconsistent
about that.)

This should be fixed eventually, but...
2019-12-02 10:34:07 +01:00
Zoltan Varga f13e6ba91a [wasm] Add beginnings of SIMD support in AOT mode. (#17971)
* [jit] Implement beginnings of SIMD support for WASM.

* [wasm] Add a DISABLE_WASM_NO_NETCORE Make.local conditional to disable non-netcore builds during development.

* [wasm] Add a --simd argument to the packager.

* Remove some duplicate defines.

* Fix an windows assertion.
2019-12-02 07:21:17 +01:00
campersau b0f0b14023 [wasm][bcl][http] Fix descriptive message for the PlatformNotSupportedException (#17919)
* Fix descriptive message for the `PlatformNotSupportedException`

* Add more descriptive message to the `Properties` `PlatformNotSupportedException`
2019-12-02 05:20:15 +01:00
EgorBo d33ce1c3ec enable only for netcore 2019-12-01 17:38:33 -06:00
EgorBo 376d3acb29 fix failing tests 2019-12-01 17:38:33 -06:00
EgorBo b77f757679 fix build 2019-12-01 17:38:33 -06:00
EgorBo 1b7c7e6075 Address feedback and fix C++ warnings 2019-12-01 17:38:33 -06:00
EgorBo e2241969f5 Optimize box(X) == null to false 2019-12-01 17:38:33 -06:00
Jay Krell fffeb23239 sockets: Remove piecemeal ifndef ENABLE_NETCORE since the file has it broadly. 2019-12-01 17:33:24 -06:00
Zoltan Varga ae4b678b94 [wasm] Avoid rebuilding corelib in build-wasm-bcl-netcore. (#17969) 2019-11-30 03:18:42 +01:00
Bernhard Urban-Forster 4d0feffa42 [interp] track valuetype stack for CEE_RET too (#17968)
[interp] track valuetype stack for CEE_RET too

Usually the stack depth is determined via basic block predecessors, but  that only works for forward branches.

Fixes https://github.com/mono/mono/issues/17924
2019-11-30 00:06:05 +01:00
Zoltan Varga 6c0bfdc3f3 [wasm] Add more netcore test suites. (#17965)
* [wasm] Handle more exceptions in the xunit runner.

* [wasm] Add more corefx test suites.

* Add more icall signatures.

* Add support for running the System.Compression test suites.
2019-11-29 18:17:40 +01:00
Zoltan Varga d1735a8356 Interpreter fixes. (#17966)
Interpreter fixes.
2019-11-29 16:26:05 +01:00
Ankit Jain c92a96ccb5 Bump msbuild to track xplat-master (#17950)
* Bump msbuild to track xplat-master

* Bump nuget.exe to 5.4.0
2019-11-29 12:01:06 +01:00
Egor Bogatov 9a18b6fd6f [netcore] Improve Jitdiff tool (#17960)
use `objdump` arguments to reduce verbosity (no raw bytes, etc) and make the dasm more diff-friendly.
On Linux all functions have a random prefix (address) - replace it with `0xD1FFAB1E` constant). On macOS it's fine by default.

Also, allow to generate diffs for a single assembly via `dump-asm-lib` rule.
2019-11-29 11:54:01 +01:00
Kenneth Pouncey 608c93643e [wasm][http] Fix race condition as a result of using TaskCompletionSource: RangeError: Maximum call stack size on Safari (#17946)
There is a race condition on Safari as a result of using TaskCompletionSource that causes a stack exceeded error being thrown during Blazor startup.

Safari throws the following error:

```
[Error] Unhandled Promise Rejection: RangeError: Maximum call stack size exceeded.
    step (blazor.webassembly.js:2530)
    fulfilled (blazor.webassembly.js:2500)
    promiseReactionJob

```

- Modify `WebAssemblyHttpMessageHandler.cs` to create the TaskCompletionSource to run continuations asynchronously.
   - `TaskCompletionSource<HttpResponseMessage>(TaskCreationOptions.RunContinuationsAsynchronously);`

- There is a second part to this on AspNetCore in `Renderer.cs` to process it's pending startup tasks to run continuations asynchronously to compliment the TaskCompetionSource of the Http response message.

The following link discusses the potential stack/race problem that can occur when using TaskCompletionSource:  https://devblogs.microsoft.com/premier-developer/the-danger-of-taskcompletionsourcet-class/
2019-11-29 07:14:19 +01:00
Zoltan Varga 59a9af6d7c Revert "Use range metadata (#17956)" (#17958)
This reverts commit fde058518d.
2019-11-28 21:52:01 +01:00
Egor Bogatov fde058518d Use range metadata (#17956) 2019-11-28 14:26:35 +01:00
Kenneth Pouncey d6dea097ca [wasm] Marshal null string correctly. (#17955) 2019-11-28 07:31:59 +01:00
Zoltan Varga 79db9b2a87 [interp] Make calli non recursive as well. (#17953) 2019-11-28 06:26:36 +01:00
EgorBo 6360ccbfe3 Add quick diff 2019-11-27 19:15:56 -06:00
EgorBo e333f13710 Add jit-diff tools 2019-11-27 19:15:56 -06:00
Ryan Lucia 9c1cae02ca Change signature, add cast 2019-11-27 19:13:30 -06:00
Ryan Lucia 0c3036abc7 [mini] Move bundled native library loading into the runtime 2019-11-27 19:13:30 -06:00
Jay Krell b2817e6404 [cxx] MonoJitICallId same between C and C++.
This is optional.
The real problem was -enable-cxx not actually compiling some files as C++.
This is still reasonable, to ward off the danger of that occuring again.
2019-11-27 19:12:32 -06:00
Jay Krell c3da0ad518 Update netcore. 2019-11-27 19:11:35 -06:00
Jay Krell 4a15e73429 Remove handles/MonoError from Mono.RuntimeMarshal and Mono.SafeStringMarshal. 2019-11-27 19:11:35 -06:00
Nikita Voronchev 7db2389846 [WinForms][UIA] Add to the PropertyGrid new internal event to track grid items expanded state update (#17731)
This PR is needed for the https://github.com/mono/uia2atk/pull/6 one.
2019-11-27 23:59:04 +01:00
abrevet-dev f135312b44 [WinForms] DrawToBitmap() did not draw children controls (#17676)
Fixes #13777
2019-11-27 23:38:04 +01:00
Sergey Podobry abc75476d7 [Mono.Security] Do not decode data beyond detected length in ASN1 parser (#17751)
If `byte[] data` has additional data at the end `Decode` will try to parse it and will generate incorrect tags or throw an exception. This fix makes `Decode` to use only `nLength` bytes from `byte[] data`.
2019-11-27 23:37:20 +01:00
Aleksey Kliger (λgeek) b6a351c330 [threadpool] Decrement max_working when worker times out (#17927)
[threadpool] Decrement max_working when worker times out

The corresponding code in CoreCLR:
https://github.com/dotnet/runtime/blob/bf88f146412d1e2d41779422337184a802f186c9/src/coreclr/src/vm/win32threadpool.cpp#L2156-L2164

The issue is that counter._.max_working is only ever changed by:
1. the monitor_thread when it detects starvation (increment by 1).
2. the hill climbing algorithm (increment or decrement).

Creating a periodic load of creating many workers at once followed by a minute
or more of quiescence increases the variable worker.counters._.max_working in
monitor_thread(). Once that counter matches worker.limit_worker_max,
monitor_thread() keeps looping without unparking or creating a thread, even
though the actual number of threads/workers is small. If the existing threads
are all waiting on work that needs a new thread, then a deadlock occurs.

With this change, when a parked worker times out, it will lower max_working to
the number of active (working + parked + starting) threads minus itself (but no
less than limit_worker_min). As a result, monitor_thread will only increment
max_working as long as none of the already running workers are timing out.

Attempt to address https://github.com/mono/mono/issues/17833

---

Also allow monitor_thread to start workers if there aren't enough active, even if the max worker limit has been reached.

Work around a pathological condition where the work_item_count is non-zero, and
the max_working limit has been reached, but the number of active threads is
still below the max.  In that case, unpark some workers and start some new
threads.

May address http://work.azdo.io/827206
2019-11-27 22:06:17 +01:00
Jay Krell 3e369ba8c9 Cleanup initialize_object_slots, remove unused GetHashCode (ghc) handling. (#17834)
Fix possible race conditions involving multiple writes to globals, by
removing one of the two globals involved in finalize.
2019-11-27 14:25:14 -05:00
abrevet-dev 6145a397d6 [WinForms] Refactor selected indexes in TabControl.Remove (#17608)
Fixes #12337
2019-11-27 19:55:00 +01:00
abrevet-dev 5944e72d72 [WinForms] Returns real installed input languages on Windows (#17653)
The method InstalledInputLanguages() always returned InvariantCulture. Now, installed input languages are detected properly on Windows.

Reproducer:

`Console.WriteLine(InputLanguage.CurrentInputLanguage.Culture.TwoLetterISOLanguageName.ToUpper());`

Related issue: #13552
2019-11-27 19:46:04 +01:00
abrevet-dev 41ba5846b3 [WinForms] DefaultCellStyle was not cloned deeply in DataGridViewRow.Clone() (#17755)
Fixes #16557
2019-11-27 19:41:15 +01:00
HebaruSan d62eed676e [WinForms] Fix X11 theming (#17888)
## Background

In Mono 5 and earlier, the active GTK or KDE theme would be sourced for system colors, allowing dark themes to be obeyed:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

This allowed WinForms apps to blend in and look more like native apps.

## Problem

In Mono 6, this capability was lost (as noted in passing in #14325). WinForms apps now only look like this, regardless of the system color settings:

![image](https://user-images.githubusercontent.com/1559108/69486114-dccce280-0e0d-11ea-8107-fe1653b2d816.png)

This looks somewhat jarring and worse than previous Mono releases, for users with a dark system theme.

## Cause

During load of the system settings (in the static constructor of `X11DesktopColors`), `Theme.SetSystemColors` passes system color data to `System.Drawing.KnownColors.Update`, which populates them into `System.Drawing.KnownColors.ArgbValues`. In Mono 5 and earlier, this flowed through to the visible UI.

It seems that as of Mono 6, this table no longer affects the visible WinForms UI (although it is still being updated!). Instead, a same-but-different table in `System.Drawing.KnownColorTable` drives the system colors now. I was not able to identify the commit in which this change was made.

## Changes

Now `Theme.SetSystemColors` accesses `System.Drawing.KnownColorTable.s_colorTable` instead, and WinForms apps once again obey GTK and KDE themes:

![image](https://user-images.githubusercontent.com/1559108/69486043-f3bf0500-0e0c-11ea-9b29-1654ba4e6d9a.png)

Fixes KSP-CKAN/CKAN#2881.
2019-11-27 19:13:11 +01:00
RizvanKuliev a73e7b8af2 [WinForms] Fix Recalculate in ScrollableControl (#17163) 2019-11-27 19:09:19 +01:00
Alexander Köplinger 9954579c2a [sdks] Call mono_gc_init_finalizer_thread() on iOS device runtimes
These runtimes are configured with `--with-lazy-gc-thread-creation=yes` so they don't start a GC finalizer threads by themselves.
XI does this because they want to delay creating that thread for some time after startup.

We can just do it immediately, the important part is that it _does_ exist.
2019-11-27 18:36:02 +01:00
Alexander Köplinger c66c7814d0 [sdks] Add option to enable debug server for iOS test app
Include .pdb files in iOS test app if the DEBUG variable is set and wait for managed debugger to attach.
2019-11-27 18:36:02 +01:00
Alexander Köplinger 18ddc2e45d [sdks] Fix loading of assemblies in assembly_preload_hook()
We need to check whether the assembly name passed to us already contains .exe/.dll and append it otherwise.
Fixes loading of assemblies via the hook, we were falling back to the usual path resolving logic (which doesn't work in netcore mode).
2019-11-27 18:36:02 +01:00
Alexander Köplinger a0d79a4886 [sdks] Add netcore option for iOS SDK 2019-11-27 18:36:02 +01:00
Alexis Christoforides 167913524f [merp] Use timeline (and generate timeline breadcrumbs) for SendExceptionToTelemetry() (#17913)
Fixes https://github.com/mono/mono/issues/17790
2019-11-27 11:56:12 -05:00
Bernhard Urban-Forster 4c595069e7 [arm] if mtriple is provided, do not set -march (#17937)
[arm] if mtriple is provided, do not set -march

E.g. when passing `-march=arm -mtriple=armv7s-ios`, `-march` will win and thus ignore `armv7s`.

Fixes https://github.com/mono/mono/issues/17931

* Caused by 5e318afd2e
* Related with 4cd5068230
* Almost fixed by e431093f73

Hopefully we are done with it.
2019-11-27 17:36:03 +01:00
monojenkins 9dfeb545d5 [bcl] Update BCL Linked Size (#17322) 2019-11-27 13:43:31 +01:00
Bernhard Urban-Forster 690af29ad9 [amd64] do not stack allocate on the application stack for the transtion from alstack handling (#17922)
### Backstory
This test is failing in the interpreter (ONLY on Debian9/amd64. Not Ubuntu/amd64. Not macOS/amd64):
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mcs/class/Mono.Debugger.Soft/Test/dtest.cs#L2484-L2499
Where the debuggee is just some unsafe code that tries to read from unmapped memory, thus causing a segfault.

So the test expects the runtime to send a CRASH event via the managed debugger interface. This happens very late in the crash handling machinery:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L1105

However, the runtime would never reach that, because it silently crashes when calling `backtrace`:
https://github.com/mono/mono/blob/0fed03ed63ed4ea742c4511d8edc3bc1c6f4044f/mono/mini/mini-posix.c#L939

At the point where the SIGSEGV happens, the native stack trace looks like this:
```
Thread 1 "mono-sgen" received signal SIGSEGV, Segmentation fault.
interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
4463                            sp[-1].data.i = *(gint32*)sp[-1].data.p;
(gdb) bt
%0  interp_exec_method_full (frame=0x555555c51f18, context=0x555555c38350, clause_args=clause_args@entry=0x0, error=0x555555c53f18, error@entry=0x7fffffffd8b0) at interp/interp.c:4463
%1  0x000055555569237a in interp_runtime_invoke (method=<optimized out>, obj=<optimized out>, params=<optimized out>, exc=0x0, error=0x7fffffffd8b0) at interp/interp.c:1900
%2  0x0000555555590745 in mono_jit_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=0x7fffffffd858, exc=0x0, error=0x7fffffffd8b0) at mini-runtime.c:3001
%3  0x0000555555778c4c in do_runtime_invoke (method=0x555555bfdc38, obj=<optimized out>, params=<optimized out>, exc=<optimized out>, error=0x7fffffffd8b0) at object.c:3052
%4  0x000055555577b7d0 in do_exec_main_checked (method=0x555555bfdc38, args=<optimized out>, error=0x7fffffffd8b0) at object.c:5184
%5  0x000055555559a2aa in mono_jit_exec_internal (argv=0x7fffffffdcb8, argc=1, assembly=0x0, domain=0x555555bf9d40) at driver.c:1320
%6  mono_jit_exec (domain=domain@entry=0x555555bf9d40, assembly=assembly@entry=0x555555c66ae0, argc=argc@entry=1, argv=argv@entry=0x7fffffffdcb8) at driver.c:1265
%7  0x000055555559b8e2 in main_thread_handler (user_data=<synthetic pointer>) at driver.c:1402
%8  mono_main (argc=<optimized out>, argv=<optimized out>) at driver.c:2622
%9  0x000055555558b757 in mono_main_with_options (argv=<optimized out>, argc=<optimized out>) at main.c:52
%10 main (argc=<optimized out>, argv=<optimized out>) at main.c:434
```
Stepping into the crash handling machinery, the picture looks a bit different:
```
Thread 1 "mono-sgen" hit Breakpoint 1, altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
871     {
(gdb) bt
%0  altstack_handle_and_restore (ctx=0x7fffffffd2d0, obj=0x0, flags=0) at exceptions-amd64.c:871
%1  0x00005555556957de in interp_exec_method_full (frame=0x555555c53f08, context=0x555555c38350, clause_args=0x0, error=0x555555c53f18) at interp/interp.c:4461
%2  0x0000000000000000 in ?? ()
```
So it looks like we messed up in the transition from altstack back to the program stack.

 ### The fix

The problem is that `backtrace` consults the unwind information emitted at the point where SIGSEGV happens in `interp_exec_method_full ()`. However at that given program code, there is no function call that allocates space to pass arguments on the stack. Still, we massage the stack to do that in the signal handler, which does not add up with the emitted unwind information by the C compiler.

Note that without the fix, crashing in `backtrace` is the worst case. In other cases we might (1) get still a correct stack trace, or (2) we get a wrong stack trace, and `backtrace` will not crash. So this PR might improve the crash experience overall.

This also reverts commit cff400f3fae9775ac734f8a691339e59a272872c.
2019-11-27 12:20:47 +01:00
Zoltan Varga c4136ebd94 [interp] Non-recursive interpreter (#16461)
* Aling alloca_size to 8.

* Prevent some false pinning in the monitor-resurrection.cs test when using the non-recursive interpreter.

* [runtime] Add a mono_compiler_barrier () function.

* Add a MONO_GET_SP opcode to pass to enter/exit_gc_safe_region, ldloca might not return a native stack address with the interpreter.

* Add an interpreter/gc callback to mark the interpreter stack.

* Copy only an integer in ENDFILTER.

* Add a native_stack_addr field to InterpFrame so the EH code has something to compare against native stack addresses.

* Add a FrameStack structure to allow allocation of InterpFrame structures and stack data outside the native stack.

* Allocate InterpFrames/stack data from a frame stack instead of the native stack.

* Make some calls non-recursive.

* Remove an assert.

* Use non-recursive calls in CALLVIRT_FAST as well.

* Fix stack size allocation for vararg calls.

* Add some micro optimizations to the calling code.

* Add more micro optimizations.

* [dtest] disable Crash test on interpreter

* Add some implicit conversions to STSFLD to fix a corefx test failure.

* Disable the new test.
2019-11-26 04:31:32 +01:00
Zoltan Varga 5b4cd5ce50 WASM + Netcore. (#17915)
* [runtime] Throw an exception from Thread::StartInternal () if threads are disabled.

* Add more test suites and a corefx exclusions file.
2019-11-26 02:59:31 +01:00
Larry Ewing e917c95053 Add the reference assembly path for the Http implementation (#17908) 2019-11-25 19:21:38 -06:00
dotnet-maestro[bot] cd190481bd Update dependencies from https://github.com/dotnet/arcade build 20191124.1 (#17899)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19574.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19574.1
2019-11-25 21:22:41 +01:00
Alexander Köplinger e437330808 Remove mkinstalldirs invocation from two places (#17910)
These can be easily replaced so we don't need to mirror over the mkinstalldirs file to the new repo
2019-11-25 21:10:26 +01:00
Alexander Köplinger 8aa1cdfe17 [configure.ac] Don't run compiler server check in netcore mode (#17911)
We don't need it there.
2019-11-25 21:10:08 +01:00
Calvin Buckley 836f453906 Fix BinaryReader on BE (#17893)
Fix BinaryReader on BE

The ints passed into the int array ctor needed to be endian swapped.
Interestingly, the BinaryWriter implementation currently being used
(not refsrc AFAIK) does this on write just fine.

Fixes ReadDecimal test on AIX, and adds another test to ensure that
Write(decimal) can be read back by ReadDecimal in case.
2019-11-25 19:15:55 +01:00
Alexander Köplinger bdf4790fa0 Fix boehm on Windows i386 builds (#17901)
After https://github.com/mono/mono/pull/16832 was merged the run-msbuild.bat
script has an additional required parameter for specifying the GC.

However the script only passed the MONO_TARGET_GC property to msbuild
if the script had no additional parameters which doesn't make sense.

Always pass the parameter, this should fix the boehm build on Windows i386.
2019-11-25 18:58:42 +01:00
Larry Ewing b42845c567 Fix issue in debugger variable inspection (#17900) 2019-11-25 11:55:21 -06:00
Marek Habersack 10795da1c0 [WIP] Make debugger agent startup faster (#17898)
Mono SDB's debugger agent takes a `host:port` parameter to specify the host and
port on which to listen for debugger connection. `host` is passed, via
`mono_get_address_info`, to the `getaddrinfo(3)` standard library call which
does the job of translating the name to the IP address. However, there's no
special treatment for situations when `host` is already an IP address. In this
case `getaddrinfo` takes a lot of time to perform at least two DNS lookups (if
there's just a single DNS server configured) which will time out as the IP
address in `host` will not be resolved by any of the servers.

`getaddrinfo` takes a "hints" parameter where caller can specify some
information about the `host:port` pair to make the lookup faster/easier.
However, Mono doesn't specify the address family in the `hints` parameter, thus
forcing the slow path of DNS lookups on `getaddrinfo`.

This commit adds a new hint named `MONO_HINT_NUMERIC_HOST` (which maps to
`AI_NUMERICHOST` for `getaddrinfo`) which will allow, if set, `getaddrinfo` to
take a faster route in address discovery. Debugger agent will, in most cases,
get an IP address to instead of host name so we call `mono_get_address_info` up
to 3 times (IPv4, IPv6 and unspecified) in order to make the lookup faster.
2019-11-25 17:52:58 +01:00
Calvin Buckley f47b7bbf4f Fix flawed BufferTest on big-endian (#17894)
Fix flawed BufferTest on big-endian

Test ignored byte order when considering integer comparison. This
tests different cases for each endian, and explains why the byte
order is different.
2019-11-25 16:25:55 +01:00
Kenneth Pouncey 5d981c2679 [wasm] [bindings] Method invoke signature argument validation. (#17853)
* Implement validation for signature and argument count consistency.

- throw error `Parameter count mismatch.` when arguments and signature counts are not consistent.

* Fix tests that have Parameter count mismatch.

* Add browser binding tests for `Parameter count mismatch
2019-11-25 08:24:33 -06:00
Kenneth Pouncey e8d70c333f Add tests for Promise library.
- Right now only Bluebird promise library is tested right now.
- If more are needed later then we can add them.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 2f1e2f8132 Add a more reliable way of identifying a Promise to be marshaled.
- This is left as generic as possible for now.
2019-11-25 08:23:45 -06:00
Kenneth Pouncey 64e2864fef Add a more descriptive error in case of fetch response not returning back what it should.
- An example of this is when a Promise library is being used and the Promise can not be identified correctly.  Thus the internal error.
2019-11-25 08:23:45 -06:00
Vlad Brezae 1ef7a29836 [interp] Resolve tokens also from wrappers during optimization (#17880)
[interp] Resolve tokens also from wrappers during optimization

Remove some duplicated code.
2019-11-25 14:45:56 +01:00
Calvin Buckley 004fd06b96 [eglib] In AIX/generic POSIX, handle null addresses better (#17892)
Fixes crash in tests on AIX.

Matches b0d966bd7a
2019-11-25 02:52:54 -05:00
Calvin Buckley a3f4def8b6 Allow runtime to be built with C++ on AIX (#17672)
* Allow runtime to be built with C++ on AIX

Many AIX/PPC/BE specific codepaths didn't do casting properly,
since C++ is much stricter than C about pointer typing.

Also specify the C99 format macros early as possible, since
inttypes may get included before eglib gets a chance to set the
macros.

Also explicitly use -pthread, since not using it is the cause of
many libstdc++ crashes on AIX.

* Oops, should have been that define instead

* Suggested changes from Jay on style

* Remove unneeded cast
2019-11-24 17:08:19 -05:00
Alexander Köplinger 280e9d2423 Fix warnings in runtime build (#17887)
This makes at least the netcore-mono build in dotnet/runtime warning-free.
2019-11-24 00:16:20 +01:00
Alexander Köplinger 165e28ae97 [netcore] Disable libmonoruntime-support.la build on netcore (#17885)
It's used for the zlib compression helpers used by System.IO.Compression in the old Mono BCL.
On corefx this is handled differently so we don't need it there.

To avoid automake complaining about missing files we include the .c files directly instead of in the Makefile.
2019-11-23 10:06:40 +01:00
Alexander Köplinger 2ccd6122bd [netcore] Disable a test which fails on the interpreter
The test is wrong, will be fixed upstream.
2019-11-22 23:00:28 +01:00
Alexander Köplinger 81e305421d Fix run-msbuild.bat invocation in pipeline-netcore-runtime.yml 2019-11-22 22:48:53 +01:00
Alexander Köplinger 0d80372173 Make configure.ac and Makefile.am work without certain directories (#17871)
Those folders won't be present in the dotnet/runtime repository.

We need to factor out AC_OUTPUT into a separate file so we can continue
to share configure.ac but not the list of files in AC_OUTPUT since
autotools doesn't support variables there.
2019-11-22 22:48:11 +01:00
Zoltan Varga 93e46c687b [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case. (#17832)
* [runtime] Treat calling a non-virtual method through an open delegate the same as the normal case instead of the OPEN_VIRTUAL case.

* [amd64] Fix the return value mapping in gsharedvt calls, the return address might be in different registers in the caller and callee.

* [runtime] Always initialize del->method_ptr even if interp_method is set, JITted code makes calls through it in mixed mode.

* Add a test for open delegates and gsharedvt.

* Increase nrgctx trampoline count, its needed by dynamic-method-churn.exe in full-aot-interp mode.
2019-11-22 20:34:14 +01:00
Jonathan Chambers cc137237d8 Replace embedded libgc with Unity fork of recent Boehm (bdwgc) (#16832)
* Initial commit to migrate libgc to bdwgc.

* Fix submodule url

* Don't force Boehm as runtime wrapper. Was just for testing locally.

* Build bdwgc as single object file. Helps with performance.

* Bump bdwgc submodule.

* Attempt to fix windows MSVC based build.

* Remove any references to GC_INSIDE_DLL as we manually managed threads on Windows.

* Allow building boehm on Windows amd64.

* Allow building boehm with msvc.

* Bump bdwgc to fix line endings

* Use LF for sh, am, m4, and ac files in bdwgc

* Bump bdwgc & libatomic_ops with fix for cygwin

* Bump bdwgc to fix mono-boehm being generated as libtool wrapper script.

* Link the static boehm library.

* Revert "Link the static boehm library."

This reverts commit 1f90d81d08308a61c890743cd79d8f6bb791bc99.

* Use --export-all-symbols on cygwin/mingw to fix issues with boehm causing mono symbols not to be exported.

* Fix
2019-11-22 18:03:23 +01:00
Filip Navara 0fed03ed63 Fix InternalGetHashCode for boehm/null GC (#17859)
It was unintentionally shifting the address to left instead of right. It should be shift right to get rid of zeroes from the alignment.
2019-11-22 07:51:43 -05:00
Alexander Köplinger 5fedc42ccf Fix automake warning 2019-11-22 13:18:10 +01:00
Kenneth Pouncey 9e1d59b8cc Remove unused GetHttpMessageHandler from WebAssembly.Net.Http.
- Remove linker.xml from project
2019-11-22 13:12:17 +01:00
Zoltan Varga ecd67a91c4 [wasm] Add aot+netcore support. (#17798)
* [wasm] Compile corelib as 32 bit.

* [wasm] Add a netcore cross compiler.

* [wasm] Add netcore+aot support.

* [aot] Load the dedup module lazily instead of at startup.

The previous version didn't work on netcore because it couldn't resolve references to corlib etc.
since they were not yet registered in the root domain at the time of loading.

* Add more icall trampoline signatures.

* Add more netcore pinvokes.

* Fix the build.

* Add more icall trampoline signatures.

* Add an --embed-file option to the packager.

* Add more netcore pinvokes.

* [interp] Return FALSE for IsDynamicCodeCompiled.

* Add a netcore project and targets for running BenchmarkDotNet.

* Fix pinvoke tables.

* Add missing files.

* Fix make clean.

* Add a comment.

* Revert "Fix pinvoke tables."

This reverts commit 998065e77614b7e621d2873510aabc3930669ca9.

* Make xunit-runner into a dotnet project. Add a netcore version.
2019-11-22 12:42:55 +01:00
Egor Bogatov 6a5c44b15e [netcore] Run tests with Interpreter (#17862)
* Run tests with interp and llvm

* fix build

* Run tests with llvm

* Add a few tests to rsp for LLVM
2019-11-22 09:14:23 +01:00
Kenneth Pouncey 202cb6a648 Add more descriptive message to the PlatformNotSupportedException (#17864) 2019-11-22 08:40:30 +01:00
Jay Krell dadc3d3117 Remove handles/MonoError from Mono.RuntimeGPtrArrayHandle. (#17806) 2019-11-21 14:15:29 -05:00
Jo Shields 9e00330af3 Stop referencing files from Mono BCL in Netcore System.Private.CoreLib (#17866) 2019-11-21 19:48:00 +01:00
Bernhard Urban-Forster 058d157648 [interp] fix array_element_size intrinsic (#17857)
[interp] fix array_element_size intrinsic

`mono_class_array_element_size` gives us the size of the provided MonoClass if it would be an array element. But here we want the element size of a given array's MonoClass. That's what `mono_array_element_size` is returning.

This leads to all kind of weird crashes otherwise, specifically here:
https://github.com/mono/mono/blob/2c20649539ac16e069a65f2a750c793eb341e50f/netcore/System.Private.CoreLib/src/System/Array.Mono.cs#L64

Which is then used later to compute the size to memset in order to clear the content of an array. If it's larger than it should be, then this will cause memory corruption. For example this would crash eventually:

```csharp
using System;
using System.Collections.Generic;

namespace HelloWorld {
    class Program {
        static void Main(string[] args) {
            for (int j = 0; j < 0x1000; j++) {
                var d = new Dictionary<string, string> ();
                for (int i = 0; i < 197; i++)
                    d.Add (i + "", i + "foo");
                d.Clear ();
            }
        }
    }
}
```

Thanks to @EgorBo for reporting.
2019-11-21 11:56:03 +01:00
Alexander Köplinger b6e4ddd73c [metadata] Remove native threadpool from netcore build (#17856)
We use the managed threadpool there.
2019-11-21 10:01:39 +01:00
Egor Bogatov 2c20649539 [llvm] Recognize GEP (#17844)
* initial impl

* improve jit

* fix copy-paste

* disable for non-netcore

* clean up
2019-11-20 21:25:24 +01:00
Zoltan Varga 4961d411e7 [interp] Move the freeing of interp method info into a callback. (#17852)
[interp] Move the freeing of interp method info into a callback.
2019-11-20 19:56:09 +01:00
Egor Bogatov 27e6e812f1 enable gss on android (#17838) 2019-11-20 10:53:14 -05:00
Alexander Köplinger 0396037d31 [debugger-agent] Fix CMD_VM_ALL_THREADS returning wrong value due to unitialized variable (#17847)
`remove_gc_finalizing` wasn't initialized so its value is undefined.
The `count_thread_check_gc_finalizer()` function only sets it in some cases so we could end up with the variable having an undefined (and probably not 0) value.
This resulted in the returned thread count from the debuggee being wrong because we'd decrement `count` even though we shouldn't.

Regression from https://github.com/mono/mono/pull/15618
2019-11-20 08:28:58 +01:00
dotnet-maestro[bot] a6db160596 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17690)
* Update dependencies from https://github.com/dotnet/arcade build 20191102.1

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19552.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19552.1

* Update dependencies from https://github.com/dotnet/core-setup build 20191104.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19554.1

* Update dependencies from https://github.com/dotnet/corefx build 20191103.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19553.8

* Update dependencies from https://github.com/dotnet/arcade build 20191108.11

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19558.11
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19558.11

* Update dependencies from https://github.com/dotnet/core-setup build 20191109.5

- Microsoft.NETCore.App - 5.0.0-alpha.1.19559.5

* Update dependencies from https://github.com/dotnet/corefx build 20191109.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19559.6

* Update dependencies from https://github.com/dotnet/arcade build 20191117.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19567.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19567.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191114.1

- Microsoft.NETCore.App - 5.0.0-alpha.1.19564.1

* Update dependencies from https://github.com/dotnet/corefx build 20191113.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19563.6

* Use NETCoreApp version 5.0.0-alpha.1.19563.3

Later versions have an issue and the new dotnet/runtime assets aren't ready yet.

* Disable failing tests

* Fix merge error
2019-11-19 16:47:57 +01:00
Steve Pfister 0c3ffb9cc9 Fixes https://github.com/mono/mono/issues/15805 at least in terms of what may be making it crash. (#17270)
Adds a null check around the sslStream before trying to dispose.
2019-11-19 14:23:51 +01:00
Zoltan Varga b8fe43da3e [interp] Add some missing netcore intrinsics. (#17782) 2019-11-18 13:55:15 +01:00
Steve Pfister 0269f7b479 Bump corefx to pick up https://github.com/mono/corefx/pull/370 (#17758)
Backport of https://github.com/dotnet/corefx/pull/34560

Added HAVE_LCHFLAGS & HAVE_STAT_FLAGS to configure.
2019-11-18 12:13:16 +01:00
Filip Navara e2ccdac3f4 [llvm] Add support for LLVM JIT on ARM64 (#17827)
* [llvm] Add support for LLVM JIT on ARM64

* Fix tabs
2019-11-18 00:24:11 +01:00
Jérôme Laban 965efade84 Update emscripten to 1.39.3 (#17816)
Includes updates for https://github.com/emscripten-core/emscripten/issues/9562 and side modules.
2019-11-16 13:59:28 +01:00
Dotnet-GitSync-Bot 6785b27567 Mirror changes from mono/runtime (#17803)
It looks like the mirror didn't fully quiesce prior to snapping coreclr/corefx for the runtime consolidation.  As a result the libraries build is breaking.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-16 11:27:32 +01:00
Nikolay Sivov 18dd48cad8 [mini] Fix NULL pointer crash in mono_decompose_vtype_opts(). (#17801)
* [mini] Fix NULL pointer crash in mono_decompose_vtype_opts().

All other call sites for mono_type_size() are already passing non-null pointer there.

* [mini] Rename variable dummy -> align.
2019-11-15 17:56:55 +01:00
Marcos Cobeña Morián 92dd4d3883 [Wasm] Forced filesystem creation (#17789)
* Forced filesystem creation

* [Wasm] Added new hello-io sample
- Goal is testing file preloading through System.IO
- Filesystem is now always forced to support later file preloading
- build-io-sample target showcases such

* Added --enable-fs flag to packager.exe

* Update packager.md
2019-11-15 15:24:50 +01:00
Jan Kotas 3ceecb8db0 Fix build breaks
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Carlos Sanchez Lopez d3fbf4a1d3 Fix NetStandard issue in System.IO.FileSystem.AccessControl in 5.0 (#42546)
Fix NetStandard issue in System.IO.FileSystem.AccessControl

We built the fix incorrectly so that it would only apply for netcoreapp3.0 but customer needs it on netstandard2.0 (desktop + core).  Customer impact is that they’ll see a PlatformNotSupported exception rather than the new API.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 00b0170bdc Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#42531)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Tanner Gooding c93801d458 Adding tests for the new apis to convert between S.Numerics and S.R.Intrinsic vectors (#27481)
* Updating Vector128.AsVector3 to have a functioning implementation

* Adding test template metadata covering the new methods for converting between S.Numerics and S.R.Intrinsics

* Regenerating the existing tests

* Generating the new Vector conversion tests

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Emmanuel AndréandStephen Toub 5645032fc6 Add span-based CopyTo and CopyToAsync methods (dotnet/coreclr#27639)
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <[email protected]>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
John adcaf88db0 Fix Vector128 and Vector256 WithUpper documentation (dotnet/coreclr#27274)
* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector128.cs

* Update Vector256.cs

* Update Vector128.cs

* Update Vector256.cs

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 1f2f9b929b Add tracing for handling of dependencies for Assembly.LoadFrom (#27833)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Elinor Fung 3242b5f354 Add tracing for Resolving/AssemblyResolve event handler invocation (#27788)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
ts2do 35d94dc8ff Address bugs in BigInteger (dotnet/coreclr#27280)
* Method Add(ref BigInteger lhs, uint value, ref BigInteger result) would store most of the result blocks into lhs instead of result.
* Method ShiftLeft(ulong input, uint shift, ref BigInteger output) with a shift argument exceeding 32 would generally compute the higher blocks incorrectly.
* Multiply(ref BigInteger lhs, uint value, ref BigInteger result) would not set result._length in some cases.
* IsZero() would incorrectly return false for non-canonical zeros with _length > 0.

Fix:
* Inline Add(ref BigInteger, uint, ref BigInteger) into Add(uint).
* Inline ShiftLeft(ulong, uint, ref BigInteger) into Pow2.
* Inline ExtendBlock and ExtendBlocks into Pow2.
* Properly handle 0 in SetUInt32 and SetUInt64.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Stephen Toub 2a108e958f Use Stream's BeginReadInternal from FileStream (dotnet/coreclr#27737)
FileStream has two modes, decided upon at construction time.  When it's created in non-async mode, the Read/WriteAsync methods end up queueing work items to invoke the synchronous Read/Write methods.  To do this, the base methods on Stream delegate to Begin/EndRead/Write (since they were around first) and then the resulting IAsyncResult is wrapped.  However, Stream has an optimization that checks to see whether the derived stream actually overrides Begin/EndXx, and if it doesn't, then it skips using those and goes straight to queueing a work item to Read/Write.  However, FileStream does override those, but when it delegates to the base implementation because it's in non-async mode (rather than because it's a type derived from FileStream), going through Begin/EndXx is just unnecessary overhead.  So, in the right circumstances, we can call to Stream's special helper instead.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-15 13:29:04 +01:00
Zoltan Varga 25b7e6bd42 [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files. (#17795)
* [runtime] Add MONO_EMPTY_SOURCE_FILE to a couple of files.

* Fix a typo.

* Revert unrelated changes.
2019-11-15 09:04:35 +01:00
Vsevolod Kukol 805c7f5436 [MacSDK] Bump xamarin-gtk-theme.py to latest revision from private bockbuild (#17785) 2019-11-14 09:27:33 +01:00
Jay Krell 19a44cd32b Remove handles from System.Diagnostics.Debugger. (#17713)
* Remove handles from System.Diagnostics.Debugger.
memset (&x, 0, sizeof (type)) to memset (&x, 0, sizeof (x)).

* Bump API snapshot submodule
2019-11-14 08:18:02 +01:00
Zoltan Varga 1d81534d27 [interp] Add some inline checks from the JIT. (#17784) 2019-11-13 22:34:01 +01:00
Zoltan Varga a0fcc29d5d [runtime] Fix some handle usage in native-library.c (#17781) 2019-11-13 18:12:09 +01:00
Jay Krell 73431403c7 Remove handles from ves_icall_System_Array_CanChangePrimitive. (#17708)
Remove handles from ves_icall_System_Array_CanChangePrimitive.
2019-11-13 17:55:52 +01:00
Thays Grazia dd18ec40dd [debugger] Assert when async debug a generic method (#17727)
* When we try to call a method to get the async_id to do an async debug and we are trying to do this in a generic method like this:

async Task<T> ExecuteAsync_Broken<T>()
 {
            await Task.Delay(2);
            return default;
  }

We need to inflate the generic type before call the method or we will get the error:  Could not execute the method because the containing type 'System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[T_REF]’, is not fully instantiated.

Fixes #17549
Fixes #17569
2019-11-13 16:29:33 +01:00
imhameed fe0f824db5 [llvm] Use explicit null checks with LLVM. (#16954)
* [llvm] Use explicit null checks with LLVM.

When the LLVM backend is used for a method, explicit null checks will
unconditionally be used at the mini IR level. We had previously used
implicit null checks at the mini IR level combined with llvm-volatile
loads and stores to avoid mis-optimization of the generated code.

During AOT compilation, LLVM's `ImplicitNullChecks` pass is enabled.
This conservatively elides these explicit null checks. This pass only
modifies branches tagged with `make.implicit` metadata.

Null check branches are now tagged with `make.implicit` metadata, with
three exceptions:

1. For platforms with backends that do not support zero page fault
handling, null check branches are not tagged with `make.implicit`.

2. This commit also adds a `MONO_DEBUG` option:
`llvm-disable-implicit-null-checks`. When it is enabled, null check
branches are not tagged with `make.implicit`.

3. This commit alters the behavior of the `explicit-null-checks`
`MONO_DEBUG` option. Enabling `explicit-null-checks` also implies
`llvm-disable-implicit-null-checks`, because this option is documented
(e.g. in https://www.mono-project.com/docs/debug+profile/debug/) to
completely disable signal/seh-based null checks.

* Add llvm-disable-implicit-null-checks to the MONO_DEBUG help string.

* Add llvm-disable-implicit-null-checks to mono.1.

* Don't tag null checks in EH regions with `make.implicit`.

This relocates the logic formerly used to guard individual LLVM-level
loads and stores with explicit null checks inside EH regions.

* Use atomic/volatile loads and stores for the terimation flag in thread-suspend-suspended.

LLVM was hoisting the load of and branch against `finished` out of
thread t1's loop, resulting in an infinite spin.

* Use ImplicitNullChecks with LLVM JIT.

* Suppress emission of the .llvm_faultmaps section.

Bump external/llvm to release_60
(7a8dc89adbe7e123220e070a527e096ee91e66d5).

* [ci] use LLVM built from repository
2019-11-13 01:35:44 +01:00
Zoltan Varga eaf895f2d0 [wasm] Change netcore support to use a prebuilt corefx runtime. (#17773)
* [wasm] Change netcore support to use a prebuilt corefx runtime.

* [wasm] Build and test netcore support on CI.

* [wasm] Disable netcore tests on CI for now.

* [wasm] Add an error check.

* [wasm] Add a linker exclusion.

* [wasm] Add netcore runtimes to the nupkg.
2019-11-13 01:30:09 +01:00
Filip Navara bd278dd00d Mirror changes from mono/corefx,coreclr (#17777)
* Rewrite the rest of the array non-generic fast paths in C# (#27703)

* Rewrite the rest of the array non-generic fast paths in C#

* Add missing resource string

* Use CorElementType and handle native ints to match the unmanaged implementation

* Avoid try pattern for Sort and Reverse

* Match the native implementation more closely

Signed-off-by: dotnet-bot <[email protected]>

* Rewrite Enum.CompareTo in C# (dotnet/coreclr#27792)

* Rewrite Enum.CompareTo in C#

* Handle floating point too for match the existing native implementation

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused argument from GetRegionDisplayName (dotnet/coreclr#27806)

* Remove unused argument from GetRegionDisplayName

* Address race conditions in CultureData

While most of the race conditions in this file are benign, in these cases, a thread could end up returning an intermediate value due to multiple potential writes to the same field.

Signed-off-by: dotnet-bot <[email protected]>

* Fix StyleCop rule SA1129 ("Do not use default value type constructor") (#27716)

Will enable the rule in corefx once these changes have propagated.

Signed-off-by: dotnet-bot <[email protected]>

* Remove non-shared Enum.CompareTo/InternalCompareTo implementation, add new Array icalls used by shared code
2019-11-12 18:46:00 +01:00
Bernhard Urban-Forster 4728cec01f [llvm] use multiple cores to build llvm if ninja is not available (#17778) 2019-11-12 18:45:45 +01:00
Jo Shields 8bceb40878 Move zlib to its own directory, and re-enable subdir-objects
This has the following benefits:

* Clearer structure for mirroring
* Slight build time improvement on bundled zlib use (as it only builds once)
* Removes pages of subdir-objects spew from autogen
2019-11-12 09:31:08 +01:00
abrevet-dev 8683fa6db7 [WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask setted (#17660)
[WinForms] Fix #10559 In MaskedTextBox wrong Lines value when Mask se…





<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-11-12 01:55:53 +01:00
abrevet-dev 2b4801d9f1 [WinForms]: Fix #17546 Fix TabPage position when enabling MultiLine (#17670)
[WinForms] Fix TabPage position when enabling MultiLine

Fixes #17546
2019-11-12 00:15:51 +01:00
abrevet-dev 61811075ca [WinForms]: Fix #16632 special values (-1 and -2) of ListView Column header Width was not handled when using List.View. (#17669) 2019-11-11 23:00:45 +01:00
abrevet-dev 9eb0bcae57 [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs (#17662)
* [WinForms] Fix #12249 scroll orientation was not defined in ScrollEventArgs of ScrollBar.cs

* [WinForms] Fix 12249 scroll orientation was not defined in ScrollEventArgs of DataGridView.cs
2019-11-11 22:59:53 +01:00
Jo Shields c256fc86b6 Delete some LLVM test cases from make dist (#17772)
Many of the tests from the llvm-project monorepo cause problems to `make dist`
2019-11-11 17:42:51 +01:00
Alexis Christoforides 722c6f7dd5 [netcore] Complete Monitor.LockContentionCount implementation (#17719)
* [netcore] Complete Monitor.LockContentionCount implementation

Fixes https://github.com/mono/mono/pull/16538
2019-11-11 13:03:03 +01:00
Jan Kotas 594bbbef25 Relax visibility
Signed-off-by: dotnet-bot <[email protected]>
2019-11-10 09:29:18 +01:00
Marek Safar d90030d273 Fixes build 2019-11-09 20:45:42 +01:00
Ryan Lucia 9664df7eb3 Move AssemblyLoadContext.GetResolvedUnmanagedDll to shared (#27775)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
rob313663 e46f29313d Fixed typos in comments. (dotnet/coreclr#27789)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 20:45:42 +01:00
Joseph Musser 5bf661d7f9 StackTrace.GetFrames should return an array of non-nullable frames (#27787)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 09:37:25 +01:00
Levi Broderick ae02b6677b Improve Encoding.UTF8.GetString / GetChars performance for small inputs (#27268)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-09 08:13:11 +01:00
Alexander Köplinger f9b3b5276d Fix llvm path in CODEOWNERS file 2019-11-09 00:44:57 +01:00
Alexander Köplinger ed2d5f69e0 [netcore] Fix RuntimePropertyInfo.GetValue() in FullAOT scenarios (#17757)
It tried to use dynamic codegen which is not available in FullAOT.
We can use the `RuntimeFeature.IsDynamicCodeSupported` to check for this, added a JIT intrinsic which returns false in FullAOT.

Used a linker friendly way to check for TypeBuilder by adding an `internal virtual IsTypeBuilder` method in the `Type` class which returns false, and overwrote it in `TypeBuilder`.
2019-11-09 00:33:00 +01:00
Zoltan Varga 73b50890cc [wasm] Bump emscripten. (#17748)
* [wasm] Bump emscipten to 1.39.2.

* Disable some code in libtest.c which causes emscripten to fail.

* [wasm] Source emscripten env before make install as well.
2019-11-08 14:43:32 -05:00
Nathan Ricci 5b0a822afb Implement GC.GetGCMemoryInfo (#17607)
Implement the GC.GetGCMemoryInfo API.
2019-11-08 13:58:37 -05:00
Ryan Lucia d5b6cf33b3 [loader] Unmanaged library refactoring and NativeLibrary implementation (#17369) 2019-11-08 13:42:56 -05:00
Ryan Lucia 68d142912d [netcore] Run individual CoreCLR test suites (#17753)
This is implemented in a lazy way for convenience when manually testing until the tests are wired up to xunit
2019-11-08 13:14:06 -05:00
Alexis Christoforides 23f2baa71e [merp] Remove extraneous waitpid invocation (#17723)
The extraneous call basically guarantees that the call right after will fail. Fixes merp stating that its process has failed when it hasn't.
2019-11-08 11:40:04 -05:00
Jo Shields 3948a065af Merge pull request #17749 from directhex/dotnet-llvm-monorepo
[LLVM] Change llvm submodule to dotnet-org fork of official LLVM git repo
2019-11-08 10:42:20 -05:00
lateralusX fad91a1c1e Fix Windows LLVM build using new repository structure. 2019-11-08 09:21:30 -05:00
Dotnet-GitSync-Bot 759a8c1847 Mirror changes from mono/corefx,coreclr,corert (#17761)
* Add EventWaitHandle creation extension method that takes an ACL (#42213)

* Add EventWaitHandle creation extension method that takes an ACL

* Call OpenExisting, add basic unit tests, move MAX_PATH in csproj for all Windows platforms, ensure same exceptions are thrown in both netcore and netfx.

* Address suggestions: using for the created handle, simplify try finally, merge methods, remove an unnecessary Debug.Assert

* simplify using, reorganize PInvokes a bit, let VS format resx as it expects it, remove BOM from csproj

* Add more unit tests

* Save basic access rights in a constant, generate name randomly

* Use FullControl (EVENT_ALL_ACCESS); update unit tests to use EventWaitHandleRights

* Apply suggestion of creating an EWH, then replacing its SWH.

* Remove using causing chaos, add more unit tests, remove null string unit test.

* spacing

* Address comments

* Remove unnecessary checks/exceptions, update unit tests

* Fix netfx x86 ut failure - different exception for mode validation.

* Small documentation fix

* Remove documentation for exception thrown with null security

* more documentation fixes

* New file for unit tests

* Dispose and remove windows only attributes in tests

* suggestion to not use var

* Remove duplicate resx after merge

* Readd cs files lost during merge

* Remove resx modification since it was untouched for this PR

* Address test comments

Signed-off-by: dotnet-bot <[email protected]>

* Move NativeLibrary.Free null check to managed (dotnet/coreclr#27744)

Signed-off-by: dotnet-bot <[email protected]>

* Arm64: Add S.P.CoreLib instrinsics UnzipEven and UnzipOdd (#27731)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 07:11:13 -05:00
Tamar Christina 771b5fb18d Arm64: Update S.P.CoreLib for ZipLow and ZipHigh (dotnet/coreclr#27730)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 11:01:17 +01:00
Carlos Sanchez Lopez 229d3cc097 Add Semaphore creation extension methods that take an ACL (#42377)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Semaphore with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Semaphore, then set its ACLs. This presents a few problems:
- Potential security hole as semaphores can be accessed between creation and modification.
- Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Semaphore and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-08 01:59:07 +01:00
Zoltan Varga c0a9d6bf76 [aot] Improve the aot mangler a bit, handle bool/char as a primitive type and avoid emitting a System prefix. (#17739) 2019-11-07 18:18:12 -05:00
Tanner Gooding 8637912a99 Updating NumberToFloatingPointBitsSlow to handle the remaining parsed fractional digits being zero. (#27688)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 23:39:21 +01:00
Jo Shields 4d579d00ff Bump LLVM submodule 2019-11-07 15:52:56 -05:00
Tamar Christina b924ffa5e0 Arm64: Add S.P.CoreLib for AddV. (dotnet/coreclr#27663)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 21:47:22 +01:00
Ryan Lucia 2c22d0de2e [netcore] Propagate ALCs through reflection functions (#16949)
* [reflection] Pass assembly load context through mono_reflection_get_type_with_rootimage

The entry points are mono_reflection_type_from_name_checked and
mono_reflection_get_type_checked which for now pass in an ambient ALC, but
should be updated to take an ALC argument.

Internally mono_reflection_get_type_with_rootimage is recursive through a half
dozen helper functions which all get an ALC argument too.

The principle place where the ALC is used is in
_mono_reflection_get_type_from_info which passes it to
mono_assembly_loaded_internal and to replace the call to mono_assembly_load by
a direct call to mono_assembly_request_byname with an explicit ALC set in the
request.

* Mark mono_assembly_load as external only

* Pass an alc through non-static reflection functions

* Fix usage in wasm debugger

* Relax assertions due to wasm
2019-11-07 15:02:23 -05:00
Marcos Cobeña Morián 8cc0402967 Enabled --preload-files without AOT (#17732) 2019-11-07 15:00:36 -05:00
Jo Shields fe69f5b699 Fix .gitmodules typo 2019-11-07 14:03:06 -05:00
Jo Shields c1c483cef8 LLVM changes for SDKs 2019-11-07 12:16:29 -05:00
Jo Shields d0d779b8d9 Update AzDo pipeine for new LLVM location 2019-11-07 12:09:58 -05:00
Alexander Köplinger 71b78536ad [sdks] Add xunit to iOS test runner and add results reporting (#17744)
- Allows the on-device runner to work with the xunit framework in preparation for netcore.

- Added a simple UI to show the test run progress

- Report the test result XML back to the host so we can process it on Jenkins/AzDO
2019-11-07 18:09:51 +01:00
Jo Shields b61263e994 [llvm] redirect from mono llvm-mirror fork to dotnet llvm-project fork 2019-11-07 12:03:29 -05:00
Jo Shields 9e6b57ce5a delete old llvm 2019-11-07 11:47:32 -05:00
Aleksey Kliger (λgeek) 2c56ac54c0 [profiler] Fix log profiling of native to managed wrappers (#17740)
* [profiler] Add test case for pinvokes and reverse pinvokes

* [profiler] Fix crash in native to managed wrappers

When we call back to managed from a pinvoke, we're in GC Safe mode and the
profiler instrumentation is inserted in the wrapper before it does the
transition to GC Unsafe.  So we need an extra transition before the profiler
calls its locking functions

Fixes https://github.com/mono/mono/issues/17687

* [coop] Mark internal GC Unsafe functions with MONO_PROFILER_API

Allow the profiler to transition to GC Unsafe mode

* [profiler] ptestrunner.pl fix profiler modules directory

we used to specify mono/mini/.libs but that's not where the profiler plugins live, they're in mono/profiler/.libs.   We probably picked up the profiler plugins from the system Mono install.
2019-11-07 11:42:52 -05:00
Thays Grazia 4bdacea237 [embed] Assert when call mono_runtime_object_init (#17589)
* Fix assert when calling externally call mono_runtime_object_init, mono_object_get_domain, mono_string_to_utf8
* Adding samples/embed as unit-test
2019-11-07 11:22:25 -03:00
Carlos Sanchez Lopez 3c84633c7c Add Mutex creation extension methods that take an ACL (#42281)
Approved API Proposal: #41662

Description
We don't currently have a way to create a Mutex with a given ACL in .NET Core. We can modify the ACL, but it would be more secure to have the proper ACL on the object from the start.

Customer impact
Before this change, customers had to create a Mutex, then set its ACLs. This presents a few problems:

Potential security hole as mutexes can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
This change addresses those problems by adding a new extension method that allows creating a Mutex and ensuring the provided ACLs are set during creation.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 12:27:50 +01:00
Elinor Fung 0886ddb3a0 Populate properties of AssemblyLoadStart/Stop events (dotnet/coreclr#27558)
* Populate properties of AssemblyLoadStart/Stop events
* Add tests for basic start/stop binder tracing events
* Add property for ALC of requesting assembly

Signed-off-by: dotnet-bot <[email protected]>
2019-11-07 09:54:23 +01:00
Jay Krell 7b4b5e72d9 Remove handles from ves_icall_System_Array_InternalCreate. (#17681)
Remove handles from ves_icall_System_Array_InternalCreate.

Also combine two allocas.
2019-11-07 07:36:07 +01:00
Zoltan Varga 0680073b87 [runtime] Fix locking in mono_get_seq_points (). (#17738)
Fixes https://github.com/mono/mono/issues/17737.
2019-11-06 19:50:59 -05:00
Johan Lorensson bda239f44c Switch away from Start-Process, Wait-Process in build.ps1. (#17730)
Running Start-Process, Wait-Process on msbuild.exe have showed a couple
of hangs in the build script. This is most likely due to msbuild.exe
spawned child processes (like compile servers) running longer than
spawned msbuild.exe process. Since there is no specific requirements
when spawning the process from build.ps1, switch to regular process
exec and LastExitCode.
2019-11-06 16:51:31 +01:00
Johan Lorensson 1a6c092019 Explicit update/init only LLVM BTLS repro on external MSVC build. (#17691)
Reduce the scope of git submodule update --init to only LLVM and/or BTLS
repositories when building external MSVC dependencies.
2019-11-06 16:51:14 +01:00
Marek Safar 965beadc1e [System.Private.CoreLib] Remove unused Array methods 2019-11-06 12:46:30 +01:00
Stephen Toub e506008ebb Add Span.Sort, and make Array.Sort span-based (dotnet/coreclr#27700)
* Add Span<T>.Sort

Shares existing Array-based implementation by changing that implementation to be span based.

* Additional work to enable span-based sorts

- Cleans up changes from the previous commit, e.g. corrects nullable annotations, removes TODOs, using GetRawSzArrayData instead of GetRawArrayData, etc.
- Passes spans around rather than spans+lo+hi.
- Deletes the native TrySZSort, preferring to use the managed implementation in all cases.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-06 12:46:30 +01:00
Bernhard Urban-Forster 29ba75c154 Bump Corefx (#17694)
* Bump Corefx

New commits in mono/corefx:

* mono/corefx@c44efe7297 [mono] limit DegreeOfParallelism to 16 (#369)

Diff: https://github.com/mono/corefx/compare/6e65509a17da898933705899677c22eae437d68a..c44efe7297f8e11cab157c7e96b998dc936d53b7

* another bump
2019-11-06 07:36:17 +01:00
Jay Krell 1e8e357831 Handles reduction -- 4 MERP functions. (#17711)
Handles reduction -- 4 MERP functions.
2019-11-06 03:56:10 +01:00
Jay Krell aa506ca1bb error_init reduction. (#17712) 2019-11-05 16:38:40 -05:00
JesperTreetopandStephen Toub 104abd7fe7 Prevent DivideByZeroException in EventCounter.ToString() (#27581)
* Prevent DivideByZeroException in EventCounter.ToString()

Maybe it should print something other than 0. Maybe "Mean" should be excluded. But at least with this it doesn't divide by zero. Ternary operator in interpolated string is not super readable; didn't know if making the code more descriptive would have performance implications.

* Update src/System.Private.CoreLib/shared/System/Diagnostics/Tracing/EventCounter.cs

Co-Authored-By: Stephen Toub <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 21:18:22 +01:00
Ryan Lucia f477bce548 [loader] Skip the full pinvoke resolution process for __Internal (#17701) 2019-11-05 14:35:00 -05:00
Tanner Gooding 3d4f44d85c Fixing the return type of the ArmBase.ReverseElementBits methods in the PlatformNotSupported file (#27672)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-05 20:20:17 +01:00
Bernhard Urban-Forster 93b97d5582 [mini] Fix Coverity CID 1455161 & 1455162 (#17615)
* [mini] Fix Coverity CID 1455161 & 1455162

> Null-checking "ji" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.

* do not rely on assert to guard ji-nullness.  refactor a bit
2019-11-05 19:09:32 +01:00
Johan Lorensson 6f73c2dab8 Fix MSVC intellisense for LLVM sources. (#17706) 2019-11-05 13:16:32 +01:00
Johan Lorensson 44e6226c31 Fix correct accumulated time for parallelized jobs in SGEN. (#17518)
* Fix correct accumulated time for parallelized jobs in SGEN.

Current implementation incorrectly accumulate time for scan jobs for
the follow metrics when running parallel minor GC:

time_minor_scan_major_blocks,
time_minor_scan_los,
time_major_scan_mod_union_blocks,
time_major_scan_mod_union_los

Commit fix so updates are atomic making sure values are correct in cases
where parallel minor GC is used.

* Implement 64-bit atommic add fallback in SGEN.
2019-11-05 01:57:06 -05:00
Nikolay Sivov 95743afa80 [mono] Fix ProcessExit handler argument. (#17680)
Problem was spotted with mono tracing hitting ProcessExit handler,
and crashing on first argument.
2019-11-05 01:56:43 -05:00
Egor Bogatov fdc6495a54 [netcore] Intrinsify Buffer.IsPrimitiveTypeArray (#17438)
* Improve IsPrimitive intrinsic

* Implement it for interpreter

* clean up

* clean up
2019-11-05 01:50:37 -05:00
Zoltan Varga ebb4598096 [wasm] Build the tests with --no-native-strip. (#17698) 2019-11-04 17:46:52 -05:00
Vlad Brezae b3b9960ed1 [interp] Use GetType instrinsic also on net4x (#17654)
We didn't do this because it had some issues when GetType was invoked through reflection. This issue would also be present with jit but, because jit doesn't use direct wrappers, the GetType call can't be intrinsified in that scenario. See https://github.com/mono/mono/pull/5350 for the context of the issue.
2019-11-04 12:58:38 -05:00
Dotnet-GitSync-Bot db49be3cbb Update only S.P.CoreLib for ReverseElementBits (dotnet/coreclr#27582) (#17695)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 12:55:57 -05:00
Egor Bogatov cf8495e2f7 [netcore] Copy Array.Clear impl from CoreCLR (#17411)
* Address feedback

* fix build error

* Fix has_reference usage in object.c

* Fix fallback impl

* Rename to ArrayHasReferences

* Rename to ObjectHasReferences

* Rollback to ArrayHasReferences

* Rename to ObjectHasReference

* remove redundant is_array variable

* ObjectHasReferences can accept any object

* undo changes in Program.cs

* undo changes in Program.cs
2019-11-04 12:48:04 -05:00
Filip Navara 07fa0bd3b4 Intrinsify Activator.CreateInstance<T> for value types with no ctor (#17688)
* Intrinsify Activator.CreateInstance<T> for value types with no ctor

* Fix failed test, address feedback
2019-11-04 12:40:13 -05:00
Zoltan Varga 929a8a7fd8 [jit] Allow Unsafe.As<TFrom, TTo> on gsharedvt types. (#17692) 2019-11-04 18:14:59 +01:00
Bernhard Urban-Forster 237f210ea2 [interp] use mask instead of bool expression (#17683)
Discovered by Coverity:
```
>>>     CID 1455205:    (CONSTANT_EXPRESSION_RESULT)
>>>     The expression "flag && 2" is suspicious because it performs a
        Boolean operation on a constant other than 0 or 1.
```
2019-11-04 15:56:46 +01:00
Alexander Köplinger 49d5c4d14d [sdks] Use Xcode 11.2 stable version for iOS/Mac SDKs
As requested by the xamarin-macios team.
2019-11-04 15:27:42 +01:00
Tarek Mahmoud Sayed 6630bcc67f Fix Buggy code in TimeZoneInfo (dotnet/coreclr#27591)
Although couldn’t reach the corner case cause executing the buggy code, the fix here is done according to the similar code in TimeZoneInfo which show the intention of the change

https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1638
https://github.com/dotnet/coreclr/blob/97c582975eed3260ef21e0bcc55c8c86d1e3bec8/src/System.Private.CoreLib/shared/System/TimeZoneInfo.cs#L1568

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 09:59:54 +01:00
Jan Kotas f3e30a0ad0 Delete two more FCalls (dotnet/coreclr#27640)
Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas deb3115b15 Use simple Array.Copy overload where possible (dotnet/coreclr#27641)
The simple Array.Copy overload does not come with a performance penalty anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Jan Kotas 6ecf9e87ba Consolidate *.Fast.cs (dotnet/coreclr#27644)
System.Memory OOB package that this split was originally done for does not exist anymore.

Signed-off-by: dotnet-bot <[email protected]>
2019-11-04 08:57:17 +01:00
Johan Lorensson c0e237b734 Mono NetCore Windows only build/test. (#17646)
Support for Windows Mono NetCore build/test wihtout any need for
cygwin/wsl/automake just MSBuild and MSVC. Windows build support mirror
technologies used by coreclr/aracade builds msbuild + powershell.

Windows only Mono NetCore build, makes a port of existing build.sh and
Makefile, implementing a build.ps1 and build.targets. So far full build
support + execution of corefx test suite is implemented. Other make file
rules can be added when needed.

MSVC build runtime seems to hits a couple of additional test failures
in corefx test suite that needs to be investigated further.

Building NetCore Mono on Windows can now be done without cygwin or wsl.
Build depends on regular VS install (or build tools), if xunit summary
is requested, python needs to be installed.
2019-11-04 08:55:36 +01:00
Egor Bogatov 137ffe8b7c [netcore] Improve Array.CreateInstance (#17673)
* Rewrite Array.CreateInstance

* Rewrite Array.CreateInstance

* Address feedback

* Convert the icall to NOHANDLES

* fix build errors, remove NOHANDLES

* Fix GC.KeepAlive usage

* Remove icall definition for GC.KeepAlive (it's no an icall)

* Fix failing tests (expect details in the AOORE)
2019-11-03 22:39:19 -05:00
Filip Navara 8131518bb1 [netcore] Improve default constructor lookup, (#17666)
* [netcore] Improve default constructor lookup, cache when no default constructor is found (common for struct)

* Address PR feedback
2019-11-03 18:28:32 +01:00
Levi Broderick 8700c04b02 Fix typo in Tuple.cs (dotnet/coreclr#27637)
Reported by https://github.com/dotnet/corefx/issues/42321

Signed-off-by: dotnet-bot <[email protected]>
2019-11-03 18:17:23 +01:00
Alexander Köplinger 4888c49c26 [netcore] Disable shared compilation for System.Private.CoreLib
There's a dotnet process hanging around which seems to lock the build when called from the SDKs Makefiles.
2019-11-03 15:43:40 +01:00
Alexander Köplinger 40c1cd3576 [netcore] Fix strong name key for System.Private.CoreLib
It's using the silverlight key: https://github.com/dotnet/coreclr/blob/34fe045a27e150bde7ee54e0d5f0df635922519d/src/System.Private.CoreLib/System.Private.CoreLib.csproj#L112
2019-11-03 15:43:40 +01:00
Alexander Köplinger d89b96a298 [netcore] Fix out of tree build for netcore/ folder 2019-11-03 15:43:40 +01:00
Jan Kotas ad385f98b3 Normalize trailing whitespaces (dotnet/coreclr#27626)
Normalize trailing whitespaces in frequently changing files (docs and sources)

Signed-off-by: dotnet-bot <[email protected]>
2019-11-02 16:51:03 +01:00
Bernhard Urban-Forster 62ee01b36b [interp] s/MONO_API_ERROR_INIT/error_init_reuse/g (#17650)
[interp] s/MONO_API_ERROR_INIT/error_init_reuse/g
2019-11-02 16:45:36 +01:00
Zoltan Varga 803c657b24 [jit] Call mono_class_setup_fields () before accessing field->offset. (#17667)
Fixes https://github.com/mono/mono/issues/17665.
2019-11-02 03:57:25 -04:00
Manuel de la Pena 36539ed67f [iOS] Match changes done in xamarin-macios in the SDK runtime. (#17664)
* [iOS] Match changes done in xamarin-maciosn in the SDK runtime.

* Address reviews.
2019-11-02 01:40:40 -04:00
Jay Krell 4b9dc08ac1 Move MonoError from managed wrappers to native wrappers. (#15894)
Move MonoError from managed wrappers to native wrappers.

Move MonoError from managed wrappers (ilgen) to native wrappers (C preprocessor).

Alternative to https://github.com/mono/mono/pull/15869.
Which says: Previously, we would emit a call to memset
since the struct is large (> 100 bytes). Only the error code
field needs to be initialized.
Which that PR and this PR both fix, in different ways.

In the past:
- This did not work due to a possible dependency on precise GC,
  interacting with what gets zeroed or not here.
  That is why I put it on the managed side.
  We'll see if that reproduces.
  (https://github.com/mono/mono/pull/11294/files#diff-efe0070415f3f0c0f24ed0221aa1962aR339
  external/coreclr/tests/src/CoreMangLib/cti/system/weakreference/weakreferenceisaliveb.exe.)

- Some indication that embedding API should expose MonoError
  and not set_pending_exception. This does not seem relevant
  presently, and the code seems better structured and
  more efficient this way. It can revisited if/when new public API
  is the dominant concern, or do it this way for these functions,
  and another way for actual public functions.
  (https://github.com/mono/mono/pull/11294#issuecomment-432443485)
2019-11-01 22:36:06 +01:00
Zoltan Varga e9d68e9485 [netcore] Cleanups. (#17661)
* [netcore] Remove the type_info field from RuntimeType, extend the TypeCache structure instead.

* [netcore] Remove some linker exclusions which are not needed.
2019-11-01 12:52:45 -04:00
Steve Pfister b9f185b74a Bump mono/corefx@6e65509 (#17631)
Pulls in PR https://github.com/mono/corefx/pull/368
2019-11-01 11:04:27 -04:00
Jo Shields 16d1a2e6f1 [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. (#17628) 2019-11-01 10:26:08 -04:00
monojenkins a4790c892d [csproj] Update project files 2019-11-01 14:25:26 +00:00
monojenkins cc28706e58 Bump API snapshot submodule 2019-11-01 14:24:11 +00:00
Vlad Brezae 166fdf60b0 [interp] Handle remoting field access same as jit (#17641)
Doing this we avoid using slow ldrmfld in the common case of loading fields from instance methods, when we execute on the real object. We could do this also for strmfld but we need to track on the stack if the instance object is this, which is not worth the trouble.
2019-11-01 14:55:58 +02:00
Vlad Brezae 170d33c35b [interp] Optimize call path (#17623)
* [interp] Optimize void calls

By avoiding setting of retval

* [interp] Avoid computing param count for every call

* [interp] Remove wrong profiler check

From the times when interp had its own EH mechanism. We raise this event from mono_handle_exception_internal for interp.

* [interp] Remove profiling check code from method exit

We emit MINT_PROF_EXIT opcode when under profiler. Unify profiling enter/exit with tracing enter/exit since they are very similar.
2019-11-01 14:55:27 +02:00
Filip Navara 8aa3654de5 Lower min/max SSE operations on x64 (#17536)
Follow up on #17511.
2019-11-01 13:30:41 +01:00
Zoltan Varga dc8d55d446 [llvm] Fix a case where we treated the dreg of a store_membase instruction as a dreg, its actually the base reg. (#17648) 2019-11-01 03:43:42 -04:00
Steve Pfister 9566027142 Modify run-jenkins.sh to adapt wasm build for the OSX CI lane (#16785)
* Modify run-jenkins.sh to adapt wasm build for the OSX CI lane

* Round 2... Go

* Make ENABLE_WASM_DYNAMIC_RUNTIME conditional on non-OSX

* Add runtime-threads target

* Ignore 'multiple rules generate' on osx

* Fix typo

* Remove blank line

* Exclude CoreXml.Test.XLinq.FunctionalTests.EventsTests.EventsReplaceNodes.XElementReplaceNodes

* Revert "Ignore 'multiple rules generate' on osx"

This reverts commit 5010721bd710d6955aaee47aa7f49ad7845763fc.

* Check if OSX lane can follow general build flow
2019-11-01 01:44:50 -04:00
Stephen Toub 8b86349c2c Disable another static analysis rule (dotnet/corefx#42265)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 22:23:35 +01:00
Bernhard Urban-Forster 0be756d609 [threadpool] cache processor count (#17642)
On top of https://github.com/mono/corefx/pull/369 this improves the execution time of `System.Core-xunit` on Linux/ARM64 by 2x, so from:

```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 131.143s
```
to
```console
$ make -C mcs/class/System.Core run-xunit-test
[...]
=== TEST EXECUTION SUMMARY ===
   net_4_x_System.Core_xunit-test  Total: 48774, Errors: 0, Failed: 0, Skipped: 6, Time: 74.636s
```

This is only relevant for non-netcore. The CoreCLR folks just recently fixed something similar (thanks to Marek sharing this link): https://github.com/dotnet/coreclr/pull/27543
2019-10-31 22:08:14 +01:00
Vlad Brezae b484979aa9 [interp] Avoid emitting MINT_SAFEPOINT for every single call (#17625) 2019-10-31 16:49:16 +01:00
Jonathan Chambers 3648cb087b Avoid caching of System.dll image and types as they may be unloaded with appdomain. (#17602) 2019-10-31 10:13:01 -04:00
Zoltan Varga dff84a2464 [wasm] Fix xunit test ninja errors. (#17639)
* [wasm] Error out on duplicate assemblies in the packager.

* [wasm] Switch search path order to avoid duplicate references to assemblies.

* Copy all xunit binaries to test_lib_dir.

* Revert "Copy all xunit binaries to test_lib_dir."

This reverts commit b15b03c16b508c04dc6121196fddca0e6d949e2f.
2019-10-31 10:10:51 -04:00
Alexander Köplinger ca72deed17 Fix build 2019-10-31 14:36:53 +01:00
Ben Adams c9b7f767a8 Don't needlessly refresh proc count (dotnet/coreclr#27543)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 14:36:53 +01:00
Vlad Brezae 1d043a31e7 [interp] Fix interp logging (#17636)
* [interp] Don't print NOPs during cprop

They can become numerous and distracting.

* [interp] Fix mono_interp_print_code

Regressed after code_size was changed to byte count instead of short count.
2019-10-31 15:32:50 +02:00
abrevet-dev 6b3aa3a05b [WinForms] Change HotkeyPrefix default value in TabControlPainter.cs (#17579)
The default behavior on .NET Framework is _HotkeyPrefix.None_ when creating a _TabControl_ element (ampersands not escaped). 
If an user wants to override this behavior, he has to declare a _DrawItemEventHandler_ (with _DrawMode = OwnerDrawFixed_).

Fixes #16206
2019-10-31 13:48:12 +01:00
Stephen Toub a27aa7afd0 Update analyzer versions (dotnet/corefx#42249)
* Update analyzer packages to latest NuGet versions

* Fix new warnings

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Stephen Toub 6ae3360342 Clean up Uri.UnescapeDataString (dotnet/corefx#42225)
- Use string.IndexOf rather than an open-coded, unsafe loop.
- Avoid an unnecessary SequenceEquals at the end: we're only at this point if a `%` was found highlighting that something escaped was found.
- Use stack memory for smaller inputs if possible, to avoid unnecessary ArrayPool interaction
- Remove an unnecessary argument to a helper function.
- Fix ValueStringBuilder.Grow to only copy the contained data.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 11:09:58 +01:00
Egor Chesakov a94dda5b5a Add BitwiseSelect (dotnet/coreclr#27430)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-31 08:51:46 +01:00
Zoltan Varga dc76a95aa9 [runtime] Add a --enable-minimal=threads configure option to disable threading support. Use it on wasm. (#17611) 2019-10-31 01:33:50 -04:00
Zoltan Varga 5811404e55 [eglib] Assert in g_strlcpy () if src/dest is NULL, instead of printing a warning. (#17507)
Related to https://github.com/mono/mono/issues/17283.
2019-10-30 16:07:44 -04:00
Martin Baulig 578e0df42f [System.Net.Http]: Bring HttpClient from CoreFX on monotouch and xammac. 2019-10-30 15:37:25 -04:00
Jo Shields 27921fbbd0 Rearrange System.Private.CoreLib for easier monorepo integration (#17517)
* [netcore] Move corlib around to match CoreCLR directory layout better

This is a prerequisite to making a sane migration of this whole
directory from mono.git to coreclr.git
2019-10-30 15:09:06 -04:00
Thays Grazia fffb1201bd [dim][regression] Explicit interface override (#17583)
* Precedence order:
-> Override interface method in class
-> Method in class
-> DIM
2019-10-30 14:47:15 -03:00
Egor Chesakov feca5e042d Add ComputeCrc32, ComputeCrc32C (dotnet/coreclr#27421)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Egor Chesakov 744d77ea7f Remove unsigned overloads of LeadingSignCount (dotnet/coreclr#27533)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 18:06:03 +01:00
Martin Baulig c6f71167a1 Bump CoreFX to pickup corefx PR #367 to fix #17133. (#17590) 2019-10-30 12:32:21 -04:00
Bernhard Urban-Forster 95cad24526 [interp] fix signature mismatch between jit<>interp for string constructor (#17595)
[interp] fix signature mismatch between jit<>interp for string constructor
2019-10-30 16:36:04 +01:00
Marek Safar e4c0b50dec [System.Private.CoreLib] Remove NativeRuntimeEventSource 2019-10-30 16:07:43 +01:00
Jan Kotas c9b88b45fe Move runtime specific methods out of the runtime partition
Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 16:07:43 +01:00
Jay Krell 10ac2750e2 Inline TLS access. (#17119)
Inline TLS access.

This attempt differs from the previous in that the TLS accessor functions are presented to far less code, i.e. not to main.c, so their data references do not need to be resolved so much.

See https://github.com/mono/mono/commit/bffd4c6b99213a0b154d7efac9b0581df67775e8 etc.
2019-10-30 16:06:24 +01:00
Zoltan Varga 79f5e6c288 Remove some unused icalls. (#17612) 2019-10-30 10:08:50 -04:00
Filip Navara 3d397805d4 [netcore] Report errors on Windows CI (#17616)
* [netcore] Report errors on Windows CI

* Fix Windows build
2019-10-30 14:39:47 +01:00
Zoltan Varga 2823ac5d95 [wasm] Print a useful error message instead of a signature mismatch error on missing icalls. (#17600) 2019-10-30 00:04:15 -04:00
Dotnet-GitSync-Bot 3695d8704e Mirror changes from mono/coreclr,corefx (#17609)
* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp() (#27473)

* Revert removal of SuppressGCTransition from SystemNative_GetTimestamp()

* Insert GC_POLL before statement with unmanaged call.

* JIT test for insertion of GCPoll

Signed-off-by: dotnet-bot <[email protected]>

* Add file creation method that takes an ACL (dotnet/corefx#42099)

Approved API Proposal: #41614
Related change for directory creation method that takes an ACL: #41834 -merged and ported to 3.1 Prev2

Description
We have extension methods in System.IO.FileSystem.AclExtensions that let the user get and set ACLs for existing files, but we do not have methods that create files with predefined ACLs.
.NET ACL (Access Control List) support is Windows specific. This change will reside inside the System.IO.FileSystem.AccessControl assembly.

Customer impact
Before this change, customers had to create a file or filestream, then set its ACLs. This presents a few problems:

Potential security hole as files can be accessed between creation and modification.
Porting difficulties as there isn't a 1-1 API replacement
Stability issues with background processes (file filters) can prevent modifying ACLs right after creation (typically surfaces as a security exception).
This change addresses those problems by adding a new extension method that allows creating a file and ensuring the provided ACLs are set during creation.
This change is expected to be backported to 3.1.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-30 00:03:47 -04:00
Zoltan Varga 9adce0b632 [wasm] Fix build problems. (#17596)
* [runtime] Add a stub for MacOsIPInterfaceProperties::ParseRouteInfo icall on non-osx platforms to fix the wasm aot build.

* [wasm] Avoid generating invalid ninja build files if an assembly is specified on the command line and its also an implicit reference.

* Put back and ifdef.

* Fix the windows build.
2019-10-30 00:03:08 -04:00
Filip Navara d49d3efd5d Temporary build fix 2019-10-29 23:34:11 +01:00
Noah Falk 67ff64a381 Support large EventSource filter args (dotnet/coreclr#27500)
1. Fix NullReferenceException. When the filter args exceeded the
hard-coded size limit GetDataFromController would return data = null.
The code previously asserted that data was not null and triggered NRE
when it was. The fix correctly null checks the value instead of
asserting and uses an empty args dictionary in this case, the same as if
filter args had been empty to begin with.

2. ETW has always limited filter args to 1024 bytes but EventPipe has no
such restriction. When using DiagnosticSourceEventSource it can be
useful to specify a larger filter arg blob. I can't do anything about
ETW's restriction but there is no need for the runtime to force
EventPipe to be equally limited. The larger size also reduces the chance
that we need to hit the fallback path above causing filter args to be
ignored.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 8a03f60897 Cleanup (dotnet/coreclr#27502)
* Delete empty FireCustomerDebugProbe FCall

* Delete duplicate C++ compiler command line options

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Elinor Fung 17cef4c4f9 Basic infrastructure for binder tracing (dotnet/coreclr#27383)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub 249522036d Fix StreamReader to pass cancellation token into ReadBufferAsync (#27464)
ReadAsyncInternal is correctly passing its token into the two call sites where it delegates directly to the underlying stream, but in one place it calls through ReadBufferAsync, which is currently not defined to take a token.  Fix that, and pass the token through.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Jan Kotas 447cb03399 Delete unnecessary abstraction from stubbed out SecurityElement (#27474)
The stubbed out SecurityElement implementation does not support extensibility that ISecurityElementFactory was designed for.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Dan Moseley 9aeef958c6 Add path to not-absolute-path exception (dotnet/coreclr#27470)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Filip Navara db02ebc88b Fix building portable thread pool on Windows (dotnet/corert#7849)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Stephen Toub f26e73b49b Move PreserveDependencyAttribute to shared (dotnet/corefx#42174)
We need to use it in corelib, too.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-29 23:34:11 +01:00
Bernhard Urban-Forster d274cfbacd [mini] Initial tiered compilation work (#17551)
[mini] Initial tiered compilation work

Enable it with `./autogen.sh --enable-experiment=tiered`.

Let's consider `Simple.cs`:
```csharp
using System.Runtime.CompilerServices;
using System;

public class Simple {
    public static void Main (string []args) {
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
        HotMethod ();
        Console.WriteLine ("cnt: " + cnt);
    }

    static int cnt = 0;

    [MethodImplAttribute (MethodImplOptions.NoInlining)]
    public static void HotMethod () {
        for (int i = 0; i <= 1000; i++)
            cnt += i;
    }
}
```

```console
$ csc Simple.cs
$ MONO_LOG_LEVEL=debug MONO_LOG_MASK=tiered ./mono/mini/mono-sgen --trace=M:Simple:HotMethod --interp=-all Simple.exe
[0x10ec275c0: 0.00000 0] ENTER:i Simple:HotMethod ()()
Mono: tiered: queued Simple:HotMethod ()
[0x10ec275c0: 0.00010 0] LEAVE:i Simple:HotMethod ()(
Mono: tiered: patching 0x7fe855803224 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x34 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
Mono: tiered: patching 0x7fe8558031f2 with patch_kind=INTERP @ tier_level=0
Mono:   -> caller= Simple:Main (string[]) [{0x7fe85420df88} + 0x2 interp]  (0x7fe8558031f0 0x7fe855803258) [0x7fe85420c880 - Simple.exe]
Mono:   -> callee=Simple:HotMethod ()
cnt: 500500
[0x10ec275c0: 0.04093 0] ENTER:c Simple:HotMethod ()()
[0x10ec275c0: 0.04095 0] LEAVE:c Simple:HotMethod ()(
cnt: 1001000
```
Note the suffix after `ENTER:`
* `i` indicates it's executed by the interpreter
* `c` indicates it's a compiled method (JIT)

Another example:
```console
$ make -C mono/mini gshared.exe
$ ./mono/mini/mono-sgen --interp=-all --stats ./mono/mini/gshared.exe
Regression tests: 84 ran, 0 failed in Tests
[...]
Tiered statistics
Methods promoted                    : 68
```

It's a basic proof-of-concept for now. An incomplete list of future work items:

* Right now it only works for direct calls, need to expand it to virtual/interface calls.
* Calls of the JIT leading into the interpreter again can't be patched yet.
* Kind of related, no concept of versioning compiled methods does exist yet. The interpreter maintains its own table of "transformed" methods, however, when doing the transition from JIT->interpreter, the wrapper+interpmethod will end up in the JIT table. We need a way to have multiple JIT compilation results for the same MonoMethod exist.
* Investigate actual performance. We might have to optimize the interp<>JIT transition.
* Patching is racy. Need to make that atomic.
* All the FIXMEs in this PR



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-29 22:36:01 +01:00
Alexander Köplinger 1340095581 [acceptance-tests] Bump mono/roslyn to get build fix
See https://github.com/mono/roslyn/commit/6bfcd4e770609a38b8523e83ce1ec943162f30bb
2019-10-29 20:27:09 +01:00
Zoltan Varga b0d966bd7a [eglib] Handle dli.dli_sname being NULL in g_module_address (). (#17592)
Should fix the warnings/assertions in g_strlcpy ().
2019-10-29 09:21:23 -04:00
Aniruddha Shastri 7f91972ded Support platform-independent newlines in line wrapping in Mono… (#17431)
* Add newLine parameter to StringCoda methods

* Update Options.cs to handle both kinds of newlines

* Add test to OptionSetTest

* Directly handle \r\n instead of using the TextWriter.NewLine
2019-10-29 13:53:41 +01:00
dotnet-maestro[bot] cddf90dda4 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17577)
* Update dependencies from https://github.com/dotnet/arcade build 20191027.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19527.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19527.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191027.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19527.2

* Update dependencies from https://github.com/dotnet/corefx build 20191027.5

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19527.5

* Update CoreFX.issues.rsp

GetGCMemoryInfo was moved to GCExtendedTests in https://github.com/dotnet/corefx/commit/d6dac9e6a7875862b91139334c8ec620f599cd4d.

GetTotalAllocatedBytes and GetAllocatedBytesForCurrentThread seem to pass now.
2019-10-29 12:51:33 +01:00
Zoltan Varga e4cc70f4b5 [runtime] Implement support for SuppressGCTransitionAttribute. (#17534)
Fixes https://github.com/mono/mono/issues/17367.
2019-10-28 10:59:52 -04:00
Johan Lorensson c5edcd1ed4 Additional and adjusted logging for parallel minor GC. (#17522)
* Additional and adjusted logging for parallel minor GC.

When running parallel minor GC we currently get incorrect timing
on several loggings and counters due to work being put on thread pool
and accumulated. This commit fix so that the logging is accurate
when running with and without parallelization. In parallel mode a new
logging is also added to show the accumulated split between major and los
remset scan jobs.

The counters time_minor_scan_remsets and time_minor_scan_roots are only
updated when not running parallel GC since they are not correct when
running parallel GC.

* Fix WASM build error.
2019-10-28 15:34:42 +01:00
Alexander Köplinger 450d474684 [netcore] Fix offsets generation when cross compiling
The offsets-tool.py uses some magic to map `typename` to `<underscore>typename` for looking up runtime types.
This caused issues on netcore because we aliased it to a different name.

To fix this let's keep the `_MonoInternalThread` name instead and just fix the one location where `_MonoThread` is used.
2019-10-28 13:57:17 +01:00
Alexander Köplinger fb7b075e49 [netcore] Don't load types in aot-compiler.c that don't exist in corefx
Fixes https://github.com/mono/mono/issues/16026
2019-10-28 13:57:17 +01:00
dotnet-maestro[bot] 6f04054541 Update dependencies from dotnet/corefx (dotnet/coreclr#27431)
* Update dependencies from https://github.com/dotnet/corefx build 20191024.13

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.13
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.13

* Update dependencies from https://github.com/dotnet/corefx build 20191024.14

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19524.14
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19524.14

* Update dependencies from https://github.com/dotnet/corefx build 20191025.6

- Microsoft.NETCore.Platforms - 5.0.0-alpha.1.19525.6
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha.1.19525.6

* Workaround https://github.com/dotnet/coreclr/issues/27465

Signed-off-by: dotnet-bot <[email protected]>
2019-10-27 17:54:12 +01:00
Alexis Christoforides db568c6fe0 [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping (#17537)
* dump_native_stacktrace should always register the sigterm handler.

Cherry picked from the https://github.com/mono/mono/pull/17310

* [merp] Introduce a new 'dump mode' that allows different signal behavior when dumping

The primary reason for this is gaining the ability to interpret SIGTERM properly as a crashing signal when outside of 'dump mode', and as a utility signal during the 'thread summarizer' dumping process.
2019-10-26 18:48:40 -04:00
Aleksey Kliger (λgeek) 0e3caf00df [runtime] Unbalanced GC Unsafe transitions before shutdown (#17566)
When embedders call mono_runtime_quit or mono_jit_cleanup, we need to do unbalanced transitions to GC Unsafe, because after mini_cleanup runs we don't have GC thread states anymore and MONO_EXIT_GC_UNSAFE will assert.

* [runtime] Do an unbalanced GC Unsafe transition in mono_jit_cleanup

After mini_cleanup, we don't have GC thread states anymore because all that
stuff was cleaned up, so MONO_EXIT_GC_UNSAFE would assert

* [runtime] Mark mono_runtime_quit external only.

Runtime should use mono_runtime_quit_internal.

After we call the quit_function (aka mini_cleanup) we don't have any GC thread
states anymore because all that stuff got cleaned up.  So MONO_EXIT_GC_UNSAFE
can't work.
2019-10-26 18:45:35 -04:00
Ben Adams 2da4e54a7f Avoid mod operator when fast alternative available (dotnet/coreclr#27299)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 19:07:15 +02:00
Jay Krell 5f3446bd12 [cxx][x86] int/ptr casts (#17570)
[cxx][x86] int/ptr casts
2019-10-26 18:26:15 +02:00
Dotnet-GitSync-Bot c5e3ca90de Mirror changes from mono/coreclr (#17571)
* Use ValueStringBuilder instead of StringBuidler{Cache} in multiple places

Signed-off-by: dotnet-bot <[email protected]>

* Avoid unnecessary allocations in Exception.ToString

Signed-off-by: dotnet-bot <[email protected]>
2019-10-26 07:48:46 -04:00
Dotnet-GitSync-Bot fec907397d Mirror changes from mono/coreclr,corert,corefx (#17565)
* Update Corelib to include Stopwatch (dotnet/coreclr#27417)

* Update Corelib to include Stopwatch

* Delete HighPerformanceCounter

Signed-off-by: dotnet-bot <[email protected]>

* Share completed Task between Task and AsyncTaskMethodBuilder (#27437)

Signed-off-by: dotnet-bot <[email protected]>
2019-10-25 21:23:46 -04:00
Caleb Cornett 1801bdad6a Add --preload-file packager option 2019-10-25 14:53:15 -05:00
Leonard 4ec9bd03e5 Fixes #17190: SerializationException on ListViewItemCount (#17195)
* Fix issue #17190: SerializationException on ListViewItemCount

ListViewGroup doesn't (always) have a serialized member "ListViewItemCount".

This patch accounts for that and also adds support for the "ItemsCount"
property that upstream [1] seems to use.

Also items should be initialized regardless of the count, otherwise we get a NPE.

[1] https://github.com/dotnet/winforms/blob/master/src/System.Windows.Forms/src/System/Windows/Forms/ListViewGroup.cs#L144
2019-10-25 19:30:08 +02:00
Kenneth Pouncey 1bd76c6069 Spacing 2019-10-25 12:19:54 -05:00
Kenneth Pouncey f1a3f3c58d Add CompileFunction to Runtime that returns a WebAssembly.Core.Function
- The code passed to `CompileFunction` is not a function definition. Instead it must create and return a function instance.
- Solves the problem of accessing `Node.js` functionality as defining a JavaScript function executes in the global space and does not get `Node.js` access.
- Returning a function instance allows the function to initialize and maintain encapsulated state.
2019-10-25 12:19:54 -05:00
Jay Krell 893486638c [cxx] Compile mini-llvm.c as C++ if configure -enable-cxx. (#17558)
[cxx] Compile mini-llvm.c as C++ if configure -enable-cxx.

This is presently catastrophic, because of `MonoJitICallId` varying between C and C++, and therefore the layout of `MonoCallInst`.

Related to:
https://github.com/mono/mono/pull/17557

https://github.com/mono/mono/pull/17445#issuecomment-546175352
2019-10-25 17:46:01 +02:00
Jay Krell 84d64d314b [jit][x86][cxx] int/ptr casts (#17382) 2019-10-25 11:34:50 -04:00
Aleksey Kliger (λgeek) d6ee245c94 [runtime] Make mono_thread_manage external only (#17554)
* [runtime] Make mono_thread_manage external only

runtime should use mono_thread_manage_internal
2019-10-25 11:28:18 -04:00
Vlad Brezae 768fefbad1 [interp] Optimize stelem (#17519)
* [interp] Optimize stelem instructions

* [intepr] Optimize stelem_ref and slowpath casts

mono_object_isinst_checked uses coop handles and is very slow. Attempt to use the straightforward mono_class_is_assignable_from_checked.

Makes stelem_ref 5x faster.
2019-10-25 16:25:08 +02:00
Egor Bogatov dc4a7f5ed7 [llvm] Use -place-safepoints in JIT mode too (#17116)
* always use -place-safepoints pass in JIT mode

* ignore loaded llvm

* remove gc.safepoint_poll after optimizations

* fix typo

* cleanup

* cleanup

* cleanup

* cleanup

* Do not emit safepoints on entry thus simple functions (without calls and loops) won't have them at all

* Undo  -spp-no-entry=true

* fix failing tests

* do not emit sp in Allocators
2019-10-25 16:10:57 +02:00
Kenneth Pouncey 6ac749ed20 [wasm][bcl] Fix System.Diagnostics.Trace.TraceError crash. (#17509)
* Fix `System.Diagnostics.Trace.TraceError` crash.

- WASM does not have access to the command line arguments through the browser so the length is 0 causing index out of range error.
- The zero index of the command line arguments is what decides the `AppName` so under WASM the AppName will be blanks.

* Address review comments

* Remove #ifdef for WASM only.

- Addresses review note.
- As per channel discussion some embedding scenarios might be the same.
2019-10-25 11:17:00 +02:00
Zoltan Varga 9d781eb78c [wasm] Propagate exit code from Main in the test runner. (#17559)
* [wasm] Propagate exit code from Main in the test runner.

* Disable some System tests as they currently fail.
2019-10-25 04:59:30 -04:00
Dotnet-GitSync-Bot 8800bfdf9c Fix build break (#17553)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 18:30:09 -04:00
Jay Krell 4ef7c566af [cxx] externC for some icalls, for wasm and ios. (#17444)
Maybe all JIT icalls?
2019-10-24 17:59:11 -04:00
Jay Krell fc390a5f24 Include signal.h unconditionally. (#17430)
Some of the includes already were unconditional, including both on Win32 and Unix.
It is part of ANSI C 89, albeit in a reduced form compared to typical Unix (no sigaction).

Remove configuration of MSG_NOSIGNAL which has no other uses.

Preserve configuration of signal.h temporarily due to other uses of ifdef HAVE_SIGNAL_H.
 i.e. LLVM?
2019-10-24 17:44:57 -04:00
Jay Krell ec81a868a1 [cxx][x86] int/ptr casts. (#17441)
* [cxx][x86] Another int/ptr cast.

* More casts. Notice these are under
`#if defined (MONO_ARCH_USE_SIGACTION) && !defined (MONO_CROSS_COMPILE)`
which would explain them not showing up.
2019-10-24 17:37:12 -04:00
Dotnet-GitSync-Bot aea7214eb4 Optimize GC.AllocateUninitializedArray and use it in StringBuilder (#27364) (#17550)
* use GC.AllocateUninitializedArray for allocating internal char buffers

* force inlining of AllocateUninitializedArray to have no perf hit on the small buffers hot path

* insrease the threshold from 256 to 2048 bytes

* use Unsafe.As instead of a cast

* remove the size precondition from AllocateNewArray method, the called AllocateSzArray is responsible for handling negative size

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:02:11 -04:00
dotnet-maestro[bot] a7287bdae5 [master] Update dependencies from dotnet/arcade dotnet/core-setup dotnet/corefx (#17458)
* Update dependencies from https://github.com/dotnet/arcade build 20191018.2

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19518.2
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19518.2

* Update dependencies from https://github.com/dotnet/core-setup build 20191021.2

- Microsoft.NETCore.App - 5.0.0-alpha1.19521.2

* Update dependencies from https://github.com/dotnet/corefx build 20191020.7

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19520.7

* Update dependencies from https://github.com/dotnet/arcade build 20191023.3

- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19523.3
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19523.3

* Update dependencies from https://github.com/dotnet/core-setup build 20191023.10

- Microsoft.NETCore.App - 5.0.0-alpha1.19523.10

* Update dependencies from https://github.com/dotnet/corefx build 20191023.8

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19523.8

* Disable a file system watcher test on mac

See https://github.com/mono/mono/issues/17547

* Reenable tests that were fixed
2019-10-24 21:06:46 +02:00
Bernhard Urban-Forster 25b53a642b [interp] fix code length for JitInfo (#17538)
* [interp] fix code length for JitInfo

* fix c++ build
2019-10-24 20:13:27 +02:00
Tanner Gooding 0595c0e6d7 Move down S.N.Vector APIs and expose new conversion support (#27401)
* Updating S.P.Corelib to start building Vector2/3/4 and supporting types

* Removing HAS_INTRINSICS usages from Matrix4x4

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 14:10:07 -04:00
Marek Safar 81a4a19f0a [netcore] Fix corerun build deps 2019-10-24 14:09:14 -04:00
Martin Baulig 1553889bc5 Cleaning up SslStream, MobileAuthenticatedStream and TaskToApm. (#17393)
### Overview

After the removal of the Legacy TLS Provider, we can make some cleanups and improvements to the `SslStream` class.  Since both the remaining providers use the internal `MobileAuthenticatedStream` and support Task-async code, we can get rid of some of the Begin/End async logic and the the underlying
`Stream` class handle it instead.

This also fixes a nasty issue with some of those task objects leaking unobserved exceptions.  The problem was that `SslStream.Dispose()` needs to clear out all resources, but in doing so also nulls out the `Impl` handle to the `MobileAuthenticatedStream` - so a subsequent `End*()` call will not reach it's corresponding `TaskToApm.End()` method.

There are also some cleanups in the internal APIs that were only ever used internally and not intended for public consumption.

### Mono.Security.dll

* `Mono.Security.Interface`: removed internal `ICertificateValidator2` interface.

* `Mono.Security.Interface.IMonoSslStream`: this internal interface is only used by the web-tests, which only use it as a way to get the `SslStream` instance, so all those `Begin/End` async methods are now gone.

* `Mono.Security.Interface.IMonoSslStream2`: removed another internal interface.

* `Mono.Security.Interface.MonoTlsProvider`: removed some `internal abstract` methods; these have been moved into a new `Mono.Net.Security.MobileTlsProvider` inside `System.dll`.

### System.dll

* `Mono.Net.Security.MobileTlsProvider`:
  New abstract internal base class, which received the aforementioned abstract internal methods.  The naming is such to make it consistent with the already existing `MobileTlsContext` and `MobileAuthenticatedStream`.

Since both `AppleTlsProvider` and `BtlsProvider` now have an abstract internal base class inside `System.dll`, they can use the `ChainValidationHelper` class directly and we can also avoid some `Mono.Security` dances like for instance that `MonoSslPolicyErrors` enum.

This will also help make the code easier to read and maintainer.
  
* `Mono.AppleTls.AppleTlsProvider` and `Mono.Btls.MonoBtlsProvider`: change base class into `MobileTlsProvider` and reflect above mentioned overload changes.

* `Mono.AppleTls.AppleCertificateHelper`: use `ChainValidationHelper` as well.

* `Mono.Net.Security.ChainValidationHelper`: only implement `ICertificateValidator`.
  The `ICertificateValidator` interface is still required because it is in use by the web-tests, but the previous `ICertificateValidator2` was an internal dance.

* `Mono.Net.Security.MobileAuthenticatedStream`: largely cleaned up this class.
  - we now implement the current slimmer version of `IMonoSslStream`.
  - moved all those overloads into `SslStream` that only fill in default values for some of their arguments and then call another overloaded version of their own.
  - removed lots of `Begin/End` methods that are not used anymore.

* `Mono.Net.Security.MonoTlsProviderFactory`: use `MobileTlsStream` everywhere.

* `Mono.Net.Security.MonoTlsStream`: cleanup `SslStream` creation logic; instead of going throw `provider.CreateSslStream()`, we can now use an internal constructor directly, thus avoiding the `IMonoSslStream` interface while also making the code a lot easier to read and understand.

* `System.Net.Mail.SmtpClient`: same here.

* `System.Net.HttpWebRequest`: use `MobileTlsProvider`.

* `System.Net.Security.SslStream`: largely cleaned up this class.
  - we now use `MobileAuthenticatedStream` directly instead of the `IMonoSslStream` interface and therefore could also get rid of `IMonoSslStream2`.
  - cleanup constructor logic to allow the internal constructor being used with a `MobileTlsProvider` instead of going through `provider.CreateSslStream()`.
  - all those overloads that are only filling in default values for some of their arguments before calling another overloaded version of their own now live here instead of in the `MobileAuthenticatedStream` class.  The reason is to simply that `MobileAuthenticatedStream` is already quite a complex and difficult class, and removing some of that complexity out of it will make it easier to understand.
  - handle the `TaskToApm.Begin()` / `TaskToApm.End()` here to ensure that we are not leaking any task objects with unobserved exceptions.
  - removed `BeginRead()` and `BeginWrite()` overloads to let the underlying `Stream` class handle these.


* Bump API snapshot submodule

* [csproj] Update project files
2019-10-24 19:42:54 +02:00
Calvin Buckley 638e2c6a86 Fix eglib on AIX/PASE (#17476)
* fix eglib on AIX/PASE

	* Work around access(2) syscall bug on PASE
	  On PASE, X_OK will always returns true. This is contrary to
	  how AIX behaves (though corresponds to documentation!). This
	  is probably because it's an upcall to the ILE version of
	  access, and that seemingly always returns true.

	* Always export the dummy function in the test suite
	  The AIX linker didn't export this properly, without using the
	  giant sledgehammer of -bexpall. Write an export file and use
	  it. This required some build system changes to expose an
	  AIX-only automake conditional.

	* Use a file that exists on PASE for tests
	  PASE doesn't use Unix-style DNS resolution, so /etc/hosts is
	  missing. Use /etc/magic, since that seems to be present.

	* Don't use /bin on AIX/PASE for tests
	  This is a symlink to /usr/bin, and it seems chdir/getcwd
	  follows links. Use /usr instead, that is unlikely to be a
	  symlink.

	* Make a note what directory we're in if we fail test_cwd

* Change manual executable access check change per Jay

* Change test on POSIXy platforms to always use root directory
2019-10-24 18:04:20 +02:00
Vlad Brezae b7ef736338 [interp] Add super instructions for field storing (#17525)
* [interp] Add also floating versions for stloc.np

* [interp] Add super instructions for stfld

When storing into a field of a local or an argument (ldloc/ldarg -> random instructions that end up pushing one value -> stfld). We add this instruction to the cprop pass because the instructions that are optimized together are not consecutive and we need the stack information to be able to access the instruction that loaded the argument / local and to make sure that the local wasn't dirtied. Alternatively we could make the super instruction pass stack aware, which we should probably do if more super instruction candidates turn out to require stack state information.

* [interp] Enable cprop for methods without locals

We do some optimizations here that don't apply to locals.

* [interp] Fix result stack type of CEE_CONV_OVF_I_UN
2019-10-24 18:05:04 +03:00
Vladimir Sadov 6fd07e4745 Moving StopWatch to src\Common\src\Corelib (dotnet/corefx#42072)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 16:54:28 +02:00
Egor Bogatov 7952e73780 Bump to mono/corefx@fb41040
Includes upstream corefx changes that makes NTLM auth work for more scenarios.
2019-10-24 10:04:24 -04:00
Bernhard Urban-Forster 861812252a [lldb] another fix for handling interp frames (#17535)
[lldb] another fix for handling interp frames

And improved error message in case it breaks again.
2019-10-24 14:07:22 +02:00
Dotnet-GitSync-Bot 2c8e0f68c1 Add AdvSimd (LeadingZeroCount, LeadingSignCount, PopCount) (#27382) (#17532)
* AdvSimd.LeadingSignCount

* AdvSimd.LeadingZeroCount

* AdvSimd.PopCount

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 07:16:49 -04:00
Zoltan Varga 951769b279 [netcore] Add a few more missing SSE intrinsics. (#17511)
* [netcore] Add a few more missing SSE intrinsics.

* Disable some intrinsics for non-llvm.
2019-10-24 06:57:13 -04:00
Johan Lorensson fe7654f745 Adjust restart world logging level to match start major/nursery logging. (#17520)
With this change we will get the following logging on level 1:

2019-10-22 15:46:59 Start nursery collection 38898 00000205A0000000-00000205A1000000, size: 16777216
2019-10-22 15:46:59 restarted (pause time: 2728 usec, max: 689233 usec)

showing the pause time and when runtime was restared, before this fix
we will only see the start log message on level 1.
2019-10-24 04:57:55 -04:00
Filip Navara 7a15b71873 [netcore] Managed ThreadPool implementation (#17387)
* Wire up the managed side of the portable thread pool

* Remove old native thread pool code

* Fix non-netcore build

* Fix Thread.IsThreadPoolThread

* Fix MSVC builds

* Code style fix

* Make the locking primitives uninterruptible

* Revert icall changes for non-netcore

* Implement Thread.ResetThreadPoolThread

* Compile out more thread pool code for netcore

* Remove ThreadPoolBoundHandle, it is shared now

* Address PR feedback on code style
2019-10-24 04:56:25 -04:00
Dotnet-GitSync-Bot 2dfd682ac9 Enable pooling for async ValueTask/ValueTask<T> methods (#26310) (#17531)
Today `async ValueTask/ValueTask<T>` methods use builders that special-case the synchronously completing case (to just return a `default(ValueTask)` or `new ValueTask<T>(result))` but defer to the equivalent of `async Task/Task<T>` for when the operation completes asynchronously. This, however, doesn't take advantage of the fact that value tasks can wrap arbitrary `IValueTaskSource/IValueTaskSource<T>` implementations.

This commit gives `AsyncValueTaskMethodBuilder` and `AsyncValueTaskMethodBuilder<T>` the ability to use pooled `IValueTaskSource/IValueTaskSource<T>` instances, such that calls to an `async ValueTask/ValueTask<T>` method incur 0 allocations (ammortized) as long as there's a cached object available.

Currently the pooling is behind a feature flag, requiring opt-in via the DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKS environment variable (setting it to "true" or "1").  This is done for a few reasons:
- There's a breaking change here, in that while we say/document that `ValueTask/ValueTask<T>`s are more limited in what they can be used for, nothing in the implementation actually stops a `ValueTask` that was wrapping a `Task` from being used as permissively as `Task`, e.g. if you were to `await` such a `ValueTask` multiple times, it would happen to work, even though we say "never do that". This change means that anyone who was relying on such undocumented behaviors will now be broken. I think this is a reasonable thing to do in a major release, but I also want feedback and a lot of runway on it.
- Policy around pooling. Pooling is always a difficult thing to tune. Right now I've chosen a policy that limits the number of pooled objects per state machine type to an arbitrary multiple of the processor count, and that tries to strike a balance between contention and garbage by using a spin lock and if there's any contention while trying to get or return a pooled object, the cache is ignored. We will need to think hard about what policy to use here. It's also possible it could be tuned per state machine, e.g. by having an attribute that's looked up via reflection when creating the cache for a state machine, but that'll add a lot of first-access overhead to any `async ValueTask/ValueTask<T>` method.  For now, it's tunable via the `DOTNET_SYSTEM_THREADING_POOLASYNCVALUETASKSLIMIT` environment variable, which may be set to the maximum number of pooled objects desired per state machine.
- Performance validation.  Preliminary numbers show that this accomplishes its goal, having on-par throughput with the current implementation but with significantly less allocation.  That needs to be validated at scale and across a variety of workloads.
- Diagnostics.  There are several diagnostics-related abilities available for `async Task/Task<T>` methods that are not included here when using `async ValueTask/ValueTask<T>` when pooling.  We need to evaluate these (e.g. tracing) and determine which pieces need to be enabled and which we're fine omitting.

Before shipping .NET 5, we could choose to flip the polarity of the switch (making it opt-out rather than opt-in), remove the fallback altogether and just have it be always on, or revert this change, all based on experimentation and data we receive between now and then.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-24 03:09:50 -04:00
Kenneth Pouncey 1b5658d590 [wasm][sdk] Bump SDK version to 0.2.2 (#17516)
* [wasm][sdk] Bump SDK version to 2.2

* Update SDK version for browser tests
2019-10-24 07:37:28 +02:00
Dotnet-GitSync-Bot e5d88dbd04 Mirror changes from mono/coreclr,corefx,corert (#17526)
* Apply SuppressGCAttribute to some SPCL functions. (dotnet/coreclr#27369)

Apply SuppressGCAttribute to some SPCL functions.

Only insert GC_POLL in first morph call.

Signed-off-by: dotnet-bot <[email protected]>

* Remove unused methods (dotnet/coreclr#27370)

Signed-off-by: dotnet-bot <[email protected]>

* Move Matrix3x2/4x4, Plan, and Quaternion to the shared CoreLib partition (#42021)

Signed-off-by: dotnet-bot <[email protected]>

* Remove Socket.InnerSafeCloseSocket (dotnet/corefx#41888)

* Remove Socket.InnerSafeCloseSocket

* Move IsInvalid checks in SetHandle function

* PR feedback

* Remove unnecessary 'SocketPal.'

* Remove SafeSocketHandle.SetHandle

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 23:01:00 -04:00
Maxim Lipnin 43a938fefb [wasm][xunit tests] Exclude some System.Data tests (#17527) 2019-10-23 22:55:56 -04:00
Thays Grazia 8fda1738f7 [debugger] Changing how debugger handles exception when hybrid suspend is enabled (#17478)
* Removing all fixes to fix #15203 and #17083, and trying a new approach to fix this #17084, together.

The thing is when we are executing on hybrid suspend the caller is executed inside try catch in the runtime_invoke_ method that is generated. And this try catch should be ignored for the debugger exception, otherwise it's the last try catch in main thread, but should always be considered to generate the stack trace.

Fixes #17084

* This test check the case that we were missing. On preemptive this test always runs perfectly, on Hybrid, never run well until this PR.

* Changing how call unit test.
Changing variable name.
2019-10-23 17:56:58 -03:00
Jay Krell 4d7adf4d2a [cxx] Put extern "C" in MONO_LLVM_INTERNAL. (#17443) 2019-10-23 15:36:35 -04:00
Johan Lorensson df30c7228c Optimize LOS for better parallelization. (#17173)
Scanning LOS list as part of minor GC didn't parallelize well since
all scan jobs needed to walk the complete LOS list. This led to touching
a lot of memory walking the list, increasing minor GC pause times
when the number of items in the LOS list increased. As an example,
stressing LOS data structure using ~600 MB's of random sized byte arrays
caused minor GC pause times between ~20ms without parallelization and ~19ms
with parallelization. NOTE, the more work each job gets (more memory to scan)
we will get a better parallelization, but still a lot of room for improvement.

Changing from a normal linked list to a SgenArrayList makes it more effective
to parallelize the scan jobs and since the list is using sequential
memory, this will also reduce cache misses for each thread iterating LOS
objects. Tagging objects in SgenArrayList with reference information reduce
cache misses dramatically in cases where objects not including references
exists on the list, since scanning jobs can skip these items without any
need to touch object memory causing cache misses.

Same scenario running with fix gives us minor GC pause times, without
parallelization in ~7ms (down from 20~ms) and with parallelization,
< 5ms (down from 19ms). NOTE, the more work each job gets (more memory to scan)
we will get a better result using parallelization, so the delta will increase
between using and not using parallelization, this scenario primarily stress
the extreme case where most LOS objects doesn't have references.

In total, this optimization reduce minor GC pause time 4x, from ~19ms when
using parallelization down to < 5ms.
2019-10-23 14:58:04 -04:00
Zoltan Varga 31b475e25a [wasm] Avoid adding an extra arg to native-to-managed wrappers. (#17508)
[wasm] Avoid adding an extra arg to native-to-managed wrappers.

Fixes https://github.com/mono/mono/issues/17389.




<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-23 17:06:18 +02:00
Jay Krell 8b8b8b5010 [coop] Fix big endian problem in MonoThradStateMachine and cleanup. (#17502) 2019-10-23 10:21:54 -04:00
Filip Navara e29f8e32b7 [netcore] Add Unsafe.SkipInit intrinsic 2019-10-23 09:26:09 -04:00
Tanner Gooding e73aa17e23 Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe (#27377)
* Adding SkipInit to Internal.Runtime.CompilerServices.Unsafe

* Updating the runtime to handle Unsafe.SkipInit

* Updating Decimal.DecCalc to use Unsafe.SkipInit

* Removing 'workaround for CS0165' comments

Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 09:26:09 -04:00
Marius Ungureanu bed1b39c6e Bump Bockbuild to pick-up gtk# binding change (#17486)
Fixes https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1005448
Which causes quite a few broken clipboard bugs and possible memory corruption

(cherry picked from commit 989333d6ae)
2019-10-23 14:36:46 +02:00
Kenneth Pouncey bc769e7376 [wasm] move jsvu repo back after fixes for v8 (#17514) 2019-10-23 13:34:36 +02:00
Kenneth Pouncey cdf8b839ae [wasm][binding] Add bindings for calling the entry point of an '.exe' (#17510)
- Add `mono_bind_assembly_entry_point`
- Add `mono_call_assembly_entry_point`
   - Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);

Example:

```
   Module.mono_call_assembly_entry_point("WasmHttpStreamSync", []);
```

```
   var main = Module.mono_bind_assembly_entry_point("WasmHttpStreamSync", []);
   main();
```
2019-10-23 12:43:16 +02:00
Vlad Brezae 479d2dff36 [interp] Add a new super instructions pass (#17489)
Together with a few very simple super instructions. We replace common instruction patterns to avoid indirection via interp stack, instruction dispatch overhead and to allow the compiler to generate better code for the instruction.
2019-10-23 11:57:07 +03:00
Vlad Brezae 4419dc919f [interp] Retry cprop pass if killing stloc (#17491)
When replacing a stloc with a pop, we should retry the pass, since it is very likely that we will be able to kill the instruction that pushed that value on the stack.
2019-10-23 10:39:27 +03:00
Jay Krell 92c08fab4d SetLastError slightly later, in case last error not preserved, and it is just as good. (#17483)
SetLastError slightly later, in case last error not preserved, and it is just as good.
2019-10-23 09:36:15 +02:00
Vlad Brezae 6159407846 [interp] Optimize ldelem instructions (#17494)
Remove the switch and have each instruction have its own code. Seems to improve speed on a benchmark that heavily uses ldelem by about 10-20%.
2019-10-23 10:23:49 +03:00
Zoltan Varga a2cd8b2495 [wasm] Enable threads again on CI, it seems to work again. (#17503) 2019-10-23 02:41:46 -04:00
Dotnet-GitSync-Bot b99a3e9514 Move Vector2/3/4 to the shared CoreLib partition (dotnet/corefx#42005) (#17505)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-23 00:36:20 -04:00
Filip Navara 1e60703a19 [netcore] Disable debugging code in lookup_intrins (#17464) 2019-10-23 00:01:51 -04:00
Aaron Robinson 11dca2a3b8 Add documentation for SuppressGCTransition (dotnet/coreclr#27368)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 21:30:52 -04:00
Zoltan Varga 745e5ba3c3 [wasm] Bump emscripten to 1.39.0. (#17484)
* [wasm] Bump emscripten to 1.39.0.

* [wasm] Fix threads build.
2019-10-22 20:09:07 -04:00
Larry Ewing d08713198f [wasm] fix package target 2019-10-22 18:19:19 -05:00
Larry Ewing 433d7c5004 [wasm] Bump DebugProxy to netstandard 2.1 / netcore 3.0.0 2019-10-22 18:19:19 -05:00
Larry Ewing 15d61d3f47 [wasm] work around jsvu / v8 changes 2019-10-22 15:50:18 -05:00
Tanner Gooding 69407517bc Adding back more arm intrinsics that existing before refactoring (#27153)
* Adding AdvSimd.And

* Adding AdvSimd.AndNot

* Adding AdvSimd.Or

* Adding AdvSimd.OrNot

* Adding AdvSimd.Xor

* Adding AdvSimd.Not

* Removing a trailing whitespace from AdvSimd.PlatformNotSupported.cs

* Adding AdvSimd.Subtract

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Filip Navara fdca1630dc Fix nullability error in portable thread pool (dotnet/coreclr#27366)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Jan Kotas b98911d4f8 Delete Uap quirks from CoreLib (dotnet/coreclr#27356)
This matches cleanup done in CoreFX.

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Adam Sitnik 0663433389 remove double bound check from StringBuilder.Append(char) (#27340)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 14:48:34 -04:00
Aleksey Kliger (λgeek) fb5b2c949d [reflection] mono_method_get_base_method on a GTD should use canonical ginst (#17475)
* [test] GetCustomAttributes on generic type definition

* [reflection] mono_method_get_base_method on a GTD should use canonical ginst

The issue is in how we pass along the generic instantiation.  So there are two
parts here: we start with a method on a GTD which has to decompose into the GTD
and it's instantiation with its own generic parameters, and then we go up to
its parent and grandparent inflating and decomposing the resulting
instantiation. The instantiations keep passing the last parameter as the number
of parameters shrinks so that the gparam index from the original type is now
out of bounds for the parent and grandparent.  If we mess up, we'll get a
BadImageFormatException from the runtime

Fixes https://github.com/mono/mono/issues/17278
2019-10-22 14:07:44 -04:00
Maxim Lipnin bc9b64b807 Get resource string 2019-10-22 12:13:38 -05:00
Aleksey Kliger (λgeek) ebb5846e0d [coop] Use bitfields for coop state machine state representation (#17298)
* [checked] Mask the thread state in GC thread state assertions

Because we have extra bits (like suspend count and the no_checkpoints bit) on
the thread state, we have to mask it to get the actual thread state enum value

* [coop] Use bitfields to represent coop thread state

Instead of using a 32-bit int, use bitfields and type-punning through a union
type.  We need to type pun because we need to access the entire state as a 32-bit int so that we can do atomic CAS.

* use atomic load of thread state in unwrap_thread_state.

Saw one stack trace on CI on the full coop lane where check_thread_state saw STATE_RUNNING
but with a positive suspend count.  Another thread was starting a GC a
the time.  Seems like we saw the state as runnning, but the suspend
count as if we were already in async_suspend_requested.

Thread 22 (Thread 0x7f43c03ff700 (LWP 39860)):
2019-10-22 12:14:05 -04:00
Marek Safar f34fd20fbf [System.Private.CoreLib] Remove unused methods 2019-10-22 11:28:22 -04:00
Filip Navara cf1a9e0033 Update Mono for changes in shared CoreLib partition 2019-10-22 09:46:29 -04:00
Jan Kotas 64c3405594 Cleanup memory copy helpers (dotnet/coreclr#27307)
- Merge RuntimeImports that has just a few FCalls into Buffer.
- Delete assembly implementation of memcpy for ARM. The implementation was outdated and it was used only on Windows ARM and only for fraction of memcpy callsites.
- Delete unnecessary PInvoke into SysStringLen. We have a managed equivalent on System.Runtime.InteropServices.Marshal.
- Delete large amount of redundant code between SecureString.Windows.cs and SecureString.Unix.cs, and switch SecureString to use Spans
- Fold null check into IsWin32Atom

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas d4a2f01064 Rewrite Buffer.BlockCopy in C# (dotnet/coreclr#27216)
* Rewrite Buffer.BlockCopy in C#

Fixes #27106

* Workaround to enable type check optimizations for BlockCopy only

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Tanner Gooding 8ea0426488 Improve codegen for IsNan (dotnet/coreclr#27272)
* Updating double.IsNaN and float.IsNaN to just `return x != x`

* Updating x86 codegen to optimize `x != x` for floating-point

* Changing the isnan optimization to only check the operand assigned registers

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas 113d855adb Delete ENABLE_WINRT define (dotnet/coreclr#27240)
.NET Native specific code not needed anymore

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Ben Adams d5b182c6d8 Improve Dictionary TryGetValue size/perfomance (dotnet/coreclr#27195)
* Dictionary avoid second bounds check in Get methods

* Add NullRef methods to Unsafe

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 274db87e60 Fix DecoderNLS.Convert to out the correct value for 'completed' (#27210)
This is a complementary fix to https://github.com/dotnet/coreclr/issues/23020

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Del Myers 0e02fa55af Enable ETW/EventSource logging of task IDs for boxed state machines (#27115)
* Wrap MoveNext Action for TPL event tracing

Wraps the MoveNext action of the AsyncStateMachineBox in a
continuation wrapper when async causality tracing is on so that the
TPL event source can find the task that is associated with a
continuation. Does not wrap otherwise.

* Clarifying comment

* removing trailing whitespace

* code review feedback.

Makes AsyncMethodBuilderCore.TryFindContinuationTask check to
whether the target of a continuation is itself a task, as a fall-back
to checking to see if the continuation is a ContinuationWrapper

* Got rid of unnecessary null checks

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Levi Broderick 5fdea0b84b Update Utf8String to match latest prototype (dotnet/coreclr#27046)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Stephen Toub d34ba480f9 Use stackalloc expressions with builder ctors (dotnet/coreclr#27194)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Aaron Robinson 296b8d40cf Conditionally remove the GC transition from a P/Invoke (#26458)
* Provide mechanism to remove the GC transition from a P/Invoke.
  New Attribute: 'System.Runtime.InteropServices.SuppressGCTransitionAttribute'

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Jan Kotas cd2388fd6f Replace primitive Array fcalls with managed implementations (#27123)
Prep-work for #27106

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Filip Navara 858fc11900 Move portable thread pool to shared partition (dotnet/corert#7828)
* Move portable thread pool to shared partition

* Rename ClrThreadPool to PortableThreadPool

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Dan Moseley 565b81e26f Annotate System.Resources.ResourceWriter for nullable ref types (#41880)
* Annotate System.Resources.ResourceWriter for nullable ref types

* ref

* Mark common as nullable enable explicitly

* _resourceList

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Eriawan Kusumawardhono f731db32dd Replace tfm based constants with SDK ones (dotnet/corefx#41601)
* replace DefineConstants with SDK ones

Signed-off-by: dotnet-bot <[email protected]>
2019-10-22 09:46:29 -04:00
Zoltan Varga b96260e603 [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now. (#17466)
* [jit] Avoid running mono_handle_native_crash () on the altstack, it can't produce a backtrace. AMD64 only for now.

* Fix checks so stack overflows work again.
2019-10-22 06:31:23 -04:00
Larry Ewing 20752df324 [debugger][wasm] Check the length of the vars array. Fixes #17473 2019-10-21 18:45:57 -05:00
Vlad Brezae c1461e4a2d [interp] Replace loading of static readonly fields with LDC (#17420)
LDC is faster and also allows for constant propagation. Cleanup code a little bit.
2019-10-21 23:42:24 +03:00
Jay Krell fd82c6036e No type_traits on Android.
In file included from /Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall.c:42:
/Users/builder/jenkins/workspace/test-mono-pull-request-amd64-osx-products-sdks-android/mono/metadata/icall-internals.h:89:10: fatal error: 'type_traits' file not found
2019-10-21 11:15:16 -05:00
Kenneth Pouncey 7e522ef179 Modify the check for streaming to also check that the browser Response object exists. 2019-10-21 10:27:05 -05:00
Jay Krell 21d7d41bfe [cxx][debug-interp] Cast once from void* to non-void*. 2019-10-21 10:22:43 -05:00
Kenneth Pouncey 2cea0a89d1 Fix host object existence check.
- The checking for the existence of the object is incorrect: Missing parens around undefined.
- Modify error description returned back to managed code.
- Fix capitalization in global object check error.
2019-10-21 10:21:16 -05:00
Thays Grazia 0d357c1347 [tests] Replace __WATCHOS__ with MONOTOUCH_WATCH (#17459)
Fixes the conditional from mono/mono#17423, we don't define __WATCHOS__ in the Mono BCL builds. We use MONOTOUCH_WATCH instead.
2019-10-21 16:20:12 +02:00
Kenneth Pouncey f07e93da2f [wasm] Make sure message pump is working when executing via NODEJS (#17453)
- Tests if we are running in ENVIRONMENT_IS_NODE
   - run message pump if we are.

- This also fixes GC not running.

- Clean up some messages.
2019-10-21 14:29:18 +02:00
Kenneth Pouncey 3ff90ac0c2 Export mono_load_runtime_and_bcl so that is available in nodejs. (#17456) 2019-10-21 13:50:57 +02:00
Jay Krell 9f50bdb2d9 [Coop] Unconvert Microsoft.Win32.NativeMethods. (#16992)
* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_CloseProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetCurrentProcessId.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetExitCodeProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessTimes.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_GetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetPriorityClass.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_SetProcessWorkingSetSize.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_TerminateProcess.

* [Coop] Unconvert ves_icall_Microsoft_Win32_NativeMethods_WaitForInputIdle.
2019-10-21 10:29:39 +02:00
Zoltan Varga 4b9226619c [interp] Small fixes. (#17442)
* [interp] Really abort in ves_real_abort ().

* [interp] Fix stack usage in TRACE_EXIT.

* [interp] Add a callback to free ThreadContext.

* Fix warnings.

* Remove an empty statement.

* Fix a double free.
2019-10-20 23:28:25 -04:00
Filip Navara 32ab6bcf00 [netcore] Implement Array.GetElementSize as intrinsic (#17437)
* [netcore] Implement Array.GetElementSize as intrinsic

* Remove unnecessary icall

* alloc_preg -> alloc_ireg

* Implement Array.GetElementSize as intrinsic in interpreter

* Address PR feedback
2019-10-20 18:39:39 -04:00
Filip Navara 67c8d0d812 [netcore] Remove unused code for Array.Unsafe[Mov/Load/Store] intrinsics 2019-10-20 12:44:52 +02:00
monojenkins 5fcd00669c [csproj] Update project files 2019-10-20 12:43:59 +02:00
monojenkins b19d9aa658 Bump API snapshot submodule 2019-10-20 12:43:59 +02:00
Zoltan Varga 1d44139d01 Bump cecil. 2019-10-20 12:43:59 +02:00
Filip Navara bc678a2c10 Use correct variable in mini_get_cpu_features (#17436)
* Use correct variable in mini_get_cpu_features

* Address PR feedback
2019-10-20 01:26:44 -04:00
Filip Navara 91f10aeee4 Remove dead code left out in #17341 (#17433) 2019-10-20 01:23:05 -04:00
Jay Krell 91413f78f3 [jit][x86][cxx] int/ptr casts in mini-x86.c (#17380) 2019-10-19 19:20:56 +02:00
Jay Krell 84f4e1dcbf Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same, and it is only used once. (#17429)
Replace ICALL_DECL_EXPORT with ICALL_EXPORT. They are always the same and it is only used once.
2019-10-19 13:25:54 +02:00
Thays Grazia 105ead8707 Changing behavior of some tests in WatchOs
Needs this ifdefs because of this https://github.com/mono/mono/pull/17260. (#17423)
2019-10-18 21:23:44 -04:00
Aleksey Kliger (λgeek) d8ae32a9ee [bcl] Add null reference checks to Interlocked.Exchange<T> (#17400)
* [bcl] Add null reference checks to Interlocked.Exchange<T>

and Interlocked.CompareExchange<T>

This is to mimic the old behavior in unmanaged when the intrinsic cannot be
used.

The issue was detected via the coreclr acceptance test:
https://github.com/mono/coreclr/blob/mono/tests/src/baseservices/threading/interlocked/exchange/exchangetneg.il

* and netcore
2019-10-18 19:11:22 -04:00
Jay Krell a2992d634b [cxx] [wasm] int/ptr casts esp. m2n-gen. (#17379) 2019-10-19 00:03:00 +02:00
Martin Baulig fffad7e3a8 Remove the Legacy TLS Provider. (#17391)
* `MonoTlsProviderFactory` now treats "legacy" as an alias for "default".

* `Mono.Net.Security/LegacySslStream`: Removed.  This class also wasn't using the `MobileAuthenticatedStream` base class, thus preventing cleanups simplifications in that area.

* `Mono.Net.Security/LegacyTlsProvider`: Removed.

* `Mono.Security/Mono.Security.Protocol.Tls*`: Removed all the old legacy code.

* `Mono.Security.Interface.MonoTlsProviderFactory`: Bump internal version to 4.
2019-10-19 00:02:48 +02:00
imhameed 8836231459 Merge pull request #17279 from imhameed/bug-gh-16935-netcore-equalsall-codegen
Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers.
2019-10-18 12:47:41 -07:00
Alex Rønne Petersen d44e0e80d7 Some updates to make RISC-V buildable again. (#17408)
Some updates to make RISC-V buildable again.

Also some updates to the toolchain setup documentation.
2019-10-18 20:16:14 +02:00
Kenneth Pouncey a0c7d73d6f [sdks] add .gitignore python __pycache__ (#17414)
This will ignore the __pycache__ files generated after the tools offset build.
2019-10-18 18:08:57 +02:00
Vlad Brezae 37f628750a [profiler] Fix coverage profiler on macos (#17396)
If we have coverage profiling enabled for a method (mono_profiler_coverage_instrumentation_enabled, with coverage_filter callback), then we should be able to allocate a coverage info structure (mono_profiler_coverage_alloc). After https://github.com/mono/mono/commit/52429673dc617fd437518757c04c7448211075e9, coverage_filter was reporting wrapper methods as ok to profile, while mono_profiler_coverage_alloc was refusing to allocate the necessary structure, leading to crashes.
2019-10-18 16:31:47 +02:00
Jay Krell 958b9b2658 [interp][wasm] Remove more varargs to conserve stack. (#17388) 2019-10-18 10:39:05 +03:00
Egor Bogatov 4908d1f19a Enable GSS on Linux and Android (#17223) 2019-10-17 19:01:44 -04:00
Alexander Köplinger e5721bbb23 [corlib] Mark YieldAwaitable struct readonly (#17386)
* [corlib] Mark YieldAwaitable struct readonly

Make it consistent with .NET Standard/Core which makes future API Compat runs report one less difference.

* Bump API snapshot submodule
2019-10-17 16:01:22 +02:00
Vlad Brezae ec653e70bf [interp] Constant folding for integers (#17362)
* [interp] Move instructions together

For easy check if an instruction is unary or binary operation. We consider conversion instructions to be unary operators and compare instructions to be binary operators.

* [interp] Add stloc.np also for longs

* [interp] Add stats for total number of emitted instructions

Also make the killed_instructions counter actually track the total amount of killed instructions. It was not previously tracking instructions killed as part of addition of stloc.np / movloc.

* [interp] Improve logging during cprop

Make dumping instruction support also InterpInst's not just compacted instructions.

* [interp] Constant folding of common unary operators

* [interp] Constant fold binary operations

* [interp] Remove MINT_CONV_U8_I4 opcode

It is equivalent with MINT_CONV_I8_U4.
2019-10-17 14:23:12 +03:00
Jay Krell e8b6fcc945 [cxx] Int vs. enum, static for efficiency, cleanup, fix typo. (#17377)
[cxx] Int vs. enum, static for efficiency, cleanup, fix typo.
2019-10-17 12:25:18 +02:00
Jay Krell e97e1f994d [cxx][x86] int/ptr casts. (#17345) 2019-10-17 12:24:46 +02:00
Vlad Brezae ae0fbd366e [interp] Always intrinsify MemoryBarrier method
As we do it on jit.
2019-10-17 12:53:19 +03:00
Vlad Brezae 0102263645 [interp] Avoid using full barriers for volatile operations
This commit avoid unnecessary memory barriers on x86 and amd64.

Volatile writes have release semantics, volatile reads have acquire semantics. Organize the code a little bit to indicate this. On x86 and amd64 we don't require barrier for acquire and release semantics since they are provided by the arch, so we can skip them. Ideally we should have 3 memory barrier interp instructions for each type, but, currently, they would all seem to call mono_memory_barrier so it is useless at this point.
2019-10-17 12:53:19 +03:00
Johan Lorensson 4f8a82c9fd Upgrade build scripts to support VS2019. (#17269)
Changes will default Windows SDK version as well as Platform Toolset to
the default versions used in the targeted VS version. If the projects are
opened up in VS2015, it should default to Windows SDK 8.1 and v140, but if
the same projects are opened in VS2019, it will default to latest Windows SDK 10
and v142. This way the project files should adapt to used VS version, meaning
that we could still build them using VS2015 (what's currently used on CI) but also
using VS2017 and VS2019. There should not be a need to install any previous versions
of build tools, unless an older version is targeted. It is also possible to set
PlatformToolset when calling msbuild and that should adapt to corresponding default
Windows SDK version for targeted toolset version.

Commit makes many changes and adjustments, aligning all vcxproj files but changes
should not affect build output.
2019-10-17 08:55:50 +02:00
Jay Krell edafa021d3 Use functions instead of macros for is_in/is_out. (#17350)
Use functions instead of macros for is_in/is_out.
2019-10-17 06:46:03 +02:00
Aleksey Kliger (λgeek) b6388e271f [bcl] Remove CompareExchange_T (#17370)
In https://github.com/mono/mono/pull/17341 we change the caller of
CompareExchange_T<T>(ref T, ...) to call CompareExchange (ref object, ...), but
didn't delete the declaration.

It's already deleted in unmanaged and in the netcore version of this file.
2019-10-16 21:11:36 -04:00
Maxim Lipnin 02bc95a51a [wasm][xunit tests] Disable System.IO.Compression.Tests.BrotliEncoderTests (#17351)
Fixes https://github.com/mono/mono/issues/17053
2019-10-16 18:47:16 -04:00
dotnet-maestro[bot] 1dbad7cd88 [master] Update dependencies from dotnet/core-setup dotnet/corefx (#17355)
* Update dependencies from https://github.com/dotnet/core-setup build 20191015.17

- Microsoft.NETCore.App - 5.0.0-alpha1.19515.17

* Update dependencies from https://github.com/dotnet/corefx build 20191015.14

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19515.14

* Reenable a test that was fixed
2019-10-16 23:12:15 +02:00
Jay Krell c0d50719ea [cxx][x86][amd64] Remove unused STORE_MEM_IMM. (#17347)
[cxx][x86][amd64] Remove unused STORE_MEM_IMM.

And there was an int/ptr cast error within.
2019-10-16 22:26:02 +02:00
Thays Grazia 3f9d501a78 Removing execution of network tests from WatchOs. (#17358) 2019-10-16 17:10:27 -03:00
Aleksey Kliger (λgeek) f017835ac1 [bcl][jit] implement Interlocked.Exchange<T> in terms of object (#17341)
* [bcl][jit] implement Interlocked.Exchange<T> in terms of object

Exchange<T> is supposed to be picked up as an intrinsic.  But if it isn't,
since T has a class constraint, call the `Exchange (ref object, ref object, ref
object)` overload instead.

This works around a limitation in the JIT:
https://github.com/mono/mono/blob/82a07273c22b996b08fa88ee2e6632d782ac2242/mono/mini/mini-trampolines.c#L704-L713

if we're in the common call trampoline, and the caller is generic method and
the callee is a generic method, if we're using generic sharing for the caller,
but there's no generic jit info for the callee, we will not patch the call
site.

In this case we had Exchange<T> calling Exchange_T<T>
where the callee is an icall (and evidently didn't have generic jit info).

So every time we called Exchange<T>, we would go through the trampoline when
trying to call Exchange_T<T> without patching the call site.

Addresses part of https://github.com/mono/mono/issues/17334

* Bump Mono

* Also drop CompareExchange_T<T>

* Sprinkle some Unsafe magic.

Mark the non-icall methods that we expect the JIT to treet as intrinsics with [Intrinsic]

* aot-runtime doesn't need Volatile.Read<T>/Write<T> code

Both of those methods are implemented as calls to the object overload of Volatile.Read/Write
2019-10-16 15:35:49 -04:00
Egor Bogatov 58b53299b7 remove GenericVectorTests from rsp (#17366) 2019-10-16 15:24:16 -04:00
Jay Krell de9ccfa63f [cxx][netcore] Goto around init. (#17348)
[cxx][netcore] Goto around init.
2019-10-16 20:56:09 +02:00
Jay Krell 7067524141 [cxx][x86] ifndef DISABLE_JIT around mono_arch_emit_prolog. (#17346) 2019-10-16 20:28:24 +02:00
Egor Bogatov c13f36bdfe fix netcore build with cygwin (#17361) 2019-10-16 14:20:48 -04:00
Egor Bogatov 886cd73784 [netcore] Implement System.IO.HasOverriddenBeginEndRead(Write)… (#17237)
* Implement HasOverriddenBeginEndRead and HasOverriddenBeginEndWrite

* fix typo

* clean up

* more clean up

* more clean up

* convert into intrinsics

* handle end* methods

* Update intrinsics.c

* make cache_System_IO_Stream_slots static

* fix race condition

* Rollback to icall

* Update icall.c

* Address feedback

* Address feedback
2019-10-16 20:16:51 +02:00
Marek Safar be6f43c80e [System] Add reflection dependency tracking
for built-in type converters
2019-10-16 20:05:03 +02:00
Alexander Köplinger cd6e574b87 [ci] Improve netcore build telemetry by running nupkg and tests through build.sh (#17349)
Follow-up to https://github.com/mono/mono/pull/17331.

We were running nuget pack and tests explicitly in the Azure Pipelines
config so we weren't capturing failure telemetry.
2019-10-16 18:52:13 +02:00
Cody Russell 37dc33a257 [GTK] Bump bockbuild for GtkViewport autoscrolling patch. (#17321) 2019-10-16 16:44:26 +02:00
Jay Krell a03027cab0 Replace mono_assembly_name_free use with mono_assembly_name_free_internal. (#16982)
Replace mono_assembly_name_free use with mono_assembly_name_free_internal.

Maybe just remove the GC mode change and remove the internal form.
We don't really equate malloc/free with GC/locks/hangs, though arguably we should. Arguably this change is right.
2019-10-16 14:25:55 +02:00
Vlad Brezae 0aabccef9b [interp] Add constant propagation of integers (#17326)
* [interp] Make StackValue more intuitive

Replace the opcode field with an enum which will represent what type of value is contained (STACK_VALUE_LOCAL, STACK_VALUE_I4 etc)

* [interp] Add the mint type to the local data

Makes the emitting code easier and we no longer compute the mint type for each local access, which is fairly expensive.

* [interp] Handle indirect locals saner during cprop

Instead of checking for the indirect flag when loading from a local, simply never store a StackValue into such a local since we can't use it. We use the same logic when not pushing on the stack values for such locals.

* [interp] Add constant folding for integers

We decode the values from LDC_I4 and LDC_I8 opcodes and track/propagate the values on the stack and in the locals using the existing infrastructure. When loading a local, we try to load using the known constant value instead, so we might get to kill that local.

* [interp] Similar to stloc.np we can optimize pop/push pairs

If the pushing instruction pushes the same value that was popped by the previous instruction, we can kill them both

* [tests] Remove issue that seems to have been fixed for a while
2019-10-16 13:56:14 +03:00
Jay Krell 7d3fc17e81 Fix C++ WebAssembly build. (#17302) 2019-10-16 10:12:41 +02:00
Filip Navara 0c02bb1207 Fix SafeHandle marshalling in ref/in/out parameters (#17330)
* Fix SafeHandle marshalling in ref/in/out parameters

* Reorder code to make it exception safe, update comments

* Revert the code reordering because it breaks the invariant that new handle returned from native code should always create new SafeHandle on the managed side

* Fix test_0_safehandle_ref test. The test was expecting incorrect behavior, verified on .NET Core 3 and .NET Framework 4.8.

* Address PR feedback

* Fix cut & paste error

* Fix build

* Really fix the test_0_safehandle_ref tests

* Add more tests for SafeHandle marshalling

* Add exclusion for broken CoreFX test
2019-10-16 01:43:42 -04:00
Zoltan Varga d4d93b9699 [wasm] Add a --native-lib option to the packager to allow linking additional native libraries. (#17344) 2019-10-16 01:31:03 -04:00
Kenneth Pouncey 2b4790a1f3 [wasm][tests] WASM Safari browser tests (#16816)
* Add support for tests via safari browser

* Run safari browser tests on CI for Mac lanes

* Update to fix syntax error using UNAME to `if [[ ${CI_TAGS} == *'osx-amd64'* ]];`

* Fix syntax error with `if`
2019-10-16 04:59:55 +02:00
Kenneth Pouncey 2ddcf8cf16 Set default StreamingEnabled property of the WasmHttpMessageHandler to be false. (#17328) 2019-10-16 04:39:06 +02:00
Zoltan Varga 27247739c6 [wasm] Bump emscripten. Remove generated python cache files. (#17340)
* [wasm] Bump emscripten to 1.38.48.

* Remove generated python cache files.
2019-10-15 21:57:52 -04:00
Zoltan Varga 08e8a7f2c3 [jit] Fix is_reference checks for intrinsics with byref parameters. (#17338)
Found by https://github.com/mono/mono/issues/17335.
2019-10-15 21:45:04 -04:00
Ryan Lucia 3fa5f6c66e [loader] Add an explicit define DISABLE_DLLMAP to control dllmap usage (#17315)
* [loader] Add an explicit define ENABLE_DLLMAP to control dllmap usage

* Update configure.ac

* Update configure.ac

* Add dllmap to enable-minimal

* Change to DISABLE_DLLMAP

* Grab some missing MonoDllMap uses
2019-10-15 19:40:24 -04:00
Bernhard Urban-Forster 73ae9dbc36 [bcl] add WriteLine(string) override to CStreamWriter needed due to corefx import (#17336)
In https://github.com/mono/mono/commit/fd424c5dd07b2dbbd4aafbf9165a42166d789438 we switched over to the `TextWriter.cs`/`StreamWriter.cs` implementation of corefx. One subtle difference is that an override for `WriteLine(string)` in `StreamWriter` was introduced:
https://github.com/mono/corefx/blob/8e3b279377d1f01d2d089f24548e33855bf6502e/src/Common/src/CoreLib/System/IO/StreamWriter.cs#L526-L531
Previously the definition in `TextWriter` was used, which would call the base case `Write(char[],int,int)`.  The new override however won't call the base implementation at all, but directly uses the `Span`-variant.

This is a problem for our `CStreamWriter` implementation, since `WriteLine(string)` won't end up in the special handling for `Write(char[],int,int)`, which accounts for terminal cursor changes.

This is the real fix for https://github.com/mono/mono/issues/16701

This also reverts commit 68f5613422. ("[TermInfoDriver] update top/left value on Console.{CursorTop,CursorLeft} access (#16745)")
While it kind of masked the issue, it also posed a performance issue on some terminals (e.g. GNOME Terminal).
2019-10-15 22:29:41 +02:00
Jo Shields dc0eea6fd1 Initial telemetry for netcore builds (#17331)
* Load VSTS telemetry helpers into build.sh script

* First cut adding some basic VSTS-compatible telemetry on failures

* Don't make `ci=true` the default, make it a build.sh flag
2019-10-15 22:21:21 +02:00
Alexander Köplinger 82a07273c2 [ci] Always initialize an AzDO variable
Otherwise the script will not replace the variable and we get confusing error messages like this:

```
[command]/bin/bash --noprofile --norc /home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh
/home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh: line 2: llvm: command not found
/home/builder/azdo/_work/_temp/ba0890e7-8c80-4245-b9dc-14253b0d399f.sh: line 2: [: =: unary operator expected
```
2019-10-15 19:44:03 +02:00
Alexander Köplinger 3b56247afd [ci] Only install android-specific brew packages when building for android 2019-10-15 13:59:34 +02:00
Steve Pfister 806f75fe4f Bumps corefx to mono/corefx@8e3b279 (#17307)
Resolves https://github.com/mono/mono/issues/17288 where RequestUri
in HttpRequestMessage did not allow file:// based Uri's the same
way its constructor did.
2019-10-15 11:06:24 +02:00
Ryan Lucia a2ef1274c3 Update codeowners
Get notifications for various files in metadata
2019-10-15 10:28:31 +02:00
dotnet-maestro[bot] 55e87320f0 Update dependencies from https://github.com/dotnet/corefx build 20191013.1
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19513.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 61db5aafdd Update dependencies from https://github.com/dotnet/core-setup build 20191014.1
- Microsoft.NETCore.App - 5.0.0-alpha1.19514.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 140750f543 Update dependencies from https://github.com/dotnet/arcade build 20191011.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19511.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19511.1
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] 5a6fb5abe7 Update dependencies from https://github.com/dotnet/corefx build 20191008.3
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19508.3
2019-10-15 09:50:26 +02:00
dotnet-maestro[bot] f068c89ca2 Update dependencies from https://github.com/dotnet/core-setup build 20191008.2
- Microsoft.NETCore.App - 5.0.0-alpha1.19508.2
2019-10-15 09:50:26 +02:00
Filip Navara ac1d05ad40 Update CoreFX.issues.rsp
See https://github.com/mono/mono/pull/17232#issuecomment-541973035.
2019-10-14 19:46:39 -05:00
Filip Navara a194de4cdf [netcore] Avoid suspending threads in Environment.Exit, it can hang process (#17318)
CoreCLR doesn't suspend threads on exit (https://github.com/dotnet/corefx/pull/41450#issuecomment-536755093, https://github.com/dotnet/coreclr/pull/1813).

Fixes #17317 
Fixes #15006
Fixes #15010
Fixes #15181

Unblocks #17232
2019-10-15 00:35:26 +02:00
Alexander Köplinger 1a21ad7dd0 [ci] Restore previous values for no Xcode in CI_TAGS
It is still used by the non-pipeline PR builds (we should unify this at some point).
2019-10-14 20:38:46 +02:00
Filip Navara 76068d8d61 Remove broken sin/cos optimization for x86 2019-10-14 12:21:15 -05:00
Filip Navara 05df3970c2 Remove broken sin/cos optimization for x64 2019-10-14 12:21:15 -05:00
Kenneth Pouncey 44e99c5eec [wasm] Continue loading app even when .pdb files are not found (#17266)
* This adds the ability to continue loading the application even though the .pdb files are not found.

When uploading an app built with mono-wasm the `.pdb` may not get committed so they will not exist when trying to load them.

- Add new boolean variable to MONO library support to control skipping these files on errors.
	- `mono_wasm_ignore_pdb_load_errors` default is true

- Modify the assembly file fetching routine `mono_load_runtime_and_bcl` within our MONO module.
	- Allows skipping the .pdb files if the `mono_wasm_ignore_pdb_load_errors` flag is true.
2019-10-14 12:10:43 -05:00
Fan Yang dcb522d181 [netcore] Update dotnet sdk version (#17312)
The previous one was broken.
2019-10-14 18:55:39 +02:00
Johan Lorensson 734b776884 Fix msvc build warning, empty source main-core.c. (#17305) 2019-10-14 18:55:07 +02:00
Alexander Köplinger 7de84059b9 [ci] Fix ios versions in Mono SDKs and fix typo 2019-10-14 18:54:25 +02:00
Ryan Lucia e7b199377c [loader] Locking and coop related cleanup; fix leak 2019-10-14 12:51:29 -04:00
Ryan Lucia bf3afd6cad [loader] Move PInvoke and DllMap code into separate source file 2019-10-14 12:51:29 -04:00
Ryan Lucia 80db55591a [netcore] Remove unused InternalLoadUnmanagedDllFromPath
As of https://github.com/dotnet/coreclr/pull/27090
2019-10-14 12:51:29 -04:00
Ryan Lucia 96050effa6 [netcore] Use a single NativeLibrary.cs for all platforms 2019-10-14 12:51:29 -04:00
Alexander Köplinger 850def2f3b [ci] Use Xcode11.1 for XI/XM Mono SDK builds (#17313)
As requested by the xamarin-macios team.
2019-10-14 18:38:06 +02:00
Marek Safar cae52aa4a6 [System.Net.Http] Clean up HttpMessageHandler setup (#17252)
* [System.Net.Http] Clean up HttpMessageHandler setup

to avoid reflection and not to be recursive

Superseded #15051 and #15922

* [csproj] Update project files
2019-10-14 17:19:22 +02:00
Dotnet-GitSync-Bot 4e81814709 Mirror changes from mono/coreclr (#17297)
* Enable fast-paths on Mono for Array (dotnet/coreclr#27144)

Signed-off-by: dotnet-bot <[email protected]>

* [CoreLib] Remove String fields moved to shared

* Stub Array Try* methods
2019-10-12 08:50:00 +02:00
Ryan Lucia 9832bbb0ac [loader] Fix gnu/lib-names.h define (#17294) 2019-10-11 20:49:24 -04:00
Jay Krell 33d62537d4 [runtime] Add portable cached array creation functions and replace gcc-specific impl. (#16855)
* Remove unused mono_class_get_field_from_name_cached.

* Replace gcc-specific:
  mono_array_class_get_cached
  mono_array_new_cached
  mono_array_new_cached_handle

with portable forms that are easy enough to use and general enough.

Before was "easier" because cache provided for you -- possibly multiple.
Before was more general because rank was a parameter -- and had to be a constant.
It was always 1.
2019-10-11 19:47:20 -04:00
Egor Bogatov 9cc5ea3862 Enable more hw intrinsics for AOT (#17160)
* Enable more hw intrinsics for AOT

* Address feedback

* does it fix the failure?

* fix loadedllvm

* Address feedback

* fix copy-paste

* forgot MONO_LLVM_INTERNAL
2019-10-11 15:09:08 -04:00
Aleksey Kliger (λgeek) 7f95f57c43 [sdks] Android runner: properly shutdown runtime for Mono.Debugger.Soft test app (#17261)
* [sdks] properly shutdown runtime for Mono.Debugger.Soft test app

mini_jit_cleanup will cause mono_runtime_try_shutdown to be called before
mono_runtime_quit which will set the shutting down flags in the runtime.

* [test] Re-enable disabled tests
2019-10-11 13:54:19 -04:00
Kyle White 481b3005c6 Update MERP event type to MonoAppCrash (#17131)
* Update parameters in MERP WerReportMetadata xml

To enable backend analysis, we need to update our parameters to match our back end event type, which is MonoAppCrash.

It doesn't make sense to me that the previous implementation constructed the XML here in Mono, but depended on the user (VSMac in this case) to add the "correct" event type.  The parameters in the XML, must match exactly what is expected for the given EventType, and so we should ensure they are always in-sync by defining them here.  Thus, I removed the usage of `eventType` in `mono_merp_enable` and hard coded the expected eventType.

* [merp] Don't pass eventType from managed

The eventType must me MONO_MERP_EVENT_TYPE_STR ("MonoAppCrash") - the backend
determines the format of the other fields based on the event type, so it
doesn't make sense to pass other values from managed since the runtime controls
the other fields.

For compatability Mono.Runtime.EnableMicrosoftTelemetry will still take a
string parameter (now called "unused" instead of "eventType_str") but it will
not pass it down to the internal call.

* Bump corlib version

* [merp] Revert paramerters in WerReportMetadata.xml

Still need event type to be `MonoAppCrash` but the parameters will be the same as before.
2019-10-11 11:15:21 -04:00
Alexander Köplinger d457346e06 [tests] Bump corefx to get Azure testhost change (#17275)
* Bump corefx to get Azure testhost change

See https://github.com/mono/corefx/pull/360

* Update ClientWebSocketTest as well
2019-10-11 09:34:28 +02:00
Tanner Gooding 5784d064e7 Refactoring the ARM intrinsics to match API review and share code with x86 (#25508)
* Rewriting the ARM64 HWIntrinsic JIT infrastructure to share with and mirror the x86 infrastructure where possible.

* Lowercasing the hwintrinsicarm64 and hwintrinsiclistarm64 files.

* Applying formatting patch.

* Fixing up some names in the PAL and VM that were missed.

* Adding support for AdvSimd.Add

* Ensure impSpecialIntrinsic returns nullptr for unsupported base types.

* Rewording various method headers and simplifying some code.

* Fixing getEmitter to GetEmitter

* Adding support for the ld1 instruction

* Applying formatting patch

* Clarifying a todo comment and removing a stray newline

* Fixing the Arm64 impSpecialIntrinsic to not assert for helper intrinsics

* Fixing some intrinsic signatures that were incorrect

* Adding the initial template files for generating Arm HWIntrinsic tests

* Generating the initial batch of Arm HWIntrinsic tests

* Replace gtRegNum with GetRegNum()

* Fixing the test template alignment checks for Arm

* Regenerating the Arm HWIntrinsic tests

* Fixing the namespace checked for Arm intrinsics

* Changing Base to ArmBase and fixing some Arm hwintrinsic metadata

* Fixing the arm emitter and lowering to not assert for hwintrinsics

* Fixing the arm intrinsics to not set the number of vector elements for SIMDScalar intrinsics

* Updating the AdvSimd.Abs tests to use negative input values

* Regenerating the arm hwintrinsic tests

* Fixing the arm hwintrinsic codegen to use the element size for simd scalar operations.

* Fixing the arm scalar intrinsic tests to check for 0 on upper bits

* Regenerating the ARM intrinsic tests

* Fixing AdvSimd.AbsScalar and AdvSimd.AddScalar to be recursive

* Marking AdvSimd.Arm64 and ArmBase.Arm64 as implemented

* Ensure that AdvSimd.Arm64.Abs is marked as variable size

Signed-off-by: dotnet-bot <[email protected]>
2019-10-11 08:14:11 +02:00
Imran Hameed 7c7d406fc0 Generate LLVM IR for OP_XEQUAL that is recognized by LLVM's vector pattern recognizers. 2019-10-10 16:05:34 -07:00
Jo Shields 5567c83049 Update Linker. This fixes a Xamarin.Android breakage (#17272) 2019-10-10 16:26:10 -04:00
Vlad Brezae 165c114aaa [interp] Kill more instructions (#17226)
* [interp] Replace STLOC to dead local with MINT_POP

* [interp] Optimize MINT_POP

It was only used for 0 and 1 values. The implementation for values larger than 1 was broken anyway.

* [interp] Remove unnecessary usage of MINT_POP1

When storing a static field from a CEE_STFLD instruction.

* [interp] Prevent optimizing away the stack, after a branch

We know the contents of the stack following a branch, so we could have optimized away some instructions preceding the branch. We shouldn't do this because the code we are branching to expects a certain stack size.

* [interp] Clear instructions which push unused values on stack

* [interp] Properly initialize stack when pushing local
2019-10-10 21:29:19 +02:00
monojenkins 3ef0544e7a [csproj] Update project files 2019-10-10 19:43:11 +02:00
Marek Safar 502115435d [System.Net.Http] Make HttpClientHandler on WatchOS always fallback
to platform handler. This simplifies HttpClientHandler in shared libraries
and the "default" socket handler would always throw PNSE anyway
2019-10-10 19:43:11 +02:00
Ryan Lucia 1b6bbfaf36 [eglib] Add newline for failure check prints (#17262) 2019-10-10 09:23:16 -04:00
Johan Lorensson 0db52289a3 Incorrect constrained virtual call in method using gsharedvt for reference type. (#17227)
* Incorrect constrained virtual call in method using gsharedvt for reference type.

For generic constrained virtual calls using reference type's, the
virtual call will be dispatched using instance vtable. When the same
virtual call is taking place in a gsharedvt method, mono_gsharedvt_constrained_call
gets called that will assist making the call, the problem is that it will
use the constrained class for reference types meaning that it will not call
the most derived implementation of the method. This difference in the behavior
compared to none gsharedvt versions of methods, gives inconsistent results.

One way this will reproduce is by using Dictionary<Type, EmptyStruct>. For
methods that will take both Type (reference type) and EmptyStruct (value type)
like Add, calling that through IDictionary (hiding details from compiler) will
use the gsharedvt version of the method. Internally that method will call
Type.GetHashCode (), and since Type really is a RuntimeType it overrides
GetHashCode. But since mono_gsharedvt_constrained_call will use the constrained
class instead of the instance class, the called method will be
Object.GetHashCode, that gives a different result. When later ContainsKey
is called using the same Type, it only takes a reference value, so not
using a gsharedvt version of the method, then the most derived version of
GetHashCode will be called, given a different hash, not finding item
in dictionary.

Fix is to make sure behavior in mono_gsharedvt_constrained_call match
behavior for regular constrained virtual calls when having a reference type.
This makes sure we always call the most derived version of the method.

Commit also adds a test for this specific scenario using a dictionary.

* Updated to a more low-level test.
2019-10-10 09:22:54 -04:00
Bernhard Urban-Forster 6ebf07f2cd [mini] trace snippet should restore return value (#17251)
* [mini] trace snippet should restore return value

This is a problem on architectures with a floating point stack (x86).

Without this PR and without `--trace` the managed-to-native wrapper for `Math.Ceiling` has a basic block like this at the end:

```
AFTER LOWER-VTYPE-OPTS  4: [IN:  BB6(5) BB3(1), OUT:  BB1(3) ]
 fmove R34 <- R16
 ldaddr R35 <- R10
 load_membase R37 <- [R35 + 0x0]
 store_membase_reg [R36] <- R37
 il_seq_point il: 0x2d
```

`R16` contains the return value. `ldaddr`/`load_membase`/`store_membase_reg` is the sequence for LMF popping.  `mono_spill_global_vars` does this with it:

```
 SPILL BLOCK 4:
 fmove R34 <- R16
         ff  34 16
        0  loadr8_membase R34 <- [%ebp + 0xffffffd4]
        1  nop
 ldaddr R35 <- R10
         ii  35 5
        1  add_imm R35 <- %ebp [-36] clobbers: 1
 load_membase R37 <- [R35 + 0x0]
         ii  37 35
        1  load_membase R37 <- [R35 + 0x0]
 store_membase_reg [R36] <- R37
          ii -1 37
        1  store_membase_reg [%edi] <- R37
 il_seq_point il: 0x2d
             -1
        1  il_seq_point il: 0x2d
```

The local register allocator takes care of the FP stack handling:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        [%fr0]
processing:     0  store_membase_reg [%edi] <- %eax
        [%fr0]
processing:     0  il_seq_point il: 0x2d
        [%fr0]
```

At the end of the basic block we have one value left on the FP stack. That's good as we are in the `cfg->bb_exit` basic block, so this is ought to be the return value.

Let's enable `--trace=M:System.Math:Ceiling`. It will insert some code in order to copy arguments/return value to some datastructure that can then be printed by the trace code:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  iconst %eax <- [11116832]
        [%fr0]
processing:     0  iconst %eax <- [52]
        [%fr0]
processing:     0  localloc_imm %eax <-
        [%fr0]
processing:     0  localloc_imm %ecx <-
        [%fr0]
processing:     0  store_membase_reg [%eax + 0x30] <- %ecx
        [%fr0]
processing:     0  x86_lea_membase %edx <- %ebp
        [%fr0]
processing:     0  store_membase_reg [%ecx] <- %edx
        [%fr0]
processing:     0  store_membase_imm [%eax + 0x28] <- [11116832]
        [%fr0]
processing:     0  storer8_membase_reg [%ebp + 0xffffffcc] <- %fr0
        []
processing:     0  x86_lea_membase %ecx <- %ebp
        []
processing:     0  store_membase_reg [%eax + 0x2c] <- %ecx
        []
processing:     0  store_membase_reg [%esp + 0x4] <- %eax
        []
processing:     0  store_membase_imm [%esp] <- [11116832]
        []
processing:     0  voidcall [mono_trace_leave_method] clobbers: c
        []
processing:     0  x86_lea_membase %eax <- %ebp
        []
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        []
processing:     0  store_membase_reg [%edi] <- %eax
        []
processing:     0  il_seq_point il: 0x2d
        []
```

At the very beginning we push some value on the FP stack, same as before. However the code inserted by the tracing code will pop that value. Now, when returning to the, it will pop again something from the FP stack, but alas it's gonna be garbage.

With this PR it looks like this:

```
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffd4]
        [%fr0]
processing:     0  iconst %eax <- [11116832]
        [%fr0]
processing:     0  iconst %eax <- [52]
        [%fr0]
processing:     0  localloc_imm %eax <-
        [%fr0]
processing:     0  localloc_imm %ecx <-
        [%fr0]
processing:     0  store_membase_reg [%eax + 0x30] <- %ecx
        [%fr0]
processing:     0  x86_lea_membase %edx <- %ebp
        [%fr0]
processing:     0  store_membase_reg [%ecx] <- %edx
        [%fr0]
processing:     0  store_membase_imm [%eax + 0x28] <- [11116832]
        [%fr0]
processing:     0  storer8_membase_reg [%ebp + 0xffffffcc] <- %fr0
        []
processing:     0  x86_lea_membase %ecx <- %ebp
        []
processing:     0  store_membase_reg [%eax + 0x2c] <- %ecx
        []
processing:     0  loadr8_membase %fr0 <- [%ebp + 0xffffffcc]
        [%fr0]
processing:     0  store_membase_reg [%esp + 0x4] <- %eax
        [%fr0]
processing:     0  store_membase_imm [%esp] <- [11116832]
        [%fr0]
processing:     0  voidcall [mono_trace_leave_method] clobbers: c
        [%fr0]
processing:     0  x86_lea_membase %eax <- %ebp
        [%fr0]
processing:     0  load_membase %eax <- [%ebp + 0xffffffdc]
        [%fr0]
processing:     0  store_membase_reg [%edi] <- %eax
        [%fr0]
processing:     0  il_seq_point il: 0x2d
        [%fr0]
```

Leaving us the return value on the FP stack before returning.

Fixes https://github.com/mono/mono/issues/16410
2019-10-10 11:21:23 +02:00
Jay Krell 8c5581a6f2 Fix g_assert_not_reached message regression. (#17263)
Fix g_assert_not_reached message regression.
2019-10-10 10:26:14 +02:00
Vlad Brezae d4fbbed106 [tests] Disable tests that crash on android sdks (#17255)
Running these tests makes the application crash, probably at shutdown. Even if the tests are reported as passing, check if disabling them fixes the suite timeout on android osx sdks.
2019-10-10 10:51:53 +03:00
Jay Krell 8718b75b7a Remove varargs from g_assert and g_assert_not_reachable. (#17254)
Remove varargs from g_assert and g_assert_not_reachable (save 200+ bytes per frame in wasm interp).

g_assertf unchanged.

I have a strong suspicion, this will save approximately 432 - sizeof (InterpFrame) bytes of stack
per function call in the WebAssembly interpreter.

It will not significantly benefit or hurt any other system.
Systems with disabled asserts will receive a small size improvement.

WebAssembly has no provision for varargs.
Emscripten emulates it.
It appears the emulation allocates room in the frame per outgoing varargs call.
No stack packing.
So each g_assert takes space.

Alternative would be to discourage or disable or remove asserts.
2019-10-09 22:56:06 +02:00
David Bueno 7df7ceb6d3 [System.Web] Now IsExpired property for FormsAuthenticationTicket compares two dates with same kind (UTC) to avoid unexpected results (#17162)
Fixes #17161
2019-10-09 20:50:46 +02:00
Steffen Kieß 30886f997f [Mono.Posix] Add support for memfd_create() and file sealing (#17225)
* [Mono.Posix] Add support for memfd_create() and file sealing

Add the linux syscall memfd_create() and add support for file sealing with
fcntl().

* Bump API snapshot submodule
2019-10-09 20:49:14 +02:00
Bernhard Urban-Forster 1cf081529e [mini] print inserted instruction in verbose logging (#17249)
[mini] print inserted instruction in verbose logging

before:
```
 fmove R11 <- R9
         ff  11 9
        1  nop
```

after:
```
 fmove R11 <- R9
         ff  11 9
        0  loadr8_membase R11 <- [%ebp + 0x8]
        1  nop
```
2019-10-09 20:16:04 +02:00
Filip Navara cbc8cc658b [mini] Fix check for FastAllocateString that used old name (#17219)
* Fix check for FastAllocateString that used old name

* Fix linker scripts

* Update the icall name for FastAllocateString to match managed name

* Update size check in managed allocator

* One more 64-bit target fix
2019-10-09 15:47:43 +02:00
Vlad Brezae 8e649c3545 [sgen] Fix invalid value passed to write barrier (#17236)
When doing memory copy dest is the destination address and src is the source address. If we are copying a single reference, we need to emit a write barrier and pass to the write barrier the value, not the pointer where the value is stored.
2019-10-09 14:37:41 +03:00
Alexander Köplinger 1e9ba3a026 Add drawing type converters to mobile profiles (#17238)
* Add drawing type converters to mobile profiles

Needed for https://github.com/xamarin/xamarin-android/issues/3726

* Bump API snapshot submodule

* [csproj] Update project files
2019-10-09 12:17:53 +02:00
Johan Lorensson 7fad4a12e2 Running --enable-msvc-only didn't find jay.vcxproj. (#17228) 2019-10-09 10:06:09 +02:00
Egor Bogatov 12ff246297 Optimize ToString() for byte, ushort, uint and ulong (dotnet/coreclr#27056)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-09 09:25:18 +02:00
Filip Navara 28b3446d9e [netcore] Fix NULL dereference when running LLVM JIT 2019-10-09 08:39:22 +02:00
Ryan Lucia 1cfb39203b Replace InternalLoadUnmanagedDllFromPath with NativeLibrary.Load (#27090)
Both just call into NDirect::LoadLibraryFromPath, and so this simplifies the code by removing a superfluous QCall

Signed-off-by: dotnet-bot <[email protected]>
2019-10-09 08:23:30 +02:00
Jay Krell cacf99b973 Revert mono_runtime_set_main_args in 44ff0597b8. (#17233)
It breaks Xamarin.Android.
2019-10-09 00:54:44 -04:00
Vlad Brezae 579e385b96 [interp] Small cleanups (#17229)
* [intepr] Add define for constant

* [interp] Pass separately pop_vt_sp argument

Cleans up the code a bit. There is no need to do fancy bit operations for such a slow opcode.
2019-10-09 00:08:44 +02:00
Filip Navara 63025d93fd [netcore] Port CoreCLR implementation of Exception.SetCurrentStackTrace (#17243)
* Port CoreCLR implementation of Exception.SetCurrentStackTrace

* Update SerializationRemoteStackTraceString

* Update Exception.cs
2019-10-08 23:45:17 +02:00
Alexander Köplinger 4ca42a933f [man] Update MONO_IOMAP docs as it no longer works with corefx System.IO (#17231)
Fixes https://github.com/mono/mono/issues/15845
2019-10-08 21:39:31 +02:00
Alexander Köplinger 209c4bae3e [netcore] Disable some SafeWaitHandle tests everywhere, not just Linux (#17230)
The tests are kinda bogus since they create a SafeWaitHandle from `new IntPtr(1)` which causes a crash during finalization when releasing that handle.

See https://github.com/mono/mono/issues/17224
2019-10-08 17:39:41 +02:00
dotnet-maestro[bot] 52c4d87633 Update dependencies from https://github.com/dotnet/arcade build 20191006.1 (#17214)
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19506.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19506.1
2019-10-08 13:30:46 +02:00
Filip Navara e8ed2494ca Fix check in fix_libc_name to trigger only for libc, not or other names with same prefix (#17222)
See https://github.com/mono/mono/pull/17210#pullrequestreview-298663220
2019-10-08 13:06:38 +02:00
Alexander Köplinger faf100da27 [support] Make sure map-impl.h ends up in the tarball 2019-10-08 12:05:47 +02:00
Steffen Kieß 8bb7b8645a Clean up map.c /map.h (#16746)
* [Mono.Posix] Move Mono_Posix_{From,To}MremapFlags() to support/sys-mman.c

Since b522eab5ff Mono_Posix_FromMremapFlags()
and Mono_Posix_ToMremapFlags() contain manual changes for NetBSD. Move the
functions from support/map.c to support/sys-mman.c so that they won't be
overwritten when create-native-map is rerun.

* [Mono.Posix] Fix prototypes in support/stdio.c

In ed892ccf27 wrapper functions for several
stdio functions were added, but the prototypes in map.h were manually
written, not using create-native-map. This commit changes the prototypes so
that they match the output of create-native-map.

* [Mono.Posix] Undefine HAVE_STRUCT_SOCKADDR_IN6 in support/map.c on MSVC

In 22b6b95814 a manual change to support/map.c
was added to prevent Mono_Posix_FromSockaddrIn6() and
Mono_Posix_ToSockaddrIn6() from being built for windows even if
HAVE_STRUCT_SOCKADDR_IN6 is defined. Instead undefine it in a header file
to avoid manual changes to map.c.

With this commit map.c, map.h and NativeConvert.generated.cs contain no
manual changes anymore.
2019-10-08 12:04:45 +02:00
monojenkins ad2132e611 [bcl] Update BCL Linked Size 2019-10-08 10:23:41 +02:00
Ryan Lucia 3355d2dd81 [netcore] Add libc name transition 2019-10-08 10:22:18 +02:00
Alexis Christoforides 37c197f225 Remove unused variable 2019-10-07 21:04:47 -05:00
Alexis Christoforides 2ca93bd2f9 [merp] Print missing status marker file for stage 1 (setup) 2019-10-07 21:04:47 -05:00
Alexis Christoforides b18d52bf1c Zeroing is not necessary, snprintf() always null-terminates in this case 2019-10-07 21:04:47 -05:00
Vlad Brezae d9c9a53332 [interp] Rerun cprop pass if stloc.np intructions were eliminated
If we eliminate these instructions, it means we removed instructions that use the top of stack. Without them we might be able to optimize away that stack slot usage. If we retraverse the instruction stream, then we must handle movloc and stloc.np separately.
2019-10-07 19:35:57 -05:00
Vlad Brezae f2006f61dc [interp] Handle more cases for stloc.np optimization
We emit stloc.np if we have "stloc local; ldloc local". Bu we can also emit if we have "stloc loc1; ldloc loc2" and loc2 == loc1.
2019-10-07 19:35:57 -05:00
Vlad Brezae fc100ae043 [interp] Skip NOPs when checking previous instruction
When emitting stloc.np, which looks for stloc/ldloc pair.
2019-10-07 19:35:57 -05:00
Calvin Buckley 01dd63c440 dladdr shim for gmodule; try to enable crash reporter on AIX (#15808)
* Introduce new dladdr wrapper in glib, AIX reimpl, and convert usage

Provides a wrapper around dladdr that should be platform-neutral.

Also provided is a reimplementation for AIX, which should enable
that platform to get some previously dladdr-specific functions like
the crash reporter. It is somewhat flawed in that it only gets info
for symbols in .text, returns non-constant allocated strings, and
allocates heap memory for buffers.

Then convert usages (except a macOS specific one) of dladdr to the
new eglib function. It has a different signature, which should
simplify intent and reduce structiness. Also free memory on AIX due
to limitations of its reimplmenetation.

* Enable (and fix build of) crash reporter on AIX

So far crashes don't seem any different, but the code for it is
built.

* typo fix (guess this isn't compiled on aix)

* attempt to fix macOS and win32 builds

* Implement more of g_module_address on Win32

Implements the filename stuff. Untested; don't have Windows.

* Look for backtrace_symbols in an IBM compat library for i

Enables i to natively dump a stack trace. (AIX "works" too if you
copy the compat lib's code and hack it in - it probably needs to be
in EGlib.) Unfortunately, it's not perfect. The backchain from the
signal handler is empty on i, and AIX sometimes mangles the stack
frame in a way that severely confuses the backtrace code, causing
it to dereference invalid memory by interpreting instructions as a
pointer. (I have seen somewhat similar for the sigaltstack case,
where the native memory dump attempts to read something it should
not.) Perhaps mincore could be used, but that feels like a sloppy
workaround, especially in the "AIX making an invalid stack frame"
case.

* If the file isn't an archive, don't format its name like one

* Fix up search for backtrace_symbols to look in libexecinfo

FreeBSD ships it in base and Haiku has a package; likely others too.

* Win32 suffers in g_module_address as well

* That should habe been guarded with AIX due to limitations

Otherwise seems to free things that shouldn't be freed

* Casting for win32

* Fix up Win32 impl of module address

* Free HMODULE to lower reference count after using it

* Change dladdr based impl to always dup for impl consistency

Means freeing afterwards shouldn't be ifdefable.

* probably shouldn't strdup NULL

Likely why macOS has failures.

* Don't dup/return const strs, but copy to caller-provided buffers

...which can be on the stack. This makes it far less risky to use
in a crash reporter scenario, because malloc can be hosed then.

Also fix profiler up better to work with this new reality.
2019-10-08 00:19:28 +02:00
Zoltan Varga c7f8e5bdb7 [offsets-tool] Update the README. (#17212) 2019-10-07 22:37:09 +02:00
Jo Shields d28d72cb96 Bump version to 6.9 (#17207)
* Bump version to 6.9

* Add missing new linker source file

* [csproj] Update project files

* No monolinker on MCS mode, new linker version needs Roslyn
2019-10-07 22:36:13 +02:00
Ryan Lucia 1d18c27dbe Don't declare mono_get_module* on non-win32 2019-10-07 09:44:41 -05:00
Ryan Lucia ca5926fa3f Use eglib win32 helper functions without MAX_PATH limit 2019-10-07 09:44:41 -05:00
Ryan Lucia d12db16723 Escape type namespaces, document that we aren't escaping names for now 2019-10-07 09:43:47 -05:00
Ryan Lucia 8760cd446c Make the mono identifier escape function work like unescape and public 2019-10-07 09:43:47 -05:00
Ryan Lucia 7c2d2a551c Unescape type names in the managed parser 2019-10-07 09:43:47 -05:00
Ryan Lucia f37c3a649f Make TypeNameParser consistently use tabs
Is this worth the line damage? Unsure.
2019-10-07 09:43:47 -05:00
Ryan Lucia fa8fbbbc36 Partially revert "Remove redundant function call of EscapeTypeName to make (#16425)"
This partially reverts commit abe48e3fd4.
2019-10-07 09:43:47 -05:00
Jay Krell f0ff9f3860 Do not zero localalloc if not asked. (#16881)
Note that we will still touch the pages (mono_emit_stack_alloc).
2019-10-07 09:42:13 -05:00
Jay Krell 957e6fa4dd [Coop] Convert Get/SetGenericValueImpl. (#17034)
This is an alternative to https://github.com/mono/mono/pull/16994
and does *not* depend on
   https://github.com/mono/mono/pull/16987
or https://github.com/mono/mono/pull/17009
or similar.
2019-10-07 09:40:22 -05:00
Alexander Köplinger 495b08a3c7 Default --host in autogen.sh on cygwin when missing. (#17164) 2019-10-07 16:38:49 +02:00
Ryan Lucia 6f72363932 [IO] Remove read-only logic in mono_w32_get_disk_free_space (#17177)
This brings us in line with GetDiskFreeSpaceExW on Windows as well as .NET Core.
2019-10-07 09:38:23 -05:00
Junil Um 026d82fac3 Fixed a bug in mono-service.cs (#17095)
There is a bug in mono-service.cs file.

`setup.ApplicationBase = Environment.CurrentDirectory;`;

General `Environment.CurrentDirectory` does returns `Directory.GetCurrentDirectory()`.
And retuened value 'current path' without last character `/` but `AppDomain.CurrentDomain.BaseDirectory`;

It have a problem in my service application.

In my service application code,

```cs
using System;
using System.IO;

namespace BaseDirTest
{
    internal class Program
    {
        public static void Main(string[] args)
        {
            var baseDir = AppDomain.CurrentDomain.BaseDirectory;
            Console.WriteLine($"baseDir={baseDir}");
            Console.WriteLine($"Path.GetDirectoryName(baseDir)={Path.GetDirectoryName(baseDir)}");
        }
    }
}
```

When I launch `mono app.exe`,
```bash
baseDir=/workspace/test/mono-owin-test/ConsoleApplication1/BaseDirTest/bin/Debug/
Path.GetDirectoryName(baseDir)=/workspace/test/mono-owin-test/ConsoleApplication1/BaseDirTest/bin/Debug
```

But when I launch `mono-service --no-daemon app.exe`,
```bash
eDir=/workspace/test/mono-owin-test/ConsoleApplication1/mono-service-umc/bin/Debug
Path.GetDirectoryName(baseDir)=/workspace/test/mono-owin-test/ConsoleApplication1/mono-service-umc/bin
```

Because my service application can not founds plugin directory and scans it.
2019-10-07 16:28:40 +02:00
Bernhard Urban-Forster 6861610662 [lldb] use mono_method_full_name helper (#16934)
[lldb] use mono_method_full_name helper

Instead of:

```
(lldb) mbt 20
* thread #11
  * frame #0: 0x1bf28f28 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x1bf9caac libsystem_pthread.dylib`pthread_kill + 300
    frame #2: 0x1be66080 libsystem_c.dylib`abort + 140
    frame #3: 0x024bdfa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="../../../../../mono/metadata/object.c:1905: Expected GC Unsafe mode but was in STATE_BLOCKING state", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #4: 0x02487f44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #5: 0x02487ee0 monotouchtest`monoeg_g_logv(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>, args=<unavailable>) at goutput.c:156:10 [opt]
    frame #6: 0x02487f74 monotouchtest`monoeg_g_log(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>) at goutput.c:165:2 [opt]
    frame #7: 0x024694c4 monotouchtest`assert_gc_unsafe_mode(file="../../../../../mono/metadata/object.c", lineno=1905) at checked-build.c:396:3 [opt]
    frame #8: 0x023d2f64 monotouchtest`mono_class_vtable_checked(domain=0x16d64800, klass=0x173c1b98, error=0x194b0ee8) at object.c:1905:2 [opt]
    frame #9: 0x024130a8 monotouchtest`get_current_thread_ptr_for_domain(domain=0x16d64800, thread=0x02db45d0) at threads.c:635:2 [opt]
    frame #10: 0x024119a8 monotouchtest`mono_thread_current at threads.c:2026:23 [opt]
    frame #11: 0x02416998 monotouchtest`mono_thread_interruption_checkpoint_request(bypass_abort_protection=0) at threads.c:5101:35 [opt]
    Messaging::void_objc_msgSendSuper @ 388435850 "calli.nat.fast" || frame #12: 0x024d5de8 monotouchtest`interp_exec_method_full(frame=0x194b11c0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3230:4 [opt]
    ObjCExceptionTest::InvokeManagedExceptionThrower @ 388435752 "vcall" || frame #13: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1380, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    ExceptionsTest::ManagedExceptionPassthrough @ 388502134 "vcallvirt.fast" || frame #14: 0x024d61c0 monotouchtest`interp_exec_method_full(frame=0x194b14e0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3325:4 [opt]
    Object::runtime_invoke_direct_void__this__ @ 388462550 "vcall" || frame #15: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1598, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    frame #16: 0x024d46ec monotouchtest`interp_runtime_invoke(method=<unavailable>, obj=0x0302c5f0, params=0x00000000, exc=0x194b166c, error=0x194b18c8) at interp.c:1766:2 [opt]
    frame #17: 0x0232b39c monotouchtest`mono_jit_runtime_invoke(method=0x174fda58, obj=<unavailable>, params=0x00000000, exc=<unavailable>, error=0x194b18c8) at mini-runtime.c:3170:12 [opt]
    frame #18: 0x023d5cc8 monotouchtest`do_runtime_invoke(method=0x174fda58, obj=0x0302c5f0, params=0x00000000, exc=0x00000000, error=0x194b18c8) at object.c:3017:11 [opt]
    frame #19: 0x023d26a0 monotouchtest`mono_runtime_invoke_checked(method=<unavailable>, obj=<unavailable>, params=<unavailable>, error=<unavailable>) at class-getters.h:24:1 [opt] [artificial]
```

It prints now:

```
(lldb) mbt 20
* thread #11
  * frame #0: 0x1bf28f28 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x1bf9caac libsystem_pthread.dylib`pthread_kill + 300
    frame #2: 0x1be66080 libsystem_c.dylib`abort + 140
    frame #3: 0x024bdfa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="../../../../../mono/metadata/object.c:1905: Expected GC Unsafe mode but was in STATE_BLOCKING state", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #4: 0x02487f44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #5: 0x02487ee0 monotouchtest`monoeg_g_logv(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>, args=<unavailable>) at goutput.c:156:10 [opt]
    frame #6: 0x02487f74 monotouchtest`monoeg_g_log(log_domain=<unavailable>, log_level=<unavailable>, format=<unavailable>) at goutput.c:165:2 [opt]
    frame #7: 0x024694c4 monotouchtest`assert_gc_unsafe_mode(file="../../../../../mono/metadata/object.c", lineno=1905) at checked-build.c:396:3 [opt]
    frame #8: 0x023d2f64 monotouchtest`mono_class_vtable_checked(domain=0x16d64800, klass=0x173c1b98, error=0x194b0ee8) at object.c:1905:2 [opt]
    frame #9: 0x024130a8 monotouchtest`get_current_thread_ptr_for_domain(domain=0x16d64800, thread=0x02db45d0) at threads.c:635:2 [opt]
    frame #10: 0x024119a8 monotouchtest`mono_thread_current at threads.c:2026:23 [opt]
    frame #11: 0x02416998 monotouchtest`mono_thread_interruption_checkpoint_request(bypass_abort_protection=0) at threads.c:5101:35 [opt]
    (wrapper managed-to-native) ApiDefinition.Messaging:void_objc_msgSendSuper (intptr,intptr) @ 388435850 "calli.nat.fast" || frame #12: 0x024d5de8 monotouchtest`interp_exec_method_full(frame=0x194b11c0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3230:4 [opt]
    Bindings.Test.ObjCExceptionTest:InvokeManagedExceptionThrower () @ 388435752 "vcall" || frame #13: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1380, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    MonoTouchFixtures.ObjCRuntime.ExceptionsTest:ManagedExceptionPassthrough () @ 388502134 "vcallvirt.fast" || frame #14: 0x024d61c0 monotouchtest`interp_exec_method_full(frame=0x194b14e0, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3325:4 [opt]
    (wrapper runtime-invoke) object:runtime_invoke_direct_void__this__ (object,intptr,intptr,intptr) @ 388462550 "vcall" || frame #15: 0x024d64dc monotouchtest`interp_exec_method_full(frame=0x194b1598, context=<unavailable>, clause_args=<unavailable>, error=<unavailable>) at interp.c:3400:4 [opt]
    frame #16: 0x024d46ec monotouchtest`interp_runtime_invoke(method=<unavailable>, obj=0x0302c5f0, params=0x00000000, exc=0x194b166c, error=0x194b18c8) at interp.c:1766:2 [opt]
    frame #17: 0x0232b39c monotouchtest`mono_jit_runtime_invoke(method=0x174fda58, obj=<unavailable>, params=0x00000000, exc=<unavailable>, error=0x194b18c8) at mini-runtime.c:3170:12 [opt]
    frame #18: 0x023d5cc8 monotouchtest`do_runtime_invoke(method=0x174fda58, obj=0x0302c5f0, params=0x00000000, exc=0x00000000, error=0x194b18c8) at object.c:3017:11 [opt]
    frame #19: 0x023d26a0 monotouchtest`mono_runtime_invoke_checked(method=<unavailable>, obj=<unavailable>, params=<unavailable>, error=<unavailable>) at class-getters.h:24:1 [opt] [artificial]
```



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-10-07 16:16:13 +02:00
Thays Grazia 18fac0a241 [debugger][exception] Debugger breaks on handled exceptions (#17106)
* If there is a perform_wait_callback in the stack there will be another catch generated by the owner thread, so we don't need to throw, we can continue and find the next catch.
Fixes #17083

* Reverting unit test changed on commit 405d521.

* Fixing assert when calling mono_jit_info_get_method if it was a trampoline.
2019-10-07 08:22:06 -04:00
Egor Bogatov 35c6b7691f [netcore] Optimize Buffer.BlockCopy and Buffer.IsPrimitiveArray (#16692)
* Optimize Buffer.BlockCopy and IsPrimitiveArray (make them icalls)

* wrap with ENABLE_NETCORE

* fix formatting

* fix check for "is primitive"

* use m_class_is_primitive

* fix build errors

* fix typo

* fix failing test

* use mono_error_set_argument_out_of_range

* fix argument names for exceptions
2019-10-07 12:29:22 +02:00
Steve Pfister 2f6d69e2e6 Re-ordered the * IsValid check in SiteIdentityPermissions. We were throwing an IndexOutOfRangeException
Fixes https://github.com/mono/mono/issues/17017
2019-10-07 09:43:53 +02:00
Jan Kotas c6dcd3de56 More CompareInfo.IndexOf globalization fixes (dotnet/coreclr#27049)
Apply earlier CompareInfo.StartsWith fixes onto CompareInfo.IndexOf as well

Signed-off-by: dotnet-bot <[email protected]>
2019-10-07 09:41:24 +02:00
Zoltan Varga dea3a20b05 [wasm] Bump emscripten version. (#17179)
* [offfsets-tool] Embed a copy of the clang bindings, the version in PIP doesn't work with newer clang versions.

* [wasm] Bump emscripten version to 1.38.47.
2019-10-06 20:10:43 -04:00
Jay Krell f57af77e69 Optimize and extend System.Array intrinsic identification. (#17047)
* Optimize and extend System.Array intrinsic identification.
i.e. do the strcmp last.
Add GetRawArrayData.

* Use [Intrinsic] attribute instead of comment.
2019-10-06 10:27:42 +02:00
Zoltan Varga d2f543b4b1 [mixed] Add interp entry wrappers for devirtualized calls as well. (#17159)
* [mixed] Add interp entry wrappers for devirtualized calls as well.

* [aot] Generate code for string:memcpy/memset/bzero even in profile mode, they are called by the generated code.

* [aot] Treat string:memcpy as always AOTed in mono_aot_can_enter_interp () as well.

* [aot] Always aot string ctors as well, the JIT doesn't generate interp entry code for them.

* [wasm] Update print-table.py.
2019-10-05 20:51:52 -04:00
Aleksey Kliger (λgeek) cb52186661 [merp] Put thread into async context before running summarizer (#17194)
Followup work for https://github.com/mono/mono/pull/17090

In order to get managed method info for unmanaged stacktraces, we call
mini_jit_info_table_find_ext which calls decode_exception_debug_info which may
allocate if it's not in async context.

Do the switch in mono_threads_summarize_execute so that non-supervising threads
all switch when they run the sigterm_signal_handler.  mono_threads_summarize
already does it for the supervisor thread.

Fixes https://github.com/mono/mono/issues/17180
2019-10-05 13:01:56 -04:00
Alexander Köplinger 0600ea1256 Bump libgdiplus to 6.0.4 2019-10-05 17:36:40 +02:00
Aleksey Kliger (λgeek) 2f7fd4f03e [merp] exit_status is 0 if we ran the uploader successfully (#17185)
Not 1 (TRUE)
2019-10-05 09:59:05 -04:00
Marek Safar fce0e64de9 [System.Private.CoreLib] Fixes build 2019-10-05 14:09:54 +02:00
Stephen Toub 2c8d5f0a39 Add ExceptionDispatchInfo.SetCurrentStackTrace (dotnet/coreclr#27004)
* Add ExceptionDispatchInfo.SetCurrentStackTrace

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub d8c115d07a Fix stackalloc reuse bug in Environment.UserDomainName (#26994)
* Fix stackalloc reuse bug in Environment.UserDomainName

According to the comment, this code is never expected to be used (so it should either be deleted or the comment fixed), but in the meantime, if it were to be executed, things could go badly due to potentially passing the same buffer into LookupAccountNameW for two different arguments.

* Fix existing ValueStringBuilder usage to dispose the builder

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub 670738b55f Use const internally in corelib for Environment.NewLine (#27013)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Stephen Toub 1e3d017479 More corelib cleanup (dotnet/coreclr#26993)
* Make relevant types static (mostly Interop)

* Fix a few stray semicolons

* Remove unnecessary locals

Some can be removed entirely, some can be replaced by discards.

* Remove unnecessary usings

* Move Unix-specific helper method to TimeZoneInfo.Unix.cs

* Remove duplicated code

* Remove duplicative checks

* Remove [ThreadStatic] initialization

These are all the default values and wouldn't be necessary for any static, but inline initialization for [ThreadStatic]s is particularly unuseful (and in non-default cases problematic) because the fields are only set to these values on the initializing thread.

* Fix methods whose return value is never used

* Use string.IsNullOrEmpty

* Remove unnecessary field initialization

All ctors overwrite these values.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Boris Parfenenkov 8205424667 Improve TimeSpan precision (dotnet/coreclr#26992)
Change multiplying by (x * (1.0 / BigValue)) on (x / BigValue).

Fix #41380

Signed-off-by: dotnet-bot <[email protected]>
2019-10-05 14:09:54 +02:00
Larry Ewing b45d0a1084 [aprofutil] Add skip regexes to simplify expression of removal matching (#17146)
* [aprofutil] Add skip regexes to simplify expression of removal matching

* [aprofutil] arguments to skip and take by count for automatic profile testing
2019-10-04 22:23:24 +02:00
Maxim Lipnin 06effa7d7e [wasm] Enable CoreXml.Test.XLinq.LoadSaveAsyncTests.RoundtripSyncAsyncMatches_Stream test (#17183) 2019-10-04 12:42:45 -05:00
Ryan Lucia 05217395db [reflection] Only duplicate MonoMarshalSpec strings for custom types (#17175)
* [reflection] only duplicate MonoMarshalSpec strings for custom types

* Add test
2019-10-04 13:41:56 -04:00
Egor Bogatov 26754eecff [llvm] Propogate --enable-llvm-asserts to llvm/build.mk (#17149)
* fix Enable llvm asserts

* fix chmod
2019-10-04 10:27:01 -05:00
Vlad Brezae d1b3ee2e32 [interp] Improve copy propagation (#17154)
* [interp] Refactor tracking of stack/local values

Previously we had just an InterpInst* inside StackContentInfo, which was representing the instruction that pushed a certain local/value on the stack. This makes many things awkward, since an instruction is logically different from a value, that a local or a stack slot have. If we clear an instruction, it doesn't necessarily mean that the value that the instruction produced can't be stored on stack or in a local.

This commit creates a new structure StackValue, which holds the value.

* [interp] Generalize contents of StackValue

StackValue contains an opcode and some data to enable reconstruction of the value. For example instead of doing a LDLOC for a local, we can see if the local has a known value and use it instead (this could be a LDLOC from another local, whose value was propagated, or in the future a LDC). This will make more sense when we also start to track constant values.

Also decouple MOVLOC instructions from the cprop pass. They serve no purpose there. They are useful though when we do deadce, since we currently don't know how to kill instructions that change the stack.

* [interp] Handle dup opcode during copy propagation

* [interp] Avoid losing track of stack values for some opcodes

For some opcodes, that access stack values below the top of the stack, we were assuming they pop everything and then push the stack back, in order to prevent optimizing away instructions that pushed some of these values, since the original instruction expects those values to reside on the stack. We handle these instruction separately and keep track of the values of the stack, so we can further propagate the stack values, even though we currently can't optimize away those instructions.

* [interp] Propagate values for ctor arguments

* [interp] MINT_CASTCLASS no longer clobbers top of stack
2019-10-04 14:20:52 +03:00
Johan Lorensson 2aa7adb269 Always do copy_stack_data on entering GC safe/unsafe mode. (#17150) 2019-10-04 09:16:05 +02:00
Piotr Zierhoffer 4e8c496082 mkbundle: fix 17139 mono aot register module fix definitions (#17152)
* [mkbundle] Fix mono_aot_register_module declaration (#17139)

This change reflects the declaration introduced in jit.h in #15132.

* [mkbundle] Remove spurious function declaration (#17139)

The comment is clearly outdated after the introduction of #15132.
2019-10-04 08:46:58 +02:00
Aleksey Kliger (λgeek) ea2a460cf6 [merp] Don't overrun buffer in copy_summary_string_safe … (#17176)
* [merp] Don't overrun buffer in copy_summary_string_safe

MonoFrameSummary:str_destr is an array of MONO_MAX_SUMMARY_NAME_LEN bytes, not
MONO_MAX_SUMMARY_NAME_LEN + 1 bytes.

Fixes Coverity CID 1454563

* [merp] Use g_strlcpy for copy_summary_string_safe

Fixes Coverity CID 1454563

We would sometimes write to MonoSummaryFrame:str_descr which is
MONO_MAX_SUMMARY_NAME_LEN bytes long at index MONO_MAX_SUMMARY_NAME_LEN which
is one past the end of the array.

* nit: rename confusing parameter names

old names were confusing - we were copying from 'out' to 'in'.  Now we copy to
'dest' from 'src'
2019-10-04 01:17:49 -04:00
Vlad Brezae 8c1ef74bb1 [arm] Fix fetching of method addresses (#17099)
After https://github.com/mono/mono/commit/9ff3b0d65ee4, in an aot image we can emit addresses either as bl offset (which uses 4 bytes) or as ldr pc, =<label>; <addr> (which uses 8 bytes). Before this commit we were dereferencing an instruction pointer, in get_call_table_entry, assuming 8 bytes per entry. This could overflow and crash if we used in fact only 4 bytes per entry. Fix this by including the entry size in the aot image.

Fixes random crashes on arm CI on full-aotmixed suite.
2019-10-03 22:29:55 +03:00
Alexander Köplinger ea9dad7249 Bump ikvm to get https://github.com/mono/ikvm-fork/pull/13 (#17169) 2019-10-03 20:46:37 +02:00
dotnet-maestro[bot] 06c3637681 [master] Update dependencies from dotnet/corefx (#17166)
* Update dependencies from https://github.com/dotnet/corefx build 20191002.6

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19502.6

* Reenable libgdiplus/System.Drawing.Common tests

* Update dependencies from https://github.com/dotnet/corefx build 20191002.7

- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19502.7
2019-10-03 20:45:29 +02:00
Bernhard Urban-Forster 40b6f1ad2b [sdks] .gitignore python virtualenv (#17168) 2019-10-03 20:34:15 +02:00
Alexander Köplinger 43ccc6c26b [netcore] Update .gitignore 2019-10-03 18:45:14 +02:00
Alexander Köplinger 5bf18642ff [netcore] Move ignored test to global ignore file instead of the Linux-specific
It affects all platform
2019-10-03 18:44:07 +02:00
Marek Safar 0ccfd7806c [System.Runtime.Serialization] Make reflection calls linker friendly (#17165)
* [System.Runtime.Serialization] Make reflection calls linker friendly

Fixes #12586

* [csproj] Update project files
2019-10-03 18:40:14 +02:00
Sam Patel 83a5343d12 Merge pull request #17105 from naricc/naricc/UnitializedArray-Comment
Add a comment explaining why GC.AllocateUninitializedArray is not an icall
2019-10-03 10:43:45 -04:00
lateralusX aaa383b3e1 Default --host in autogen.sh on cygwin when missing. 2019-10-03 12:25:30 +02:00
Johan Lorensson 55be8289ff Optional only build MSVC runtime using make file system on Windows. (#16915)
* Build MSVC only runtime using make file system.

Add a new option to only build MSVC version of mono runtime and native
tools, --enable-msvc-only. In the past, --enable-msvc build both
mingw as well as MSVC runtime, this commit adds an option to only build
MSVC version of runtime and native tools using make file system.

Commit updates all native make file making sure mingw/gcc builds are
not done if --enable-msvc-only has been used. It also makes sure we
build and use MSVC based tools as part of build and tests.

Result will be a build without any mingw/gcc artifacts, but where all
existing make files still works with the MSVC build artifacts.

This commits is also preparing to switch the .NETCore build over to use
MSVC build runtime in order to get LLVM support, something that is
currently only supported on Windows MSVC build.

* Add link to mono-sgen.exe and clean windows binaries.

* Fix failures in mono/tests test-aot on Windows.

* Add fullaot-mixed tests on Windows x64.

* Run msbuild in parallel.

* Default to x64 mingw host on cygwin build.
2019-10-03 11:21:36 +02:00
Nathan Ricci d0461a9d8d Spelling corrections. 2019-10-02 13:14:48 -04:00
Jo Shields d8040ba865 Fix missing corerun/ directory in make dist tarballs (#17126)
```
 configure.ac:6695: error: required file 'netcore/corerun/Makefile.in' not found
 Makefile.am:24: error: required directory ./netcore/corerun does not exist
```
2019-10-02 10:46:42 -04:00
Jay Krell 6fb01aa269 Some external_only/internal split, and inlining. (#17012)
mono_type_get_type
 mono_type_get_signature
 mono_type_is_byref
 mono_type_get_class
 mono_type_get_array_type
2019-10-02 12:59:45 +02:00
Jay Krell 8d84b6fd38 [jit amd64] Add const and remove x87. (#16700) 2019-10-02 11:51:59 +02:00
Vlad Brezae 0ce8c78e0c [mini] Use clr memory model by default (#17136)
* [mini] Use clr memory model by default

Otherwise we can have potential crashes in the bcl on arm targets, since a lot of code that assumes this memory model is now shared with mono.

Keep the old clr-memory-model option alive for now, to avoid potential build problems, if building older mono with newer mono.

* [build] Remove clr-memory-model option

It is default now

* [man] Add entry for weak-memory-model
2019-10-02 09:23:45 +02:00
Alexis Christoforides 10e9159eaa [merp] Include any managed methods in the 'unmanaged_frames' portion … (#17090)
* [merp] Include any managed methods in the 'unmanaged_frames' portion of the output also

This can help correlate the two lists when we have mixed managed & unmanaged stacks in our crash output.

* Wehn symbol/function address is not found, still output the IaP

* Don't set native_offset to ip, that's not what it means

* [merp] Bump protocol version

* Minor formatting fix

* Remove paths from assembly image names

* Remove unused var

* Refactor filling in managed info for stack frame

* Revert "Remove paths from assembly image names"

This reverts commit defe86b471f37e12fa66a104747cac932943dca1.
2019-10-02 01:11:35 -04:00
Kyle White 0004522028 [merp] Use macOS version not Darwin version in MERP reports (#17130)
* Use macOS version not Darwin version in MERP

Fixes #17004

* Update mono-merp.c

Ensure this is only for Mac

* Don't allocate in macos_version_string

Use a static buffer.

Also some formatting fixes.
2019-10-01 22:01:35 -04:00
Tanner Gooding c76b84e80c Fixing AdvSimd.Arm64 to have an internal constructor in the "supported" file. (#26953)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 19:22:28 +02:00
Maxim Lipnin 41a89af31d [wasm][xunit tests] Exclude some tests due to PNSE-implemented members. (#17137) 2019-10-01 10:13:46 -05:00
Vlad Brezae 5af0ad900c Update codeowners
Get notifications for profiler and exceptions.
2019-10-01 16:40:48 +02:00
Maxim Lipnin db2e8ea79f [wasm][xunit test] Exclude RemoteExecutorTests in bcl profile (#17120)
`System/RemoteExecutorTests.cs` tests are excluded in the mobile test profiles. It makes sense to do the same for wasm.

Fixes https://github.com/mono/mono/issues/17038.
2019-10-01 15:34:49 +02:00
Adam SitnikandJan Kotas 20ab9bf581 allow ASCII equality ordinal fast path for en-* cultures (#26905)
* allow ASCII equality ordinal fast path for en-* cultures

* dont use StartsWith in the initialization execution path

* Update src/System.Private.CoreLib/shared/System/Globalization/CompareInfo.Unix.cs

Co-Authored-By: Jan Kotas <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 11:36:07 +02:00
Vlad Brezae f9ca01e3a5 [sgen] Fix allocator spelling (#17111) 2019-10-01 12:09:54 +03:00
Marek Safar e5f218786d [netcore] Update coreclr tests package name 2019-10-01 10:57:02 +02:00
Santiago Fernandez Madero b74f7b936e Make ResolveEventArgs.Name not nullable (dotnet/corefx#41415)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 10:13:53 +02:00
Youssef Victor 4d6596359c Typo: retreive => retrieve (dotnet/coreclr#26915)
Signed-off-by: dotnet-bot <[email protected]>
2019-10-01 10:13:53 +02:00
Egor Bogatov 5537a7c2ba Enable hw intrinsics in AOT mode (#17005)
* Introduce -mattr flag

* fix build on arm targets

* remove mono_memory_barrier

* Address feedback

* Address feedback

* cleanup

* fix crash

* fix "bmi" (it's not "bmi1")

* cleanup

* ignore System.Drawing.Tests.IconTests.CorrectColorDepthExtracted test

* fix build on arm

* fix build

* fix build
2019-09-30 21:28:55 -04:00
Maxim Lipnin e416137dfb [wasm][xunit tests] Other fix for "MissingMethodException: Default constructor not found" issue (#17122)
Fixes https://github.com/mono/mono/issues/17040
2019-09-30 15:21:32 -04:00
Ankit Jain 410cb28177 Bump msbuild to track xplat-master (#16975)
.. to pick up https://github.com/mono/msbuild/pull/142 and
https://github.com/mono/msbuild/pull/139 .
2019-09-30 21:11:09 +02:00
Vlad Brezae eb887a8c74 [sgen] Add stats for allocated gchandles (#17074) 2019-09-30 21:15:42 +03:00
dotnet-maestro[bot] 5818100a80 Update dependencies from https://github.com/dotnet/roslyn build 20190926.6 (#17103)
- Microsoft.Net.Compilers - 3.3.1-beta4-19476-06
2019-09-30 18:40:57 +02:00
Egor Bogatov 3768181879 Update NETStandard2.1 APIs (#17080)
Contributes to #17064

* Add System.Data.DataSetExtensions to mobile profiles
2019-09-30 16:07:02 +02:00
Alexander Köplinger 7f37f76b6d [netcore] Disable System.Drawing.Tests.IconTests.CorrectColorDepthExtracted test
It is broken on newer libgdiplus.
2019-09-30 15:32:18 +02:00
Marek Safar f88da6cc14 [netcore] Add target for running coreclr tests 2019-09-27 21:27:08 +02:00
Nathan Ricci bc63dc68f8 Updated comment in GC.cs about unitialized array. 2019-09-27 14:48:00 -04:00
Tanner Gooding bf5f6174ad Refactoring the ARM Hardware Intrinsics based on the latest design decisions. (#26895)
* Moving the Arm64 intrinsic files into the Arm folder.

* Refactoring the ARM Hardware Intrinsics based on the new design.

* Adding support for AdvSimd.Add

* Adding LoadVector64 and LoadVector128 APIs to Arm.AdvSimd

* Marking the LoadVector64 and LoadVector128 methods as unsafe

* Removing the ARM64 HWIntrinsic tests

* Renaming Arm.Base to Arm.ArmBase, as per the design

* Fixing Base.cs to ArmBase.cs in the shared projitems

* Fixing the doc comments for the AdvSimd.LoadVector methods

* Marking various IsSupported methods as new, now that they inherit from ArmBase

* Fixing a malformed doc comment that was missed

* Marking AdvSimd.Arm64 as [Intrinsic] and new

Signed-off-by: dotnet-bot <[email protected]>
2019-09-27 20:47:17 +02:00
Jo Shields 2072e4ce81 Special case in GetFullPath for UNC roots of \\.\ (#17066)
A valid path might be something like \\\\.\\pipe\\pipename (i.e. \\\\.\\ is a valid root). If we blindly strip all instances of `.\\` then we end up confusing our path cleaning algorithm. So permit a directory component of `.` in UNC path roots.

Closes: https://github.com/mono/mono/issues/16460
2019-09-27 13:35:18 -05:00
Zoltan Varga ac2ee5b8fc [wasm] Bump emscripten to 1.38.46. (#17094) 2019-09-27 13:34:02 -05:00
Maxim Lipnin 643bcbc3fc [wasm][xunit tests] Fix issue "System.MissingMethodException: Default constructor not found for type" (#17082)
Fixes https://github.com/mono/mono/issues/16879
2019-09-27 13:59:15 -04:00
Alexander Köplinger 1b9d8ec9eb Fix mono-extensions build (#17100)
It needed an update because https://github.com/mono/mono/commit/5b108b5726ec196f4b61d30f406919ce70211754 touched code the patch modified.
2019-09-27 16:45:57 +02:00
Vlad Brezae 194506d495 [mini] Expand clr-memory-model effect (#17093)
* [mini] Add memory barriers to more stores inside objects

We should now have membars for all stores of an object ref inside another object (for the stores done by the jit). Used by clr-memory-model debug option.

* [mini] Add memory barrier for storing objref in static fields

For clr-memory-model debug option.
2019-09-27 12:36:04 +03:00
dotnet-maestro[bot] ba093a07b7 Update dependencies from https://github.com/dotnet/arcade build 20190923.5
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19473.5
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19473.5
2019-09-27 10:32:57 +02:00
dotnet-maestro[bot] d5c49121b6 Update dependencies from https://github.com/dotnet/corefx build 20190926.2
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19476.2
2019-09-27 10:32:57 +02:00
dotnet-maestro[bot] e062ebf096 Update dependencies from https://github.com/dotnet/core-setup build 20190926.2
- Microsoft.NETCore.App - 5.0.0-alpha1.19476.2
2019-09-27 10:32:57 +02:00
Aleksey Kliger (λgeek) 1dc63ced73 [runtime] Respect runtime_version in mono_init_internal (#17085)
In 6d2c77fb33 we changed the representation of
the runtimes that we will probe from a NULL-terminated array to a linked list.
Unfortunately we made a typo, and in the case where the caller of
mono_init_internal provided a runtime_version, we still picked
DEFAULT_RUNTIME_VERSION instead of the given runtime_version.

This is an issue for embedders like XA that use the "mobile" runtime - all
their assemblies are compiled with the "mobile" version number (2.0.5.0) but
Mono running with the default runtime configuration will remap to 4.0.0.0.  As
a result, when there are references to 2.0.5.0 assemblies, the load requests
will be remapped to 4.0.0.0 which will never see the already loaded 2.0.5.0
assemblies, and we will fall back on the embedder's preload hook or filesystem
probing every single time.  This is a performance fail.
2019-09-26 17:30:37 -04:00
Alexander Köplinger bffd4c6b99 Revert "Inline TLS access. (#16882)" (#17092)
Reverts mono/mono#16882

This broke the Mono Android SDKs build in master though for some reason it doesn't show up in the PR build:

```
[2019-09-26T03:19:55.366Z] Undefined symbols for architecture x86_64:
[2019-09-26T03:19:55.366Z]   "_mono_tls_key_jit_tls", referenced from:
[2019-09-26T03:19:55.366Z]       _mono_tls_get_jit_tls in libmain_a-main-sgen.o
[2019-09-26T03:19:55.366Z]   "_mono_tls_key_lmf_addr", referenced from:
[2019-09-26T03:19:55.366Z]       _mono_tls_get_lmf_addr in libmain_a-main-sgen.o
[2019-09-26T03:19:55.366Z] ld: symbol(s) not found for architecture x86_64
[2019-09-26T03:19:55.366Z] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[2019-09-26T03:19:55.366Z] gmake[6]: *** [Makefile:1634: mono-sgen] Error 1
[2019-09-26T03:19:55.366Z] gmake[6]: Leaving directory '/Users/builder/jenkins/workspace/archive-mono/master/android/debug/sdks/builds/android-host-Darwin-debug/mono/mini'
```.
2019-09-26 23:10:51 +02:00
Vlad Brezae bada362454 [sgen] Report roots from togglerefs (#17063) 2019-09-26 22:38:59 +03:00
Marek Safar 8da9eebcc6 [netcore] Correctly send exit code to the caller 2019-09-26 20:59:50 +02:00
Jan 3132cbd98b Add loading the default Mono configuration file. (#16961)
Add loading the default Mono configuration file.

Add loading the default Mono configuration file in the examples of Mono Embedded.
Also include header in existing usage of `mono_config_parse`.

This fixes problems under Raspbian, when extending the sample with your own code. 

The `teste.c` is still not compiling, because the `mono_set_allocator_vtable` function is not up to date. But I was not able to find the documentation for this function.
2019-09-26 20:46:16 +02:00
Vlad Brezae 470e53f6b4 [sgen] Fix heavy binary protocol build (#17037)
[sgen] Fix heavy binary protocol build
2019-09-26 19:16:04 +02:00
imhameed 5e30301ff3 Merge pull request #17031 from EgorBo/bmi-mulx
[LLVM] Implement Bmi2.MultiplyNoFlags
2019-09-26 09:29:54 -07:00
Alexis Christoforides f52678864d [crashing] Remove Mono signal handlers when starting to handle a crash
Fixes https://github.com/mono/mono/issues/16803
2019-09-26 16:41:35 +02:00
Alexis Christoforides 4727b2475f [merp] Output native library module name when 'whitelist all' mode is enabled (#16899)
Partially fixes https://github.com/mono/mono/issues/16689
2019-09-26 09:30:01 -04:00
Vlad Brezae 4a17f5ca11 [Mono.Debugger.Soft] Fix VirtualMachine detaching (#17020)
* Revert "[Mono.Debugger.Soft] Properly close connections"

This reverts commit 2da7707373.

This was previously reverted because it was causing issues and it still does. It is a hack fix. We signal the connection for closing and next we send the dispose command over the network. This is obviously racy.

* [Mono.Debugger.Soft] Make sure receiving thread always closes

When we want to close the connection, setting the close flag is not enough, because a thread can be already stuck in a blocking receive, without getting to check the flag. We fix this by additionaly breaking the connection, by using socket.Shutdown.

Fixes https://github.com/mono/mono/issues/7377

* Bump API snapshot submodule
2019-09-26 14:40:42 +02:00
Johan Lorensson 9b094e660c BeginConnect complete to early when not using callback. (#17035)
* BeginConnect complete to early when not using callback.

The following commit, https://github.com/mono/mono/commit/068e8fe0424806753d7ab72a7b9bf0e54b58408b#diff-33b28a9dc0b8032650ed77213481a99f
changed the way we do DNS resolve in BeginConnect using host name. Instead
of doing a blocking DNS resolve, it switched to an async DNS resolve, followed
by an async connect. The returned IAsyncResult was just representing the
completion of the DNS resolve task and not the underlying connect. It
also turns out that part of doing EndConnect, we did some validation
of the used IAsyncResult making sure it was an instance of SocketAsyncResult,
meaning that if EndConnect was called on the returned IAsyncResult from
BeginConnect, EndConnect would throw and exception.

NOTE, this only happens if you use pattern like this:

var result = sock.BeginConnect (hostName, port, null, null);
result.AsyncWaitHandle.WaitOne (2000);
sock.EndConnect (result);

It did not happened if you used callbacks, since then EndConnect would
be called as part of callback and that IAsyncResult was of correct type,
or if you did a BeginConnect with an EndPoint or IPAddress[] since that
won't trigger the DNS resolve.

Fix is to make sure we hand out the SocketAsyncResult instance that will
be used in our internal connect calls from BeginConnect. We also make
sure, we chain the DNS resolve task together with a task doing the async
connect (using the same SocketAsyncResult instance). This will make sure
the IAsyncResult returned from BeginConnect is the same one as used in
internal connect call, so it will only complete when the connect call
completes.

* Adding test for BeginConnect using host name.

* Bump API snapshot submodule
2019-09-26 14:34:28 +02:00
monojenkins e5b7473a21 [sdks] Bump min iOS version to 7.0. (#17073)
Because that's the min iOS version in all current versions of Xamarin.iOS.
2019-09-26 14:20:32 +02:00
Bernhard Urban c81b6f5ca9 [interp] always require GC Unsafe mode in exception checkpoint (#17016)
[interp] always require GC Unsafe mode in exception checkpoint

Follow-up for https://github.com/mono/mono/pull/16955

/cc @BrzVlad
2019-09-26 10:45:56 +02:00
Ben Adams 65dc4c364a CollectionsMarshal.AsSpan allow null refs (dotnet/coreclr#26903)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 10:10:42 +02:00
Levi Broderick 23fac785db Introduce Utf8Span, which is a span of UTF-8 text (dotnet/coreclr#26711)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 10:10:42 +02:00
Vlad Brezae f5bb650f78 [debugger] Properly close debugger thread when connection hangs
When we detach from the debugger, we reset the state by starting a new debugger thread (why don't we just use the existing thread?). As per commit https://github.com/mono/mono/commit/540ef385faa230bc226c7a83bfabd0730520911e we should shut down the debugger thread when the connection drops. However, since that commit was acting as a Dispose command was sent instead, it would end up relentlessly spawning and closing new debugger threads up until the runtime is shut down. This commit makes sure we don't respawn once the connection hangs.
2019-09-26 09:53:36 +02:00
Egor Bogatov 4a72dadb34 LLVM: Add comments (string metadata) for calls (#16956)
* Add string metadata to all calls (comments)

* Update mini-llvm.c

* rename tramp_%d to full name too
2019-09-26 09:51:36 +02:00
Jan Kotas 519380d009 Fix CoreRT build break
Signed-off-by: dotnet-bot <[email protected]>
2019-09-26 08:59:20 +02:00
Bernhard Urban 1faf71f446 [offsets-tool] install clang python package into a virtualenv (#17013)
[offsets-tool] install clang python package into a virtualenv

So it doesn't mess with the Python installation of the user.

/cc @mandel-macaque
2019-09-26 08:55:33 +02:00
Thays Grazia 5b108b5726 [threading] embedded mono hangs (#16907)
* [threading] embedded mono hangs

When embedded mono is executing a C++ code, it didn't start to execute a C# code, we don't need to suspend the thread to run GC.
Fixes #16192
Fixes #14725

* Changing what was discussed with Aleksey during our 1on1

* Changing what was suggested by Aleksey

* Separating mono_init_version that is being used on tests from the embedded ones
2019-09-25 17:12:33 -03:00
Jay Krell ed0fa35dc2 [Coop] Unconvert ves_icall_Mono_TlsProviderFactory_IsBtlsSupported. (#16995) 2019-09-25 14:40:38 -04:00
Stephen Toub c256e409c9 More Corelib cleanup (dotnet/coreclr#26872)
* Remove unnecessary asserts

* Make several classes static

* Use is instead of as+null check

* Use T? instead of Nullable<T>

* more static classes

* Mark locals as const

* Merge declaration and initialization of some variables

* Remove unnecessary casts

* Remove unnecessary "unsafe"s

* Simplify several lambda expressions

* Remove redundant parentheses

* Remove redundant '== true'

* Remove redundant empty lines

* Simplify boolean comparison with '== false'

* Replace if-statement with return statement

* Use while for infinite loop

* Add static to all partial static class declarations

* Use ++/-- operator instead of assignment

* Use string.IsNullOrEmpty

* Use coalesce expression

* Simplify lazy initialization

* Use coalese expression

* Join string expressions

* Use regular string literal instead of verbatim string literal

* Optimize StringBuilder.Append calls

* Remove redundant assignment

* Remove unnecessary unsafe context

* Merge processor directives

* Use String.Equals instead of String.Compare

* Use Debug.Fail instead of Debug.Assert(false

* Remove Attribute suffix

* Use predefined type

* Use compound assignment

* Use while statement to create an infinite loop

* Remove redundant base ctor call

* Avoid using catch (Exception)

* Remove empty regions

* Span comparison to null

* Avoid unnecessary boxing of value type

* Expression is always equal to 'true'

* Remove unused method

* update coalesce assignment

* fix unsafe

* fix redundant parens

* Fix whitespace errors introduced

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 20:13:47 +02:00
Egor Bogatov 39ff579432 [netcore] Disable annoying SourceLink warrnings (#17055)
Currently netcore-build doesn't need submodules (except LLVM) so a task somewhere in arcade/microsoft.build.tasks.git always complain if those are not cloned.

Set EnableSourceControlManagerQueries=false to ignore the warnings.
2019-09-25 12:28:53 -04:00
Alexander Köplinger ea6ea67351 Bump bockbuild to get recent fixes (#17054)
- https://github.com/mono/bockbuild/pull/121
- https://github.com/mono/bockbuild/pull/122
2019-09-25 16:52:42 +02:00
EgorBo 2ceb062c9b revise all X86 intrinsics 2019-09-25 14:58:53 +03:00
Alexander Köplinger aeca4a6cc3 Update System.csproj 2019-09-25 12:32:42 +02:00
Jay Krell baa9adc06a Fix comment about mono_thread_interruption_request_flag. (#17036)
Fix https://github.com/mono/mono/pull/16906/files#r327660329.
2019-09-25 12:30:27 +02:00
Ben Adams e48a285cfd Add CollectionsMarshal (dotnet/coreclr#26867)
* Add CollectionsMarshal

* Feedback

* Feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 09:55:20 +02:00
Levi Broderick e082719e3f Pack additional useful data into char's "Unicode categories" RVA static info (#26848)
Also improves the performance of char.IsWhiteSpace / char.IsUpper / char.IsLower

Signed-off-by: dotnet-bot <[email protected]>
2019-09-25 07:48:24 +02:00
EgorBo ba471607a3 Implement Bmi2.MultiplyNoFlags 2019-09-25 04:04:38 +03:00
Steve Pfister da02f1330f Enable additional wasm corefx xunit tests (#16860)
Add the other corefx xunit test assemblies to the build and adjust jenkins to run them on CI.
2019-09-24 17:27:50 -04:00
Egor Bogatov 37e46aa9ae LLVM: Intrinsify MathF.Round (#16840)
* improve MathF.Round

* prefer vroundss

* enable round only for jit
2019-09-24 20:53:23 +02:00
Bernhard Urban 153b5def01 [checked-build] update allowed thread states (#16971) 2019-09-24 14:53:10 -04:00
Marius Ungureanu 394d8d1f21 Add RenamedEvent* to FSW sources from CoreFX (#16755)
* Add RenamedEvent* to FSW sources from CoreFX

Fixes #16486 - This was only partially fixed until now

* [csproj] Update project files

* Add test files which test these code paths on corefx

* Remove old RenamedEvent* files
2019-09-24 20:49:02 +02:00
Egor Bogatov 8f85967612 Add loop related optimizations to LLVM JIT (#16436)
* Add loop-related optimizations

* Update llvm-jit.cpp

* Update llvm-jit.cpp

* Update llvm-jit.cpp
2019-09-24 20:45:32 +02:00
Egor Bogatov ed622e9112 [netcore] Implement missing Bmi1/Bmi2 intrinsics (#16919)
* Finalize BMI1

* cleanup

* Update simd-intrinsics-netcore.c

* Update simd-intrinsics-netcore.c

* fix LLVM-AOT
2019-09-24 20:44:03 +02:00
Stephen Toub c4aa67cbd9 Change several internal/private instance methods to be static (#26835)
* Replace ToLower(CultureInfo.InvariantCulture) with ToLowerInvariant()

* Mark several members static

Signed-off-by: dotnet-bot <[email protected]>
2019-09-24 20:06:06 +02:00
Jan Kotas 639f30f99b Remove IsFastSort optimizations from String (dotnet/coreclr#26759)
* Remove IsFastSort optimizations from String
- Fix bugs in the Span-based globalization fast paths. Note that some of these bug fixes are going to affect performance of the globalization fast paths.
- Use Span-based globalization fast paths for strings
- Avoid static array allocation for HighCharTable

Fixes #26758

Signed-off-by: dotnet-bot <[email protected]>
2019-09-24 20:06:06 +02:00
Maxim Lipnin f91dcb9c10 Fix time zone issue when jumping into DST (#16430)
Addresses an issue with jumping into DST for some time zones when the incorrect date-time offset is returned for date-time in UTC (which comes from DateTime.Now). The fix is to just check if the incoming date-time is in UTC.

Also added a set of tests for some time zones to verify jumping into DST in general.

Fixes https://github.com/mono/mono/issues/16395
2019-09-24 13:58:59 -04:00
Alexander Köplinger 0a3c9c451b [ci] Make ios/mac sdks archive URL more predictable
Requested by xamarin-macios team.
2019-09-24 15:26:32 +02:00
Bernhard UrbanandAleksey Kliger 749493d4b3 [threads] clear small_id_key TLS when unregistering a thread (#16973)
* [threads] clear small_id_key TLS when unregistering a thread

Fixes
```
* thread #12, name = 'tid_a507', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1be66144)
  * frame #0: 0x1be66144 libsystem_c.dylib`__abort + 184
    frame #1: 0x1be6608c libsystem_c.dylib`abort + 152
    frame #2: 0x003e1fa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="* Assertion at ../../../../../mono/utils/hazard-pointer.c:158, condition `mono_bitset_test_fast (small_id_table, id)' not met\n", fatal=4, user_data=0x00000000) at runtime.m:1251:3
    frame #3: 0x003abf44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt]
    frame #4: 0x003abfb4 monotouchtest`monoeg_assertion_message(format=<unavailable>) at goutput.c:184:22 [opt]
    frame #5: 0x003904dc monotouchtest`mono_thread_small_id_free(id=<unavailable>) at hazard-pointer.c:0:2 [opt]
    frame #6: 0x003a0a74 monotouchtest`unregister_thread(arg=0x15c88400) at mono-threads.c:588:2 [opt]
    frame #7: 0x00336110 monotouchtest`mono_thread_detach_if_exiting at threads.c:1571:4 [opt]
    frame #8: 0x003e7a14 monotouchtest`::xamarin_release_trampoline(self=0x166452f0, sel="release") at trampolines.m:644:3
    frame #9: 0x001cdc40 monotouchtest`::-[__Xamarin_NSTimerActionDispatcher release](self=0x166452f0, _cmd="release") at registrar.m:83445:3
    frame #10: 0x1ce2ae68 Foundation`_timerRelease + 80
    frame #11: 0x1c31b56c CoreFoundation`CFRunLoopTimerInvalidate + 612
    frame #12: 0x1c31a554 CoreFoundation`__CFRunLoopTimerDeallocate + 32
    frame #13: 0x1c31dde4 CoreFoundation`_CFRelease + 220
    frame #14: 0x1c2be6e8 CoreFoundation`__CFArrayReleaseValues + 500
    frame #15: 0x1c2be4c4 CoreFoundation`CFArrayRemoveAllValues + 104
    frame #16: 0x1c31ff64 CoreFoundation`__CFSetApplyFunction_block_invoke + 24
    frame #17: 0x1c3b2e18 CoreFoundation`CFBasicHashApply + 116
    frame #18: 0x1c31ff10 CoreFoundation`CFSetApplyFunction + 160
    frame #19: 0x1c3152cc CoreFoundation`__CFRunLoopDeallocate + 204
    frame #20: 0x1c31dde4 CoreFoundation`_CFRelease + 220
    frame #21: 0x1c304a80 CoreFoundation`__CFTSDFinalize + 144
    frame #22: 0x1bfa324c libsystem_pthread.dylib`_pthread_tsd_cleanup + 644
    frame #23: 0x1bf9cc08 libsystem_pthread.dylib`_pthread_exit + 80
    frame #24: 0x1bf9af24 libsystem_pthread.dylib`pthread_exit + 36
    frame #25: 0x0039df84 monotouchtest`mono_threads_platform_exit(exit_code=<unavailable>) at mono-threads-posix.c:145:2 [opt]
    frame #26: 0x0033bb84 monotouchtest`start_wrapper(data=0x1609e1c0) at threads.c:1296:2 [opt]
    frame #27: 0x1bf9b914 libsystem_pthread.dylib`_pthread_body + 128
    frame #28: 0x1bf9b874 libsystem_pthread.dylib`_pthread_start + 44
    frame #29: 0x1bfa3b94 libsystem_pthread.dylib`thread_start + 4
```

* Update mono/utils/mono-threads.c

Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>
2019-09-24 13:07:19 +02:00
Maxim Lipnin 9988ebc7f2 [System.Xml] Fix deserialization issue (#16965)
[This ](https://github.com/mono/mono/pull/11194/files#diff-ae54d9565e735005686191de0f19ed50R431-R435)change in `XmlTypeMapElementInfo.GetElement`  introduced the logic for handling the case of derived types deserialization. 
For some cases (like https://github.com/mono/mono/issues/16918) it breaks existing functionality because it returns wrong element for given element name after searching withing derived types collection.

![image](https://user-images.githubusercontent.com/3258267/65416085-bedc0500-ddff-11e9-941c-f24045c64121.png)

Unfortunately, what I can do here is just reverting the related changes.

Fixes https://github.com/mono/mono/issues/16918
2019-09-24 11:35:15 +02:00
Thays Grazia 7e6d863686 [debugger] New way to filter exceptions to support VSWin features (#16825)
* Creating a new version of MOD_KIND_EXCEPTION_ONLY to support VsWin features, discussed with Joaquin Jares.
Because we don't have this functionality of get every other exception that is not specified, VsWin was getting every exception and filtering later. But this causes a slowness during the debugger because we stop all thread on each exception that we get, even if they are caught and we don't want to stop.

* Fixing other unit test because now we have 2 nested classes.
Fixing indentation.

* Creating a flag in the MOD_KIND_EXCEPTION_ONLY protocol event and not creating a new protocol event.

* Changing what was suggested by Zoltan.

* Coding style

* Bump API snapshot submodule
2019-09-23 18:16:59 -03:00
Alexander Köplinger 9b98db9ca9 Correctly check for HAVE_STAT_BIRTHTIME in configure.ac checks for System.Native (#16999)
The check was done incorrectly, the include files need to be separated by newlines otherwise we would get a false negative result.

config.log contained this:

```
conftest.c:279:23: warning: extra tokens at end of #include directive [-Wextra-tokens]
#include <sys/types.h>, #include <sys/stat.h>
                      ^
                      //
```

This means that when doing File.GetCreationTime() we'd fall back to returning the last modified time.

It regressed when we switched to System.IO.File to the CoreFX implementation i.e. System.Native in https://github.com/mono/mono/commit/8f5cef936491e8b20888bbf18d426482c890637c.

Fixes https://github.com/mono/mono/issues/16974
2019-09-23 22:44:59 +02:00
Alexis Christoforides bc43f32c9d [merp] Use function names even in 'private crashes' mode (#16897)
Partially fixes https://github.com/mono/mono/issues/16689
2019-09-23 15:49:53 -04:00
Steve Pfister 15e939d542 Restore Previous ExecutionContext Capture Behavior (#16945)
Restores previous ExecutionContext Capture behavior where we were copying a new SynchronizationContext as opposed to a reference to the existing one.

The old behavior caused SynchronizationContextAwaitTaskContinuations to not post to a SynchronizationContext.

Fixes https://github.com/mono/mono/issues/16759

Reverts a fix for https://github.com/mono/mono/issues/12421
2019-09-23 10:52:23 -04:00
Alexander Köplinger 8db7d198b2 [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:53:57 +02:00
Alexander Köplinger 93c67f2a3c [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:48:43 +02:00
Alexander Köplinger 70e266a3af [ci] Fix typo in sdks-archive.groovy 2019-09-23 15:45:02 +02:00
Alexander Köplinger 6ad929c317 [ci] Build iOS/Mac Mono sdks archive using Xcode 11
This was requested by the xamarin-macios team.
2019-09-23 15:42:36 +02:00
Jo Shields 1a8cc98160 Include Mono.WebBrowser.dll in mkbundle bundles (Closes: #7454) (#16972)
* Include Mono.WebBrowser.dll in mkbundle bundles (Closes: #7454)

Because System.Windows.Forms.dll includes an assemblyref on
Mono.WebBrowser.dll, bundles need to be able to include it
in order for WinForms to work (even if they don't actually
use a browser component)

Compare:

```
$ mkbundle hellowin.exe -o hellowin --cross mono-6.0.0-ubuntu-18.04-x64 --i18n all
From: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64
ERROR: Unable to load assembly `Mono.WebBrowser' referenced by `/home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Windows.Forms.dll'
```

to

```
$ mkbundle hellowin.exe -o hellowin --cross mono-6.0.0-ubuntu-18.04-x64 --i18n all
From: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64
Using runtime: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/bin/mono
     Assembly: /tmp/hellowin.exe
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Windows.Forms.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/mscorlib.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.Security.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Configuration.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Xml.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Security.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Core.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Numerics.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Drawing.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Runtime.Serialization.Formatters.Soap.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Accessibility.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.WebBrowser.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/Mono.Posix.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Data.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.Transactions.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/System.EnterpriseServices.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.CJK.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.MidEast.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.Other.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.Rare.dll
     Assembly: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/lib/mono/4.5/I18N.West.dll
systemconfig: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/etc/mono/config
machineconfig: /home/directhex/.mono/targets/mono-6.0.0-ubuntu-18.04-x64/etc/mono/4.5/machine.config
Generated hellowin
```

For the test code:
```
using System;
using System.Drawing;
using System.Windows.Forms;

public class HelloWorld : Form
{
    static public void Main ()
    {
        Application.Run (new HelloWorld ());
    }

    public HelloWorld ()
    {
        Button b = new Button ();
        b.Text = "Click Me!";
        b.Click += new EventHandler (Button_Click);
        Controls.Add (b);
    }

    private void Button_Click (object sender, EventArgs e)
    {
        MessageBox.Show ("Button Clicked!");
    }
}
```

* Also add Novell.Directory.Ldap (consumed by System.DirectoryServices)
2019-09-23 09:24:05 -04:00
Jay Krell 7497e4933c [interp] Improve debuggability with enum. (#16721) 2019-09-23 11:10:23 +02:00
Thays Grazia 1a277c31a0 [debugger] Crash when debugging iOS application that throws (#16958)
* Doing the same fix from Android to IOS.
* On mini-exceptions.c check if there is no ji we can continue and generate the exception without checking wrapper_type.

Fixes #16824
2019-09-22 16:12:10 -03:00
EgorBo 6257369adc netcore: fix debug build 2019-09-22 09:09:35 +02:00
Jay Krell aca55a42aa Inline TLS access. (#16882)
The same functions are referenced by JIT and runtime.
The JIT uses cannot be inlined. The runtime can.
Since I could not get anything else to build and the change is simple and mechanical, now two sets of functions are provided, inline and extern. Extern just calls inline.
This PR appears to change all the uses, but it is only the JIT uses. The runtime ones just work and keep the old names.

We should also consider building desktop with LTO/LTCG, which would do all this automatically, smarter, more generally. A quick experiment shows it grows the runtime size, which is why I say desktop.
2019-09-21 18:35:19 +02:00
Jay Krell b20c690e3e Partially inline mono_method_signature_checked/internal. (#16927)
* Inline common fast paths of mono_method_signature_checked and mono_method_signature_internal.

* PR: Restore profilers to use mainstream functions.
2019-09-21 11:09:41 +02:00
Bernhard Urban 0f181a9dc4 [arm64_32] fix pointer size in CallInfo (#16970)
Note that in the output below `a`, `b`, etc. are NSObjects, so they are really passed as pointer.

Before:
```
* thread #1, name = 'tid_303', queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x04f301d4 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=0x00000000) at nsstring-localization.m:81:9
   78   void *
   79   xamarin_localized_string_format_9 (NSString *format, id a, id b, id c, id d, id e, id f, id g, id h, id i)
   80   {
-> 81           return [NSString localizedStringWithFormat: format, a, b, c, d, e, f, g, h, i];
   82   }
   83
   84   }
(lldb) mbt 4
* thread #1
  * frame #0: 0x04f301d4 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=0x00000000) at nsstring-localization.m:81:9
    frame #1: 0x04d9984c aWatchOSExtension`interp_to_native_trampoline + 156
    frame #2: 0x04f41d5c aWatchOSExtension`ves_pinvoke_method(frame=0x05167af8, sig=0x1518afd0, addr=(aWatchOSExtension`::xamarin_localized_string_format_9(NSString *, id, id, id, id, id, id, id, id, id) at nsstring-localization.m:80), string_ctor=0, context=0x14550960, save_last_error=0) at interp.c:1411:2 [opt]
    NSString::xamarin_localized_string_format_9 @ 68 "calli.nat" || frame #3: 0x04f3bfa4 aWatchOSExtension`interp_exec_method_full(frame=0x05167d38, context=<unavailable>, clause_args=0x00000000, error=0x05168540) at interp.c:3290:5 [opt]
```

After:
```
* thread #1, name = 'tid_303', queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x04c2c1e0 aWatchOSExtension`::xamarin_localized_string_format_9(format="hello%@%@%@%@%@%@%@%@%@", a=0, b=1, c=2, d=3, e=4, f=5, g=6, h=7, i=8) at nsstring-localization.m:81:9
   78   void *
   79   xamarin_localized_string_format_9 (NSString *format, id a, id b, id c, id d, id e, id f, id g, id h, id i)
   80   {
-> 81           return [NSString localizedStringWithFormat: format, a, b, c, d, e, f, g, h, i];
   82   }
   83
   84   }
```
2019-09-20 18:02:20 -04:00
Jay Krell 12d3dcecc6 [Coop] ves_icall_property_info_get_default_value. (#16614)
This includes resolving a somewhat deep problem involving
a void* that is usually to a int/float, but can be a MonoObject** or at least MonoString**.
An additional MonoStringHandleOut parameter is pass around to address this.
Perhaps it should be MonoObjectHandleOut for more generality.
2019-09-20 17:58:16 -04:00
Bernhard Urban e27159174a [interp] avoid exception checkpoint when in thread is in GC Safe state (#16955)
[interp] avoid exception checkpoint when in thread is in GC Safe state 

Usually the interpreter operates in GC Unsafe state. There is an exception however, and that is when it executes a `managed-to-native` wrapper in cooperative (or hybrid) suspend.

The wrapper does essentially:
1. enter GC Safe state via `mono_threads_enter_gc_safe_region_unbalanced`
2. call native function
3. exit GC state via `mono_threads_exit_gc_safe_region_unbalanced`

Usually upon return of a native function call, the interpreter would check for thread interruption and, if applicable, throw an exception.  That, however, would mess with the GC state as the interpreter operates in GC Safe state at this moment.

Note that ignoring thread interruption at that point shouldn't be a problem, because after 3. the `managed-to-native` wrapper will check for thread interruption anyway.

Contributes to https://github.com/mono/mono/issues/16819
2019-09-20 22:15:55 +02:00
Alexander Köplinger 6433ea9cce [ci] Add SymStore directory for netcore publishing stage
It's required by arcade in some channels.
2019-09-20 20:06:11 +02:00
Jay Krell 8b4c1ef828 Use ld -Bsymbolic when linking AOT. (#14511)
The runtime already does this itself.
2019-09-20 13:12:01 -04:00
Alexander Köplinger 6bc988b087 [netcore] Move to new arcade post build using yaml stages (#16966)
This is required by the newer arcade builds.
We can simplify our scripts since we no longer need the special publishwitharcade.proj.

Added copyright and license to the .nuspecs since the new arcade validation now
verifies this against a set of Microsoft policies.

We also need to make the .nupkg pre-release by appending the -preview suffix as stable versions are not permitted in non-stable feeds.
2019-09-20 17:27:15 +02:00
Aleksey Kliger (λgeek) 3757a7296c [System] Make FileSystemWatcher backend non-static (#16922)
* [System] Make FileSystemWatcher backend non-static

The FileSystemWatcher `watcher` field is the backend IFileWatcher that's
supposed to be used for the actual operations.

Each backend has a `bool GetInstance (out watcher)` method that returns a
singleton object, so each new instance of FileSystemWatcher gets the same
IFileWatcher instance.  (They will get different `watcher_handle` instances
which are used by some backends to uniquely identify this FileSystemWatcher).

However when the first FileSystemWatcher instance is `Dispose`d, it will set
`watcher = null` which will prevent the remaining instances from calling

    watcher?.StopDispatching (watcher_handle);
    watcher?.Dispose (watcher_handle);

which means that the backend won't properly cleanup resources for any other FSW
instances that have other `watcher_handle` values.

Addresses https://github.com/mono/mono/issues/16709
2019-09-20 11:21:30 -04:00
Rolf Bjarne Kvinge 95d17b16a8 [offsets-tool] Install clang into the user-specific python directory.
Because the system location isn't world writeable on Catalina.
2019-09-20 16:19:32 +02:00
monojenkins c8450814a2 [bcl] Update BCL Linked Size 2019-09-20 15:02:00 +02:00
Alexander Köplinger 74b9ca426a [ci] Remove more XCODE32_DIR usages (#16964)
Follow-up to https://github.com/mono/mono/pull/16936
2019-09-20 14:00:22 +02:00
Egor Bogatov 32321375ee Implement WriteCore and ReadCore in DeflateStream (#16951)
No idea why these were left unimplemented (probably were not used at that point).
Fixes #16950
2019-09-20 12:06:45 +02:00
Alexander Köplinger 857284b412 Add net_4.8.xml to EXTRA_DIST and bump binary-reference-assemblies again
The file was missed in https://github.com/mono/mono/pull/16912.
Also pull in another reference assemblies update.
2019-09-20 11:59:51 +02:00
Filip Navara 338170ecce Fix check for SafeHandle types 2019-09-20 11:44:25 +02:00
monojenkins 6d3f5732f0 [sdks] Remove the mac32 build. (#16952)
Xamarin.Mac doesn't need nor use it anymore.

This also removes the last part of the mono archive that required Xcode 9.4.
2019-09-19 21:45:57 -05:00
Jay Krell d49d7b4716 [Coop] InterlockedCompare/Exchange (object/T) (#16873)
* [Coop] InterlockedExchange (object and T), InterlockedCompareExchange (object and T).

 - Usually intrinsified.
 - Always?
 - Avoid coop handles by manually always using ref to object.
 - For efficiency on input, and efficiency wrt MonoError.
 - For correctness on output (due to aliasing in the wrappers).
 - To probably avoid assertion failure in runtime around icall marshaling of generics.
 - Unclear on location.

* Balance parens in prints, so they are balanced in the code.

* Reduce local arrays from 16 to auto-sized 1.

* Update and reformat System.Threading.Interlocked signature recognition.

* Rename the generic again to "_T" (or "Generic"?) on the theory
that the aot-runtime.c signature check is too strict/incorrect,
but that if we loosen it, it is not clearly specific to the generic.

* Revise corlib version again.

* Bump API snapshot submodule
2019-09-19 22:36:18 -04:00
Zoltan Varga c730a2a2e3 [llvm] Handle recursive calls as a direct call. (#16944)
Fixes https://github.com/mono/mono/issues/16943.
2019-09-19 21:43:49 -04:00
Jay Krell d02c121f79 Inline mono_thread_interruption_request_flag. (#16906)
Note that the name mono_thread_interruption_request without trailing flag
was already taken.
2019-09-19 15:49:43 -05:00
Filip Navara 81b5819528 [netcore] Show line numbers when xunit tests fail (#16931) 2019-09-19 14:08:07 -04:00
Alexander Köplinger f6a080bb79 Add .NET 4.8 reference assemblies (#16912)
Brings in https://github.com/mono/reference-assemblies/pull/20
2019-09-19 14:06:41 -04:00
Egor Bogatov 5e5033f5e5 Fix StackOverflow in ApplicationTrustEnumerator
ApplicationTrustEnumerator used to call itself. The following snippet crashed with SO:

using System.Security.Policy;
public class Program {
        public static void Main (string[] args)
        {
            ApplicationTrustCollection col = ApplicationSecurityManager.UserApplicationTrusts;
            col.GetEnumerator ();
        }
}
2019-09-19 14:05:15 -04:00
Jay Krell 06a814e66f Balance parens in prints, so they are balanced in the code. (#16930) 2019-09-19 12:19:59 -04:00
Martin Baulig 1256d6c8cf Use NetworkStream from CoreFX. (#16889)
* Use `NetworkStream` from CoreFX.

This brings the `NetworkStream` class and related tests from CoreFX.

* `SocketAsyncEventArgs`: partially replace the following properties
  with the CoreFX version: `Buffer`, `BufferList`, `Offset`, `SetBuffer ()`
  as well as the internal `CopyBufferFrom ()`.

* `SocketAsyncResult`: change `Buffer` from `byte[]` to `Memory<byte>`.

* Add internal `Socket` overloads:
   - `Receive (Memory<byte> buffer, int, int, SocketFlags, out SocketError)`
   - `ReceiveFrom (Memory<byte>, int, int, SocketFlags, ref EndPoint, out SocketError)`
   - `SendTo (Memory<byte>, int, int, SocketFlags, EndPoint)`

* Implement async task overloads from CoreFX (`Socket.Tasks.cs`) to `Socket`:
   - `int Read(Span<byte>)`
   - `Task<int> ReadAsync(byte[], int, int, CancellationToken)`
   - `ValueTask<int> ReadAsync(Memory<byte>, CancellationToken)`
   - `int ReadByte()`
   - `void Write(ReadOnlySpan<byte>)`
   - `Task WriteAsync(byte[], int, int, CancellationToken)`
   - `ValueTask WriteAsync(ReadOnlyMemory<byte>, CancellationToken)`
   - `void WriteByte(byte)`

* remove internal unused `PooledStream` and `ConnectionPool` APIs.

* bring new tests from CoreFX:
  - `System.Net.Sockets/tests/FunctionalTests/NetworkStreamTest.cs`
  - `System.Net.Sockets/tests/FunctionalTests/NetworkStreamTest.netcoreapp.cs`

* corlib: add `System.IO.FileSystem` and `System.Net.Sockets` resources and update SR.

* Update Socket.cs

* [csproj] Update project files

* Bump API snapshot submodule
2019-09-19 14:46:29 +02:00
Maxim Lipnin 12949cf2e6 [wasm] Switch to PNSE version of FileSystemWatcher and exclude xunit tests (#16892)
Fixes https://github.com/mono/mono/issues/16876
2019-09-19 08:13:02 -04:00
Zoltan Varga a3d05c2f88 [ci] Precompile runtime tests on the mac-sdk lane before running them with llvm. (#16916) 2019-09-19 12:58:31 +02:00
Jay Krell 8d23dc71b6 Inline mono_object_unbox_internal and mono_object_get_data. (#16856) 2019-09-19 12:41:29 +02:00
Filip Navara 85056b086f Implement Marshal.SetLastWin32Error 2019-09-19 09:59:49 +02:00
Johan Lorensson ad997d3725 Fix build warnings in libmonoruntime. (#16802) 2019-09-19 08:45:44 +02:00
Johan Lorensson 077e76c470 Explicit disable hybrid suspend when cooperative suspend is enabled. (#16798)
hybrid suspend won't get disabled when using --enable-cooperative-suspend
triggering configure error:

Hybrid suspend and Cooperative suspend cannot be both enabled.

Fix explicit set hybrid suspend default behavior to no if cooperative
suspend has been configured.
2019-09-19 08:45:00 +02:00
Ryan Lucia 529a11cf5b Make case-insensitive class lookup work correctly with exported types (#16652)
* Make case-insensitive class lookup work correctly with exported types

* Search all matching namespaces

* Skip AOT cache for case-insensitive lookups
2019-09-18 17:18:30 -04:00
Egor Bogatov 9a4a624501 [netcore] Implement more System.Runtime.Intrinsics.X86.Bmi1/Bmi2 (#16891)
* Implement more BMI1/BMI2

* implement more bmi1 intrinsics

* Update simd-intrinsics-netcore.c
2019-09-18 15:08:35 -04:00
Alexander Köplinger 2cc7c410a7 [ci] Allow running only msbuild tests in "OSX Extended" runs
Needed for mono/msbuild CI so we can skip the lengthy runtime LLVM tests.
2019-09-18 17:35:57 +02:00
Neale Ferguson 86f7cef7f4 - Some more prep for SIMD on s390x (#16903)
- Use __builtin_bswapxx for s390x rather than inline assembler
- s390x ABI specifies that %f8-%f15 are protected between calls so avoid using or save/restore as required
- Save the "protected" FP registers in the context structure
2019-09-18 15:54:27 +02:00
Vlad Brezae b2739b5196 [mini] Add missing membars when initializing rgctx entries (#16904)
[mini] Add missing membars when initializing rgctx entries

Whenever we are publishing changes to the rgctx arrays, we are racing with rgctx_lazy_fetch_trampoline, so we need to make sure any pointers we set there must have their contents initialized.
2019-09-18 09:16:17 -04:00
Jay Krell 12c48df242 All existing safepoints appear to be mooted by pinned coop handles. (#16695)
Perhaps they are never needed then.
2019-09-17 23:39:09 +02:00
Александр Хухлаев f91b64d9ad [WinForms] fix #16626 ControlBox property didn`t hide Close button and icon when set to false (#16627) 2019-09-17 14:26:05 -07:00
Alexander Köplinger e1f5738371 [netcore] Fix compiler warnings (#16883)
This fixes the compiler warnings when building netcore-only on macOS.
2019-09-17 21:18:54 +02:00
monojenkins 2b98112ce1 Rename bundle identifier for the various Mono.frameworks we create for Xamarin.iOS. Fixes xamarin/xamarin-macios#7005. (#16902)
This is the equivalent of
https://github.com/xamarin/xamarin-macios/commit/a9abc07cb357f9f012e3da66c70fa09d96bf02f7,
just for the Mono.frameworks in the mono archive.

Fixes https://github.com/xamarin/xamarin-macios/issues/7005.
2019-09-17 21:18:22 +02:00
Alexander Köplinger f99acb8fa6 [corlib] Fix building nunit-lite twice (#16894)
We sometimes saw the following build error in Mono SDKs builds on Jenkins:

```
[2019-09-16T01:03:30.709Z] CSC     [xammac] nunit-lite-console.exe
[2019-09-16T01:03:30.709Z] error CS0009: Metadata file '/Users/builder/jenkins/workspace/archive-mono/2019-08/mac/release/mcs/class/lib/xammac/nunitlite.dll' could not be opened -- PE image doesn't contain managed metadata.
```

Taking a closer look we found out that nunit-lite was trying to be built twice for the same profile which caused a race where the file was corrupted.

The reason for this is that the corlib Makefile defines two test assemblies: the normal PROFILE_corlib_test.dll and BinarySerializationOverVersionsTest.dll and
integrating the BinarySerializationOverVersionsTest.dll via the `test-vts` target incorrectly overwrote the `test` target instead of using `test-local`.
2019-09-17 20:25:52 +02:00
Vlad Brezae 906c71b8bf [interp] Fix stack size regression (#16887)
Partly revert https://github.com/mono/mono/commit/b18840e92cb5b1afef6c0e5df05cc026eb32efbf
2019-09-17 19:27:51 +03:00
Stephen Toub 569878fe14 Remove allocations from Dns.* (dotnet/corefx#41061)
This started as an effort to reduce the size of System.Net.NameResolution.dll when publishing a trimmed app.  It's not that big to begin with, but it's carrying around a copy of all of the IAsyncResult helper types, because the Get*Async methods are currently wrappers for the Begin/End* methods.

This PR inverts that, wrapping the Begin/End* methods instead around the Get*Async methods, using the same TaskToApm helper we use in other places in corefx for the same purpose.  This makes the Get*Async methods faster and lighterweight, but it does increase the number/amount of allocation in the Begin/End* APIs.  Since these are considered legacy, I normally would consider that a good trade, however we still use these Begin/End methods in a few places in System.Net.Sockets, and I didn't want to regress those use cases.

So, this also then trims some additional fat, which helps the Get*Async cases even further, and gets the Begin/End* to be even better than before the change.  This includes not allocating an IPHostEntry when we're just going to unwrap it and return its addresses, computing the exact IPAddress[] size we need rather than using a List<> to grow it and ToArray to create the actual array, avoiding creating the HostName if we don't need it, and avoiding an unnecessary SafeHandle allocation.

As part of this, I also noticed that we had some bugs in how some of our interop structures on Windows were defined.  In particular, fields that in the native types were size_t were defined as int rather than IntPtr in the managed code.  It appears we were saved by padding, but I fixed it regardless.

And as long as I was changing pretty much everything else, where I was touching code I also cleaned up some legacy style stuff.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 16:13:18 +02:00
dotnet-maestro[bot] a195b746c6 Update dependencies from https://github.com/dotnet/roslyn build 20190912.11 (#16885)
- Microsoft.Net.Compilers - 3.3.1-beta4-19462-11
2019-09-17 13:18:45 +02:00
Johan Lorensson e4998f5fc4 Exclude AppleTypes from orbis System.dll. (#16814)
* Exclude AppleTypes from orbis System.dll.

* [csproj] Update project files
2019-09-17 13:03:16 +02:00
Jay Krell fb71d47543 [interp] Use +=, *= etc. Seems to generate identical code though. (#16848)
i.e. a = a op b is *not* double evaluating a.
2019-09-17 13:23:19 +03:00
Ryan Lucia 6252794ebe Temporarily disable embedded ppdb data decompression (#16868)
This addresses #16373. We will work on a proper fix, but this at least unblocks Roslyn CI
2019-09-17 11:23:05 +02:00
Bernhard Urban 578e1a9dda [arm64_32] make "Debug Mode" work on Watch series 4 with --interpreter (#16865)
[arm64_32] make "Debug Mode" work on Watch series 4 with --interpreter

Main objective for reviewers: This PR shouldn't change anything for `arm64`.

Context: "Debug Mode" on the Apple Watch allows execution of regular machine code and thus is pretty similar to FullAOT on iPhone, i.e. it doesn't require bitcode, but _still_ doesn't allow `signal(3)` so cooperative suspend is required.

Apple Watch Series 4 and onward use an ARM64 chip, but with a different ABI, also known as `arm64_32`. This PR changes the AOT compiler and related trampolines so that the AOT compiler can generate all necessary code to run the interpreter on this target.


Changes:

--------------------------------------

[arm64_32] set register size to 8 in cross compiler too 

-----------------------------------------

 [machine] update `target_mgreg_t` definition

`SIZEOF_REGISTER` is usually the same as `TARGET_SIZEOF_VOID_P`, except when `MONO_ARCH_ILP32 is defined.

`target_mgreg_t` should represent the width of the hardware register, that is, `SIZEOF_REGISTER`.

---------------------------------------------

[arm64_32] fix AOT image layout 

---------------------------------------------

 [arm64_32] make trampolines bitwidth aware

* replace pointer loads/stores with bitwidth-dependent macro
* reflect `specific` trampolines so that only two instructions are used for ILP32, since the related pair in the data page only needs 2 * 32bits too
* replace some `sizeof (target_mgreg_t)` with `TARGET_SIZEOF_VOID_P` (former is hardware register size, latter is pointer size)

---------------------------------------------
These regression tests are _not_ passing: https://github.com/mono/mono/issues/16864 Also some of monotouch-tests are crashing, see https://github.com/mono/mono/issues/16819

Here is a screenshot of a debug session in VSMac:
<img width="1321" alt="Screenshot 2019-09-16 at 19 13 22" src="https://user-images.githubusercontent.com/75403/64981326-1dfbc000-d8bc-11e9-9d16-48036f36b638.png">


Contributes to https://github.com/mono/mono/issues/10641
2019-09-17 05:15:19 -04:00
Nikolay Sivov 08521a1c43 [cominterop] Default to [in] parameter direction when not specified explicitly.
Treat [In, Out], [In] and unspecified as input arguments. Important change from
current behavior is to allow parameter without directional attributes.

Use case example is _MethodInfo.Invoke() method called from native client.
2019-09-17 11:06:17 +02:00
Jay Krell af185d2c1e [interp] Cleanup dispatch macros. (#16784)
- Factor out reptition into already existing macros.
 - Remove the default for computed goto which is dead.
   Computed goto has no notion of default. It is just
   an unchecked array access.

Extracted from https://github.com/mono/mono/pull/16721.
2019-09-17 10:55:45 +02:00
monojenkins 16ed5ec3aa [csproj] Update project files 2019-09-17 10:53:46 +02:00
Radek Doulik e7f3db9796 [android sdk] Add aprofutil tool 2019-09-17 10:53:46 +02:00
Zoltan Varga 1031c4650d [llvm] Use IntPtrType () for byref arguments instead of a pointer, the latter causes problems when pointer operations are used along with phi nodes. Fixes https://github.com/mono/mono/issues/16808. (#16874) 2019-09-17 03:53:52 -04:00
Vlad Brezae aaaa2bdd5e [interp] Optimize multidimensional array access (#16822)
* [interp] Pass rank instead to LDELEMA

It feels more intuitive and uses less computations.

* [interp] Avoid unnecessary type check

Loading element address of array requires type checks only if the elements of the array are references.

* [interp] Avoid using MINT_CALLRUN for Get intrinsic

It is very slow. Use ldelema/ldobj pair instead.

* [interp] Optimize MINT_LDELEMA

* [interp] Rename ldelema_fast to ldelema1

ldelema_fast was handling arrays with a single dimension. Rename it to better suggest this.

* [interp] Avoid using MINT_CALLRUN for Set intrinsic

* [interp] Remove some duplicated and confusing code

* [interp] Fix stack type of MINT_NEWARR result
2019-09-17 10:32:20 +03:00
Stephen Toub 87072b55c0 Update typo in comment in CultureInfo.cs
Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 09:19:11 +02:00
Tarek Mahmoud Sayed 77473d11b7 Fix calendar instance inside the culture after cloning (#26721)
When creating a new culture, we’ll have the equality CultureInfo.DateTimeInfo.Calendar == CultureInfo.calendar be true. After cloning such culture, this equality would not be true. The fix here is to ensure if the equality is true before cloning then should be true after cloning.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-17 09:19:11 +02:00
Zoltan Varga 38b2d4f4fe [wasm] Bump emscripten to 1.38.45. (#16871) 2019-09-16 23:37:30 -04:00
Johan Lorensson e8570d9500 Fix warnings in eglib gspawn.c on platforms lacking fork/execv. (#16801) 2019-09-16 17:18:59 -04:00
Filip Navara 17fe4757a3 [netcore] Make Environment calls thread-safe (#16852)
* [netcore] Make Environment calls thread-safe

* Address PR feedback

* Address PR feedback
2019-09-16 17:16:14 -04:00
Ryan Lucia a23da4b8b3 [debugger] remove extraneous ErrorCode declaration (#16862)
With the single point of exit, this means the main ErrorCode isn't being written to and uninitialized memory is returned.
2019-09-16 17:14:42 -04:00
Zoltan Varga 422d559ea4 [wasm] Initial netcore support. (#16718)
* [runtime] Fix configure support for netcore+out of tree builds. Add wasm support for generating netcore/config.make.

* [runtime] Add bundle support for the netcore assembly loader.

* [netcore] Add missing linker exclusion.

* [wasm] Add netcore builds, enabled by defining ENABLE_WASM_NETCORE in Make.config.

The targets are 'package-wasm-runtime-netcore' and 'package-wasm-bcl-netcore'.
The binaries are copied to out/wasm-runtime-netcore-<CONFIG> and out/wasm-bcl/netcore.

* [wasm] Add netcore support to the build system and packager.

* Fix the build.

* Fix the dynamic build.
2019-09-16 15:42:57 -04:00
Kenneth Pouncey e8b2b43f9b Bump sdk and bindings versions (#16809) 2019-09-16 09:34:41 -05:00
Kenneth Pouncey d435a85658 Bump Browser tests Karma version (#16810) 2019-09-16 09:31:44 -05:00
Nikolay Sivov 800e2cc4b4 [corlib] Improve reflection interface definitions (#16324)
* [corlib] Fix _Type interface definition.

* [corlib] Fix Type class attributes.

* [corlib] Fix _MethodInfo interface definition.

* [corlib] Fix MethodInfo class attributes.

* [corlib] Merge MethodInfo partial definitions.

* [corlib] Remove leftovers from testing.

* [corlib] Do not define new attributes for MOBILE build.

* Bump API snapshot submodule

* [corlib] Add explicit _MethodInfo.GetType() implementation.

* Bump API snapshot submodule

* [csproj] Update project files
2019-09-16 13:00:40 +02:00
Jay Krell 9f3effc177 Strike a size and conditional branch balance between (#16836)
https://github.com/mono/mono/pull/16835
and
 https://github.com/mono/mono/pull/16834

The main 4 call opcodes should have no conditionality here.
The non-virtuals no extra branch.
The other opcodes all have one condition and branch into common code.

Everything could be smaller or faster but not both.
2019-09-16 10:10:47 +03:00
Jay Krell 67fb575d11 [Coop] ParseRouteInfo, fix some leaks, cleanup. (#16815) 2019-09-16 00:10:12 -04:00
Johan Lorensson 7fa706a316 Fix infrequent hangs in test-runner. (#16793)
* Fix infrequent hangs in test-runner.

test-runner has been plagued with very infrequent hangs causing long
timeouts on CI. Turns out that there is a race in AsyncStreamReader
used by Process class for handling stdout/stderr when closing down
process. Since AsyncStreamReader::Dispose didn't make sure no pending
async read requests were still in flight before closing the stream
and underlying handle, it led to a race condition deadlocking the
complete process, hanging test-runner. A similar race has also been
observed but instead of causing a deadlock, it could also manifest as an
System.ObjectDisposedException when doing EndRead while another thread
is disposing the underlying stream. Hitting that race didn't deadlock
the process but failed to complete invoke of test-runner.

Fix is to better protect the async callback and the thread doing the
dispose call. In order to make sure we don't close underlying handle
while still having async reads in flight, fix also checks async result
for completion and if not completed when trying to dispose AsyncStreamReader
it will try to cancel outstanding pending IO and wait for completion
before closing the stream.

Since the issue has only been seen on Windows, the fix will try
to cancel pending IO and wait for completion before closing stream
on Windows. On other platforms, old behavior will be preserved, meaning
that the implementation will still be vunarable to race between pending
reads and close of stream, but since problem has not been observed on
other platforms and since there is no good way of cancel outstanding
blocking reads, old behavior is kept.

At some point in future we should look into replacing the implementation
of AsyncStreamReader and use cancelation tokens, but since we don't have
all corefx support around cancelation in Mono BCL, that currently won't
solve the issue, meaning that we need to bring in more classes. Since the
problem also occur in test-runner using the build BCL profile, even more
needs to be added to that profile in order to get similar support. But due
to the infrequency of the problem and that it has only been observed on
Windows, the isolated fix seems more reasonable.

* Build error fix.

* Review feedback.

* Fix type cast not working on linux builds.

* Adjust icall naming in icall-def.h.
2019-09-15 19:51:07 -05:00
Jay Krell 8da3616e7f [Coop] GetValueInternal. (#16811) 2019-09-15 19:50:25 -05:00
Johan Lorensson 9249d38079 Fix warnings in libmono. (#16813) 2019-09-15 19:49:53 -05:00
Zoltan Varga cdb04c042f [netcore] Change an ArrayList to a List<T>. (#16842)
* [netcore] Change an ArrayList to a List<T>.

* [netcore] Call the generic version of Array.IndexOf () in the ICollection wrappers. Update exclusions file.
2019-09-15 20:45:13 -04:00
Egor Bogatov 0438397b8a LLVM: Dump Module instead of Function in JIT mode (#16837)
Since each function has its own module (in JIT mode) it makes sense to dump the whole module for MONO_VERBOSE_METHOD instead of just LLVM::Function.
It adds metadata, attributes and global variables (so it will be possible to copy-paste it to godbolt)

Example

static int Test(int x)
{
    return x / 10;
}

Diff for Unoptimized IR: https://www.diffchecker.com/NrF413Ts
Diff for Optimized IR: https://www.diffchecker.com/jbEkeFFS

So now we can see global variables and metadata for each function (and can easily paste them to godbolt.org)

NOTE: optimized IR for this case contains "dead" variables (dump_module allowed us to see such), we probably need some Module-wide optimizations to strip dead code such as -globaldce

NOTE2: why do we emit exceptions in the first place for this case at all?
2019-09-15 20:43:10 -04:00
Jay Krell 304f3add20 [interp] Speed up passing this parameter. (#16846) 2019-09-15 22:26:41 +03:00
Dotnet-GitSync-Bot 189af5d0c2 Remove some boxing from tuples with >= 8 elements (dotnet/coreclr#26584) (#16844)
Take advantage of https://github.com/dotnet/coreclr/pull/14698 to avoid boxing the TRest argument and improve devirtualization.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-15 12:48:15 -04:00
Calvin Buckley babe881bf2 Fix small sized argument output with tracing enabled (#16841)
This is a legacy of before the profiler was used for the trace
output; it would apparently get the raw stack pointer and try to
dereference things by recalculating the offset as needed (no for
little, yes for big). However, because the profiler returns the
actual fixed-up pointer, this recalculation isn't needed on BE,
but instead grabs garbage a bit ahead on the stack.

To fix, use what was the little endian case unconditionally.

Example before:

[1: 20.70598 14] ENTER: System.Globalization.CultureInfo:.ctor (int,bool,bool)(this:700000000408fb0[System.Globalization.CultureInfo platform.exe], 271824864, -32, -32, )

Example after:

[1: 1.93377 14] ENTER: System.Globalization.CultureInfo:.ctor (int,bool,bool)(this:700000000408fb0[System.Globalization.CultureInfo platform.exe], 127, 0, 1)
2019-09-15 11:50:32 -04:00
Jay Krell 9608debd7b [Coop] ves_icall_InternalExecute. (#16611) 2019-09-13 08:57:45 -05:00
Radek Doulik 131b3a9a92 [aot] Do not wrap tool_prefix path when calling strip (#16792)
[aot] Do not put quotes around wrapped path

Should fix: https://github.com/xamarin/xamarin-android/issues/3619

As it results in double quoting, which vanishes the effect and breaks
AOT for XA on Windows.

The actual error:

    [aot-compiler stdout] Stripping the binary: ""C:\Program Files (x86)\Microsoft Visual Studio\2019\Stable\MSBuild\Xamarin\Android\ndk\arm-linux-androideabi-"strip" --strip-symbol=\$a --strip-symbol=\$d obj\Release\90\aot\armeabi-v7a\libaot-Xamarin.Android.Arch.Core.Common.dll.so.tmp
    [aot-compiler stderr] '""C:\Program' is not recognized as an internal or external command,
    [aot-compiler stderr] operable program or batch file.
    [aot-compiler stderr] AOT of image C:\Users\peter\source\repos\App9\App9\App9.Android\obj\Release\90\android\assets\Xamarin.Android.Support.CoordinaterLayout.dll failed.
2019-09-13 09:05:27 -04:00
Filip Navara 10e72a7ab6 Improve alias analysis for pointer sized variables (#16812) 2019-09-13 08:02:50 -05:00
Vlad Brezae 6908a1941b [interp] Propagate locals and kill dead ones afterwards (#16791)
* [interp] Add flag for locals that have their address taken

The previous handling of LDLOCA was broken because if the address was taken before the instructions that would be optimized we would miss any potential changes to the local value. Right now we just skip optimizing locals that have their address taken. This is achieved simply by not pushing LDLOC on the evaluation stack, for those locals.

* [interp] Add separate counters for optimized instructions

To provide more insight.

* [interp] Propagate locals

Whenever we generate a movloc we remember what local was copied. When loading a local we check if it's currently storing the value of another local, in which case we load the original local instead.

* [interp] Kill instructions that write to dead locals

We track the liveness of a local by counting how many times the local is loaded using ldloc. When such instructions are optimized, the ldloc counter is updated accordingly.

* [interp] Don't optimize code if under debugger
2019-09-13 10:35:44 +03:00
Jay Krell 735b2cd20c [interp] Replace frame_objref with a volatile store to local. (#16790)
* [interp] Replace frame_objref with volatile stores to non-volatile locals.
Non-volatiles subject to volatile read/write still get stack-packed.
Volatiles do not.

This will possibly conserve stack, unless MINT_NEWOBJ_FAST is critical path.
 It looks like it is -- enabling this for Linux/gcc/amd64 grows
  frame from 0x98 to 0xA8.

In either case, it is more efficient as the reads do not have to be volatile.
The value can be on the stack and in registers. The stack value will pin it.

* PR: Comments.

* PR: Rearrange comment for correctness, same length.
2019-09-12 21:39:02 -05:00
Maxim Lipnin fe19944155 [ci][wasm] Use only V8 step to run corlib tests, which can reduce build time 2019-09-12 19:32:01 -05:00
Vlad Brezae aa18379968 [interp] Add percentage to opcode count stats 2019-09-12 19:29:13 -05:00
Vlad Brezae 6d75b066ee [interp] Disable computed goto when debugging interp
It makes tracing and following the code more painful.
2019-09-12 19:29:13 -05:00
Vlad Brezae 54a7e61115 [interp] Fix COUNT_OPS
And make it print the sorted opcode counts when runtime is shutting down
2019-09-12 19:29:13 -05:00
Next Turn 9f93d5d15c Modernize Array constructor (dotnet/coreclr#26684)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 19:27:58 -05:00
naricc bfe5e1efd3 Merge pull request #16388 from naricc/naricc/GetTotalAllocatedBytes
Naricc/get total allocated bytes
2019-09-12 18:12:26 -04:00
Jay Krell b2a1c6cd92 [Coop] Debugger single point of exit. (#16620)
* [Coop] Debugger single point of exit.
Extracted from https://github.com/mono/mono/pull/16614.

* Use a label per error for code size.
2019-09-12 17:57:02 -04:00
Sung Yoon Whang 3f26b9aa6c Improve the performance of Environment.WorkingSet in Windows (#26522)
* Use win32 api directly for workingset counter

* Fix build warnings

* Removing useless code

* more cleanup

* remove size annotation

* remove useless comment

* Move all the changes to Environment.WorkingSet and remove it from RuntimeEventSourceHelper

* removing useless usings

* Use kernel32.dll instead of psapi.dll

* Code review feedback

* Remove newline change

* More code review nits

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Jan Kotas 0055195d1c Change Vector<T> to use more compact code using S.R.CS.Unsafe (#26637)
S.R.CS.Unsafe is used in number of places in Vector<T> already. This change modifies more places to use it to make the code more compact.

Updated Vector.tt template to match while I was on it

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Marek Safar 43c5aa0dad Remove unused Memcpy methods from shared Buffer implementation (#26586)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
obligaron 1b48b44661 Improve logging for EventSource.WriteEvent parameter mismatch (#25533)
* improve logging for EventSource.WriteEvent

* Fix IndexOutOfRangeException in SerializeEventArgs when incorrect arguments got passed

* simplify arg-check; remove nullable-todo

* switch from Debugger.Log to ReportOutOfBandMessage

* Remove unused flush-Paramater from ReportOutOfBandMessage

* fix parameter type nullable-check for reference-types

* fix parameter type nullable-check for nullable-types

Signed-off-by: dotnet-bot <[email protected]>
2019-09-12 13:27:35 -05:00
Vlad Brezae 72405f68af [interp] Emit more stloc.np (#16796)
Use STLOC_NP_O also for MINT_TYPE_P. It has the same behavior.
2019-09-12 12:46:22 -05:00
Filip Navara 017f21c2c6 [mini] Fix couple of R4 constant folding checks (#16782)
* Fix couple of R4 constant folding checks

* Address feedback

* Split RMOVE and FMOVE
2019-09-12 12:37:52 -04:00
Jay Krell 9bb3912590 [interp] Remove unused parameters. (#16773) 2019-09-12 15:42:23 +03:00
Jay Krell c7df01524a [interp] Add assert in jmp that sp == stack at start. (#16788)
It makes it true sometimes, only if not calling alloca.
Throwing invalid IL if false might be preferred here.
2019-09-12 15:06:31 +03:00
Egor Bogatov 977a37844b [netcore] Add LLVM to nuget (#16524)
* nuspec for llvm

* Add llvm to matrix

* fix scripts/ci/pipeline-netcore-runtime.yml

* fix scripts/ci/pipeline-netcore-runtime.yml

* fix scripts/ci/pipeline-netcore-runtime.yml

* run tests with llvm

* add macOS lane

* disable llvm tests for now
2019-09-12 13:09:03 +02:00
Maxim Lipnin c241b25bb5 Avoid creating un-representable DateTime for transition point (#16753)
Fixes https://github.com/mono/mono/issues/16742
2019-09-12 05:11:22 -04:00
Steve Pfister 95cd0e74b1 Bump to mono/corefx@e79cf5b 2019-09-12 08:57:15 +02:00
EgorBo 1d324e617a Fix DynamicMethodToString.ToStringTest 2019-09-12 08:55:12 +02:00
EgorBo 313ce184be use full name for llvm ir and asm dumps 2019-09-12 08:55:12 +02:00
Jay Krell 2235e37470 [interp] Fix buffer overreads and overwrites found by valgrind. (#16726)
* [interp] Fix buffer overreads and overwrites found by valgrind.

* Attempt to deal with inlining correctly.

* PR: Dial back IR changes. Only IL relevant.

* PR: Dial back more -- only needed changes to .NET IL manipulation, not interpreter IR manipulation.

* PR: Remove one more line.
2019-09-11 19:35:05 -05:00
Zoltan Varga 6a1ac97dc7 [netcore] Create a separate version of threadpool.c for netcore with the domain support removed. (#16250)
* [netcore] Create a separate version of threadpool.c for netcore with the domain support removed.

* Fix the windows build.

* Fix the build.
2019-09-11 20:04:44 -04:00
Maxim Lipnin a67212d34c [wasm] update run-jenkins.sh (#16771)
* replaced set of run-*-mini targets with more common run-all-mini one;
* removed redundant run-v8-corlib (there is run-all-corlib)
2019-09-11 15:28:07 -04:00
Nathan Ricci 4acfcef038 White space, naming convetions. 2019-09-11 14:43:06 -04:00
Nathan Ricci b88d67d564 White space issues, typos. 2019-09-11 14:28:16 -04:00
Jay Krell f4c1358ed0 [interp] Reduce register pressure in endfinally.
Eventually this could contribute to stack savings. On its own, nothing.
The compiler apparently likes to create temporaries, like ip and ip - 1.
Which then have to survive functions calls, on the stack, if nonvolatile registers are exhausted.
This combats that and in a larger PR helped.
2019-09-11 11:19:44 -05:00
Steve Pfister 5ad1d62367 Use an init-tools script to install a local dotnet for wasm. (#16688)
Similar to what we use in netcore
2019-09-11 09:14:05 -04:00
Zoltan Varga feb0b4ab9a [wasm] Add a separate test for profiled aot. 2019-09-11 07:44:34 -05:00
Zoltan Varga 52d24a57d2 [aot] Emit interp entry code for methods which are not AOTed when using AOT profiling. Also emit most wrappers. 2019-09-11 07:44:34 -05:00
Radek Doulik 52aa2cd3e1 [aprofutil] Add -p and -f options (#16766)
-p PORT allows easier retrieval of AOT profile from the application
 running with aot profiler, using the socket connection on local PORT

-f instructs the tool to try setup adb port forwarding to/from Android
 device or emulator

Example usage, retrieve the AOT profile from Android:

    > mono aprofutil.exe -s -v -f -p 9998 -o my.aprof
    Calling 'adb forward tcp:9998 tcp:9998'...
    Reading from '127.0.0.1:9998'...
    Read total 94903 bytes...
    Summary:
    	Modules:          6
    	Types:          262
    	Methods:      1,162
    Going to write the profile to 'my.aprof'
2019-09-11 07:42:58 -05:00
Tarek Mahmoud Sayed f97ceb5c3e Fix Running .NET Core Under Windows App Compat Shim (dotnet/coreclr#26605)
* Fix Running .NET Core Under Winndows App Compat Shim

.NET Core uses Windows features called sort handles for perf reason to speed up the collation operations (e.g. string comparisons). When enabling Windows app compat shim (e.g. Windows 7 compatibility mode), the calls uses sort handles will fail which cause all collation operations to fail.
The fix here is to detect if the sort handles work before using it to ensure successful run.

* Address the feedback

* More feedback addressing

Signed-off-by: dotnet-bot <[email protected]>
2019-09-11 09:05:40 +02:00
Zoltan Varga 2e78f0c739 [wasm] Add missing aot-profile/runtime.js. (#16762)
* [wasm] Add missing aot-profile/runtime.js.

* [aot] Avoid writing out a zero AOTID in deterministic mode.

* [wasm] Add --aot-profile= option to packager.
2019-09-10 22:24:40 -04:00
Vlad Brezae f1a57de621 [docs] Improve sgen write barrier documentation (#16761) 2019-09-10 20:54:07 -05:00
Zoltan Varga a0ed95fc98 Enable the wasm targets even if ENABLE_WASM is not defined. (#16747) 2019-09-10 20:53:25 -05:00
Larry Ewing b26fd07ff9 [msbuild][roslyn] Bump msbuild and roslyn to pull in new versions (#16744) 2019-09-10 20:30:11 -05:00
Kenneth Pouncey 0847a6d218 [wasm] Dynamic Release RUNTIME configuration (#16749)
* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`

* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`

* Modify configuration flag `ENABLE_WASM_DYNAMIC_RUMTIME` to `ENABLE_WASM_DYNAMIC_RUNTIME`
2019-09-10 18:04:08 -05:00
Zoltan Varga 697ab15c12 [netcore] Disable more features. (#16748) 2019-09-10 15:36:29 -04:00
Larry Ewing 4790582812 [wasm] test cleanup (#15727)
* [wasm] stop excluding test_0_large_nullable_invoke on wasm

* [wasm] remove wasm exclusion from test_0_simple_double_casts

* Add category MultiThreaded to more of the concurrent collection tests

* Add more MultiThreaded annotations to HttpListener2Test

* Add category MultiThreaded to HttpWebRequestTest

* [wasm] flag more tests that are taking down the runtime

* [wasm] force sockets to throw PNSE in the ctor for now

* [wasm] enable corlib interpreter tests

* [wasm] Increase timeout for corlib tests

* try to fix wasm-cross-win-release-configure on macos
2019-09-10 13:08:58 -04:00
Maxim Lipnin e78ab0b1c0 [wasm] Fix several typos (#16760)
* [wasm] Fix several typos

* add a comment about an error in test class name

* Remove the comment about test class name because it could be passed on xunit command line
2019-09-10 12:36:13 -04:00
Bernhard Urban 68f5613422 [TermInfoDriver] update top/left value on Console.{CursorTop,CursorLeft} access (#16745)
It can be outdated due to writes to `stdout`.

Fixes https://github.com/mono/mono/issues/16701
2019-09-10 17:17:15 +02:00
Kenneth Pouncey eea1ab5994 Remove extraneous -s from EMCC_DEBUG_FLAGS (#16754) 2019-09-10 13:50:20 +02:00
Marek Safar 9894bfa6fa Update tests exclusion 2019-09-10 13:09:56 +02:00
Stephen Toub b434981755 Remove use of WeakReference from corelib (dotnet/coreclr#26597)
Only remaining use is the one place it shows up in public API, in GC.GetGeneration(WeakReference).

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Tyler Brinkley 05d4f301e9 Make StringComparer.Create throw ArgumentNullException (#26570)
* Make StringComparer.Create throw ArgumentNullException

Throws proper exception.

* Update CoreFX.issues.rsp

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Stephen Toub 5f9d09bc90 Remove remaining ToLower() use from corelib (dotnet/coreclr#26602)
Also happens to save a string allocation, but that's a drop in the bucket in this function.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Stephen Toub 3c503413e0 Remove sub-array allocation from GetManifestResourceNamesList (#26601)
Also removes dependency on System.Range and RuntimeHelpers.GetSubArray in a simple console app such that they can be trimmed away.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 13:09:56 +02:00
Marek Safar 96b7fead33 [netcore] Update System.Private.CoreLib public APIs 2019-09-10 12:11:31 +02:00
Bernhard Urban e40e24b4cd [aot] move method_addresses to data.rel.so section to avoid text relocations (#16729)
[aot] move method_addresses to data.rel.so section to avoid text relocations

Another revert of a revert! 🎉 

After some discussions with @grendello and @radekdoulik, we came up with this solution. It's a better attempt to fix https://github.com/mono/mono/issues/16369 as it (1) works within the restrictions of Android 10 and (2) still works for armv7 AOT. 


`.text` relocations aren't allowed in Android 10 (and this change shouldn't hurt on Linux).

AOT compilation:
```console
$ MONO_PATH=/Users/lewurm/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/ \
MONO_ENV_OPTIONS="" \
./sdks/out/android-cross-arm-release/bin/armv7-linux-android-mono-sgen \
--aot=keep-temps,outfile=here.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-,ld-flags= \
~/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll

Mono Ahead of Time compiler - compiling assembly /Users/lewurm/Downloads/xamarin.android-oss-v10.0.99.136_Darwin-x86_64_pr_766265b9-Debug/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll
AOTID 7DDD05DA-37E0-813A-8C27-3E1634391BB3
Compiled: 144382/144382
Executing the native assembler: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-as"   -mfpu=vfp3 -o /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ.o /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ
Executing the native linker: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-ld"  -shared -o here.dll.so.tmp  /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ.o
Stripping the binary: "/Users/lewurm/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-strip" --strip-symbol=\$a --strip-symbol=\$d here.dll.so.tmp
Retained input file.
JIT time: 15031 ms, Generation time: 7374 ms, Assembly+Link time: 75130 ms.
```

Before:
```console
$ grep -C 6 'method_addresses:' /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_550yHJ
        .align 3
jit_code_end:

        .byte 0,0,0,0
.text 1
        .align 3
method_addresses:
        .local method_addresses
        .type method_addresses,#function
ldr pc,=.Lm_0
.ltorg
ldr pc,=.Lm_1
.ltorg
$ arm-linux-androideabi-readelf -a here.dll.so | grep -i textrel
 0x00000016 (TEXTREL)                    0x0
 0x0000001e (FLAGS)                      TEXTREL
```

After switching to `.data.rel.so`:
```console
$ grep -C 6 'method_addresses:' /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/mono_aot_1YhAgn  # assembly file produced by AOT compiler
jit_code_end:

        .byte 0,0,0,0
.section ".data.rel.ro"
.subsection 0
        .align 3
method_addresses:
        .local method_addresses
        .type method_addresses,#object
ldr pc,=.Lm_0
.ltorg
ldr pc,=.Lm_1
.ltorg
$ arm-linux-androideabi-readelf -a here.dll.so | grep -i textrel
$ echo $?
1
```
2019-09-10 04:16:07 -04:00
Marek Safar e819930ea2 Disable failing tests 2019-09-10 10:03:42 +02:00
Stephen Toub cfc7988318 Replace HashHelpers.Combine with HashCode.Combine (dotnet/coreclr#26589)
* Replace HashHelpers.Combine with HashCode.Combine

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-09-10 10:03:42 +02:00
Jérôme Laban 11f75e5260 [Wasm] Restore release-dynamic configuration (#16655)
* [Wasm] Restore release-dynamic configuration

* Add conditional dynamic linking support, update documentation
2019-09-09 17:21:51 -04:00
Nathan Ricci f7dcc20bc6 Refactored where the varaibles for allocation count live. 2019-09-09 16:39:51 -04:00
Zoltan Varga f5d43ad341 Enable the wasm targets even if ENABLE_WASM is not defined. Update README. (#16743) 2019-09-09 16:37:05 -04:00
Thays Grazia 055818f344 Vtable [i] can be null so this should be check before use it. (#16733)
Fixes #16712
2019-09-09 17:07:34 -03:00
Maxim Lipnin ba2bb209ba [wasm] update build.sh (#16737) 2019-09-09 12:37:54 -05:00
Jay Krell 83f965eca5 [interp] Let compiler decide clear_resume_state inlining. (#16704)
Manual inlining shows no additional stack use any longer.
2019-09-09 12:31:37 -05:00
Zoltan Varga 798af66137 [llvm] Mark Vector128/Vector256 types as SIMD.
These are not currently supported, but marking them as simd helps llvm eliminate dead variables with these types, since
XZERO is easier to eliminate than VZERO.
2019-09-09 19:09:29 +02:00
dotnet-maestro[bot] e563d3c76c Update dependencies from https://github.com/dotnet/arcade build 20190906.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19456.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19456.2
2019-09-09 19:03:51 +02:00
Nikolay Sivov 8590a25b30 Default to SAFEARRAY(VARIANT) as CCW argument for object[]. (#16673)
* [cominterop] Add a helper to create default marshal spec for ccw arguments.

* [cominterop] Default to SAFEARRAY(VARIANT) as CCW argument for object[].

* [cominterop] Fix build with --enable-cxx.
2019-09-09 19:02:46 +02:00
Jay Krell 1f76ab9270 [interp] Embrace that ip is const guint16* and remove casts to that type, (#16708)
[interp] Embrace that ip is const guint16* and remove casts to that type.

or to gint16*. Or is there a reason for this style?

Maybe could change type but still want to cast for the deref?
2019-09-09 11:06:16 -04:00
Vlad Brezae b9dbd2ff91 [interp] Defer local offset allocation to compacted emit phase (#16714)
Before this commit, the only information associated with a local was just the offset at which it resides and instructions had this offset embedded from the start. This makes it awkward to have data structures indexed on the local, or have additional information per local, which we might care at some point (for example if it is multi basic block local, if the address of the local was taken etc). In addition to this, if we will want to remove some instructions that make some locals dead, it means that we will have to compact the whole locals space and update all the offsets referenced by all instructions. We solve all these problems by resolving (and allocating) the local offset at the latest time, while emitting the instruction in the compacted byte stream. At this point dead locals will remain dead, with no additional work needed to handle them.

For now, IL locals of the current method that we are compiling are immortal. We can address that later if needed.
2019-09-09 17:10:50 +03:00
Filip Navara 320f91129d [netcore] Tweak inlining heurestic for memcpy 2019-09-09 10:56:20 +02:00
David Karlaš 13391beeed Fix Xaml reader to parse escaped values correctly 2019-09-09 10:05:57 +02:00
Michal Strehovský d81ec88406 Mark unsafe intrinsics as reflection blocked
Keeping these reflection enabled pulls in quite a bit of junk (generic dictionaries, standalone method bodies, bookkeeping) that adds up to 1.5% of the size of Hello World...

Signed-off-by: dotnet-bot <[email protected]>
2019-09-09 09:24:01 +02:00
Filip Navara cf2c8575e1 [netcore][x64] Implement lowering of new SIMD OPs into SSE opcodes (#16672)
* [netcore][x64] Implement lowering of new SIMD OPs into SSE2 or SSE4.1 opcodes.

* Implement LZCNT/POPCNT in mini JIT

* Fix C++ build
2019-09-08 14:57:21 -05:00
Stephen Toub e15efa37b7 Fix race conditions in CultureInfo.GetCultureInfoHelper (#26567)
The implementation reads the current value of the static cache field and then later overwrites that field with the originally read value.  If ClearCachedData was called in the interim to clear the cached data, this will end up effectively undoing part of that clearing, as it'll end up restoring the previous cache. The fix is to write back the new reference only when we need to, which we could do when we detect it's null.

There's a secondary race condition this fixes as well.  When a name and an altname are used with CultureInfo.GetCurrentCulture(string, string), the implementation was publishig the newly created instance into the dictionary under the lock, but then after the lock was released, it was setting that TextInfo to be read-only.  That provides a window where another thread could see it as non-read-only and mutate it.  The fix is to ensure it's marked as read-only fully before it's published.

As I was doing this, it struck me that the implementation was overly complicated, trying to merge three different code paths for three different callers, presumably to share code, but most of the code ended up not being shared anyway, which just made the code harder to follow for little gain.  So, I split it back out into the three calling functions and removed the helper.

I also changed the locking mechanism.  It had been using a single shared lock to protect all state, but the purpose of the lock is really just to synchronize access to an individual dictionary, so we can just lock on the relevant dictionary.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-08 11:22:09 +02:00
Jay Krell 8c77bcdfaf [interp] Fix debug build. (#16720) 2019-09-08 09:25:33 +03:00
Zoltan Varga d9aaed760f [sdks] Change the DISABLE_ conditionals to ENABLE_ to avoid building extra stuff. (#16719) 2019-09-07 17:30:48 -04:00
Jay Krell 5ba104cf39 [interp] Fix recent warnings I added. (#16722) 2019-09-07 15:30:47 -04:00
Jay Krell b18840e92c [interp] Assign sp once is enough in leave. (#16694)
Move vt_sp there too.
2019-09-06 20:48:15 -05:00
Dotnet-GitSync-Bot 381c187df9 Mirror changes from mono/coreclr (#16713)
* Remove stale SuppressMessage attributes (dotnet/coreclr#26501)

Signed-off-by: dotnet-bot <[email protected]>

* Use null coalescing assignment operator in more places

There's currently no auto-fix for this, so the fixes were done manually.

Signed-off-by: dotnet-bot <[email protected]>

* Fix IDE0026 (use expression-bodied members for indexers)

Signed-off-by: dotnet-bot <[email protected]>
2019-09-06 15:58:38 -04:00
Alexis Christoforides 18fb728c65 Remove accidentally committed debug line 2019-09-06 14:27:57 -04:00
imhameed b4886817ef [jit][llvm] Track variable nullness separately from array length ranges/index ranges in abcrem. (#16499)
* [jit][llvm] Track variable nullness separately from array length ranges/index ranges in abcrem.

"Nullness" is tracked via a separate field that is stored adjacent
variable/array value ranges; values may be "maybe-null" or "not-null";
the intersection of maybe-null and not-null yields not-null, the union of
maybe-null and not-null yields maybe-null. The existing graph traversal logic
is reused as much as possible, which makes the implementation of this
null value approximation slightly more complicated than it would be
otherwise.

Fixes https://github.com/mono/mono/issues/16310, which was caused by
the use of non-empty array length ranges as a way to encode non-null
values.

This also allows null check elision for zero-sized arrays.

* Fix C++ compatibility. Fix a typo in a comment.

* Add symmetric nullness relations.

* Make ldlen imply that the source register is not-null, similar to CHECK_THIS or any other null check.
2019-09-06 08:55:21 -04:00
Vlad Brezae e49be5b9ec [metadata] Fix leaks when handling a few attributes (#16675)
Callers of mono_reflection_create_custom_attr_data_args_noalloc were leaking some of the returned information. Accessed attributes are FixedBufferAttribute and UnmanagedFunctionPointerAttribute.
2019-09-06 10:31:39 +03:00
Thays Grazia c23a8906f3 [debugger][android] It was not initialising seq_points on MonoCompile on Android, so when was compiling dynamic methods, seq_points wasn't created and we got the assert when try to single step. (#16691) 2019-09-05 22:16:50 -03:00
Vlad Brezae f0dabc6618 [interp] Kill InterpFrame.ex (#16666)
* [interp] Reduce unnecessary uses of frame->ex

* [interp] Remove ex from InterpFrame

We save stack space and move the exception where it belongs, in ThreadContext. It was confusing what frame->ex really meant, it was probably not used / initialized properly. We move it now to ThreadContext where it represents the current exception being thrown during EH (when we have a resume state set).
2019-09-05 23:10:13 +03:00
Filip Navara 673c8835fd [netcore] Bump CoreFX, core-setup dependencies (#16681)
* [netcore] Bump CoreFX, core-setup dependencies

* Update .deps.json hacks

* Update test exclusions
2019-09-05 20:03:47 +02:00
Jay Krell 7fc0f8842e [interp] Localize endfinally_ip to leave, and it cannot be null (barring (#16676)
[interp] Localize endfinally_ip to leave, and it cannot likely be null (unless offset carefully formed).
2019-09-05 13:46:12 -04:00
Jo Shields a67839540e Fix distclean (#16667)
* Disable `subdir-objects` mode in Automake

This forces creation of subdirectories' files (e.g. .deps/*.Plo) in the
parent directory listing the source file. This works around a situation
where two different files specify files in a different parent directory
(i.e. ../../support/adler32.c in mono/metadata/ and adler32.c in
support/) both emit files into the same place (support/.deps), causing
`make distclean` to try and delete support/.deps from two different
places, breaking distclean.

This unfortunately adds a pile of Automake warnings, during autogen.

The "correct" fix is to stop specifying zlib as a pile of flat .c files,
and instead put it in its own subdirectory, conditionally including
its .la files as required.

Partially fixes https://github.com/mono/mono/issues/16605

* Ship netcore/Makefile with an empty distclean rule

We need a netcore/ directory because we write to it during configure,
and we need a netcore/Makefile with a distclean rule, so top-level
recursive distclean doesn't fail. So specify both here.

Partially fixes https://github.com/mono/mono/issues/16605
2019-09-05 13:25:48 -04:00
Fan Yang abe48e3fd4 Remove redundant function call of EscapeTypeName to make (#16425)
* Remove redundant function call

* Remove commented out line of code
2019-09-05 18:12:57 +02:00
Maxim Lipnin f617a2692f [wasm] Add build.sh (#16682) 2019-09-05 11:10:05 -05:00
Jay Krell ee85b695c8 [interp] Remove unused parameter to ves_pinvoke_method. (#16677)
[interp] Remove unused parameter to ves_pinvoke_method.

It might actually cause stack pressure (but on its own, this PR does not save stack).
2019-09-05 11:30:59 -04:00
Zoltan Varga c3541c2738 [netcore] Fix the range checks for Vector.CopyTo(). (#16663)
* [netcore] Fix the range checks for Vector.CopyTo().

* [netcore] Fix Vector.get_Item () for u1/u2, the llvm type is a vector of i1/i2, so we need to zero extend.
2019-09-05 13:55:53 +02:00
Next Turn cb1a228a54 Remove unnecessary !s for [DoesNotReturn] (dotnet/coreclr#26512)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen Toub 0770a3d82b Fix StyleCop tuple warnings (dotnet/coreclr#26514)
A few PRs crossed such that PR validation passed but master is issuing lots of warnings-as-errors.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen Toub 1dc58da785 Remove readonly from GCHandle field on Gen2GcCallback (#26509)
GCHandle is mutable, and this type does call Free on the handle, but with it marked as readonly, the mutations performed by Free will be done on a copy rather than on the original.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Stephen ToubandAdam Sitnik ff7dd88cfa Use expression-bodied members in more places (dotnet/coreclr#26500)
* Fix IDE0027 (use expression body for accessors)

* Fix IDE0023 / IDE0024 (use expression body for operators)

* Apply suggestions from code review

Co-Authored-By: Adam Sitnik <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-09-05 10:46:55 +02:00
Zoltan Varga db626fe9d3 [llvm] Use branch weight metadata in some cases. (#16668)
* [llvm] Add functions to create weighted branches.

* [llvm] Use weighted branches in GC_SAFE_POINT, some llvm passes doesn't seem to recognize llvm.expect so the LLVM JIT generates the slow inline.
2019-09-04 18:17:07 -04:00
Sam Patel c0a36b62af Update CodeOwners (#16662)
* Update CodeOwners

Add Imraan as code owner for LLVM related files

* Update CODEOWNERS
2019-09-04 19:32:11 +02:00
Jay KrellandLarry Ewing 1f21aae8cb Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 9fc2ade131 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 4359ad66d0 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing e25d2e5536 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 557645fea3 Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b3f996d45d Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 8232c85e4c Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing bd9ae10c7c Take stars out of types where they make more sense.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b071aa9192 more star placement
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 46ea28c95b Move stars to sometimes but not always onventional and wierd location. C syntax experiment takes it toll for decades on.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing 90e2563523 Move stars to conventional but wierd place.
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing b8313d586a Update mono/eglib/gmodule-win32.c
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay KrellandLarry Ewing d0817ccdc6 Update mono/eglib/glib.h
Co-Authored-By: Larry Ewing <[email protected]>
2019-09-04 12:31:13 -05:00
Jay Krell a7357e8177 Remove unnecessary cast. 2019-09-04 12:31:13 -05:00
Jay Krell 595b66c0eb Allocate MAX_PATH to start, but it usually wastes memory.
Double later for clarity.
2019-09-04 12:31:13 -05:00
Jay Krell 8723e49b0e Fix C++? 2019-09-04 12:31:13 -05:00
Jay Krell fb6e2c75cd cleanup 2019-09-04 12:31:13 -05:00
Jay Krell da2471d001 Test invalid module. 2019-09-04 12:31:13 -05:00
Jay Krell 74f1312cd3 Skip psapi.dll and go right to the Windows 7 or newer versions in kernel32.dll. 2019-09-04 12:31:13 -05:00
Jay Krell 2db0c8c2d4 Remove cast. 2019-09-04 12:31:13 -05:00
Jay Krell 06f4f93aa5 comments 2019-09-04 12:31:13 -05:00
Jay Krell 3ca5b2687f Remove accident. 2019-09-04 12:31:13 -05:00
Jay Krell f8be8c6de7 MAX_PATH reduction. 2019-09-04 12:31:13 -05:00
Jay Krell a55da2a903 Tweak names and fix preexisting leak under low memory. 2019-09-04 10:31:22 -05:00
Jay Krell 14afe79a0a [Coop] GetStackTraces. 2019-09-04 10:31:22 -05:00
Egor Bogatov 30625b25df Implement more Math LLVM intrinsics (#16578)
* Implement more Math intrinsics

* Test on CI

* Update pipeline-netcore-runtime.yml

* undo "test on CI"
2019-09-04 09:54:52 -04:00
Jay Krell 350dd0e2de [Coop] ves_icall_System_IOSelector_Add 2019-09-04 08:36:42 -05:00
Stephen Toub 036ab58b42 Fix several recently added StyleCop tuple rules (dotnet/corefx#40794)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-04 11:15:23 +02:00
Egor Bogatov 1558316cec Throw NRE if src or dst in Memmove are nulls (#16650) 2019-09-04 11:01:53 +02:00
Zoltan Varga 55ac86fe8b [netcore] Fix the exception thrown by the Vector<T>.CopyTo() intrinsic. (#16618)
Fixes https://github.com/mono/mono/issues/16616.
2019-09-04 10:57:39 +02:00
Dotnet-GitSync-Bot dcb90fa1c0 Mirror changes from mono/coreclr,corefx (#16654)
* Refactor BinaryReader.ReadChars using Span (dotnet/coreclr#26325)

Avoids unsafe code and makes the code simpler

Signed-off-by: dotnet-bot <[email protected]>

* Remove unnecessary parens from `return (true);` and `return (false);` (#26488)

Just a search and replace.

Signed-off-by: dotnet-bot <[email protected]>

* Avoid some unnecessary initialization in DateTimeParse (#26487)

Signed-off-by: dotnet-bot <[email protected]>

* Update analyzer versions (dotnet/corefx#40778)

In particular to pick up StyleCop fixes related to C# 8 and nullability.

Signed-off-by: dotnet-bot <[email protected]>
2019-09-04 00:20:35 -04:00
Jay Krell 82d4d8d81d Fix overly sensitive g_ascii_strncasecmp tests. 2019-09-03 22:46:10 -05:00
Jay Krell b66ec468da Add g_asciiz_equal_caseinsensitive, g_ascii_equal_caseinsensitive, g_asciiz_equal.
The default takes a length.
"z" means zero terminated, the usually slower not preferred.
2019-09-03 22:46:10 -05:00
Jay Krell 8c288f0666 Cleanup and augment string handling based on https://github.com/mono/mono/pull/15522.
1. When comparing strings, check pointer equality.
Faster when they match, slower when they do not.

2. Provide string lowercasing into a provided buffer, which can equal
the input, to avoid extra malloc/free.

3. define G_STRING_CONSTANT_AND_LENGTH(x) (x), G_N_ELEMENTS (x) - 1
e.g. strncmp (foo, G_STRING_CONSTANT_AND_LENGTH ("version"))
2019-09-03 22:46:10 -05:00
Zoltan Varga e1b568b529 [netcore] Enable Vector<T>.get_Item () intrinsics for floats/doubles. 2019-09-03 22:05:15 -05:00
Zoltan Varga 4107a15389 [netcore] Disable socket support to fix warnings. 2019-09-03 21:24:23 -05:00
Alexis Christoforides e50a7360e1 Debugging 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4840f4161f [merp] Fix logic of 'is_managed' attribute 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4a52bd6da0 Fix pointing to stack on string && some add tracing 2019-09-03 20:30:30 -05:00
Alexis Christoforides 8c0dc66d25 Guard writing to out_module 2019-09-03 20:30:30 -05:00
Alexis Christoforides 1b66666cc7 Update MONO_CORLIB_VERSION 2019-09-03 20:30:30 -05:00
Alexis Christoforides 001398e05b Remove problematic assert from SIGILL handler
mono_handle_native_crash() function can actually return if the native crash has triggered or has been trigerred by  more than one signal. So it is possible to reach the point of the assert
2019-09-03 20:30:30 -05:00
Alexis Christoforides 8d4c0f3553 Add stub of new function when crash reporting is disabled 2019-09-03 20:30:30 -05:00
Alexis Christoforides 4b02c3eb9d Set out_module, to avoid getting no useful information in the printout
It seems that without this, most/all entries in the unmanaged frames list are have no useful information.
2019-09-03 20:30:30 -05:00
Alexis Christoforides 40a8d88d8e [merp] Add API method that whitelists all native libraries
This is so that complex consumers such as Visual Studio for Mac can forego attempting to enumerate all loaded libraries, register them one at a time and potentially keep the list updated
2019-09-03 20:30:30 -05:00
Next Turn 0fc4d68d8e Use Span<T> in Encoding (dotnet/coreclr#25860)
Signed-off-by: dotnet-bot <[email protected]>
2019-09-03 19:42:55 -05:00
Zoltan Varga 83953ef83d [wasm] Disable threads tests as well. 2019-09-03 18:06:36 -05:00
Zoltan Varga aad5494258 [wasm] Avoid aligning the stack bounds, they are not page aligned on wasm. 2019-09-03 18:06:36 -05:00
Zoltan Varga ea512c1604 [wasm] Bump emscripten to 1.38.43. 2019-09-03 18:06:36 -05:00
Zoltan Varga 5f79b69e34 [wasm] Disable threads support, it doesn't work with 1.38.42. 2019-09-03 18:06:36 -05:00
Larry Ewing 68b7af0539 [wasm] update emscripten 2019-09-03 18:06:36 -05:00
Jay Krell 1780e1d1f8 [coop] CustomAttributeBuilder_GetBlob, etc. (#16579)
mono_reflection_get_custom_attrs_blob_checked
 ModuleBuilder_WriteToFile
 ModuleBuilder_build_metadata
2019-09-03 18:52:41 -04:00
Jay Krell 651eb7696e [Coop] ves_icall_InternalInvoke. (#16625)
* [Coop] ves_icall_InternalInvoke.

(Part taken from ves_icall_InternalExecute https://github.com/mono/mono/pull/16622.)

* error_init_reuse.

* Remove stars from types.
2019-09-03 17:18:21 -05:00
Thays Grazia 1046dd746f [debugger][android] Fixing single step inside Exception on Android (#16596)
* Fixing single step inside Exception on Android...
The method is null because on ss_create_init_args the IL is -1 when calling method Mono_UnhandledException_internal what I think it's expected, because of that I ignore the single step on android if there is no method. (debugger-engine.c)
After that the sequence point cannot be found because it's calling a method of class DynamicMethodNameCounter what I think it's expected, because of that I remove the assert on android. (debugger-agent.c)
Fixes #14772

* Changing comment to make it clear...
2019-09-03 16:28:37 -04:00
Egor Bogatov 625af20cef [netcore] Use CoreRT's Overlapped (#16106)
Made it Nullability friendly

Interop.MemFree  -> Marshal.FreeHGlobal
Interop.MemAlloc -> Marshal.AllocHGlobal

Fixes https://github.com/mono/mono/issues/15308
Fixes https://github.com/mono/mono/issues/15310

Also, copy ThreadPoolBoundHandle.PlatformNotSupported.cs
2019-09-03 16:22:04 -04:00
Jay Krell 37e49876b9 Fix non-netcore warning about add_assembly_to_alc. (#16622) 2019-09-03 16:09:24 -04:00
Filip Navara 4ae804bb33 [netcore] Add intrinsic implementation for Unsafe.InitBlockUnaligned 2019-09-03 18:40:06 +02:00
Vlad Brezae 930a5d7d44 [interp] Transform ldloc/stloc pairs to a movloc instruction (#16546)
* [interp] Add stack information for each instruction

For each instruction PopN indicates how many values from the top of the stack are used by the instruction. PushN signals how many values are pushed back. These are also used to prevent values from the stack being optimized out, since they might still be used by an instruction. VarPop and VarPush mean that the number is not constant, so we need to inspect the data fields of the instruction (typically for an imethod or csignature) to figure out the stack usage.

* [interp] Convert ldloc/stloc pairs to movloc

The movloc instruction operates directly on locals. This is faster due to avoiding dispatch and stack indirection. More important is that it makes it easy to do copy propagation of locals and it has no stack usage, which means that it could even be killed if the destination local is no longer referenced (as well as STLOC_NP). In the future we could reuse this mechanism of tracking the value of a stack slot to do constant propagation and kill even more instructions (we might need to have _IMM versions of opcodes for this to work).

* [interp] Enable disabling of interp optimization

* [interp] Add time tracking stat for cprop optimizations

* [interp] Move stloc.np transformations to the cprop pass

It makes more sense to do such transformations here and we also don't want transformations in the generate phase prevent us from detecting alternative transformations when running the full pass.

* [interp] Add counter for instructions that were optimized away

* [interp] Make it explicit that we are dealing with IR code

It can be confusing in verbose output.
2019-09-03 19:36:21 +03:00
Nikolay Sivov f8718c7a05 [cominterop] Maintain object domain around ccw wrapper calls. 2019-09-03 18:03:42 +02:00
Ryan Lucia 5377f9655d Remove MAX_PATH usage in w32process (#16553)
* Remove MAX_PATH usage in w32process

* Fixes

* Shrink buffer temporarily

* More fixes

* Reset buffer size to 260

* Fix length check and add notes

* Actually free things

* Add tracing and zero len at start

* Typo

* Fix another leak

* Simplify logic
2019-09-03 11:41:27 -04:00
Zoltan Varga 98a4d5ec1e [llvm] Implement more SIMD intrinsics on netcore. (#16449)
* [netcore] Add intrinsics for Vector128/Vector256.Count.

* [netcore] Implement Vector<T>.get_Item () as an intrinsic.
2019-09-03 08:31:16 -04:00
Egor Bogatov 416a31796e Improve Fast-Math mode for LLVM (#16599)
It turns out it's not enough to set global fast math parameters, it's also required to mark some instructions as "fast" (see https://llvm.org/docs/LangRef.html#fast-math-flags) like, for instance, clang does, see https://godbolt.org/z/U8TFST
E.g. all optimizations inside SimplifyLibCalls pass (it's a subpass of InstCombine) need it.
2019-09-03 08:19:03 -04:00
Egor Bogatov 5692180ea2 [netcore] Makefile clean up (#16630)
Remove ASP.NET sample (we currently have to download apsnet runtime everytime)
instead we can do:

dotnet new webapp

and do make patch-app APP_PATH=/path/to/myasp (works for any kind of apps/templates)
2019-09-03 07:55:22 -04:00
Bernhard Urban f23113afcd Relax assembly checking for 64 bit targets to allow running mixed-mode MSVC code. (#16330)
* [jit] Relax assembly checking for 64 bit targets to allow running mixed-mode MSVC code.

Fixes #37913

* STACK_PTR|I8 should be the same as I8|STACK_PTR tuple

* fix i4 to ptr conv on 64bit
2019-09-03 12:14:02 +02:00
Vlad Brezae f491af13f4 [interp] Remove varargs from InterpFrame and recompute it instead (#16598)
It is a waste of stack space.
2019-09-03 11:21:56 +03:00
Filip Navara b9b13f0fc1 Use CMP_GE/CMP_LE/CMP_GE_UN/CMP_LE_UN instead of manual decomposition 2019-09-03 09:43:17 +02:00
Filip Navara 03899588ba [netcore] Add intrinsics for Vector.GreaterThanOrEqual/LessThanOrEqual 2019-09-03 09:43:17 +02:00
Dotnet-GitSync-Bot a6b91f4295 Mirror changes from mono/coreclr (#16547)
* Move FormatTypeName to Type (dotnet/coreclr#25631)

Signed-off-by: dotnet-bot <[email protected]>

* Fix for mono
2019-09-02 16:02:59 -04:00
Egor Bogatov 73c8017b1f Merge pull request #16624 from filipnavara/hebrew
[corlib] Add hack for Hebrew calendar data
2019-09-02 20:43:08 +03:00
monojenkins eaa525ab41 [csproj] Update project files 2019-09-02 10:33:47 +00:00
Filip Navara 6ca4e84758 Fix month names for the Hebrew calendar 2019-09-02 11:19:24 +02:00
Filip Navara ca389a3cae [corlib] Add hack for Hebrew calendar data 2019-09-02 02:45:45 +02:00
Alexander Köplinger 07b7c7d824 [Coop] Monitor. (#16608)
* [Coop] Monitor.

* PR: Rename function; cleanup slightly.

* PR: And then rename the icall back.

* Rename parameter back.
2019-09-01 15:14:17 +02:00
Jay Krell b3e48bcf95 [Coop] Fully inline NULL_HANDLE (do not even access a global variable). (#16612) 2019-09-01 15:13:20 +02:00
Ryan Lucia c9a6dd351a [netcore] Add handle frame to loop body in GetTopLevelForwardedTypes (#16594)
* [netcore] Add handle frame to loop body in GetTopLevelForwardedTypes

* No early returns

* Move scope of cols

* Fix C++ builds, probably
2019-08-31 21:40:52 -04:00
Egor Bogatov 6327274b54 LLVM: Make Buffer.Memmove intrinsic (#16610)
* Add memmove intrinsic

* Fix typo

* Add -lower-expect
2019-08-31 19:42:44 -04:00
Jay Krell 24180e9106 Rename parameter back. 2019-08-31 10:35:14 -07:00
Jay Krell 5d9ca20646 PR: And then rename the icall back. 2019-08-31 10:34:32 -07:00
Jay Krell ea0db154b4 PR: Rename function; cleanup slightly. 2019-08-31 10:13:47 -07:00
Zoltan Varga fb26a68fd1 [aot] Avoid crashing with --aot=llvm is used in a runtime without llvm support. (#16577) 2019-08-31 09:03:45 -04:00
Jay Krell 91445e2286 [Coop] Monitor. 2019-08-31 05:22:45 -07:00
Jay Krell 9b193b9ec4 [coop] Allow SETUP_ICALL_FRAME (i.e. a loop) to reference calling frame w/o HANDLE_FUNCTION_ENTER or SETUP_ICALL_FUNCTION. (#16498)
This is based on my read of https://github.com/mono/mono/pull/16487.
2019-08-31 14:20:44 +02:00
Nikolay Sivov 2f5c8cba6e [corlib] Simplify file: protocol check in Assembly::Load(). (#16494)
Mirrors change made with 068660732f.
2019-08-31 08:19:41 -04:00
Vlad Brezae bf53a3a53b [interp] Fixes for il_offsets associated with instructions (#16587)
* [interp] Remove some some false is_bb_start markings

Having fewer basic blocks makes potential optimizations more effective.

* [interp] Persist variables in TransformData

Thate are not used by inlined code because we don't inline code that has multiple basic blocks or clauses.

* [interp] Enable emitting of MINT_NOP instruction

This instruction will be optimized away when compacting the instruction stream.

* [interp] Clear instructions insead of removing them

Makes everything easier.

* [interp] Fix some instructions size inconsistencies

* [interp] Add more is_bb_start markings

* [interp] Add nop instead of extending bb

We should always have an IR instruction associated with an IL offset, especially when that instruction is a bb_start since we will need to branch to it and resolve the IR instruction from the il_offset.

* [interp] Simplify resolving in offset

We should always have an in_offset associated with the il_offset we are branching to.

* [interp] Add correct il_offset for all instructions

Except for special prolog instructions that will have -1 as the offset. We don't make much use of il_offset for non bb_start instructions, but they might be useful for debugging.

* [interp] Fix il_offset of some conv opcodes
2019-08-31 13:37:06 +03:00
Jay Krell a768b15330 Follow convention -- "static inline" is only for .h files, "static" is for .c. (#15502)
inlining is up to the compiler to decide.
It is used in .h files for linkage/dedup, not to affect codegen.
2019-08-31 09:40:33 +02:00
Jay Krell cba94f28be [coop] InternalGetHashCode (#16583) 2019-08-31 09:38:12 +02:00
Aleksey Kliger (λgeek) 8bc27f6665 [runtime] Make eglib tests run on CI (#16187)
[runtime] Make eglib tests run on CI

Update mono/eglib SUBDIRS to build tests after eglib is built.

Add a check-local automake rule.

Add CI steps to run the eglib tests.
2019-08-31 02:06:25 -04:00
Jay Krell f735571f27 Cleanup x86_patch and amd64_patch. (#16474)
* Cleanup x86_patch and amd64_patch.

amd64_patch has two incorrect looking probably unused cases.
 g_assert(0) and fix them.
 call and jmp indirect (one case)
 move reg RIP relative

x86_patch is large-ish inline macro.
Presumably for sharing with amd64, not inlining.
Restructure as an inline function, called once.
Try to make the code a little clearer also, with same functionality and efficiency
 (modulo the optimized perf of "if i == 4" vs. "if i")
i.e. speak of instruction_size and offset_size, not pos and boolean size.

Add to x86_patch the ability to patch jmp/call indirect,
as amd64_patch has, but which was incorrect.
This would seem useful for a later change (using "thunks"
to patch nearby data instead of patching aligned instructions,
indirect call/jmp vs. direct).

amd64_patch largely delegates to x86_patch.
One case asserts and delegates.
Move the assert to the x86 side, where an optimizing
compiler should know to remove it for x86 anyway.

Change some assert to g_assert.

Remove some unnecessary casts.

Fix an assert (https://github.com/mono/mono/pull/16471).

* Change g_assert back to assert except where making larger changes
and less expansion.
2019-08-31 00:45:55 +03:00
Bernhard Urban 06e63b345f [mini] publish global patches after JitInfo has been added (#16589)
[mini] publish global patches after JitInfo has been added

Fixes https://github.com/mono/mono/issues/14080

Consider the following example:

```csharp
static void CommonCallTarget () { }

static void TailA () {
    tailcall CommonCallTarget ();
}

static void TailB () {
    tailcall CommonCallTarget ();
}
```

Since `TailA` and `TailB` are tailcalling into `CommonCallTarget`, the resolution at patch-time is a bit tricky, that is, since it's a jump-like instruction the patching machinery won't know where it was called from. That's why we maintain a global hashtable `jump_target_hash` where each jump-site is signed up to be patched. At patch-time we know the target method (in the example `CommonCallTarget`), but since we don't know where we are coming from, we will just apply all patches for that target.

This works since ages, so why did it crash on arm64 sometimes?
When the patching happens, we check if the displacement between jump-site and target fits into it (24bit). If not, which happens not very often, we have to allocate a _thunk_:
https://github.com/mono/mono/blob/36296ce291f8a7b19de3eccb7a32c7e4ed9df8f2/mono/mini/mini-arm64.c#L928-L942

So instead of jumping to the target directly, we'll branch to the thunk. This is a little trampoline that loads the full address of the target and then finally branches to it. This one will live close-by the jump-site, because during compilation we will reserve specifically for that scenario some space after the generated code. For this, however, we need the JitInfo of the jump-site. And that's where the origin of the race is. Let's say:

* Thread A compiles `TailA`, and then jumps into it. Thus one patch point is in the `jump_target_hash`.
* Now Thread B compiles `TailB`, registers the patch point but has _not_ yet registered its JitInfo.
* Then Thread A continues, does the tailcall into `CommonCallTarget`, enters the patching machinery, which sees two patches. Now assume when applying the patch for `TailB` the displacement is too large, thus it tries to allocate a thunk but can't find the relevant JitInfo for it that it needs to emit the thunk. So it crashes as reported in https://github.com/mono/mono/issues/14080

As far as I can tell this only affects ARM64, ARM and PPC.



<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-08-30 16:46:10 -04:00
Steve Pfister dbce6ac447 Turned the compiler server off on Windows because Named Pipes aren't fully working (#16552)
* Turned the compiler server off on Windows because Named Pipes aren't fully working

* Added win32 to the CS is turned off message
2019-08-30 16:27:14 -04:00
Bernhard Urban 5eb67ac88a Revert "[android] add hardware specific workaround for Nexus9 in armv7 mode" (#14691)
Revert "[android] add hardware specific workaround for Nexus9 in armv7 mode"

This reverts commit ee90fc601e (also see https://github.com/mono/mono/pull/4878 ).

The device was released November 2014.  I think it's fair to assume it isn't widely used anymore.
2019-08-30 16:26:12 -04:00
Zoltan Varga 4997d83291 [netcore] Fix make patch-local-dotnet if .dotnet has more than one netcore install. (#16590) 2019-08-30 14:08:41 -04:00
Jay Krell 86c45ff645 [coop] mono_runtime_class_init_full (#16580) 2019-08-30 13:30:58 -04:00
Steffen Kieß 80b7813d54 [Mono.Posix] Add FcntlCommand.F_OFD_{GETLK,SETLK,SETLKW} (#16571)
Add F_OFD_GETLK, F_OFD_SETLK and F_OFD_SETLKW to FcntlCommand. These
commmands are similar to F_GETLK, F_SETLK and F_SETLKW but operate on a
per-open-file-description lock instead of a per-process lock and avoid the
problem that the locks are released when close(dup(fd)) is called. See
https://lwn.net/Articles/586904/ for more information.

This commit also reruns create-native-map but keeps manual changes from
ed892ccf27,
b522eab5ff and
22b6b95814.

In [create-native-map.diff.txt](https://github.com/mono/mono/files/3557077/create-native-map.diff.txt) you can see the difference between the commited `map.[ch]` and the one `create-native-map.exe` wants to create:
- Manually inserted changes for NetBSD (`MAYMOVE` is turned into a negated `FIXED`)
- `Mono_Posix_ToSockaddrIn6()` and friends get skipped for `HOST_WIN32` (not sure why, when `sockaddr_in6` exists then `Mono_Posix_ToSockaddrIn6()` should work)
- At the bottom of `map.h` some functions are reordered and the parameters change (`const char*` vs. `char*`)

Running create-native-map.exe also moved some O_NOATIME-related stuff (from 8b9033e4d2) to the correct position and added it to Mono_Posix_ToOpenFlags(), which I kept in the commit.
2019-08-30 19:13:33 +02:00
Nikolay Sivov c8707c6e84 [cominterop] Respect ComDefaultInterfaceAttribute when constructing ccw. (#16584) 2019-08-30 18:29:54 +02:00
Nikita Voronchev 94797b95d6 [WinForms] Fix UIA event on node insertions into a TreeView object (#16559)
The UIA event doesn't emited by means of insertion a new `TreeNode` to a `TreeNodeCollection` of the respective `TreeNode`.
2019-08-30 18:29:18 +02:00
Jay Krell 9db00e7495 [Coop] ves_icall_RuntimeFieldInfo_GetRawConstantValue. (#16582) 2019-08-30 10:53:40 -04:00
Bernhard Urban 07045604cf [configure] remove autodection of llvm-config in $PATH (#16569)
[configure] remove autodection of llvm-config in $PATH

Instead `./autogen.sh --with-llvm=<path>` should be used if a build from `external/llvm` is not preferred.

The old behavior was confusing when a `llvm-config` is on `$PATH`, which can be the case when you have LLVM packages installed on a Linux distribution. It picked that up when `./autogen.sh --enable-llvm` was used.

/cc @brianrob
2019-08-30 09:16:03 -04:00
Jay Krell 36296ce291 [interp] Compress interpreter opcode names in usual way, offsets vs. pointers. (#16350)
* [interp] Compress the interpreter opcode names in the usual way, by changing
from an array of pointers to an array of 16 bit offsets into one large "string".

* PR: Make mono_interp_opname a function.
2019-08-30 11:27:19 +03:00
Jay Krell 3db78e829c [coop] Convert System_Runtime_Remoting_Messaging_AsyncResult, etc. (#16566)
* [coop] Convert System_Runtime_Remoting_Messaging_AsyncResult_Invoke, etc.

* Remove mono_load_remote_field_checked from this PR.

* Pin ac->msg earlier (should not matter).

* Add missing barrier.

* Pin slightly earlier -- still should not matter.
2019-08-29 22:51:15 -05:00
Jay Krell cbb9c645eb [Coop] mono_runtime_class_init_full mostly single point of exit. 2019-08-29 22:00:46 -05:00
Zoltan Varga 796a575131 Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5. (#16567)
* Bump ikvm to pick up caa8e7f54279a47422626005d228447f5d3670d5.

* [monodis] Add a missing cattr parent case.
2019-08-29 20:46:30 -04:00
Egor Bogatov c3ccb6e027 Make Math.Pow, Min and Max for doubles intrinsics (#16561)
* Add new Math intrinsics

* call EB.setTargetOptions only for fast-math

* Address feedback

* rename float opcodes

* Rename opcodes

* Update CoreFX.issues.rsp
2019-08-29 19:01:15 -04:00
Zoltan Varga 210b324347 [simd] Implement .ctor(T[]) and CopyTo () as intrinsics. (#16558) 2019-08-29 19:00:59 -04:00
Jay Krell 5390b796dd Cleanup trampoline size checks. (#15624)
Cleanup trampoline size checks.

- named `int` everywhere
- `assert` everywhere
- assert `<=` everywhere instead of `<`
- `const` almost everywhere
Names not consistent, following existing code (`size`, `buf_len`, `kMaxCode`.)

Only touching x86/amd64, granted.

Could be better -- make them two pass and avoid the presizing.
2019-08-29 16:16:14 -04:00
Alexander Köplinger 309cf1bcee Build pkg-config as 64bit only, libtiff as universal and remove gtk-quartz-engine.py (#16548)
See https://github.com/mono/bockbuild/pull/116
2019-08-29 20:24:14 +02:00
Ryan Lucia 3b57849541 [netcore] Implement AssemblyLoadContext.LoadUnmanagedDllFromPath (#16525)
* [netcore] Implement AssemblyLoadContext.LoadUnmanagedDllFromPath

* Fixes
2019-08-29 12:35:15 -04:00
Nikolay Sivov 421bd79f06 [cominterop] Implement native-to-managed safearray marshalling. (#16483)
* [cominterop] Implement native-to-managed safearray marshalling.

* [tests] Fix index variable type.

In attempt ot fix MSVC build failure (int* vs LONG* on Windows).

* [cominterop] Use proper statement block for newly added switch entry.

* [marshal] Conditionally compile in mono_cominterop_emit_marshal_safearray().

Hopefully should fix remaining build failures.

* [cominterop] Don't free user array argument
2019-08-29 08:19:08 -04:00
Jay Krell 52c1d4acfc [Coop] mono_load_remote_field_checked single point of exit. (#16526)
Extracted from earlier version of #16504.
Baby steps.
2019-08-29 11:00:44 +02:00
Egor Bogatov 2e9c5df855 LLVM: Add more Math intrinsics, introduce --cpu flag (#16309)
* Add Math Intrinsics, introduce --cpu flag

* cpu -> mcpu
2019-08-29 04:32:43 -04:00
Jay Krell 2f2d9541e5 [coop] Cleanup mono_reflection_get_custom_attrs_blob_checked. (#16535)
- single point of exit
 - local variable cheaper than strcmp
2019-08-29 04:23:20 -04:00
Jay Krell 07c6b1c9f1 Should fix many warnings: sgen-protocol-def.h adding unsigned long to a string does not append to the string (#16288) 2019-08-29 10:08:42 +03:00
Filip Navara e32fb11dbe [netcore] Add Monitor.LockContentionCount icall (#16538)
* Add Monitor.LockContentionCount icall

* Add NOHANDLES around the new icall
2019-08-28 20:35:07 -04:00
Marius Ungureanu 1d23ca14e1 Use the right FileSystemWatcher sources (#16539)
* Use the right FileSystemWatcher sources

The corefx code for FileSystemEventArgs has some changes to support
constructing one with a null filename.

Fixes #16486

* Remove the mono sources for the newly replaced files

* [csproj] Update project files
2019-08-28 19:45:25 -04:00
Jay Krell 7ca965e09c Read only function pointers (MonoThreadInfoRuntimeCallbacks). (#16545)
Read only memory is preferred, for efficiency and security.

Even if it requires relocation first.

If the runtime is loaded in more than one process, physical pages can be shared (at least on Windows, where the kernel does the relocation).

If there is memory corruption, writes to here will sigsegv/accessviolate.

The older code also copied the pointers. For lifetime purposes that could be useful, but generally is not. i.e. they live with the code they point to.

Writable function pointers are also particularly targets for attackers.
There's been tons of defenses put around them through the years.
Read only ones are not vulnerable -- unless the attacker escalates and calls mprotect first.

When there is a JIT, arguably there is just tons of attack surface.
But there is also AOT.

Structs of function pointers are usually read only in the wider world.
2019-08-29 00:10:40 +02:00
Steve Pfister 7c4c30ec00 Implement Read and WriteAsyncMemory in Mono DeflateStream
Resolves https://github.com/mono/mono/issues/16122

Since we put in the corefx version of GZipStream, it is expected that
our version of DeflateStream actually implements Write+ReadAsyncMemory.

This change does that by just calling the plain WriteAsync & ReadAsync
methods until we decide (or not) to pull in the corefx DeflateStream.
2019-08-28 16:20:09 -04:00
Zoltan Varga c4695b8b44 Merge pull request #16323 from EgorBo/ffast-math
Fast Math mode
2019-08-28 16:16:18 -04:00
Jay Krell 056d5ca545 [interp] Fix remaining MonoError reuse. (#16533)
* [interp] Fix remaining MonoError reuse.

This finishes off:
 https://github.com/mono/mono/commit/38aebeae38704df53ef8f2991ff63e293a3ac4de

which is *ok*, but leaves alone the other approach:
 https://github.com/mono/mono/commit/4d9e6ae20e536394c9ff65c61238045db294c08e
 https://github.com/mono/mono/commit/b654aa1ca858666539a677e96d00f594fd81c630
 https://github.com/mono/mono/commit/fd4956fbf277ec56a13137e01b48efb98a5556f5

Several other options are possible.

* self and peer PR: No need to cleanup successful error.
2019-08-28 21:33:48 +02:00
Jay Krell ff73c84482 Portably pass a full pointer and not just an integer to terminate vararg pointer lists. (#15629)
i.e. because of define NULL 0.
Though I think at least Visual C++ special cases this and passes a full pointer to varargs.
2019-08-28 13:35:10 -04:00
Jay Krell a0ba2cbce5 [interp] Possible code size savings via goto instead of macros. (#16531)
[interp] Possible code size savings via goto instead of macros.

Possibly the compilers do this already.
2019-08-28 12:59:11 -04:00
Egor Bogatov ffc3081c3b Merge remote-tracking branch 'origin/master' into ffast-math 2019-08-28 19:01:34 +03:00
imhameed bd3cc61301 Merge pull request #16242 from lateralusX/lateralusX/windows-fix-local-valuetype-copy-llvm
OP_LLVM_OUTARG_VT doesn't always create temporary vt needed by some ABI's.
2019-08-28 08:23:43 -07:00
Bernhard Urban 71f0f0ff82 [mini] do not re-AOT whole BCL for fullaotcheck targets (#16536)
[mini] do not re-AOT whole BCL for fullaotcheck targets

And enable `llvmfullaotcheck` target on CI.

Fixes https://github.com/mono/mono/issues/15999

On my armv7/linux machine with a `--with-runtime-preset=fullaot_llvm` configured build, before this PR:
```console
$ time make -C mono/mini llvmfullaotcheck -j4
[...]
real    9m42.362s
user    23m2.000s
sys     0m18.480s
```
After:
```console
$ time make -C mono/mini llvmfullaotcheck -j4
[...]
real    3m1.984s
user    4m51.648s
sys     0m8.248s
```
2019-08-28 11:19:39 -04:00
Alexander Köplinger 2152f0a16d [netcore] Add missing 'prepare' dependency to run-tests-corefx 2019-08-28 16:17:31 +02:00
Phil Jaenke 1bee4e354c [corlib] Remove/Update deprecated TZs for #16415 in TimeZoneInfoTests (#16450)
This fixes #16415 

All timezones indicated as deprecated have been updated to the current IANA specification.
2019-08-28 16:08:20 +02:00
Zoltan Varga f562c7ef6a Merge pull request #16517 from vargaz/wasm-xunit4
[wasm] Enable xunit tests for corlib/System.Core.
2019-08-28 09:46:09 -04:00
Jay Krell c80a1eca63 Fix assert in x86_patch regarding second byte of Jcc instructions being 0x80-0x8F vs. 0x70-0x8F (6byte conditional branch with 32bit offset). (#16471) 2019-08-28 15:42:44 +02:00
Aleksey Kliger (λgeek) fc21168065 [merp] Initialize hang_watchdog_path earlier; be lax if it's missing (#16298)
* [merp] Initialize hang_watchdog_path earlier; be lax if it's missing

Initialize hang_watchdog_path earlier - not just when
ves_icall_Mono_Runtime_EnableCrashReportingLog is called.  That function is
only expected to be called by apps that explicitly want progress reports from
MERP when it's collecting a crash report.  But we want the mono-hang-watchdog
to execute for all mono crashes (when crash reporting isn't ifdef'd out).

Also if we try to exec mono-hang-watchdog and fail for some reason, print a
nice message and then exit.  We used to call g_assert_not_reached which would
kick off another MERP of the _forked child process_ which would then get
confused because it really can't run in the forked process at all.
2019-08-28 08:34:30 -04:00
Jay Krell 2ca04a9743 [interp] Outline box. (#16456)
With additional reasonable changes this should conserve stack.
A combined PR that demonstrates actual savings probably should/will be presented.
2019-08-28 13:51:23 +03:00
Jay Krell 56b83f489b [interp] Minor cleanup of stackval_from_data. Compiler can do it. (#16534)
I missed this earlier cleaning stackval_to_data.
2019-08-28 12:11:56 +02:00
Jay Krell c870df0aa7 [interp] Asserts pessimize clang. Removing saves 16 bytes on Linux/amd64/clang. (#16530)
Paired with https://github.com/mono/mono/pull/16456, saves 16 bytes on Linux/amd64/clang.
No change with gcc.
Contributes to https://github.com/mono/mono/issues/16172.
2019-08-28 12:10:30 +02:00
Eric Erhardt 39ddea3983 Don't validate empty TransitionTimes on deserialization. (#26405)
On Unix, TransitionTimes are not used, so they are left empty. When deserializing an empty TransitionTime, it is throwing an exception because those are not normally valid. Fix deserialization to not validate empty TransitionTimes.

Fix dotnet/corefx 40578

Signed-off-by: dotnet-bot <[email protected]>
2019-08-28 11:55:07 +02:00
Jay Krell c534511003 Remove dead Boehm code. (#15224)
* Remove dead Boehm code.

Unity looked similarly dead.

The key points making it dead were:

create_allocator:
  g_assert_not_reached() covering everything but local variables

mono_gc_get_managed_allocator:
  always returning NULL, killing a bunch of code
  after it

mono_gc_is_critical_method:
   looking at data that was never filled in

This should have no impact, not even on Boehm.
Boehm should continue working just the same.

It is nice because it is the few/only uses of CEE_MONO_TLS.
2019-08-28 10:56:08 +02:00
Jay Krell 0375acd7c8 [mini] Cleanup MONO_MMAP_32BIT. (#16469)
- It _presumably_ means nothing on 32bit ARM and 32bit MIPS (was there mono/mips64?).
 - Leaving only PowerPC32 and PowerPC64 (in this vicinity).
   - It presumably means nothing on PowerPC32.
   - Leaving PowerPC64 as the only user in this vicinity -- is it needed?
- Notice this is varyingly absent on x86, arm64, amd64, s390x.
  Subject of another PR maybe.

Perhaps this was to enable the debugger to be 32bit against a 64bit target
on a biarch system, so it could do a cross-process manipulation of these pages?
That makes some sense on Windows at least.
(32bit debugger cannot debug 64bit target as of Vista, but that is using the Win32
"hard" debugger API. Mono could do otherwise).

There remains amd64 mono-codeman.c define ARCH_MAP_FLAGS MONO_MMAP_32BIT left alone.
2019-08-28 10:54:00 +02:00
Zoltan Varga 162849f972 Merge pull request #16520 from vargaz/fix-llvm-2
[llvm] Fix ctlz intrinsics, they have 2 arguments.
2019-08-28 03:24:01 -04:00
Bernhard Urban d6fa5ff9c6 fix cast and signature for mono_rconv_u4 2019-08-27 20:35:42 -05:00
Bernhard Urban 460bfa9beb [amd64] use opcode emulation for fconv/rconv 2019-08-27 20:35:42 -05:00
Bernhard Urban 8e6b967391 [llvmaotcheck] require sse3 being available for regression tests 2019-08-27 20:35:42 -05:00
Bernhard Urban 5b20322a84 [arm] use opcode emulation for fconv/rconv 2019-08-27 20:35:42 -05:00
Bernhard Urban ba824cb921 [arm] account only 4 bytes on stack for single precision arguments
Fixes `test_0_arm64_small_stack_args` in gshared.cs when
* `test_0_arm64_small_stack_args` is compiled with LLVM
* `Foo3.Floats` is compiled with Mini
2019-08-27 20:35:42 -05:00
Bernhard Urban 475defca4d [arm64] use opcode emulation for fconv/rconv
LLVM eagerly optimizes constants such as NaN in a way that does not align with .NET behaviour. In the future we can change it so that we only do opcode emulation when LLVM is used.

Fixes https://github.com/mono/mono/issues/16411
2019-08-27 20:35:42 -05:00
Bernhard Urban d164550a61 [ci] add make -C mono/mini llvmaotcheck to FullAOT LLVM lanes 2019-08-27 20:35:42 -05:00
Jay Krell 645c01066c Cleanup arm64 SeqPointInfo. 2019-08-27 20:34:42 -05:00
Stephen Toub 55002c3e02 Fix pragma warning restore (dotnet/coreclr#26389)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-28 00:16:03 +02:00
Jay Krell f516e79dd9 Inline mono_string_length_internal and mono_string_chars_internal (#16374)
* Inline mono_string_length_internal and mono_string_chars_internal for everyone.

* Add MONO_REQ_GC_UNSAFE_MODE.
2019-08-27 17:39:18 -04:00
Zoltan Varga cb5470b40a Fix timeouts. 2019-08-27 17:37:59 -04:00
Tarek Mahmoud Sayed 9ad1f08077 Fix Abbreviated Genitive Month Names Parsing (dotnet/coreclr#26379)
* Fix Abbreviated Genitive Month Names Parsing

In 3.0 we have fixed the formatting to use the abbreviated genitive month names when having "d" format specifier followed by "MMM". This fix is good as original formatting specs required but we needed to support the parsing when we have such genitive names.

* Addressing the feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 22:56:08 +02:00
Filip Navara f4145ce108 Update version information in System.Private.CoreLib to mimic CoreCLR and make BenchmarkDotNet happy (#16515)
BenchmarkDotNet recently updated the algorithms for .NET version detection (https://github.com/dotnet/BenchmarkDotNet/pull/1230) which made it incompatible with the version presented by Mono.

This updates the value of [ProductName](https://github.com/dotnet/coreclr/blob/1f428dbc5554191dadd27511cd5cd181e4a60efb/Directory.Build.targets#L7) and [Version](https://github.com/dotnet/coreclr/blob/c9007e73286456434ba12744e778e87a89fc1d90/eng/Versions.props#L5) to match what CoreCLR is currently reporting and makes the [BDN detection code](https://github.com/dotnet/BenchmarkDotNet/blob/b6d0e0f1d80ed60aa157149a0dfb704a2da4ca41/src/BenchmarkDotNet/Environments/Runtimes/CoreRuntime.cs#L73-L81) happy.
2019-08-27 22:55:37 +02:00
Ryan Lucia f9f5fff40c [netcore] Change exception thrown for SetTypedReference (#16518)
This brings us in line with CoreCLR behavior.
2019-08-27 22:53:37 +02:00
Zoltan Varga 340289d4ae [llvm] Fix ctlz intrinsics, they have 2 arguments. 2019-08-27 16:34:27 -04:00
Zoltan Varga ab1e9be35e [llvm] Fix the mixing of LLVM modules in the llvm JIT, use ctx->lmodule instead of module->lmodule. 2019-08-27 16:31:18 -04:00
Zoltan Varga 44d33673a5 Fix the netcore build. (#16519) 2019-08-27 22:30:16 +02:00
Zoltan Varga 0c015aa815 [wasm] Enable xunit tests for corlib/System.Core. 2019-08-27 15:46:26 -04:00
Zoltan Varga 3108d43051 Merge pull request #16487 from vargaz/cattr-data-handles
[runtime] Convert mono_reflection_create_custom_attr_data_args () to use handles.
2019-08-27 15:22:30 -04:00
Zoltan Varga 79475574a6 [wasm] Add more xunit test exclusions. 2019-08-27 13:41:11 -05:00
Egor Bogatov e9dff75d2f use gunichar2 2019-08-27 20:21:39 +02:00
Alexander Köplinger f914d900f6 Remove AppContext.cs that was moved to shared 2019-08-27 20:21:39 +02:00
Egor Bogatov f97d699183 Move AppContext.CoreCLR.cs to shared, remove static ctor (#26045)
* Remove AppContext.CoreCLR.cs (move to shared)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 20:21:39 +02:00
Steve Pfister 580defd967 Allow file:// based Uri's to be used in HttpClient when the BaseAddress is set
Fixes https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems/edit/967582

Bumps the HttpClient part in mono/corefx at: https://github.com/mono/corefx/commit/ee3acec16b37bec5016da129e1e3840097ca629b

The test that would have caught this failure was disabled previously via:
https://github.com/mono/mono/commit/2f411750acfd2132f9b4970ef8cf11d9a0f62df9
2019-08-27 13:54:52 -04:00
Alexander Köplinger 8d422436f5 [acceptance-tests] Bump roslyn test repo to remove a problematic nuget feed 2019-08-27 19:48:08 +02:00
Vlad Brezae ccc4bf8536 [interp] Remove unnecessary recomputation (#16509)
Seems to have been added by mistake in https://github.com/mono/mono/pull/16317, which was meant just to remove the locals variable from InterpFrame.
2019-08-27 20:27:22 +03:00
Jay Krell 00c1ec421a [interp] Error handling refactor (optimize non-exception path) (helps clang save stack) (#16371)
While working on stack reduction, I believe I found that
the compiler could not tell how THROW_EX and/or NULL_CHECK terminate.

That is, essentially, the success paths preserve volatile registers.
They do not have to be spilled/filled.
This tweak should make it clearer to the compiler, optimized common paths, and not slow down uncommon paths.

This also cleans it up a little and reduces macro expansion, and then possibly should allow for reinlining what is now used only once.

Also add some const so it can be casted away less.

Believed to save 32 bytes of stack with Linux/amd64/clang, nothing for gcc.
And a nice cleanup.
2019-08-27 15:31:10 +02:00
Zoltan Varga fa1288794b Merge pull request #16493 from EgorBo/impl-lzcnt
Implement System.Runtime.Intrinsics.X86.Lzcnt
2019-08-27 06:51:03 -04:00
Johan Lorensson 70aba13173 Fix multiple SGEN_LOG to correctly scale elapsed time in usec. (#16485)
Several SGEN_LOG entries logged usec based on result returend from TV_ELAPSED.
On SGEN's implementation of TV_ELAPSED, the result is not scaled to usec
but returned as 100ns ticks. This was not handled correctly by SGEN_LOG,
but other measures, like profile counters and SGEN binary protocol handles
it correctly.

Fix will adjust the SGEN_LOG calls currently presenting usec to correctly
scale the result. Fix also adds one additional logging measuring total
amount of time spend scanning thread data.
2019-08-27 12:14:12 +02:00
Jay Krell bfe1b6182c [interp] Replace imethod with frame->imethod. (#16459)
This conserves stack on Linux/amd64/gcc.
2019-08-27 11:51:21 +02:00
Jay Krell 6cf8b6e738 CodeManager cleanup: (#16468)
- Chunks do not need flags, only codemanagers do.
 - Sort by size to minimize padding for alignment.
 - Booleans only need one bit.
 - Chunk is not spelled chunck.
 - Suggest foo accessors be named foo() and set_foo(), not get_foo() and set_foo().
   i.e. the more common "get" is implied.
- Allow free(0) w/o initialization.
2019-08-27 10:40:35 +02:00
Jay Krell 9f9e5f7803 [monodis] Combine adjacent identical cases. (#16501) 2019-08-27 10:36:59 +02:00
Zoltan Varga c7d785f329 Add SETUP_ICALL_FUNCTION. 2019-08-27 03:57:59 -04:00
Jay Krell 7ca15e5a42 mono_attach_load_agent: Fix leaks, alloc, copy. (#15950)
Extracted and slightly modified from https://github.com/mono/mono/pull/15868.
2019-08-27 09:46:35 +02:00
Stephen Toub a0307b1e34 Fix comment typo in ManualResetValueTaskSourceCore
Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Vladimir Sadov 88ac3856eb Do not use AllocateUninitializedArray in private array pools. (#26338)
User may have extra expectations for the privatly owned array pools.
For example there could be an expectation that array never contains negative numbers (since user never puts them there). Uninitialized allocations can break such expectations.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Stephen Toub 82f618023f Disable SA1001 for LocaleData.Unix.cs
Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Stephen Toub 39d62eb8c8 Separate Async*MethodBuilder types into their own files (#26364)
* Separate async method builder types into their own files

* Minor cleanup of AsyncTaskMethodBuilder types

* Split AsyncValueTaskMethodBuilder types into their own files

Signed-off-by: dotnet-bot <[email protected]>
2019-08-27 01:24:57 +02:00
Nikolay Sivov 515ec75154 [git] Set csharp diff pattern for *.cs sources. (#16495) 2019-08-27 01:24:30 +02:00
Jay Krell 67c97d30b4 Add missing const, particularly on functions "like" memcpy. (#16477)
But do not change the signatures of external functions.
2019-08-27 01:04:46 +02:00
Zoltan Varga 4f8c0b9b46 Merge pull request #16451 from vargaz/gc-handle-pin
[runtime] Make the local gc handles pin the object they point to by scanning the handle area conservatively.
2019-08-26 17:31:50 -04:00
Aleksey Kliger (λgeek) 78edafd6e2 [threadpool-io] Throw if backend won't be able to register a FD (#16396)
The poll+select i/o selector backend can't handle file descriptor ids greater
than FD_SETSIZE.  This can happen if too many files are open and we want to
wait on it.

Previously, mono would fail in the i/o selector thread by which point it was
too late to do anything.

With this change we will fail eagerly on the thread that calls IOSelector.Add
by throwing a NotSupportedException.

Addresses https://github.com/mono/mono/issues/15931
2019-08-26 17:13:45 -04:00
Zoltan Varga 65a71a2912 Fix review comments. 2019-08-26 14:17:09 -04:00
Egor Bogatov 044d1519bd Implement Lzcnt hw intrinsics 2019-08-26 21:08:53 +03:00
Egor Bogatov 85265d884c [netcore] Implement missing ThreadPool methods (for metrics) (#16076)
* Implement missing ThreadPool methods

* Address feedback

* ifdef for mono_threadpool_worker_get_completed_threads_count

* remove GetPendingUnmanagedWorkItemCount
2019-08-26 11:35:38 -04:00
Zoltan Varga a908b72037 [runtime] Convert mono_reflection_create_custom_attr_data_args () to use handles. 2019-08-26 11:17:15 -04:00
Johan Lorensson f9be7a67d3 Fix special case tramp assert in x86/amd64. (#16476)
https://github.com/mono/mono/pull/16408 temporary fixed a tramp issue on
x86 (also exist on amd64). This commit fixes the underlying issue and re-enable
the use of noreturn outside of netcore.

The problem hit by the use of OP_NOT_REACHED on x86/amd64 is due to a specific
case when the address to patch happens to fall at the start of a different
managed method. Since the use of OP_NOT_REACHED can end a method
with the call instruction on x86/amd64 in combination with how patch location
(using the return address from stack) is resolved, the implementation in
mono_arch_patch_callsite didn't take this case into account, reading incorrect
patch data, triggering assert oh x86. So, based on timing we could end up with
the following code:

025715B3 call 02570D98
025715B8 push ebp

since the patch target is 025715B8, but also the start of a completely different
method, method_start == origin_code. The fix is to detect this case on x86/amd64
(done on each arch since the way origin_code is detected is arch specific)
and then use a method_start of NULL, that is already a supported scenario.
2019-08-26 16:46:43 +02:00
monojenkins 80504b8c98 [bcl] Update BCL Linked Size (#16427) 2019-08-26 14:01:25 +02:00
Jan Kotas 501b9cc3dd Fix BinaryReader.ReadChars for fragmented Streams (dotnet/coreclr#26324)
BinaryReader.ReadChars incorrectly read more than necessary from the underlying Stream when multi-byte characters straddled the read chunks.

Fixes https://github.com/dotnet/corefx/issues/40455

Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Next Turn 366f2750cd Remove unnecessary !s for [DoesNotReturn] (dotnet/coreclr#26350)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub 37c6d154ef Contribute to IDE0055 (fix formatting)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub 2f11eff09b Contribute to StyleCop SA1002 (semicolon spacing)
However, I'm not enabling the rule permanently, as it disagrees with VS IDE0055 about the spacing around semicolons at the end of empty while and for loops.

Because this also impacts `for (;;)` loops, I also searched and replaced those with `while (true)`, to standardize on that across the codebase.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Stephen Toub a86fb8c847 Fix StyleCop warning SA1001 (spaces around commas)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-26 13:56:53 +02:00
Alexander Köplinger 2f2771fcfa [utils] Don't use MAP_32BIT on Apple platforms, fixes crash with XCode 11 beta6 (#16441)
As of XCode 11 beta6 the MacOSX SDK defines the MAP_32BIT symbol:

```diff
--- /Applications/Xcode11-beta5.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/mman.h    2019-07-25 17:43:49.000000000 -0400
+++ /Applications/Xcode11-beta6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/mman.h    2019-08-06 21:03:07.000000000 -0400
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2000-2002 Apple Computer, Inc. All rights reserved.
+ * Copyright (c) 2000-2019 Apple Computer, Inc. All rights reserved.
  *
  * @APPLE_OSREFERENCE_LICENSE_HEADER_START@
  *
@@ -145,6 +145,10 @@
 #define MAP_RESILIENT_CODESIGN  0x2000 /* no code-signing failures */
 #define MAP_RESILIENT_MEDIA     0x4000 /* no backing-store failures */

+#if !defined(CONFIG_EMBEDDED)
+#define MAP_32BIT       0x8000          /* Return virtual addresses <4G only: Requires entitlement */
+#endif  /* !defined(CONFIG_EMBEDDED) */
+
 #endif  /* (!_POSIX_C_SOURCE || _DARWIN_C_SOURCE) */

 /*
```

This causes the mono_valloc() function to try to use the MAP_32BIT flag for mmap().

However as mentioned in the comment for the symbol in mman.h it seems to require a special entitlement
which isn't available/documented anywhere yet.
This in turn causes the mmap call to fail presumably because we're missing that entitlement.

Instead we now skip setting this flag on Apple platforms to make mmap() behave like it did before.
2019-08-26 13:53:40 +02:00
Egor Bogatov 6aaec2fe9b Optimized LLVM IR (#16448)
LLVM JIT: dump IR after optimizations 

```csharp
[MethodImpl(MethodImplOptions.NoInlining)]
static int Test(int a, int b, int c) => a * b + a * c;
```
`MONO_VERBOSE_METHOD="Test"` now prints both unoptimized and optimized IR:
```llvm
*** Unoptimized LLVM IR for P:Test ***

; Function Attrs: noinline uwtable
define monocc i32 @"P:Test (int,int,int)"(i32 %arg_a, i32 %arg_b, i32 %arg_c) #0 {
BB0:
  br label %BB3

BB3:                                              ; preds = %BB0
  br label %BB2

BB2:                                              ; preds = %BB3
  %t22 = mul i32 %arg_a, %arg_b
  %t25 = mul i32 %arg_a, %arg_c
  %t27 = add i32 %t22, %t25
  br label %BB1

BB1:                                              ; preds = %BB2
  ret i32 %t27
}
***


*** Optimized LLVM IR for P:Test ***

; Function Attrs: noinline uwtable
define monocc i32 @"P:Test (int,int,int)"(i32 %arg_a, i32 %arg_b, i32 %arg_c) #0 {
BB0:
  %t251 = add i32 %arg_b, %arg_c
  %t27 = mul i32 %t251, %arg_a
  ret i32 %t27
}
***
```
2019-08-26 07:36:04 -04:00
Jay Krell 6671316e35 [interp] Cleanup stackval_to_data and stackval_to_data_addr. (#16458)
No functional change, except for debugging, look up a frame
for the prior value.
2019-08-26 10:26:07 +02:00
Bernhard Urban b6ca76c484 [android] remove text relocations in AOT/arm32 (#16432)
[android] remove text relocations in AOT/arm32

Remove support for "big assemblies" for AOT on Android running on ARM 32bit. This will remove relocations in `.text` which are not allowed in the upcoming Android Q release anymore.

_If_ a customer should run into this problem, the suggested workarounds are:
* Switch to ARM64, or
* Do not use AOT compilation.

Fixes https://github.com/mono/mono/issues/16369
2019-08-26 03:46:12 -04:00
Ryan Lucia 510f3c7d70 Detect additional invalid assembly names when parsing (#16446) 2019-08-25 23:13:39 -04:00
Jay Krell e95ef822a1 Add MONO_LLVM_INTERNAL to mono_jit_compile_method. (#16463) 2019-08-25 23:52:17 +02:00
Jay Krell 2224d6b55d [interp] Outline mono_interp_box_vt. (#16454)
Contributes to https://github.com/mono/mono/issues/16172.

Seen to reduce frame from 0xB8 to 0xA8 on Linux/amd64/gcc against master.
Possibly similar elsewhere.
(Possibly 8 added to a rounded 8.)

Add const, add braces, they might help compiler and/or programmer.
2019-08-25 19:11:02 +02:00
Zoltan Varga 7d43ffda1a [runtime] Make the local gc handles pin the object they point to by scanning
the handle area conservatively.
2019-08-24 15:46:33 -04:00
Jay Krell 9d1eec6bf8 [interp] Outline STRMFLD_VT. (#16355)
Without being entirely scientific about it, this case seems
to be one that uses more than typical locals and therefore
be on a sort of critical path to reduce frame size.

I have a change that reduces frame to 0x78 bytes (albeit
not yet working) and to achieve that, in this case I refetched
the locals after function calls.

This achieves similar but perhaps more elegantly.

If we rest at larger frame size then this case might not be
on the critical path.

https://github.com/mono/mono/issues/16172
2019-08-24 09:56:40 +02:00
Jay Krell 99e9d80642 [windows gc] Trim one syscall per thread in garbage collection. (#16003)
* Skip one syscall per thread in garbage collection.
Specifically do not query for guard pages.
There are better ways:
 1. Visual C++ specific, __try / __except, get the guard page exception,
    put back the guard page. This is a little slower (exception + two syscalls),
    but exceedinly rare. __try itself is cheap on x86, free on the rest.
    This was also not allowed on UWP. It will be. And is VirtualProtectFromApp.

 2. Record the TEB and get from it the maximum commited page and just use MAX, trivial.

The comments are also not completely clear as to what is going on here, but they were close.
Augment them.
2019-08-24 09:56:00 +02:00
Jay Krell ab6a09741d Remove dlmalloc from Win32. (#15773) 2019-08-24 09:52:30 +02:00
Alexander Köplinger 3a409e8a7f Pass --clr-memory-model in csc/vbc wrapper script (#16439)
Should help with random CSC crashes on ARM (we've been using it in the Mono build for a while).
2019-08-24 00:08:25 +02:00
Zoltan Varga ab9c58d76d Merge pull request #16414 from vargaz/llvm-gc-poll-wrapper
[llvm] Add a mono wrapper method for the gc poll cold cconv wrapper so the EH code can unwind through it.
2019-08-23 18:06:17 -04:00
Ryan Lucia 83cf007ae3 Return an empty string for assemblies loaded from stream (#16345)
* Return an empty string for assemblies loaded from memory

* Blind attempt to fix wasm
2019-08-23 15:58:10 -04:00
Ryan Lucia 7615ba3e45 [netcore] Use requesting assembly's context for InternalLoad calls 2019-08-23 15:30:04 -04:00
Ryan Lucia 0661bf0424 [netcore] Random loader cleanup 2019-08-23 15:30:04 -04:00
Nathan Ricci f6b2befe8d Removed trailing whitespace. 2019-08-23 13:57:01 -04:00
Nathan Ricci 707db63569 Added stub for boehm. 2019-08-23 13:45:23 -04:00
Nathan Ricci cdb77f1391 Fixed fomatting issues. 2019-08-23 13:42:34 -04:00
Tarek Mahmoud Sayed cb1f7c0677 Cleanup GregorianCalendarHelper leap month checks (dotnet/coreclr#26321)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-23 15:43:54 +02:00
Alexander Köplinger 278fd48274 Bump corefx to get https://github.com/mono/corefx/pull/336 2019-08-23 14:43:05 +02:00
Zoltan Varga f0cbb79b7f Merge pull request #16431 from kjpou1/wasm-maindir-cleanup
[wasm] Small cleanup of main wasm directory
2019-08-23 08:13:21 -04:00
Zoltan Varga 97d2d97615 [jit] Make mono_unwind_frame () ignore registers which are greater than NUM_DWARF_REGS.
These registers can be encountered in the unwind info for functions with a cold
cconv, like xmm0 on amd64. Since they are caller saved, they don't have to
be restored during EH.
2019-08-23 07:49:20 -04:00
Zoltan Varga fdd0d7f027 Merge pull request #16429 from vargaz/llvm-jit-module
[llvm] Don't pass a module argument to the code in llvm-jit.cpp, we create a new LLVM module for each method.
2019-08-23 06:40:09 -04:00
Zoltan Varga 81f736e774 Fix the bitcode build. 2019-08-23 06:28:40 -04:00
Zoltan Varga 4ed1c3335f [llvm] Add a cold wrapper and mono wrapper for gc poll for JITted code as well. 2019-08-23 05:34:52 -04:00
Kenneth Pouncey f5717d21bb Address review comments wasm/support to wasm/src 2019-08-23 11:11:57 +02:00
Zoltan Varga 0d603261a0 Fix the build. 2019-08-23 04:41:50 -04:00
Zoltan Varga f6b189ecab [llvm] Add a mono wrapper method for the gc poll cold cconv wrapper so the EH code can unwind through it. 2019-08-23 04:41:50 -04:00
Zoltan Varga 8ace375450 Merge pull request #16407 from vargaz/fix-llvm-simd
[llvm] Fix type issues in the llvm IR emitted for SIMD. Fix a warning.
2019-08-23 04:41:10 -04:00
Zoltan Varga 14a76b03c5 Merge pull request #16408 from vargaz/noreturn-methods
[jit] Special case the ThrowHelper methods only on netcore.
2019-08-23 04:40:28 -04:00
Egor Bogatov 47efe6bab0 Merge pull request #16412 from EgorBo/fix-ci4
[netcore] Fix CI, ignore failing tests
2019-08-23 11:17:31 +03:00
Kenneth Pouncey 8b338a9619 [wasm] Small cleanup of main wasm directory
- Move source support files for custom building the wasm runtimes to a new directory called `wasm/support`
- Modified `packager.exe` to build from this new subdir.
- Updated the README.md to document the change.
2019-08-23 09:59:31 +02:00
Zoltan Varga c357de87ec [llvm] Don't pass a module argument to the code in llvm-jit.cpp, we create a new LLVM module for each method. 2019-08-23 03:27:41 -04:00
Zoltan Varga c97bfe2167 Merge pull request #16379 from EgorBo/improve-dump
Improve MONO_VERBOSE_METHOD
2019-08-23 02:08:12 -04:00
Michal Strehovský 9bffb810ee Mark methods specially handled in JitInterface Intrinsic (#26209)
* Mark methods specially handled in JitInterface Intrinsic

For CPAOT compiler's sake.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-23 00:51:24 +02:00
coypoop f626f3e7aa If either sched_get_priority_{min,max} failed, don't try to set a priority (#15898)
netbsd and linux both require that if SCHED_OTHER is used, a fixed value is used for sched_param ([0 for linux](http://man7.org/linux/man-pages/man7/sched.7.html), -1 for netbsd)

the netbsd case can also be detected by the [POSIX-defined failure](https://pubs.opengroup.org/onlinepubs/007908799/xsh/sched_get_priority_min.html) return value for sched_get_priority_min, a return value of -1.
2019-08-23 00:46:08 +02:00
EgorBo 71dce6502a ignore FSW test 2019-08-23 01:12:47 +03:00
Martin Baulig 473d07a86c Fix issue in SafeHandle.DangerousReleaseInternal() that caused #16034. (#16244)
Using the CoreCLR version of this line fixed the problem and my investigations
also revealed why.

The `SafePipeHandle` contains a reference to both the Socket as well as the
`SafeSocketHandle`, taking ownership of them.  If there are no additional
references to either of them, so that both are collected in the same GC pass,
their finalizer may be invoked in any order.

If the Socket gets finalized first, it's `SafeSocketHandle` will have a `_mstate`
of 8 (two references) at the beginning of `DangerousReleaseInternal()` - which
will set it to 6 (one reference plus disposed flag).

Then when the `SafePipeHandle` gets collected, it will call `DangerousRelease ()`
on it's `SafeSocketHandle`.  As mentioned above, it's `_mstate` will be 6.
Now we own the handle, so we'll decrease it's reference count by one and set the
`Closed` flag - the new `_mstate` should thus be 3.

However, using

    new_state = (old_state & RefCount_Mask) - RefCount_One;

would set it to 1 - thus effectively clearing the `Disposed' flag.  And since it's
reference count is now also zero, the next finalize pass will then throw.

The CoreCLR version of this class uses

    new_state = old_state - RefCount_One;

which fixes this.
2019-08-22 23:46:31 +02:00
Nathan Ricci 93ff9dab7b Removed start/stop the world gc api. 2019-08-22 16:50:58 -04:00
EgorBo 692f70cb67 ignore more tests 2019-08-22 23:08:44 +03:00
EgorBo 40cf68aa98 Disable the whole System.ComponentModel.Composition namespace 2019-08-22 21:47:45 +03:00
Stephen Toub f5a1fa4744 Fix IDE0034 (simplify default expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f256a4525d Fix IDE0054 (use compound assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub e2c7b4200a Fix (mostly) IDE0055 (formatting)
Lots of various spacing issues.  I say "mostly" because I explicitly undid some changes that added to rather than reduced inconsistency, e.g. region indents.  We can revisit that later.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 9be0c2217c Fix StyleCop warning SA1114 (parameter list should follow declaration)
Fixed just be removing stale/unnecessary comments.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub d3817c9302 Fix StyleCop warning SA1006 (empty statements)
This is just running the rule and auto-fixing everything.  I undid changes that converted e.g. `while (...);` to `while (...) { }`.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 35d8ca894b Fix StyleCop warning SA1005 (single line comment spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 55bc2fee8a Fix (mostly) StyleCop warning SA1003 (operator symbol spacing)
The "mostly" here is that I undid changes where it was forcing formatting around a `: this` or `: base` call for a ctor, in particular where it was moving one up from the next line to be at the end of the current line.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub e23c35f730 Fix StyleCop warning SA1024 (colon spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 26656df25e Fix StyleCop warning SA1015 (closing generic brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f3b0e98aae Fix StyleCop warning SA1013 (closing brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub ac42e06fea Fix StyleCop warning SA1012 (opening brace spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 8c01fd5189 Fix StyleCop warning SA1008 (opening paren spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub f83a8da0ce Fix StyleCop warning SA1002 (semicolon spacing)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Stephen Toub 46302d54aa Fix StyleCop warning SA1001 (commas followed by whitespace)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 20:21:18 +02:00
Nathan Ricci bfce92d9dd Added tracking for detached threads. 2019-08-22 14:14:39 -04:00
EgorBo 7678654781 fix macOS 2019-08-22 20:53:14 +03:00
Steve Pfister fbdad78101 Added a workaround for an issue with the compiler server on 32-bit linux. (#16405)
Currently, if the process is 32-bit, roslyn tries to determine how much
memory is available.  To do that, it tries to pinvoke into GlobalMemoryStatusEx,
which is not supported on mono.

Without it, the compiler server bombs on https://github.com/dotnet/roslyn/blob/0e63260c5afb3fb5b74c357dd250e500172bcd63/src/Compilers/Server/VBCSCompiler/NamedPipeClientConnection.cs#L55-L59
2019-08-22 18:22:22 +02:00
EgorBo 779fda5103 update rsp 2019-08-22 19:02:53 +03:00
EgorBo f809fe55a8 ignore some tests 2019-08-22 18:52:52 +03:00
Stephen Toub 4dd0a137a4 Avoid race condition in DiagnosticCounter (dotnet/coreclr#26260)
* Avoid race condition in DiagnosticCounter

If while a DiagnosticCounter instance is being created the timer fires to process counters, the timer may end up evaluating a DiagnosticCounter whose constructor or derived types' constructor hasn't finished configuring the instance's state yet.

* Remove unnecessary #if

Signed-off-by: dotnet-bot <[email protected]>
2019-08-22 16:59:43 +02:00
Nikolay Sivov 068660732f [corlib] Improve file:// url handling in AppDomainSetup. (#16161)
* [corlib] Improve file:// url handling in AppDomainSetup.

WiX 3 installer toolkit uses file:/// prefix when setting ApplicationBase for newly created domain.
Currently this leads to invalid path exception:

file:///C:\... -> /C:\... -> full path expansion with current workdir on different drive -> Y:\C:\...

Later it fails check for multiple ':' -> NotSupportedException ("The given path's format is not supported.").

This patch uses logic already present in AppDomain::Load().

* [Test] Add a test for file:/// application base.

* [Test] Fix test failure on Windows.

* [corlib] Simplify file: protocol check for AppDomainSetup.
2019-08-22 16:54:32 +02:00
Zoltan Varga 49f913ac8a [jit] Special case the ThrowHelper methods only on netcore. 2019-08-22 10:36:43 -04:00
Zoltan Varga f2f58e522b [llvm] Fix type issues in the llvm IR emitted for SIMD. Fix a warning. 2019-08-22 10:28:47 -04:00
Aleksey Kliger (λgeek) e74736a471 [android] mono_dl_open_file: use g_file_test only on absolute paths (#16387)
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is
the name of a shared library we want to open is to speed up probing for
non-existent libraries.

See https://github.com/mono/mono/pull/12074

The problem is that if file is just a simple "libdl.so" then `dlopen (file)`
doesn't just look for it in the current working directory, it will probe some
other paths too.  (For example on desktop linux you'd also look in all the
directories in LD_LIBRARY_PATH).  So the g_file_test() call is not a robust way
to avoid calling dlopen if the filename is relative.

But it actually broke more things: https://github.com/xamarin/xamarin-android/issues/3388

When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try
prepending some paths that it knows about and we end up calling
`dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security
restrictions on what code can dlopen something from /system/lib with an
absolute path.  Eventually mono_lookup_pinvoke_call will go back to trying the
bare "libdl.so" which hits `g_file_test` and returns NULL.

The new code only does the file test if we pass it an absolute path, which
gives Bionic's dlopen a chance to deal with relative paths however it needs to.
2019-08-22 10:10:09 -04:00
Zoltan Varga f6b0c9b772 Merge pull request #16300 from EgorBo/llvm-fpm
Use basic LLVM optimizations (PassManager) in JIT mode
2019-08-22 09:07:02 -04:00
Jay Krell 5368fef5ee [interp] Replace endsp with preexisting child_frame.retval which appears equivalent. (#16378)
This should conserve stack. I didn't verify it.
child_frame being address-taken, compiler would struggle to remove it.
So we have the same value in two places.

https://github.com/mono/mono/issues/16172
2019-08-22 14:57:38 +02:00
Zoltan Varga 87a6364d58 Merge pull request #16390 from EgorBo/intptr-intrinsic
Make IntPtr.Zero intrinsic
2019-08-22 08:43:07 -04:00
Zoltan Varga 341cc0dd83 Merge pull request #16386 from vargaz/llvm-inline-limit
[llvm] Use a different inline limit (30) for llvm+aot.
2019-08-22 08:26:10 -04:00
Zoltan Varga 3f21a2c8f3 Merge pull request #16384 from vargaz/fix-16381
[llvm] Fix an assert.
2019-08-22 07:26:36 -04:00
Zoltan Varga 254d6d3758 Merge pull request #16389 from vargaz/fix-16380
[aot] Simplify a check to avoid calling into the assembly loading code while loading the aot image for corlib.
2019-08-22 05:36:17 -04:00
Zoltan Varga b7ca80ed58 Add comments. 2019-08-22 00:34:32 -04:00
Egor Bogatov 279676f43e Update llvm-jit.cpp 2019-08-22 01:46:10 +03:00
Egor Bogatov d02695ef47 cleanup 2019-08-22 01:43:56 +03:00
Egor Bogatov 1822f2faab Merge remote-tracking branch 'origin/master' into llvm-fpm 2019-08-22 00:47:40 +03:00
Nathan Ricci 6caa530970 Removed unneeded declarations. 2019-08-21 17:31:34 -04:00
Bernhard Urban 5c9be2dc2b [netcore] Use Release configuration for default System.Private.CoreLib build (#16391)
It seems the Configuration set inside of the System.Private.CoreLib.csproj is just completely ignored, making it default to "Debug".

Maybe because we're using Microsoft.NET.Sdk ?
2019-08-21 23:16:03 +02:00
Nathan Ricci b65443f324 Added stubs in null, boehm. 2019-08-21 16:55:51 -04:00
Nathan Ricci f913945e76 Implemented precise. 2019-08-21 16:27:02 -04:00
Nathan Ricci dc8aeba593 stopped the world for precise. 2019-08-21 16:05:29 -04:00
Nathan Ricci 56f44379d0 Took changes from miguel's branch. 2019-08-21 15:54:07 -04:00
Nathan Ricci 65360da5d9 Working on precise. 2019-08-21 15:51:45 -04:00
Egor Bogatov cb34178cd6 use EMIT_NEW_PCONST 2019-08-21 21:48:14 +03:00
Egor Bogatov d3ecd70ead Make IntPtr.Zero intrinsic 2019-08-21 21:23:19 +03:00
Zoltan Varga c4de19ef09 [aot] Simplify a check to avoid calling into the assembly loading code while loading the aot image for corlib.
Fixes https://github.com/mono/mono/issues/16380.
2019-08-21 14:15:25 -04:00
Nathan Ricci 0754f5ebe5 Fixed alphabitization error. 2019-08-21 13:59:50 -04:00
Nathan Ricci cfb4afff43 Got rid of some debug code. 2019-08-21 13:55:20 -04:00
Zoltan Varga 433b9a7f70 [llvm] Use a different inline limit (30) for llvm+aot. 2019-08-21 12:20:50 -04:00
Zoltan Varga 0e2da3c2ab [llvm] Fix an assert.
Fixes https://github.com/mono/mono/issues/16381.
2019-08-21 12:03:20 -04:00
Nathan Ricci 53939d0790 Implemeneted imprecise. 2019-08-21 11:20:27 -04:00
EgorBo 26d73b94f6 cleanup 2019-08-21 14:34:00 +03:00
EgorBo 467a561dcb Merge remote-tracking branch 'origin/master' into ffast-math
# Conflicts:
#	mono/mini/llvm-jit.cpp
#	scripts/ci/pipeline-netcore-runtime.yml
2019-08-21 14:30:07 +03:00
EgorBo ad713a70ef Merge remote-tracking branch 'origin/master' into llvm-fpm 2019-08-21 14:20:57 +03:00
Egor Bogatov 50b57fe926 Update mini.c 2019-08-21 14:07:59 +03:00
Egor Bogatov 6bb1e5ffc4 Update mini-llvm.c 2019-08-21 14:07:02 +03:00
Egor Bogatov e5acc9be7a undo change in mini.c 2019-08-21 14:04:57 +03:00
Egor Bogatov b0b3743f36 Improve MONO_VERBOSE_METHOD 2019-08-21 13:35:40 +03:00
Kenneth Pouncey 935d0a54e7 [wasm] Make sure message pump is working when run via webworker. (#16325)
- Tests if we are running in ENVIRONMENT_IS_WORKER
   - run message pump if we are.

- This also fixes gc not running.

issue: https://github.com/mono/mono/issues/16319
2019-08-21 06:24:06 +02:00
Zoltan Varga 977ccf7cb2 Merge pull request #16363 from vargaz/tp-thread-name
[runtime] Avoid resetting the tp thread name on every call to worker_callback ().
2019-08-20 23:14:44 -04:00
Zoltan Varga 285f33efe6 Merge pull request #16286 from vargaz/netcore-simd
[netcore] Add a netcore version of simd-intrinsics.c.
2019-08-20 23:11:27 -04:00
Nathan Ricci 37086aa1c4 Put things in correct order. 2019-08-20 18:28:35 -04:00
Nathan Ricci bf04a1946f Modified sample project. 2019-08-20 17:38:23 -04:00
Nathan Ricci fb7895dddf Added ical to netcore. 2019-08-20 17:37:58 -04:00
Alexander Köplinger 38c524cc2a Fix Windows build 2019-08-20 22:55:39 +02:00
Alexander Köplinger 55f2c4d6ad [profiler] Rename helper functions to have common prefix
Unfortunately https://github.com/mono/mono/pull/16335 didn't fix the new symbols showing up in the log profiler module.
The reason is that we actually pass `--disable-visibility-hidden` for iOS builds in https://github.com/mono/mono/blob/0b6d95e8ce7c882e2db9e89fa0dec850c687f65e/sdks/builds/ios.mk#L98.

Rename the helper functions instead to have a common prefix so they can be ignored in xamarin-macios tests.
2019-08-20 22:55:39 +02:00
Aleksey Kliger ed0063b4d5 [w32socket] Translate some errno codes to WSA_ errors imprecisely.
These aren't a perfect match, but better than falling into the `default` case
where Mono asserts and crashes.

Addresses part of https://github.com/mono/mono/issues/16024
2019-08-20 16:13:41 -04:00
Aleksey Kliger c38044d74f [w32error] Add WSA_INVALID_PARAMETER and WSA_INVALID_HANDLE 2019-08-20 16:13:41 -04:00
Ryan Lucia fc5e3fa039 [netcore] Remove local copy of static alc resolve methods 2019-08-20 21:25:00 +02:00
Ryan Lucia 3b47d28e37 Move AssemblyLoadContext VM-invoked static resolve methods to shared (#26248)
Shared between CoreCLR and Mono. Mono began using a copy of these methods with https://github.com/mono/mono/pull/16256/

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 21:25:00 +02:00
Nathan Ricci c1d01679c5 Merge branch 'master' of github.com:naricc/mono into naricc/GetTotalAllocatedBytes 2019-08-20 15:22:18 -04:00
Alexander Köplinger a147402d07 [ci] FreeBSD: Make sure we have gmake in /usr/bin 2019-08-20 21:08:54 +02:00
Alexander Köplinger ac3aa7c1c7 Bump corefx to get https://github.com/mono/corefx/pull/332 2019-08-20 20:52:02 +02:00
Stephen Toub 5f38497e19 Remove unused usings from Corlib (dotnet/coreclr#26250)
* Fix IDE0065 (misplaced using)

* Fix IDE0005 (remove unnecessary usings)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 19:45:30 +02:00
madmir 4dd016004b Re-enable xunit warning 1019 (dotnet/corefx#40415)
Make MemberData reference IEnumerable<object[]>

Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 19:45:30 +02:00
Ryan Lucia 0b6d95e8ce [netcore] Remove unnecessary calls to mono_domain_get (#16336)
Should use mono_alc_domain where possible
2019-08-20 12:55:54 -04:00
Alexander Köplinger 5dc4a86d9d [netcore] Consolidate running tests between AzDO pipeline and Makefile (#16348)
* [netcore] Consolidate running tests between AzDO pipeline and Makefile

There was some unnecessary duplication between the two that we can remove.

* Add timeout to test execution
2019-08-20 17:59:55 +02:00
Zoltan Varga 8a413fad71 Fix the build. 2019-08-20 11:32:46 -04:00
Zoltan Varga b5cfeb3b5b [runtime] Avoid resetting the tp thread name on every call to worker_callback (). 2019-08-20 11:31:12 -04:00
Phil Jaenke b7dc4f0a46 Fix Jenkins build failure due to missing dependencies (#16362)
Jenkins builds are failing due to missing gettext; also realized that graphics unit tests will fail without some additional packages that will not be pulled in automatically.

Added:
`gettext-runtime gettext-tools cairo libdrm mesa-dri mesa-libs`
2019-08-20 16:19:36 +02:00
Jo Shields 3070ab6a96 Pass -fvisibility=hidden to profiler modules. (#16335)
* Pass -fvisibility=hidden to profiler modules.

This appears to not break the profilers:

```
directhex@breakfast:/tmp$ nm -g /tmp/ponyponyponypony/lib/libmono-profiler-log.so | grep close_socket_fd
directhex@breakfast:/tmp$
directhex@breakfast:/tmp$ /tmp/ponyponyponypony/bin/mono --profile=log:report bottle.exe
99 bottles of beer on the wall, 99 bottles of beer.
Take one down and pass it around, 99 bottles of beer on the wall.

[..]

No more bottles of beer on the wall, no more bottles of beer.
Go to the store and buy some more, 99 bottles of beer on the wall.

Mono log profiler data
	Profiler version: 3.0
	Data version: 17
	Arguments: log:report
	Architecture: x86-64
	Operating system: linux
	Mean timer overhead: 86 nanoseconds
	Program startup: Mon Aug 19 14:18:30 2019
	Program ID: 17715
	Server listening on: 41539

JIT summary
	Compiled methods: 753
	Generated code size: 228046
	JIT helpers: 0
	JIT helpers code size: 0

GC summary
	GC resizes: 0
	Max heap size: 0
	Object moves: 0

Metadata summary
	Loaded images: 3
	Loaded assemblies: 3

Exception summary
	Throws: 0
	Executed finally clauses: 3

Thread summary
	Thread: 0x7fcd968c9700, name: "Finalizer"
	Thread: 0x7fcd966c8700, name: "Profiler Sampler"
	Thread: 0x7fcd9b036780, name: "Main"
	Thread: 0x7fcd964c7700, name: "Profiler Helper"
	Thread: 0x7fcd954c5700, name: "Profiler Dumper"
	Thread: 0x7fcd95cc6700, name: "Profiler Writer"

Domain summary
	Domain: (nil), friendly name: "bottle.exe"

Context summary
	Context: (nil), domain: (nil)

Counters:
	Mono System:
		User Time                      : 70.000ms
		System Time                    : 0.000ms
		Total Time                     : 140.000ms
		Working Set                    : 22032384
		Private Bytes                  : 63455232
		Virtual Bytes                  : 449630208
		Page Faults                    : 3004
		CPU Load Average - 1min        : 1.870000
		CPU Load Average - 5min        : 2.670000
		CPU Load Average - 15min       : 2.900000
	Mono JIT:
		Methods from AOT               : 536
		Methods JITted using mono JIT  : 154
		Methods JITted using LLVM      : 0

```

* Add missing annotations and includes
2019-08-20 16:17:26 +02:00
Calvin Buckley b57d52ce77 Fix minizip compile on Haiku (#16301)
it doesn't have f*64 funcs, it seems
2019-08-20 16:00:24 +02:00
Jay Krell 169205813b [interp] Outline box_nullable. (#16356)
Without being entirely scientific about it, this case seems
to be one that uses more than typical locals and therefore
be on a sort of critical path to reduce frame size.

I have a change that reduces frame to 0x78 bytes (albeit
not yet working) and to achieve that, in this case I refetched
the locals after function calls.

This achieves similar but perhaps more elegantly.
If we rest at larger frame size then this case might not be on the critical path.

https://github.com/mono/mono/issues/16172.
2019-08-20 15:53:25 +02:00
Jan Kotas 1592d2a8fa Change missed by the mirror (dotnet/coreclr#26251)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-20 15:37:10 +02:00
Phil Jaenke 317e741c02 Many updates to freebsd.sh (#16342)
* Many updates to make it better

Too much to list by many measures. This makes sure appropriate kernel modules, file systems, build packages, supplemental packages, and directories are installed or created as appropriate. This also addresses a known failure condition in System.Security-xunit due to a missing directory resulting in ownership problems.

* Make corrections for Azure

I don't have an Azure to test with, so I missed that the script is calling with `-e` and that Azure uses a non-`GENERIC` kernel with procfs baked in (likely required for Azure drivers.)
2019-08-20 15:35:18 +02:00
Egor Bogatov c231a357a7 Merge branch 'master' into ffast-math 2019-08-20 14:19:06 +03:00
Jay Krell ea29349f77 [interp] Reduce register/stack pressure esp. around vtable initialization. (#16349)
Part of #16172 though not necessarily any gain from this change, at this time.
2019-08-20 13:03:12 +02:00
Alexander Köplinger b205f1bd9c [interp] Reduce register/stack pressure slightly, i.e. so that field does not have (#16339)
to be preserved across `mono_get_special_static_data ()` if non-volatile registers
are all otherwise allocated.
2019-08-20 12:04:30 +02:00
Jay Krell 33153ddd2d [interp] Relieve register/stack pressure around mono_thread_internal_current. (#16341)
Part of #16172 though not necessarily any gain from this change, at this time.
We might want to expose ifdef and inline mono_thread_internal_current also or instead.
2019-08-20 12:03:48 +02:00
Alexander Köplinger 5ebce09c51 Bump bockbuild to get gtksharp 2019-08-20 11:54:59 +02:00
Jay Krell bc2accfc7e Fix warnings. (#16287)
debug-mono-ppdb.c ppdb_compressed_size may be used uninitialized
   ppdb_size may be used uninitialized
 image.c iinfo set but not used
 icall.c klass set but not used
2019-08-20 11:47:38 +02:00
Egor Bogatov 8800f7c166 Update pipeline-netcore-runtime.yml 2019-08-20 12:14:23 +03:00
Egor Bogatov 4a5e2a3f79 Update pipeline-netcore-runtime.yml 2019-08-20 12:13:59 +03:00
Alexander Köplinger 9a5ae23f7e [netcore] Use Release configuration for CI builds (#16329)
* [netcore] Use Release configuration for CI builds

Fixes https://github.com/mono/mono/issues/16308

* [ci] Ensure the artifacts/logs/Release folder exists

Some arcade steps need the path to exist and we only create artifacts/logs/Debug right now,
because that folder is created by eng/common/tools.sh used by init-tools.sh defaulting to Debug.
2019-08-19 23:52:01 +02:00
Jay Krell 31aed632ba [interp] Inline MINT_NEWOBJ_VTST_FAST (#16314)
For purposes of stack savings, only MINT_NEWOBJ_VT_FAST w/o the "ST"
needed to be outlined, and inlining also the ST form increased
its stack use, w/o savings elsewhere.

That is, the recursive cases should generally not be outlined.
We were outlining two. We still outline one.

This is partially undoing my recent change, where I outlined both of these, but only one was justified.
2019-08-19 23:48:20 +02:00
Egor Bogatov feac6c0d04 temp: run tests on CI 2019-08-19 23:30:41 +03:00
Jay Krell 2313f1a884 [interp] Reduce register/stack pressure slightly, i.e. so that field does not have
to be preserved across `mono_get_special_static_data ()` if non-volatile registers
are all otherwise allocated.
2019-08-19 13:02:49 -07:00
Ryan Lucia 7c3dfbcbdf [netcore] Update ALC test exclusions (#16332) 2019-08-19 14:50:15 -04:00
Aleksey Kliger (λgeek) 2f0cbb5aa5 [coop] Add a handle frame in construct_culture (#16299)
Otherwise we see warnings like these:

```
ves_icall_System_Globalization_CultureInfo_internal_get_cultures_raw USED 680 handles
```
2019-08-19 11:01:09 -04:00
Alexander Köplinger 0ad1f5cca1 [ci] Add CI VM provisioning script for FreeBSD to the repo (#16326)
We'll download it from GitHub instead of storing it in Jenkins admin config.

Trial run with FreeBSD for moving all the other provisioning scripts to the repo too.
2019-08-19 14:44:00 +02:00
Martin Baulig a3c31b9ef7 Add missing Dispose() call on error in MonoTlsStream.CreateStream(). (#16233)
* Add missing `Dispose()` call on error in `MonoTlsStream.CreateStream()`.

* Remove the `finally` block to make it more easy to read.
2019-08-19 14:23:28 +02:00
Vincent Povirk 9f790cd5e2 eglib: Remove broken executable test on Windows. (#16295)
The strcmp is wrong, and it doesn't test anything meaningful anyway.

This flag is unused since #16156 was merged.
2019-08-19 13:09:14 +02:00
Jan Kotas a723047f68 Fix CoreRT build breaks
Signed-off-by: dotnet-bot <[email protected]>
2019-08-19 13:07:13 +02:00
Stephen Toub ad63045907 Remove useless GetLongPathNameW call (dotnet/coreclr#26210)
Immediately after calling this we're going to loop around and call it again at the beginning of the while loop.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-19 13:07:13 +02:00
Egor Bogatov 77a9c82523 Introduce ffast-math 2019-08-19 13:07:43 +03:00
Jay Krell 8886f71b2c [interp] Remove InterpFrame.locals, replace with reasonably efficient frame_locals macro.o (#16317)
Contributes to #16172.
2019-08-19 11:50:20 +02:00
Alexander Köplinger e754bb1c43 [interp] Put finally data back in locals instead of frame to save 16 bytes of stack. (#16307)
* Put finally data back in locals instead of frame to save 16 bytes of stack.

Partial undo of f8fd60b5c0.

* Add comments explaining goto main_loop vs. MINT_IN_DISPATCH.
2019-08-19 11:24:44 +02:00
Jay Krell 60cd1b3556 [interp] Outline ves_array_element_address to save 16 bytes of stack on Linux/amd64/clang. (#16304)
Contributes to #16172.
2019-08-19 11:24:10 +02:00
Alexander Köplinger 9001c4fa83 thread names: coop, constants, usually ignore-error. (#16272)
- Convert setting thread name to be coop-compatible.

- For constant thread names, just retain the constant, not a copy.
  This includes producing and using constant unicode thread names on Windows.

- Make setting a thread name often non-fatal, except where mandated by public API.

- From earlier PR: mono_free (mono-publib.c) would no longer be referenced and therefore
  no longer exported. That broke profilers and maybe other externals.
  Choices:
     1 Use it instead of g_free randomly sometimes.
     2 Call it randomly sometimes.
     3 include it in external-only.c or object.c
     4 Mark it external only (breaks profiler).
     5 Use a .def file or Unix equivalent.

     5 is best, 2 is done here, the advantages/disadvantages
     among most choices are subtle. 3 is probably better than 2
     but is slightly bigger change, to put off -- you'd stop
     compiling mono-publib.c and move it to include/extra_redist.

     The advantage of not-.def file is perhaps that it is exposed
     by the compiler, so maybe easy to port and work with.
     People are more comfortable with obscure C extensions than
     any linker options, and there are multiple linkers to contend
     with. Arguably there are fewer compilers.
2019-08-19 11:23:23 +02:00
Jay Krell cf0ce4f55a Reduce size of internal MonoErrors by 3 pointers, while trying to be compatible with old size.(#16218)
There is an incompatibility here.

If Mono hands off an InternalError and it is accessed
like an ExternalError, like memset(sizeof()), bad.

Hopefully nobody does that.

There is also compatability -- embedders can continue to embed MonoError
in their own data. Its size and hopefully alignment is unchanged.
2019-08-19 11:20:37 +02:00
Jay Krell 047c7d35fa [interp] Replace frame.args with frame.stack, the same except for type (#16316)
Contributes to #16172.
2019-08-19 11:19:10 +02:00
Vlad Brezae 975b543e92 [interp] Make sure we allocate stack for MINT_LD_DELEGATE_INVOKE_IMPL (#16224)
* [interp] Make sure we allocate stack for MINT_LD_DELEGATE_INVOKE_IMPL

This opcode is pushing a function pointer from the delegate, so it needs to have stack reserved.

* [interp] Fix overflow of stack space
2019-08-19 11:59:21 +03:00
Jay Krell cecea63691 warning: mono-threads-wasm.c no previous prototype for function mono_background_exec (#16289) 2019-08-19 05:22:06 +02:00
Stephen Toub 04909573e7 Undo use of Index/Range in files compiled into netfx-compat assemblies
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 22:12:06 +02:00
Stephen Toub 424af0fb98 Fix DecoderExceptionFallbackBuffer message (dotnet/coreclr#26211)
When the number of unknown bytes is exactly 20, the message includes a misleading " ..." at the end.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 22:12:06 +02:00
Jay Krell d1f47e7b48 [interp] Reduce register/stack pressure slightly. (#16303)
No savings on Linux/amd64/gcc/clang.
2019-08-18 17:36:33 +03:00
Zoltan Varga e1b57c371a Fix the build. 2019-08-18 08:48:15 -04:00
Zoltan Varga 4503b67a0b Implement a subset of Intel BMI1/BMI2 intrinsics used by corelib. 2019-08-18 06:08:31 -04:00
Zoltan Varga 7337a3e405 Add support for Bmi1 instrinsics. 2019-08-18 05:25:04 -04:00
Zoltan Varga 157bb9f4e3 Make simd-methods-netcore.h less verbose. 2019-08-18 05:23:15 -04:00
Zoltan Varga 121d9ca4a9 [jit] Don't mark ThrowForUnsupportedVectorBaseType<T> as do-not-return. Assert that do-not-return methods actually don't return. 2019-08-18 03:18:24 -04:00
Zoltan Varga 7e76abb4ca [llvm] Add support for 32 byte AVX types, not enabled yet. 2019-08-18 02:01:10 -04:00
Zoltan Varga 1a9f9fb927 [llvm] Compute cpu features lazily. 2019-08-18 01:57:14 -04:00
Zoltan Varga 3c6e8de0da [jit] Enable alias analysis dce even if there are no ldaddr instructions eliminated, they could have been eliminated earlier. Enable only for netcore for now. 2019-08-18 01:55:25 -04:00
Zoltan Varga c0e03ef28f [llvm] Fix support for 32 byte simd values. 2019-08-18 01:40:28 -04:00
Stephen Toub c48dd0608d Fix IDE0025 (use expression body for properties)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 2e9134f1a6 Fix IDE0056 (use index operator)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 08b942ab68 Fix IDE0057 (use range operator)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 9595189a0a Fix IDE0054 (use compound assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub c8eb6972b5 Fix IDE0060 (unused parameter)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 64f4ba27f0 Fix IDE0059 (unnecessary assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 71fcdf53e3 Fix IDE0029 (use coalesce expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 828f89360f Fix IDE0031 (use null propagation)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 8af19427ad Fix IDE0028 (collection initializers)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 20adb69c73 Fix IDE0008 (use explicit type instead of var)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub 8dd17900cc Fix IDE0065 (using directives should be outside of a namespace)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Stephen Toub c0191b2575 Fix IDE0019/20 (Use pattern matching)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-18 07:01:54 +02:00
Kenneth Pouncey 02126de318 Add regression test for issue https://github.com/mono/mono/issues/12917
- IL Linker not working correctly with IQueryable extensions
2019-08-17 17:50:39 -05:00
Jay Krell 9a24e9a389 Add comments explaining goto main_loop vs. MINT_IN_DISPATCH. 2019-08-17 15:45:21 -07:00
Zoltan Varga 9df02d7a2d Fix the build. 2019-08-17 17:22:41 -04:00
Stephen Toub 2e618f0e63 Fix IDE0066 across corefx (dotnet/corefx#40288)
* Fix IDE0066 across corefx

"Fix all", but then manual tweaking to accommodate places the rule doesn't work quite right.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-17 23:07:24 +02:00
Zoltan Varga 44cfea466f Fix the build. 2019-08-17 12:26:35 -04:00
Zoltan Varga 7418248835 Add support for the System.Runtime.Intrinsics.X86.Popcnt intrinsics. 2019-08-17 12:25:59 -04:00
Zoltan Varga bfe4155083 [llvm] Add support for querying the cpu attributes from the LLVM jit. 2019-08-17 12:25:58 -04:00
Zoltan Varga b39abe5365 Fix the build. 2019-08-17 12:25:31 -04:00
Zoltan Varga 9eb687f2b9 Fix the windows build. 2019-08-17 12:25:31 -04:00
Zoltan Varga 98a02059e8 [jit] Add a netcore version of simd-intrinsics.c.
Differences from the old versions:
- Only Vector<T> from corelib is supported.
- Only LLVM is supported.
- It uses more general opcodes like XBINOP instead of having one opcode
  for every op+type combination which is not useful for LLVM.
2019-08-17 12:25:30 -04:00
Zoltan Varga 2c32eb1fa8 [jit] Optimize away box+unbox combinations which are common in corefx code. 2019-08-17 12:25:30 -04:00
Zoltan Varga 13fb0134a0 Merge pull request #16210 from EgorBo/opt-llvm-jit
Auto-detect current cpu in LLVM JIT
2019-08-17 12:22:07 -04:00
Jay Krell 3fb7834afb Put finally data back in locals instead of frame to save 16 bytes of stack.
Partial undo of f8fd60b5c0.
2019-08-17 00:05:50 -07:00
Jay Krell 0c6cb9fbc3 thread names: coop, constants, usually ignore-error.
- Convert setting thread name to be coop-compatible.

- For constant thread names, just retain the constant, not a copy.
  This includes producing and using constant unicode thread names on Windows.

- Make setting a thread name often non-fatal, except where mandated by public API.

- From earlier PR: mono_free (mono-publib.c) would no longer be referenced and therefore
  no longer exported. That broke profilers and maybe other externals.
  Choices:
     1 Use it instead of g_free randomly sometimes.
     2 Call it randomly sometimes.
     3 include it in external-only.c or object.c
     4 Mark it external only (breaks profiler).
     5 Use a .def file or Unix equivalent.

     5 is best, 2 is done here, the advantages/disadvantages
     among most choices are subtle. 3 is probably better than 2
     but is slightly bigger change, to put off -- you'd stop
     compiling mono-publib.c and move it to include/extra_redist.

     The advantage of not-.def file is perhaps that it is exposed
     by the compiler, so maybe easy to port and work with.
     People are more comfortable with obscure C extensions than
     any linker options, and there are multiple linkers to contend
     with. Arguably there are fewer compilers.
2019-08-16 15:24:30 -07:00
Egor Bogatov 6847508d2a Update llvm-jit.cpp 2019-08-17 00:25:19 +03:00
Jay Krell 16a81cf520 [Coop] Convert mono_attach_load_agent. (#15954)
Based on https://github.com/mono/mono/pull/15868.
Most cleanup and leak fixes moved and expanded to https://github.com/mono/mono/pull/15950.
2019-08-16 17:16:03 -04:00
Ryan Lucia 1e457cdd1e [netcore] Enable netcore resolution logic for mono_assembly_byname_lookup (#16256)
* Use netcore_load_reference in mono_assembly_request_byname

* Add ALC static Resolve event code from CoreCLR

* Wire up the AssemblyResolve event

* Enable newly passing test

* Set requesting assembly in InternalLoad

* Remove infinite recursion

* Only call netcore_load_reference from mono_assembly_request_byname

* Remove test completely
2019-08-16 16:54:47 -04:00
Nathan Ricci 325bbf541b Changes GC.cs 2019-08-16 11:48:35 -04:00
Nathan Ricci 029467271e Implemented imprecise version of get total bytes allocated. 2019-08-16 11:26:39 -04:00
Jay Krell e54f03168d Ignore NULL in mono_marshal_free_dynamic_wrappers and mono_free_method. (#14150) 2019-08-16 09:33:04 -04:00
Jay Krell 66afab5803 Fill in more MONO_PATCH_INFO_NONE support. (#13990) 2019-08-16 09:32:48 -04:00
Jay Krell f52bfb94d5 [interp] Localize c, i32, o as suggested. (#16268)
This saves 16 stack bytes for clang, gcc is unmoved probably figured it out on its own.
2019-08-16 09:32:07 -04:00
Jay Krell 4d9e6ae20e [interp] Fix more MonoError reuse (mono_object_new_checked). (#16263)
* [interp] Fix more MonoError reuse (mono_object_new_checked).

* Inline mono_interp_new/mono_object_new_checked now that we are in outlined mono_interp_newobj.
2019-08-16 16:26:49 +03:00
Steve Pfister da2c0159e9 [corlib] Fix sharing mode validation rules (#16247)
Fixes https://github.com/mono/mono/issues/16032

When we switched to corefx File.Copy on unix, the behavior changed where it opened a source and destination stream in order to pass the safe handles to Interop.Sys.CopyFile. This exposed a flaw in our sharing validation rules in MonoIO.Open, which threw an exception in this scenario:

new FileStream("source.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite, 2048, FileOptions.DeleteOnClose);
new FileStream("source.txt", FileMode.Open, FileAccess.Read, FileShare.Read, 2048, FileOptions.None);

Simplified the FileShare check down to:

1. If the caller specifies FileShare.None, then it's an exclusive lock and no
   sharing is allowed.

2. If the caller specifies some other FileShare option, then FileShare.None
    cannot be specified.
2019-08-16 07:52:22 -04:00
Egor Bogatov 4dca4f6619 [netcore] Add patch-local-dotnet-aot-llvm rule (#16120)
* Add patch-local-dotnet-aot-llvm rule

* Update Makefile

* Add run-sample-local-dotnet-llvm
2019-08-16 12:43:08 +02:00
Jay Krell 0c637a73a9 intepreter: remove last element from various arrays (#16134)
add const
 combine lines for searchability
2019-08-16 11:37:20 +03:00
Jay Krell 35fc20b637 [interp] Fully outline enum_hasflag to save stack. (#16228)
Alternative to https://github.com/mono/mono/pull/16197.
If you are going outline, you sometimes might as well take it all with you
instead of bouncing back and forth for a partial inlining.
2019-08-16 10:13:22 +03:00
Zoltan Varga 5461fb78c7 Merge pull request #16261 from vargaz/tp-no-random
[runtime] Use a PRNG in the threadpool instead of the os random number generator.
2019-08-15 23:48:35 -04:00
Jo Shields b6c499631e Exclude Authenticode tests on Watch (#16278)
1002a4de2b added Authenticode cert support on non-WatchOS. The tests,
however, were not non-WatchOS-only. Fix this, so Watch doesn't try to
run Authenticode tests.
2019-08-15 22:07:18 -04:00
Zoltan Varga 764091677e Merge pull request #16257 from EgorBo/llvm-ptr
Align C# pointers with LLVM pointers for primitive types
2019-08-15 21:59:42 -04:00
Jay Krell 6c05cfab59 [mini] Cleanup alloca: call twice instead of once, and shrink allocation slightly. (#16158)
It is an actual function call at least on Windows.
2019-08-15 18:14:37 -04:00
Jay Krell b654aa1ca8 [interp] Fix more MonoError reuse. (#16262) 2019-08-16 00:06:50 +02:00
Jay Krell dfbf1a1bd3 [Coop] Convert emit_invoke_call. (#15964)
Removing calls to mono_string_new_checked.
Extracted from https://github.com/mono/mono/pull/15900.
2019-08-15 17:56:33 -04:00
Jay Krell 44ff0597b8 Wrap more external-only functions in GC mode transitions. (#15895)
mono_compile_method
mono_runtime_set_main_args
mono_runtime_run_main
mono_runtime_exec_managed_code
mono_load_remote_field
mono_load_remote_field_new
mono_store_remote_field
mono_store_remote_field_new
2019-08-15 17:48:18 -04:00
Zoltan Varga 870aec4aad Llvm optimizations (#16191)
* [jit] Disable inlining for ThrowHelper methods for non-netcore as well. Add an OP_NOT_REACHED after calls to these methods to let LLVM know they don't return.

* [llvm] Use the cold calling convention instead of the preserveall one, it has the same effect and its better supported.

* [llvm] Call the aot init function using the cold cconv.

* [llvm] Make the condition in the poll function unlikely.

* [llvm] Add a wrapper around the GC poll icall so it can be called with a cold calling convention.

* Move the noreturn check before the code which is shared with calli.
2019-08-15 21:46:00 +02:00
Stephen Toub 133bdb5fa2 Fix non-ASCII characters across corefx (dotnet/corefx#40318)
* Fix several non-visible characters in test files

* Replace Unicode surrogate pairs with escaped hex values

* Fix non-ASCII characters in most C# files

Signed-off-by: dotnet-bot <[email protected]>
2019-08-15 21:44:47 +02:00
Jo Shields 4ec4d7a3b3 Add missing FCONV_TO_U8 on ILP32 (WatchOS 5) (#16276) 2019-08-15 13:56:04 -04:00
Jay Krell 7b64f1cd00 Retain thread names as UTF8 instead of only UTF16. (#15945)
Windows can still have its Unicode thread names, since we have UTF16 initially, just don't retain it.

This does penalize ves_icall_System_Threading_Thread_GetName_internal (non-netcore only),
which has to convert back to UTF16 each time.

But seems like a net win overall.

There is also a little prep work here for constant thread names and some temporary measures as multiple changes head toward the same code.

Rename `mono_thread_set_name_internal` to `mono_thread_set_name`.
Close possible race conditions.

Extracted from https://github.com/mono/mono/pull/15859.
2019-08-15 19:37:34 +02:00
Jay Krell f35c755fea [interp] Outline newobj and newobj_vt. (#16267)
In this variation, the exception handling and resume are split between the main function and the helper function, and no address-taken occurs on ip/sp/vt_sp.

The order is a little tangled but convincing.
The helper function returns MonoException* or checks context->has_resume_state and then returns, and then the caller does the checks again.

Based on past analysis this will conserve stack, but that was not double checked here.
i.e. due to `&valuetype_this`, though it is conditional and the code could be split up into inline and outline.
2019-08-15 19:23:24 +02:00
Egor Bogatov 2f7a06d208 [netcore] Mark "Run Tests" step as failed if there are failures (#16230)
Currently "Run CoreFX tests" step is always green
2019-08-15 18:06:15 +02:00
Jo Shields dd6bd568a0 Fix failing OSX Android lane (#16274)
Linker order matters when using .a files, so zlib.a MUST come after
sgen.a if sgen.a was compiled with support for compressed ppdb
2019-08-15 10:37:04 -04:00
Vlad Brezae 6fc089bf8b [exceptions] Don't debug hang on ThreadAbortException (#16240)
It is pretty much useless.
2019-08-15 14:57:26 +03:00
Jay Krell 63e1828f7e Reduce redundant setting of thread name. (#16253)
Maintain a counter and only set if the counter has changed.
This cannot be fully thread safe, so is not.
Any thread can set any other thread name at any time.
You could be atomic to do better, and loop to do even better,
but the thread name could still be changed again right after the loop exists.

The previous code was also not reliable in this way, since again the name
could be changed right away.

#16248
2019-08-15 10:56:38 +02:00
Jay Krell 5ec452666a [interp] [wasm] p vs. o on newobj (#16264) 2019-08-15 10:42:11 +02:00
Jay Krell fd4956fbf2 [interp] Fix some, not all, of the error management. (#16232)
From #16190.
2019-08-15 10:41:49 +02:00
Jay Krell 7766f9b926 [interp] Short-circuit MINT_STELEM_REF on NULL. (#16229)
Presumably this is not a useful semantic change,
to skip those functions, given NULL.

(We will probably want to outline all of this also, later).
2019-08-15 10:39:45 +02:00
Jay Krell fc724e6cf0 [interp] Compute locals later to reduce register/stack pressure and conserve stack. (#16231)
i.e. do not def before a function call and use after, do both after.
2019-08-15 10:39:27 +02:00
Jay Krell dde3452690 [interp] Split leave into pieces, partially outlined, to conserve stack, (#16213)
but without changing how exception handling macros access labels and locals.

This saves 16 bytes of stack on Linux/amd64/gcc and likely similar everywhere,
as it takes an address-taken local out of interp_exec_method_full (stackval tmp_sp).

See
https://github.com/mono/mono/pull/16171
https://github.com/mono/mono/pull/16171/commits/a6a8f0d5ffa875da9cda63444e401c37810a690f for a
full outlining.
2019-08-15 10:39:03 +02:00
Nikolay Sivov 5b66d3d518 [cominterop] Fix copy-paste error in ComVisibleAttribute testing helper.
This type was introduced with 29a95bafd8.
2019-08-15 09:29:14 +02:00
Thays Grazia dcfec3966c Enable and document debug with sdb with netcore config. 2019-08-15 09:23:02 +02:00
Zoltan Varga 928aab1b84 [runtime] Use a PRNG in the threadpool instead of the os random number generator, its good enough, and its much faster. 2019-08-15 02:14:06 -04:00
Egor Bogatov 93eceec724 address feedback 2019-08-15 02:39:13 +03:00
Jay Krell 2463605f01 Remove ampersand from cfg->error per old agreement. (#16219) 2019-08-15 01:07:56 +02:00
Martin Baulig 934c4f4c87 X509CertificateImplBtls.PrivateKey should allow to actually set the key. (#16212)
The `X509CertificateImplBtls.PrivateKey` property setter was previously ignoring all values and always used `null`.  This should help with #16189.
2019-08-15 01:07:35 +02:00
Egor Bogatov ed4be03b79 handle primitive pointers in llvm 2019-08-15 01:38:58 +03:00
Vlad Brezae 4e793ad30c [interp] Fix icall signature (#16227) 2019-08-15 00:18:45 +03:00
Jo Shields f2d813c90d Fix linking of OS-provided zlib (#16177)
* Fix linking of OS-provided zlib

This is a reduced version of https://github.com/mono/mono/pull/16091
without the Z_PREFIX stuff

* Get rid of HAVE_ZLIB, be explicit about HAVE_SYS_ZLIB vs HAVE_STATIC_ZLIB

* Proposed fixes from Alex
2019-08-14 16:58:07 -04:00
Stephen Toub a103b78163 Delete dead _pauseTicks from TimerQueue (dotnet/coreclr#26160)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 21:19:04 +02:00
Jan Kotas e46b3efbd0 Undo readonly on _isLastFrameFromForeignExceptionStackTrace
This makes the field symmetric with other fields and fixes build break for CoreRT

Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 21:19:04 +02:00
Aleksey Kliger (λgeek) ce92c4376f [metadata] Create strongly typed versions of mono_assembly_request_prepare (#16094)
* [metadata] Create strongly typed versions of mono_assembly_request_prepare

Addresses the family of issues similar to Coverity #1443346.

This came up as part of the Coverity review, and Coverity notices that this is
writting beyond the end of the structure.  While this is correct, this requires
that the idiom be properly followed - that is, that the structure being passed
is of the given size.

Replaced instead with a method that eliminates the room for error with methods
that will initialize the whole structure, and the actual size computation is
done inside the method with the proper type sizes.

While I used methods, I feel that this should use macros.
2019-08-14 15:07:26 -04:00
Aleksey Kliger (λgeek) a6b8494f0a (mono.1) Update the MONO_THREADS_SUSPEND description (#16236) 2019-08-14 19:23:36 +02:00
lateralusX 2d06126dc9 OP_LLVM_OUTARG_VT doesn't always create temporary vt needed by some ABI's.
LLVM lowering passes vt as reference to managed methods. On platforms where
the value type ABI is passing an address to the value type, either in register
or on stack (Windows x64) we must make sure we always use a temporary of
the value type or we will be breaking the value type semantics.

In JIT lowering of OP_OUTARG_VT we always make sure we use a temporary
in cases where argument storage is set to ArgValuetypeAddrInIReg/ArgValuetypeAddrOnStack.
On LLVM lowering this was currently not the case, but it still worked in many scenarios.
This was because IL load of arguments before doing a call did a
local copy and then we pass that byref. However, there are cases where this
copy could get optimized away and when that happens, we will change the
local value type in the caller’s frame, breaking the value type semantics.

Added test case in winx64structs hits such a scenario that will generate IL
breaking the value type semantics before the fix:

...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %49, %winx64_vector3Struct* %3, %winx64_vector3Struct* %16)
...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %64, %winx64_vector3Struct* %3, %winx64_vector3Struct* %15)
...

The two add calls will use the same reference (to a local value type in caller frame)
on both calls, this will break the value type semantics since they should both
be temporaries, like second argument is.

After the fix, the same code will be lowered into:

...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %55, %winx64_vector3Struct* %19, %winx64_vector3Struct* %18)
...
notail call monocc void @winx64_vector3Struct_Add_winx64_vector3Struct_winx64_vector3Struct
(i64 %72, %winx64_vector3Struct* %16, %winx64_vector3Struct* %15)
...

Each argument gets its own temporary that, according to the ABI, gets passed
by address down to function.

There will be situations where we could end up with two temporaries, one from
the load and one from the lowering of OP_LLVM_OUTARG_VT (same as we get on JIT),
this will however be detected by LLVM optimization pass, removing one of the copies.
2019-08-14 19:23:32 +02:00
Egor Bogatov abb43f9361 ignore stacktrace tests 2019-08-14 14:33:44 +02:00
Jay Krell d1c0974e46 [interp] Reduce frame size by partially outlining load remote field. (#16196)
Extracted from https://github.com/mono/mono/pull/16171.
Ultimately over 100 bytes can be saved.

Tactically not outlined or indirected:
 vt_sp change
 ip advancement
 error handling

 Also partially fixes the error reuse from https://github.com/mono/mono/pull/16190.
2019-08-14 12:34:54 +02:00
Jay Krell 60ac98ee29 Make small_id_mutex not recursive. (#15485)
(Later non-recursive Windows mutexes should be SRWLOCK and only statically initialized.)
2019-08-14 12:33:58 +02:00
Jay Krell 8c5d0a790e Use ERROR_DECL. (#16220) 2019-08-14 12:28:00 +02:00
Jay Krell 74b0d01e07 [interp] Remove address-taking of vt_sp. (#16215)
Theoretically could help reduce frame size but it has not been seen to.
2019-08-14 12:27:22 +02:00
Zoltan Varga 4c71a3e892 [wasm] Disable running xunit test suites on wasm, its not working reliably yet. (#16223) 2019-08-14 12:26:31 +02:00
Jay Krell 4c17442dc3 [interp] Outline mono_interp_calli_nat_dynamic_pinvoke to reduce stack. (#16201)
Now that we have that function, also change from malloc to alloca.

This is part of #16171 and ultimately over 100 bytes were saved.
2019-08-14 12:25:44 +02:00
Jay Krell 136205ab45 Completely replace mono_error_ok with is_ok and make first external_only. (#16217) 2019-08-14 12:24:50 +02:00
Jay Krell 12de10007c Skip degenerate qsort: num < 2, size == 0, base == 0 (#16016)
Fixes https://github.com/mono/mono/issues/15994

Some is redundant for some qsort implementations.
2019-08-14 12:22:40 +02:00
Jay Krell 6873d5370d Fix warning LNK4221 warning in assembly-load-context.obj (#16004)
LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
2019-08-14 12:19:08 +02:00
Jay Krell 9123ac0197 [interp] Outline get_virtual_method_fast to save 16 bytes of stack. (#16198)
Ultimately over 100 bytes are savable in a few steps.
2019-08-14 12:17:23 +02:00
Jay Krell dbf13301c8 [interp] Outline set_resume_state to conserve stack in main interpreter loop (#16200)
This is not intuitive but I definitely saw it.
2019-08-14 12:17:01 +02:00
Marek Safar 9ee80c6d3b Bump linker 2019-08-14 10:17:54 +02:00
Stephen Toub 3803a00421 Fix StyleCop warnings for rules enabled in corefx
Signed-off-by: dotnet-bot <[email protected]>
2019-08-14 10:15:38 +02:00
Thays Grazia f812d90fa1 [debugger] Debugger tries to execute a single step when the thread is already terminated (#16208)
* Debugger tries to execute a single step when the thread is already terminated.

* Fixing how to get thread tls.
2019-08-13 22:47:28 -03:00
Stephen Toub 50d3fc69d8 Convert several static readonly byte[]s to ReadOnlySpan<byte> (#26138)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 9e13a7e097 Fix IDE1005 (simplify delegate invocation)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub db4441bfe7 Fix IDE0062 (make local functions static)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 15ea431db7 Contribute to IDE0060 (unused argument)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub d927c98e03 Contribute to IDE0059 (unnecessary assignment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub d82bc555c9 Contribute to IDE0019 (use pattern matching) and IDE0066 (use switch expression)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub e19d95b6de Contribute to IDE0044 (make field readonly)
I used the auto-fix and then undid changes in areas where the fields were potentially mutated in native, e.g. I undid many of the changes in reflection.  We can revisit those individually later.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 6c95cbadf9 Fix StyleCop warning SA1400 (missing access modifiers)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Stephen Toub 0814b9de4d Enable StyleCop rules 1205, 1411, 1026, and 1000 (dotnet/corefx#40254)
* Fix StyleCop warning SA1411 (attribute ctor shouldn't use unnecessary parens)

* Fix StyleCop warning SA1026 (new[] spacing)

* Fix StyleCop warning SA1000 (consistent spacing around if, for, switch, etc.)

* Fix StyleCop warning SA1205 (partial elements should declare an access modifier)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-13 21:35:04 +02:00
Egor Bogatov 6c51440538 auto-detect cpu in llvm jit 2019-08-13 22:24:54 +03:00
Aleksey Kliger (λgeek) 592de53a1f [runtime] Initialize thread suspend policy at startup, inline getter (#16149)
* [runtime] Initialize thread suspend policy at startup, inline getter

On some benchmarks we saw mono_threads_suspend_policy() using a lot of CPU
time.

Rather than doing an init-on-first-use, initialize the policy once upfront when
the JIT it starting up and make the getter method static inline.

* move mono_threads_suspend_policy_init into mono_thread_info_init

* [mini] Initialize suspend policy for `mono --version`

Need to initialize the suspend policy before printing its name.  Otherwise the
suspend policy is unset since no other runtime initialization is done for `--version`.
2019-08-13 15:16:05 -04:00
Alexander Köplinger ff77b427f4 [netcore] Cleanup System.Private.CoreLib.csproj (#16199)
- The eng/Versions.props was already imported by the arcade targets causing a double-import warning:

> warning MSB4011: "/Users/alexander/dev/mono/eng/Versions.props" cannot be imported again. It was already imported at "/Users/alexander/.nuget/packages/microsoft.dotnet.arcade.sdk/1.0.0-beta.19379.1/tools/DefaultVersions.props (17,3)". This is most likely a build authoring error. This subsequent import will be ignored.

- Remove BaseAddress and TRACE from the project file, this was done upstream at https://github.com/dotnet/coreclr/commit/bc430cdd2010bf7790854a60380d00c506c346a6

- Remove unecessary DebugSymbols/DebugType settings, we already set them in the global PropertyGroup
2019-08-13 16:41:03 +02:00
Aleksey Kliger (λgeek) 1f4ac5d767 [sre] Outline typebuilder_setup_fields loop body (#16184)
The loop body uses coop handles, so wrap it in HANDLE_FUNCTION_ENTER/HANDLE_FUNCTION_RETURN
2019-08-13 09:31:31 -04:00
Jay Krell ef199425a2 [interp] Replace mono_error_ok with is_ok. (#16193)
The locally unknown function, despite being small/fast, forces saving values in stack and nonvolatile registers
around calling it.
Both nonvolatile registers and stack are at a premium here, whereas usually stack is cheap.
The macro is just a read from offset 0 and checking for zero.
2019-08-13 15:30:28 +02:00
Alexander Köplinger b60aaea791 [pkg] Bump libgdiplus to 6.0.2 2019-08-13 12:12:22 +02:00
Egor Bogatov c36f0a126e [netcore] decimal should have alignment=8 (#15666)
* Fix System.Runtime.InteropServices tests

* Update marshal.c

* Update marshal.c

* Update marshal.c
2019-08-12 21:57:47 +02:00
Zoltan Varga 5bc4944bf0 Merge pull request #16151 from spouliot/ios-intr-enable-sre
[sdk] Enable the SRE runtime code inside iOS builds
2019-08-12 14:44:54 -04:00
Stephen Toub b25395bde3 Re-enable StyleCop warning SA1400 (elements should specify access) (#40240)
One of the more common things we "nit" in PRs.  Now the compiler will flag it for us.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Aaron Robinson 66f7436d6f Remove unnecessary error check for QueryUnbiasedInterruptTime call (#26120)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Stephen Toub 4f821c3e8d Enable FxCop and StyleCop analyzers (dotnet/coreclr#26100)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-12 19:46:54 +02:00
Aleksey Kliger (λgeek) ad64c781eb [interp] Fix stub_get_resume_state write to out parameter (#16145)
Addresses Coverity CID 1452245
2019-08-12 13:09:59 -04:00
Ryan Lucia cadd114f64 [netcore] Pass MONO_ASMCTX_LOADFROM on LoadFrom calls (#16075) 2019-08-12 11:46:12 -04:00
Ankit Jain c028454bef [msbuild] Bump to track xplat-master and update roslyn (#16155)
Prompted by mono/msbuild#126:

This includes:

- merge mono-2019-06
- SDK update to track dotnet release/3.0.100-preview8
- Roslyn updated to 3.3.0-beta2-19381-14
2019-08-12 17:03:13 +02:00
UnknownShadow200 69ebdd8b93 [WinForms] Fix NotifyIcon not showing in systray on 64bit linux (#16164)
https://bugzilla.xamarin.com/show_bug.cgi?id=14976

Credit goes to datnhanlz for noting that removing XChangeProperty(DisplayHandle, hwnd.whole_window, _XEMBED_INFO, _XEMBED_INFO, 32, PropertyMode.Replace, atoms, 2); would also fix the bug. 

Note that the type of Atom is actually unsigned long (see usr/include/X11/X.h), however the code used a C# int for the atoms. As such the code worked without a problem on 32 bit linux, but broke on 64 bit linux.

I switched to using IntPtr instead to make the code also work on 64-bit.
2019-08-12 16:09:08 +02:00
Aleksey Kliger (λgeek) dd3f6e0d31 [debugger] Cleanup MonoError on in isFixedSizeArray (#16146)
Cleanup MonoError on error paths.  Also change `return FALSE` to `return 0`.

Addresses Coverity CID 1452205
2019-08-12 09:50:58 -04:00
Nikolay Sivov 612e0644b6 [mono] Fix some compiler warnings (#16159)
* [mini] Remove unused variables.

* [mini] Remove unused function.

Single call site is conditionally compiled.

* [sgen] Remove unused variable.

* [utils] Fix misleading indentation warnings.
2019-08-12 12:42:59 +02:00
Nikolay Sivov f9052ed78e [System.Drawing] Use CoreFX implementation of BufferedGraphics on Windows (#16163)
The issue was made visible with .NET Core WinForms. Some controls draw text with DrawTextEx to intermediate Graphics based on Bitmap, that later it's moved back to main Graphics with DrawImage. This results in a lost of transparency component along the way, making antialiased glyph bitmaps draw and B/W. Corruption is visible with gray antialiasing, but is more severe in subpixel modes.

I verified that same happens with gdiplus on Windows, code from CoreFX works correctly.
2019-08-12 12:42:07 +02:00
Jay Krell c70b4cc3cc [remoting+interp] [remoting+debugger] Fix use of dangling local. (#16173) 2019-08-12 12:35:30 +02:00
Jay Krell 4e5544f02d [interp] Remove unnecessary loops, that optimizer might handle. (#16169) 2019-08-12 12:34:46 +02:00
Vincent Povirk 7413bb84ec [w32process-win32] Don't duplicate CreateProcess search logic. (#16156)
When given a null application name, CreateProcess already
searches the path and appends .exe in some cases. .NET's
Process class behaves in the same way (see
https://github.com/mono/mono/pull/16089#issuecomment-519984244).

The duplicated logic is unnecessary and broken in several ways.
Failures are ignored and the command line arguments are
executed. Relative paths are expected not to end in "exe".
Many corner cases work differently from .NET and CreateProcess.

This change makes it impossible for process_complete_path to fail,
so I removed the gboolean return and pulled out the output
parameter. Its caller wasn't handling the return value correctly
anyway.
2019-08-12 12:05:05 +02:00
Zoltan Varga 835282004c Merge pull request #16136 from EgorBo/fix-inlining
[netcore] Fix inlining
2019-08-11 22:32:01 -04:00
Zoltan Varga 7c511c9937 Merge pull request #16160 from vargaz/wasm-xunit-4
[wasm] Xunit testing work.
2019-08-10 22:56:05 -04:00
Alexander Köplinger ec73f9ba17 Bump corefx to fix build break 2019-08-10 23:45:11 +02:00
Zoltan Varga c6054d5aa1 [wasm] Run System.Core xunit tests on CI. 2019-08-10 15:18:26 -04:00
Zoltan Varga 95b705d8c1 [wasm] Add argument conversions to the xunit runner. 2019-08-10 15:18:26 -04:00
Alexander Köplinger 738b6988b5 Bump corefx to get https://github.com/mono/corefx/pull/327 2019-08-10 16:19:01 +02:00
Nikita Voronchev 000430002a [System.Drawing.Design] Allow context be a null in the ColorEditor.EditValue (#16141)
If one call `ColorEditor.EditValue(IServiceProvider, Object)` ([MSDN](https://docs.microsoft.com/ru-ru/dotnet/api/system.drawing.design.uitypeeditor.editvalue?view=netframework-4.8#System_Drawing_Design_UITypeEditor_EditValue_System_IServiceProvider_System_Object_)), then `context` is `null` and drop-down menu doesn't appear.
2019-08-10 16:01:37 +02:00
Sebastien Pouliot 360c1c44c7 [sdk] Enable the SRE runtime code inside iOS builds
This will allow the interpreter to use System.Reflection.Emit.

Note: There is a small size increase (about 50kb) for an application
built in release mode with the LLVM backend (even if the interpreter
is now used).
2019-08-09 13:57:54 -04:00
Vlad Brezae b52830cf11 [interp] Fix overflow of execution stack (#16115)
We were not tracking max_stack_height for a few opcodes.
2019-08-09 20:28:30 +03:00
Zoltan Varga 54c07051a8 Merge pull request #16139 from steveisok/run-single-ios-sdk-test
Added the option to run a single test in the iOS SDK
2019-08-09 12:58:01 -04:00
Ryan Lucia a1bb37cd52 [netcore] Make mono_assembly_request_prepare take an ALC (#16061)
* Make mono_assembly_request_prepare take an ALC

* Typo

* Fix pedump

* Use default ALC

* Make v1 preload hook external-only

* Convert pedump and monodis

* fixup: use default ALC in netcore preload hook

trusted platform assemblies should be loaded into the default ALC
2019-08-09 11:44:50 -04:00
Alexander Köplinger 0970996f75 Bump eng/Versions.props again
Try to fix the netcore build.
2019-08-09 17:03:12 +02:00
Alexander Köplinger 75c21692a0 Add mono-hang-watchdog.in to tarball
Remove custom distdir target and use standard EXTRA_DIST to do so.
2019-08-09 16:48:16 +02:00
Stephen Toub 86a0ad2f88 Fix FxCop warning CA1823 (unused fields)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub f39ac1d32d Fix FxCop warning CA2229 (private deserialization ctor)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 2d3a6ea2bb Fix FxCop warning CA1825 (Array.Empty)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub cce1583769 Fix StyleCop warning SA1620 (generic XML comment ordering)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 15963fe836 Fix StyleCop warning SA1617 (void return XML comment)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 4ce53c845b Fix StyleCop warning SA1113 (comma position for parameters)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 442e9ad144 Fix StyleCop warning SA1206 (modifer ordering)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 01fa50943f Fix StyleCop warning SA1121 (use built-in types)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 9c3037fc59 Fix StyleCop warning SA1200 (cref prefixes)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 1fcf7a1ab4 Fix StyleCop warnings SA1028 and SA1518 (trailing whitespace and blank lines)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Stephen Toub 441257339f Fix FxCop warning CA1018 (attributes should have AttributeUsage)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 16:23:42 +02:00
Alexander Köplinger 74338e0f2d Bump corefx to get https://github.com/mono/corefx/pull/323 2019-08-09 16:21:12 +02:00
Alexander Köplinger 09e7c4509c Update dependencies from corefx (#16137) 2019-08-09 15:19:16 +02:00
Steve Pfister 9e9697c978 Added the option to run a single test in the iOS SDK 2019-08-09 09:10:13 -04:00
Alexander Köplinger f15fe47075 [build] Remove ILDISASM variable from platform Makefiles
It was written by configure.ac to config.make and is unused anyway.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 652f6c64b6 [build] Remove PLATFORM_CORLIB from platform Makefiles
Changing the corlib assembly name isn't supported anywhere else in the build.
2019-08-09 12:53:14 +02:00
Alexander Köplinger e80f3449da [build] Remove MBAS from the platform Makefiles
It no longer exists.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 8697b31082 [build] Remove EXTERNAL_MCS setting in platform Makefiles
Setting an explicit mcs is no longer supported by the rest of the build scripts.
2019-08-09 12:53:14 +02:00
Alexander Köplinger d7174c9aa6 [build] Remove confusing commented out code 2019-08-09 12:53:14 +02:00
Alexander Köplinger ab1d61509d [build] Remove PLATFORM_MAKE_CORLIB_CMP
The corlib_cmp doesn't exist anymore.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 3e77e01071 [build] Replace PLATFORM_ ifdefs with HOST_PLATFORM checks
Building Linux BCL assemblies from a macOS machine didn't work since we
were enabling some Apple features if the build machine was macOS.
2019-08-09 12:53:14 +02:00
Alexander Köplinger 175e345122 [tools] Add .gitignore for mono-hang-watchdog 2019-08-09 12:39:34 +02:00
Stephen Toub 67a89b3a6f Move CodeAnalysis.ruleset to shared (dotnet/corefx#40153)
So that the same settings can be applied to corelib.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 12:30:05 +02:00
Egor Bogatov a7e428d82f fix inlining 2019-08-09 12:55:50 +03:00
Egor Bogatov 7deb53514d Make System.Runtime.Intrinsics.X86.*.IsSupported intrinsic 2019-08-09 09:34:29 +02:00
Egor Bogatov 9442ed95d7 fix minor issues 2019-08-09 09:34:29 +02:00
Alexis Christoforides 2c4b60fe24 [merp] Use a separate program as the hang supervisor. (#15715)
* [merp] Use a separate program as the hang supervisor.

Fixes https://github.com/mono/mono/issues/15646

macOS does not like signals being sent from the forked supervisor process, possibly towards anywhere but definitely when sent to the parent process. The following message is currently spewed after the supervisor process attempting to send a SIGSEGV to a hanged Mono process:
"The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec()."

We follow that direction and introduce a new binary that, when available in the mono executable's binary directory, is used to abort the parent process for us.
2019-08-09 10:31:57 +03:00
Ryan Lucia 6a459c25d7 [netcore] Don't return REFERENCE_MISSING in netcore_load_reference 2019-08-09 09:23:10 +02:00
Jeffrey Zhao a49ecebd4e Catch InvalidCastException properly in Collection (dotnet/coreclr#25953)
The explicit implementations of IList.Add/Insert/Set methods catch
InvalidCastException for casting input argument to type T to throw
an ArgumentException. The issue here is they put the core collection
modification operation in the try block, which could be overridden
by sub-classes and produce an InvalidCastException from user logic,
even the argument passed to IList.Add/Insert/Set is valid.

To fix the issue, we just need to move the collection modification
outside the try block, and leave the casting in try block only. In
this way, we'll get an ArgumentException when the argument type is
invalid, but propagate the InvalidCastException thrown from user
code as it should be.

dotnet/corefx#39919

Signed-off-by: dotnet-bot <[email protected]>
2019-08-09 09:22:28 +02:00
Zoltan Varga e71604d695 Merge pull request #16125 from vargaz/wasm-keep-config-cache
[wasm] Add a KEEP_CONFIG_CACHE option to Make.config to avoid deleting the autoconf cache file on make clean.
2019-08-09 03:20:03 -04:00
monojenkins 1369ebcfaf [bcl] Update BCL Linked Size 2019-08-09 09:19:32 +02:00
Alexander Köplinger fde0a5b628 [ci] Trigger PR builds for netcore only on master branch
The work is happening in master only for now.
2019-08-09 00:53:19 +02:00
Alexander Köplinger 6075417d88 Bump corefx so it points to correct branch 2019-08-09 00:14:18 +02:00
Alexander Köplinger ec35e14636 [tests] Fix accessing field of MarshalByRefObject in remoting1.cs (#16119)
We need to copy it to a local variable first.
This was exposed by a new Roslyn optimization for String.Concat (see https://github.com/dotnet/roslyn/issues/37830).
2019-08-09 00:07:41 +02:00
Stephen Toub da2c1cde97 Re-enable StyleCop warnings SA1028 and SA1518 (trailing whitespace and blank lines) (#40113)
Mechanical/automated change.  The only manual changes were a) removing the two rules from the CodeAnalysis.ruleset file, b) removing spaces from one line of a .targets file that is used to generate a .cs file, and c) reverting the changes in a few test files where the naive replacement happened in the middle of a verbatim string where the whitespace actually mattered.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 23:44:46 +02:00
Alexis Christoforides 1127974861 [System] Guard against attempting to creating out-of-range spans in macOS FileSystemWatcher 2019-08-09 00:24:06 +03:00
Zoltan Varga adeef6c0fc [wasm] Add a KEEP_CONFIG_CACHE option to Make.config to avoid deleting the autoconf cache file on make clean. 2019-08-08 17:08:51 -04:00
Aleksey Kliger (λgeek) 36af44e3be [perfcounter] Define mono_perfcounter_foreach even if perfcounters are disabled. (#16100)
Fixes netcore build on win32
2019-08-08 14:06:21 -04:00
monojenkins 18917b885c [csproj] Update project files 2019-08-08 19:10:03 +02:00
Marek Safar 2ef50f54ee Bump corefx for #15259 fix 2019-08-08 19:10:03 +02:00
Adam Sitnik b15518dd5e make DateTime.UtcNow 5% faster to minimize the leap second performance impact (#26046)
* make few methods used by DateTime.UtcNow inlinable to minimize the leap second performance regression impact, related to #25728

* apply suggested micro-optimizations

Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 18:55:19 +02:00
Jan Kotas 7c59cd14fe Cleanup ProjectN ifdefs (dotnet/corert#7675)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 18:55:19 +02:00
Alexander Köplinger dfa5a6ce4d [aprofutil] Install the tool correctly (#16112)
It needed a few tweaks to be usable when Mono is installed to the prefix.
2019-08-08 16:46:55 +02:00
Steve Pfister 64ea10fbcd Fixed RuntimeType.GetConstructorCandidates("") to return 0 elements. (#16054)
https://github.com/mono/mono/issues/16010

The current behavior for GetConstructorCandidates when the name is null, "",
or "*" is to return the set of constructors for the type. For "", .NET Core and
.NET Framework return 0 elements.  This PR adds a check for "" to match that
behavior.
2019-08-08 12:14:39 +02:00
Stephen Toub c189afd4f5 Delete PROJECTN ifdefs from Corelib (dotnet/coreclr#26058)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 10:10:47 +02:00
Sung Yoon Whang 240202cd90 Improve ArgumentNullExceptions on DiagnosticCounter.DisplayName/Unit (#26035)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-08 10:10:47 +02:00
monojenkins 53b9c88795 [bcl] Update BCL Linked Size 2019-08-08 10:09:17 +02:00
imhameed fc4d234ef6 [sdb][interp] Use the interpreter's resume state to compute the IL offset in compute_frame_info. (#15936)
[sdb][interp] Use the interpreter's resume state to compute the IL offset in compute_frame_info.

This recovers the relevant bits of the "resume state" from the interpreter and uses that to recalculate the `il_offset` for the top frame.

Fixes https://github.com/mono/mono/issues/15687.
2019-08-07 20:05:55 -04:00
imhameed 4462f10fb6 [llvm] Explicitly check for errors when calling LegacyIRCompileLayer::addModule. (#16082)
[llvm] Explicitly check for errors when calling LegacyIRCompileLayer::addModule.

LLVM built with LLVM_ENABLE_ASSERTIONS=On will crash if errors are not
explicitly consumed.
2019-08-07 15:15:55 -04:00
Ryan Lucia aece3efc99 Relax locking (#15986)
* Remove usage of loader lock

Originally added in https://github.com/mono/mono/commit/c4cbd3b700eea52c84c68c9450d80357b3be681d but no longer relevant

* Make get_agent_domain_info lock-free and relax loader lock

* Syntax errors

* Undo first commit, other than the commit message

* Fixes
2019-08-07 14:17:19 -04:00
Marek Safar b3344fa0b2 Bump linker (#16014)
* Bump linker

* Bump api-doc-tools and add NEW_CECIL define to mdoc

* Bump linker

* Bump API snapshot submodule

* [csproj] Update project files
2019-08-07 19:37:19 +02:00
Sylvain 4780486237 Memcpy instead of memcpy1 in System.Buffer.MemoryCopy (void*, void*, long, long). (#16056)
In the same way it is done in System.Buffer.MemoryCopy (void*, void*, ulong, ulong).

Correct what appears to be a typo in System.MemoryCopy with "long, long" size arguments.

Consequently, copy is not done byte per byte for the last int.MaxValue bytes when MemoryCopy is called with long argument (most common case). This should provide noticeable performance increase.
2019-08-07 18:36:47 +02:00
Radek Doulik 1f2b37ed37 [aotprof-tool] Initial import of AOT profiler tool (#15384)
* [aotprof-tool] Initial import

Imported https://github.com/radekdoulik/aotprofile-tool and updated it
to use Mono.Profiler.Log

* [aotprof-tool] Added man page

* [aotprof-tool] Use the shortened name in README.md
2019-08-07 18:24:51 +02:00
Ryan Lucia dcd5715161 [netcore] Re-enable IsAssignableFrom test
This was fixed a while back but never removed from the exclusions
2019-08-07 17:27:35 +02:00
Zoltan Varga d18daefdbc Merge pull request #16085 from kjpou1/wasm-sdk-v0.2.0
[wasm][sdk] Bump SDK Version 0.2.0
2019-08-07 11:17:12 -04:00
Jay Krell e8d28feb1e There is no need to zero static data, and it is racy (#15699)
if there are multiple threads calling (unlikely).
This zeroing was made moot by 470354fa81 January 2019.
2019-08-07 10:40:57 -04:00
Thays Grazia e690fb0240 Allow generating DIM bitcode (#16070) 2019-08-07 11:06:56 -03:00
Kenneth Pouncey 4767c581c7 Update sdk version for tests 2019-08-07 15:57:45 +02:00
Alexander Köplinger 521d4c8395 [utils] Move MAP_ANONYMOUS check after sys/mman.h import
The flag is defined in that header on OSX. Followup to 2abe569ede
2019-08-07 15:54:09 +02:00
Aleksey Kliger (λgeek) aea457cdb3 [w32socket] Translate ELOOP and ENAMETOOLONG (#16039)
Translate to WSAELOOP and WSAENAMETOOLONG, respectively.

See https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2

Addresses part of https://github.com/mono/mono/issues/16024
2019-08-07 09:42:39 -04:00
Nikolay Sivov 10c5e6407e [trace] Improve enter/leave output formatting (#16036)
* [trace] Use specific integer types for return value traces.

* [trace] Improve argument list formatting for ENTER lines.

Return value changes are especially visible for bools, where currently 4 bytes integers are printed.
2019-08-07 15:05:44 +02:00
Kenneth Pouncey b0611a5ee9 Bump SDK Version 0.2.0 2019-08-07 14:59:53 +02:00
Nikolay Sivov d34381f72e [win32] Fix some compilation warnings (#15956)
* [mini] Remove unused variable.

* [mini] Mark win32 exception handling functions static.

* [mini] Remove unused variable.

* [mini] Suppress unused function warning.

* [metadata] Fix unused function warning for Win32 build.
2019-08-07 14:45:19 +02:00
iamphi 2abe569ede Fix compilation on OSX 10.9 (#14411)
Found solution from: 
https://github.com/eclipse/omr/issues/902#issuecomment-288559044
2019-08-07 14:42:57 +02:00
Alex Thibodeau 24bbb94b6b Swap bytes for tcp port number because windows API gives us port in network order. (#16030)
Addresses the FIXME in Win32IPGlobalProperties.cs :: FillTcpTable

Fixed Unity issue: issuetracker.unity3d.com/issues/ipglobalproperties-dot-getactivetcpconnections-return-different-port-numbers-when-compared-to-cli-netstat-a-output
2019-08-07 14:30:07 +02:00
Zoltan Varga ba1c949a24 [runtime] Fix an autoconf warning. (#16078)
[runtime] Fix an autoconf warning.

<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->
2019-08-07 06:06:00 -04:00
Vlad Brezae e702e8286f [interp] Fix DEBUG_INTERP build (#16057) 2019-08-07 12:47:41 +03:00
Marek Safar 25b84915e1 Remove code moved to shared partition 2019-08-07 09:48:03 +02:00
Stephen Toub c5027781de Remove some unused locals (dotnet/corefx#39956)
* Remove some unused locals

Trying out various analyzers, including one to flag unused locals.  There are more than this, but this is a first batch.  Some of these are just clean-up.  Others are saving unnecessary cost.  At least one appears to be a real functional bug.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Stephen Toub 824351f49c Fix CounterGroup to use built-in types
Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Aleksey Kliger (λgeek) b2ab9198d3 Move AssemblyLoadContext VM-invoked resolve methods to shared (#25989)
Share between CoreCLR and Mono

Mono began using a copy of these methods with
https://github.com/mono/mono/pull/15946

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Tanner Gooding 0378116ffc Updating Math.Round and MathF.Round to be IEEE compliant so that the intrinsic and managed form are deterministic. (#25901)
* Updating Math.Round and MathF.Round to be IEEE compliant so that the intrinsic and managed form are deterministic.

* Fixing the Math.Round and MathF.Round handling for values greater than 0.5 and less than 1.0

* Applying formatting patch.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
John SalemandNoah Falk a76f6fa9b5 Allow for interface implementations in EventSource.WriteEventVarArgs (#25844)
* Allow for interface implementations in EventSource.WriteEventVarArgs

* Also account null ref types and nullable types

* fix error in comment and simplify first part of boolean logic

* Update src/System.Private.CoreLib/shared/System/Diagnostics/Tracing/EventSource.cs

Co-Authored-By: Noah Falk <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-08-07 09:48:03 +02:00
Ryan Lucia 022fc4ea37 [netcore] Set flag for processor arch when parsing assembly names 2019-08-07 09:22:04 +02:00
Bernhard Urban fe0f311a84 [arm] respect I1 and I2 on gsharedvt out transition (#16000)
[arm] respect I1 and I2 on gsharedvt out transition

In order to trigger the problem, we must call into gsharedvt code ("in" transition) and then escape the gsharedvt world ("out" transition). The bug happens in the latter. Also some other stars need to align (I haven't fully understood it), thus the repro is quite long.

The provided repro only triggers with LLVM, which we don't have enabled on CI yet: https://github.com/mono/mono/issues/15999

Fixes https://github.com/mono/mono/issues/15261
Fixes https://github.com/mono/mono/issues/15262
Fixes https://github.com/mono/mono/issues/15263
Fixes https://github.com/mono/mono/issues/15307
2019-08-06 18:25:51 -04:00
Zoltan Varga add830f9cb Merge pull request #16071 from vargaz/wasm-xunit-3
[wasm] More xunit fixes.
2019-08-06 17:57:22 -04:00
Zoltan Varga 476af240d2 [wasm] Compile interp.c with -mllvm -join-liveintervals=false to fix compilation times. 2019-08-06 16:46:19 -05:00
Jay Krell 9bf703b031 [Coop] Convert mono_object_isinst_icall. (#15963) 2019-08-06 15:26:55 -04:00
Zoltan Varga 59a285a546 [wasm] More xunit fixes. 2019-08-06 15:24:32 -04:00
coypoop 5a4283d1b3 Update NetBSD support (#15938)
Based on the FreeBSD variation.
Needs some additional changes I'm still ironing out.
2019-08-06 20:32:38 +02:00
Ryan Lucia 64bf7ef9db [netcore] Update test exclusions to reflect ALC work (#16059) 2019-08-06 14:08:18 -04:00
Zoltan Varga 3c88e9f251 [netcore] Avoid calling load hooks during AOT.
Fixes https://github.com/mono/mono/issues/16046.
2019-08-06 19:40:23 +02:00
Zoltan Varga 662b72f913 Merge pull request #16058 from vargaz/wasm-requests-2
[wasm] Update WASM-REQUESTS.md.
2019-08-06 12:53:44 -04:00
Zoltan Varga dda05d0a09 [wasm] Update WASM-REQUESTS.md. 2019-08-06 10:48:42 -04:00
Egor Bogatov 0fbc5d8315 DeflateStream: check IsInvalid before disposing (#16048) 2019-08-06 14:53:24 +02:00
Ryan Lucia 85296ef1be [netcore] Make the load hook ALC-aware (#16012)
The meat of this PR is what the title suggests, and with this commit all the basic ALC functionality should be working properly. Probably worth checking if this makes any new tests pass.

Additionally, this PR includes a second commit that switches over the search hook to use loaded_assemblies from the ALCs. This may break some existing tests due to remaining unimplemented ALC behavior, notably the native loading mechanism needed for Openssl in some of the cryptography tests. My inclination is to include the commit regardless and just disable the failing tests so that any future ALC work can watch for regressions in the loader, but I would appreciate input from steveisok on how important those tests are. I've not disabled anything yet in order to see exactly what new failures are introduced.
2019-08-06 12:16:15 +02:00
Jay Krell 474092add6 UWP now allows LoadLibrary. (#16015) 2019-08-06 12:14:38 +02:00
Martin Baulig 1c13b30225 SocketAsyncEventArgs cleanups and preparations for the real thing. (#16001)
Cleaning up the `SocketAsyncEventArgs` class and preparing it for it's eventual
adaption from CoreFX.

There are a few instance fields and property accessors with `internal` visibility,
which the CoreFX version does not have.  Replaced these with accessor methods.

The `PolicyRestricted` property and it's related `SocketAsyncEventArgs (bool)`
constructor is not used anywhere and the CoreFX version has a constructor with the
same signature, but different semantics.

In the CoreFX version, `Complete ()` does not call `DoComplete (this)` - renamed it
into `Complete_internal ()` to avoid any ambiguities and help the transitioning.
2019-08-06 12:13:40 +02:00
Alexander Köplinger 7ab3811b93 Bump api-doc-tools to latest master (#16035)
* Bump api-doc-tools to latest master

Disable building mdoc.exe in mcs mode since it no longer builds because of new C# features.

* Fix mcs build by disabling more mdoc invocations

* Bump monodroid_tools to compile against v4.7 reference assemblies

mdoc needs ValueTuple which is not in v4.6.

* [csproj] Update project files
2019-08-06 10:40:24 +02:00
Zoltan Varga d57f3f55f0 Merge pull request #16043 from kjpou1/wasm-sdk-small-fix
[wasm][sdk] Fix misspelling and trailing comma
2019-08-06 02:29:20 -04:00
Kenneth Pouncey ca51e5b5cc Fix misspelling and trailing comma 2019-08-06 07:30:47 +02:00
Zoltan Varga 9b4bc613e4 Merge pull request #16006 from vargaz/as-gsharedvt
[jit] Avoid asserting on Unsafe.As<T> in gsharedvt methods.
2019-08-05 21:04:44 -04:00
Kenneth Pouncey 69aa5dcaa7 Fix indentation 2019-08-05 15:03:08 -05:00
Kenneth Pouncey 357a3aaec5 [wasm][sdk] Fix issue with WasmLinkAssemblies task wrongly using reference assemblies
The issue is that when executing the `WasmLinkAssemblies` task the reference assemblies are currently used as input to the linker step and output in the published application instead of the actual framework library file.

My findings so far on this is that the package references are resolved in the MSBuild [ResolvePackageAssets](https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/ResolvePackageAssets.cs#L205.)

Those are then remapped from [here](https://github.com/dotnet/sdk/blob/ec9f5fc673f798dd6a89c7d11a4e62727837757c/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.PackageDependencyResolution.targets#L270) to the `RuntimeCopyLocalItems`.

What I did was pass the `RuntimeCopyLocalItems` to the `WasmLinkAssemblies` task and do a remap to the runtime versions.  This works against everything I have thrown at it so far.

Of course this could fail if the nuget has somehow set `copy to local` false maybe but I do not see anything in the code referenced above checking for that.
2019-08-05 15:03:08 -05:00
Jo Shields c8c52f7e9f Bump version to 6.7 (#16031)
* Bump version to 6.7
2019-08-05 15:31:47 -04:00
Alexander Köplinger 807e4b3d6f [ci] Disable 32bit llvm sdks build
It was removed in https://github.com/mono/mono/commit/a6e6c72c8fc3702db37e1572694a49902f63195f
2019-08-05 21:29:51 +02:00
Kenneth Pouncey ac80d041e6 [wasm][sdk] Decoupling the WebAssembly runtime debug and release versions. (#16028)
* Update templates

- Add PropertyGroups for Debug and Release to set the DebugSymbols and DebugType by default.

* Allow the selection of the WebAssembly runtime version Debug | Release

- Decouple the reliance on `_DebugSymbolsProduced` to decide which runtime version to publish.
- Add support for <MonoWasmRuntime>Debug | Release</MonoWasmRuntime>
- Add validation target of the property to Mono.WebAssembly.Build.targets.

* Indentation
2019-08-05 19:03:35 +02:00
Zoltan Varga 93a6bc3151 [netcore] ifdef out unused icalls/files. (#16009)
* [netcore] ifdef out unused icalls/files.

* Add MONO_EMPTY_SOURCE_FILE.
2019-08-05 17:13:40 +02:00
Egor Bogatov d1700fc93f Add finalizer to DeflateStream (#15879)
Fixes #15825
2019-08-05 13:20:29 +02:00
Zoltan Varga 17b35dc4fc Merge pull request #15640 from vargaz/wasm-xunit
Add beginnings of support for running xunit tests.
2019-08-04 17:05:59 -04:00
Zoltan Varga f239e4e68b [wasm] Add basic support for running xunit tests. 2019-08-04 15:59:36 -04:00
Zoltan Varga 60bfa99043 [jit] Avoid asserting on Unsafe.As<T> in gsharedvt methods.
Fixes https://github.com/mono/mono/issues/15992.
2019-08-03 15:06:58 -04:00
Stephen Toub f532f93ea6 Enable stylecop warnings 1205, 1212, and 1617 (dotnet/corefx#39941)
* Enable stylecop warning 1617 (void return value should not be documented)

* Enable stylecop warning 1212 (get/set accessor ordering)

* Enable stylecop warning 1205 (modifier ordering)

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Stephen Toub 3385433b60 Enable several StyleCopAnalyzers (dotnet/corefx#39921)
* Fix ordering of keywords in member signatures

* Add StyeCop.Analyzers

Most rules are disabled.  Many of them we'll want to leave disabled, however there are a handful it'll be good to enable over time.

* Use built-in type alias

* Fix straggling stylecop warnings in all configurations

* Replace tabs with spaces

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Stephen Toub 3fde98da3e Re-enable CA1823 (unused private fields) (dotnet/corefx#39899)
Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Sung Yoon Whang 66a7646232 Make counters use dedicated thread instead of timer (dotnet/coreclr#25864)
* Make a dedicated polling thread for EventCounters

* Cleanup

* remove debug prints

* nit

* Fix comment

* addressing issues from code review

* Fix an assertion from getting fired when we have multiple listeners

* Fix a test issue

* code review feedback

* Fix a potential deadlock

* Fix another deadlock

* Allow s_sleepDurationInMilliseconds to be reset to a larger value

* More issues fix

* Let thread wake up from sleep if needed

* Some suggestions for the counters fix.

The resulting behavior should be the same as what is there now (assuming I didn't mess anything up), these are all just code simplifications that hopefully make it easier to review and maintain going forward. In total I think this reduces the change size in CounterGroup.cs by ~30%.
- With the addition of the AutoResetEvent the ManualResetEvent is no longer needed, removed it.
- Removed the various if foo != null checks for the shared state, it is all initialized once when then thread is created and then assumed to be non-null elsewhere.
- Removed a 2nd lock acquisition inside OnTimer
- Replaced an if with Math.Min in PollForValues

* fix test

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
ts2do 5104fd4b2f Eliminate extra time zone conversions in DateTimeOffset (#25658)
* Eliminate extra time zone conversions in DateTimeOffset

Revised members dealing specifically with local time by removing delegation to DateTime methods to avoid an extra call to TimeZoneInfo to determine offset:
Now
FromFileTime(long)
ToLocalTime()

* Fix DateTimeOffset.ToLocalTime(bool)

* Consolidate conversion from UTC DateTime to Local DateTimeOffset

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Stephen Toub 6345e18108 Fix use of AddTo/RemoveFromActiveTasks in async methods (#25911)
When we rewrote the async methods implementation with AsyncStateMachineBox, we neglected to call AddToActiveTasks if the debugger is paying attention to tasks.  This hasn't mattered as the debugger's support for Tasks hasn't worked for other reasons in .NET Core, but there is now a renewed focus on it, and this is preventing some of that support from working.

This change is a minimal fix to ensure that we're adding the state machine box task when it's created and removing it when it completes.  Post-3.0, we should look at overhauling this, e.g. to clean up a lot of this logging and tracking that's done, to use a weak table in order to avoid keeping task objects alive artificially if they're dropped without completing by the developer code, etc.

This only affects when the debugger is attached, as the s_asyncDebuggingEnabled field is only ever set by the debugger.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Stephen Toub fa0c0bb2e2 Fix nullability annotations for static (dotnet/coreclr#25913)
The compiler is now analyzing statics.  Get compliant.

Signed-off-by: dotnet-bot <[email protected]>
2019-08-03 20:55:40 +02:00
Egor Bogatov e08bddaa63 [netcore] Minor fixes around System.Reflection (#15905)
* Minor fixes

* Update HelloWorld.csproj
2019-08-03 20:49:38 +02:00
Ryan Lucia 848811e4b2 [netcore] Enable ALC loaded_assemblies (#15850)
* Typo

* Enable loaded_assemblies and lock

* Make mono_domain_assembly_open_internal ALC-aware

* Populate loaded_assemblies


Switch from prepend to append

* Some notes

* Exclude add_assembly_to_alc on non-netcore

* Cleanup appdomain.c

* Remove mono_alc_cleanup for now

* Update comment

* Disable using loaded_assemblies in the search hook

* Adjust alc cleanup
2019-08-03 20:47:38 +02:00
Peter Collins 920e03f732 [corlib] Ignore TimeZoneTest.TestCtors on Android GMT
TimeZone.CurrentTimeZone.DaylightName returns an empty value on Android
when the system timezone is set to GMT.
2019-08-03 20:35:27 +02:00
Zoltan Varga 9da563509e Remove error_init from native icall wrappers since managed does it. (#15943)
Preferable would be native only, i.e. for interpreter and for when
it is actually unused, but ok for now.

Between removing the managed and the native, managed is preserved
in case there uses of MonoError in C# other than icall wrappers.
Keep them safe.

But again ideally move it to native and remove the managed type,
pending debugging WebAssembly failure doing so.
2019-08-03 12:00:46 -04:00
Kenneth Pouncey a61d8ea767 [wasm][framework] Rework framework linker descriptions (#15952)
- Remove the fully preserved assembly references from `WebAssembly.Bindings` and `WebAssembly.Net.Http`

Close https://github.com/mono/mono/issues/15365
2019-08-03 12:00:19 -04:00
Jb Evain 283c0557d3 Fix usage of mono_debug_get_seq_points to avoid leaks (#15977) 2019-08-03 11:59:25 -04:00
imhameed 3ab93f97e6 [llvm] Add support for JIT + LLVM 9.0. (#15997)
* [llvm] Add support for llvm 9.0.

Works with this branch:
https://github.com/mono/llvm/tree/release_90

* Add support for JIT + LLVM 9.

* Don't assume that all memset targets will be aligned; fixes LLVM 6 on amd64.
2019-08-03 11:58:40 -04:00
Calvin Buckley e70ac8c6ae Enable MONO_ARCH_EMULATE_FCONV_TO_U4 on PPC (#15991)
Fixes regression related to floating point introduced in #15930.

This does mean we have to take a slower path instead of using the
PPC instruction, but it provides consistent results. Other CPUs
have to do this too, so it's not like PPC is impacted unfairly.
(i.e, NaN gives an inconsistent result, at least with POWER7 and
some amd64 CPUs tested)

Fixes #15990.
2019-08-02 19:00:02 -04:00
Jo Shields 9a7b6fd5ee Bump libgdiplus to new release (#15996) 2019-08-03 00:48:26 +02:00
Calvin Buckley 9ea07610ee Don't use clock_nanosleep on PASE (#15988)
* Don't use clock_nanosleep on PASE

PASE exports this AIX syscall so autoconf detects it, but using it
will trigger SIGILL (for unimplemented syscall), which emits a LIC
log entry type of 4700-000F.

* Change ifdef guard for clock_nanosleep in threads

It only used clock_nanosleep on Linux, but enable it for all except
PASE. Android used to be guarded against, but the reasoning was
unknown, so @akoeplinger requested to have that guard removed.
2019-08-02 23:04:50 +02:00
Aleksey Kliger (λgeek) 713df6a9c4 [netcore] Fix comments in ALC to use Mono terminology (#15998) 2019-08-02 16:31:06 -04:00
Martin Baulig 6818163383 Use MultipleConnectAsync from CoreFX. (#15972)
* Use `MultipleConnectAsync` from CoreFX.

* [csproj] Update project files
2019-08-02 15:10:03 -04:00
Alexander Köplinger 2f21664fc2 [ci] Remove the downloaded pr-files.txt 2019-08-02 21:07:06 +02:00
Zoltan Varga 0104cc6a8f [ci] Skip lanes on ppc/wasm changes. (#15995) 2019-08-02 20:56:11 +02:00
Aleksey Kliger (λgeek) 2f2f6c104b [netcore] Implement referenced assembly resolving and ALC.GetLoadContext (#15946)
* [netcore] Implement GetLoadContext

* [netcore] Implement ALC-aware referenced assembly resolution.

The process is described in
https://docs.microsoft.com/en-us/dotnet/api/system.runtime.loader.assemblyloadcontext?view=netcore-3.0#usage-in-the-runtime

We try four things until one of them returns a non-null assembly.

1. Check if the assembly is already loaded in the ALC of the requesting
assembly.
2. If the ALC is not the default ALC, invoke the Load override.
3. Try the default ALC (TPA - trusted platform assemblies) loading.
4. Invoke the Resolving event of the original ALC.

The implementation borrows some managed code from
https://github.com/dotnet/coreclr/blob/8ba2e15201361402acd0ae9710bd37d50785cdfa/src/System.Private.CoreLib/src/System/Runtime/Loader/AssemblyLoadContext.CoreCLR.cs#L84-L187

It would be nice if that code was shared between the runtimes.

* [runtime] Implement mono_assembly_name_culture_is_neutral

* [netcore] Add mono_alc_invoke_resolve_using_resolve_satellite

* [netcore] Use ResolveSatelliteAssembly for satellite assembly refs

* [netcore] Some AssemblyLoadContextTest tests now pass

System.Runtime.Loader.Tests.AssemblyLoadContextTest.PublicConstructor_Default
System.Runtime.Loader.Tests.AssemblyLoadContextTest.GetAssemblyNameTest_NullParameter
System.Runtime.Loader.Tests.AssemblyLoadContextTest.LoadFromAssemblyName_AssemblyNotFound
System.Runtime.Loader.Tests.AssemblyLoadContextTest.PublicConstructor_Theory (with isCollectible: False)
System.Runtime.Loader.Tests.AssemblyLoadContextTest.GetAssemblyNameTest_ValidAssembly
System.Runtime.Loader.Tests.AssemblyLoadContextTest.GetLoadContextTest_SystemPrivateCorelibAssembly
System.Runtime.Loader.Tests.AssemblyLoadContextTest.DefaultAssemblyLoadContext_Properties
System.Runtime.Loader.Tests.AssemblyLoadContextTest.GetAssemblyNameTest_AssemblyNotFound
System.Runtime.Loader.Tests.AssemblyLoadContextTest.GetLoadContextTest_ValidTrustedPlatformAssembly
2019-08-02 14:06:44 -04:00
Thays Grazia 0b8a0e09d3 [Debugger] Fixing test with mcs (#15985) 2019-08-02 18:48:42 +02:00
Alexander Köplinger 67aed94e2b [ci] Set apidiff/csprojdiff statuses even when we're skipping the build
They are required GitHub statuses so we need to set them.
2019-08-02 17:19:14 +02:00
Zoltan Varga 5bb4f1eea8 [ci] Skip jenkins lanes for PRs which only contain changes to the netcore/ subdir. (#15978) 2019-08-02 17:12:46 +02:00
Egor Bogatov d0aa0310ee [netcore] Use mono-netcore to run real world apps using dotnet run (#15942)
So currently we have two .NET Core:

1) `mono-repo/.dotnet` - we use it to bootstrap and build stuff (the SDK version is specified in `global.json`) - if you open `netcore/Makefile` it's referred there as `$(DOTNET)` variable.
2) `mono-repo/netcore/.dotnet` - it's just a runtime + CoreFX libs (it doesn't contain SDK stuff, templates, msbuild, etc) - we use it only to run tests because this runtime is synchronized with corefx tests in `eng/Versions.prop` file which is updated by a bot and `netcore/shared` sources for our System.Private.CoreLib also depend on that exact runtime version (the sources also updated by the bot).

This PR introduces a rule to copy mono bits to the `mono-repo/.dotnet` folder to be able to run real world apps (at your own risk) using mono-netcore (so when you run `mono-root/.dotnet/.dotnet run -c Release` for an app - even MSBuild will use mono runtime to build that app). However in theory we need exact SDK for the runtime version we have but I have no idea where and how to get one.

BTW, currently MSBuild crashes when it's powered by mono on `$(DOTNET) build` command for a hello world.
```
monoeg_assertion_message
mono_domain_assembly_preload
invoke_assembly_preload_hook
...
at <unknown> <0xffffffff>
at System.Reflection.RuntimeAssembly:GetExportedTypes <0x000a2>
at AssemblyInfoToLoadedTypes:ScanAssemblyForPublicTypes <0x0008e>
```
2019-08-02 17:06:37 +02:00
Egor Bogatov 40f28b2e63 Make System.Private.CoreLib.csproj VS friendly (#15965)
Now it's possible to open System.Private.CoreLib.csproj in an IDE and build (use Roslyn version from eng/Versions.props).
2019-08-02 10:05:22 -04:00
Thays Grazia 3a9df67a64 [debugger] Invoke method with fixed size array as an attribute of this (#15766)
Implement debugger invoke when the this has an attribute that is a fixed size array.

Fixes #15556
2019-08-02 15:47:25 +02:00
Vlad Brezae 4883b56c10 [interp] Remove unconditional branches to next instruction (#15939)
[interp] Remove unconditional branches to next instruction

These are very common, at least in debug builds. Optimize them away so it doesn't distort performance numbers.
2019-08-02 05:16:07 -04:00
Kenneth Pouncey 4ed2aadbdf [wasm][sdk] Support for Custom Linker Configuration (#15953)
- Provide extra definitions to the linker to ensure the type, methods and/or fields are not eliminated from your application.

* Use a <LinkDescription> item as follows:

```
  <ItemGroup>
    <LinkDescription Include="customlinker.xml" />
  </ItemGroup>
```

This will then be added to the `WasmLinkAssemblies` step.
2019-08-02 08:58:50 +02:00
Martin Baulig 455b051ce5 Fix compiler warnings in System.dll. (#15971)
* Merge the `System.Diagnostics/ProcessStartInfo.cs` part into the referencesource file to avoid CS0282.

* Remove duplicate `using`.

* Fix unreachable code warning that got introduced in my last PR.
2019-08-01 18:46:15 -04:00
Phil Jaenke fcfd3e54a5 Fix patch step in mdoc tests for non-Linux (#15974)
Non-Linux systems largely do not support `--binary`. Since the patch appears to be text anyway, it's probably safe to remove for all platforms.

With `patch -p0` , mdoc test now provides accurate pass/fail on FreeBSD.
2019-08-01 23:42:05 +02:00
Egor Bogatov 26f3df9881 [netcore] Minor fixes, fix GetTypeTests.GetType_EmptyString (#15933)
* Fixes https://github.com/mono/mono/issues/15023
 
* Fixes https://github.com/mono/mono/issues/15189 (were marked as "require large array
support" in dotnet/corefx by @filipnavara)

* Move InternalGetHashCode to RuntimeHelpers
2019-08-01 16:56:56 -04:00
Zoltan Varga 6f05192669 [netcore] Add a README. (#15934)
* [netcore] Add a README.
2019-08-01 16:27:40 -04:00
Jay Krell 75a99b7e84 Bump monolite version. (#15969)
It got out of sync for some reason.
2019-08-01 21:38:52 +02:00
Vlad Brezae 87ba12e13f [sgen] Fix check-nursery-pinned debug option (#15865)
* [sgen] Remove dead unpin_queue argument

* [sgen] Fix check-nursery-pinned debug option

And rename it to better reflect what is is doing now (which is to check that we don't leave any vtables tagged at the end of a collection). It used to also check that nursery objects are pinned, back when we were doing a nursery collection as part of the finishing pause of a concurrent collection.
2019-08-01 21:48:23 +03:00
Filip Navara 3c92cd8c94 [jit] Align float/double Nan/Inf to uint conversion behavior with CoreCLR/LLVM/Roslyn (#15930)
* [jit] Align float/double Nan/Inf to uint conversion behavior with CoreCLR/LLVM

* Fix float/double -> ulong conversion for edge cases

* Fix mono_fconv_u8 / mono_rconv_u8 to also return 0 for PositiveInfinity

* Enable support for platform specific implementations of OP_RCONV_U8/OP_FCONV_U8/OP_FCONV_U4

* Restore OP emulation on WASM target

* Update tests

* Disable failing tests for now

* Adjust tests

* Adjust tests

* Fix float -> uint conversion on WASM

* Make interpreter consistent with compiled code

* Blind attempt to fix WASM test leg

* Implement remaining conversions in LLVM

* Fix ARM32

* Workaround for Windows x64 FullAOT+LLVM

* Bump AOT version
2019-08-01 13:37:41 -04:00
Maxim Lipnin d72ca69693 Bump corefx (#15932)
This PR brings changes from mono/corefx#313.
Fixes #15887.
2019-08-01 18:36:29 +02:00
Filip Navara 98cb4aed20 [System.Drawing] Fix comparison in MeasureString_Wrapping_Dots test (#15958) 2019-08-01 17:09:38 +02:00
Jay Krell a7b74fc69c Support proper Windows kernel Unicode thread names. (#15919)
These do not require a debugger, can be retrieved, appear in ETW, etc.

Extracted from https://github.com/mono/mono/pull/15859.
2019-08-01 15:53:41 +02:00
Zoltan Varga 6a66472815 [wasm] Add normal GC support. (#15577)
* [llvm] Spill all ref variables to the stack on wasm, so GC stack scanning can scan all references.

* [runtime] Add an option to enable on-demand GC.
2019-08-01 15:50:28 +02:00
Vlad Brezae e3ca891235 [marshal] Free delegates with target that are passed to native code. (#15935)
* [marshal] Always use gchandles in delegate_hash_table

Makes the code easier to follow and it also fixes race from https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b with boehm.

* [marshal] Free delegates with target that are passed to native code.

For static method delegates, we have a unique delegate_trampoline that is shared among all delegates. We always keep alive the first static method delegate passed to native, by creating a normal gchandle to it and storing it in delegate_hash_table. For instance methods, each delegate will create a separate wrapper and these wrappers are never shared (which was wrongly assumed in https://github.com/mono/mono/commit/caa4a753ca8e15d43baaa01adb0f56f374b74a2b). We shuldn't keep the delegate alive and this commit reverts the behavior for delegate with instance methods introduced in the mentioned commit.

Fixes https://github.com/mono/mono/issues/15751
2019-08-01 16:06:05 +03:00
Vlad Brezae ce398411a0 [mini] Ignore conv.r.un if top of stack is already float (#15818)
Fixes https://github.com/mono/mono/issues/15794
2019-08-01 15:21:50 +03:00
Martin Baulig e92ecdc9ce Fix Socket.ConnectAsync(SocketAsyncEventArgs) behavior. (#15947)
We need to clearly distinguish between synchronous and asynchronous completions
and avoid any ambiguities and race conditions when it comes to error handling.

On synchronous completion, we must not invoke the async completion delegate.

In `System.Net.Sockets/Socket.cs`, the internal `BeginSConnect()` and `BeginMConnect()`
now return a `bool` indicating whether or not an async operation is pending.

If any error happens prior to starting the async operation, then we shall call
`SocketAsyncResult.Complete(..., true)` and return `false`.  And since this is
a synchronous completion, it's `AsyncCallback` will not be invoked.

In `referencesource/System/net/System/Net/Sockets/_MultipleConnectAsync.cs`,
I compared the entire file against the CoreFX version and then copied the
`AttemptConnection()` method from their implementation to adapt their behavior.

The difference is that on synchronous completion, the `InternalConnectCallback(null, args)`
should be invoked.
2019-08-01 12:31:25 +02:00
monojenkins 0e6965f6ba [csproj] Update project files 2019-08-01 07:45:48 +03:00
Alexis Christoforides 4abc400244 [corlib] Allow copying/moving/replacing broken symlinks
Fixes https://github.com/mono/mono/issues/14971
2019-08-01 07:45:48 +03:00
Andy Gocke 863ab55d48 Add Roslyn maestro dependency (#15944)
* Add Roslyn maestro dependency

This should bring down the Microsoft.Net.Compilers package,
which contains the Roslyn binaries needed for mono packaging.

* Add Version to Versions.props as well
2019-08-01 02:04:04 +02:00
Jay Krell 180fcfe1fd Factor copy/pasted code into mono_gstring_append_thread_name. (#15920)
Extracted from https://github.com/mono/mono/pull/15859.
2019-08-01 01:25:31 +02:00
Jay Krell 3db9635bf5 Remove error_init from native icall wrappers since managed does it.
Preferable would be native only, i.e. for interpreter and for when
it is actually unused, but ok for now.

Between removing the managed and the native, managed is preserved
in case there uses of MonoError in C# other than icall wrappers.
Keep them safe.

But again ideally move it to native and remove the managed type,
pending debugging WebAssembly failure doing so.
2019-07-31 15:49:54 -07:00
Vlad Brezae 985a1c33f2 [tests] Bump sleep for pinvoke3 test (#15937)
On some operating systems, this test can take forever because the worker starves the other threads by holding the gc_lock. Bump sleeping so it doesn't rely on OS scheduling that much and guarantees other threads can progress in reasonable amount of time.
2019-08-01 00:24:06 +03:00
Alexander Köplinger cf0e2dd972 [zlib] Fix typo in readme and remove deleted file from msvc projects 2019-07-31 22:58:21 +02:00
Jay Krell 431bb95c71 Update zlib from 1.2.5 to 1.2.11. (#15480)
Circa 2010 to circa 2017.

Retrieved with apt-get source zlib on WSL Ubuntu 18.04.2 LTS.

This matches Debian or Ubuntu (they differ by one commit, and one of them restores a dummy function for compat).

---

* [minizip] Ensure we don't ever accidentally change calling convention

Make sure we always #define ZCALLBACK to empty so we don't change
the calling convention on windows by including a header which defines
'CALLBACK'. This will keep things as cdecl as required by the c#
pinvokes.

* Update zlib readme.

* Update zlib from 1.2.5 to 1.2.11.

* [zlib] Small fix for Visual C++.
see https://github.com/madler/zlib/pull/435/files#diff-0558e84b0c24a83ca790dcb05cd26212R1087
and https://github.com/madler/zlib/pull/435/files#diff-0558e84b0c24a83ca790dcb05cd26212R1104

* Add gzguts.h to Makefiles.

* Add .h files to Visual Studio project files just for GUI purposes.
2019-07-31 22:56:34 +02:00
Zoltan Varga 1d9a87e449 Fix the netcore build. (#15941) 2019-07-31 22:04:37 +02:00
turbo 8b9033e4d2 Add O_NOATIME to OpenFlags (#8993)
O_NOATIME (Linux > 2.6.8): Do not update the file last access time (st_atime in the inode) when the file is read(2). Programs that merely index the existence of a file (backup programs e.g.) can reduce disk load using this flag.
2019-07-31 21:24:53 +02:00
Zoltan Varga 60c277cb4e [runtime] Convert more icalls to use handles. (#15862)
* [runtime] Convert Volatile to use the same approach as the netcore version, get rid of most Volatile icalls.

* [runtime] Reduce the number of handles used by construct_culture ().

* [runtime] Convert some icalls to handles.

* Fix signature of FAMNextEvent.

* [aot] Avoid aoting the gsharedvt versions of Volatile:Read<T>/Write<T>.

* Bump API snapshot submodule

* Convert Marshal:UnsafeAddrOfPinnedArrayElement to use coop handles.
2019-07-31 14:08:50 -04:00
Aleksey Kliger (λgeek) 349a3d5eef [threads] Enter GC Unsafe before pumping HP queue in unregister_thread (#15890)
If the thread info TLS key dtor runs unregister_thread from a foreign thread
and we have work in the hazard pointer queue, we need to first switch to GC
Unsafe mode because some of the free methods passed to
mono_thread_hazardous_try_free need to be coop-aware.

Also, added checked mode assertions that conc_table_free and
jit_info_table_free are called from GC Unsafe mode.

Fixes https://github.com/mono/mono/issues/15878
2019-07-31 13:30:30 -04:00
Jay Krell 499a02d6a1 Cleanup and resynchronize Thread/InternalThread fields. (#15910)
- Remove unused.
- Add missing that just happened to work.
- Synchronize size between netcore and regular. `#if` considered bad.

Extracted from https://github.com/mono/mono/pull/15859.
2019-07-31 18:30:50 +02:00
Jay Krell 9acd4c8004 mono_thread_set_name_internal change unreadable boolean parameters to enum flags. (#15921)
Extracted from https://github.com/mono/mono/pull/15859.
2019-07-31 18:30:28 +02:00
Zoltan Varga 6223e74688 [jit] Optimize the initialize of MonoError locals. (#15869)
Previously, we would emit a call to memset since the struct is large (> 100 bytes). Only the error code field needs to be initialized.
2019-07-31 14:26:24 +02:00
Calvin Buckley 1059b3e6f5 Update Haiku support (#15674)
* Build System.Native.

* Disable `pthread_mutexattr_setprotocol` use, since it returns EINVAL on Haiku.

* Disable the `strerror` cache on Haiku. On Haiku, errors begin at `0x8000_0000` like a Win32 HRESULT. This means the indexing would be on negative numbers, which is a no-no. There are a few alternative ways to keep it though: hashtable, tuple array, offset...

Unfortunately, doesn't build all the way through. Has a tendency to bomb out during building `build-unix`. Will investigate why.
2019-07-31 14:23:48 +02:00
coypoop f86d46d136 configure.ac: Use = as a test(1) operator. (#15876)
== is a bash extension
= is POSIX and equivalent
2019-07-31 14:22:26 +02:00
Zoltan Varga 4170bef2c3 [wasm] Small changes. (#15918)
* [wasm] Touch the dummy .exe files created during linking if they already exist.

* [wasm] Display longer stack traces on v8.

* [wasm] Add a --native-strip option to the packager, defaults to true.

* Add targets for separate interpreter test projects. Normalize main.exe command line arguments.
2019-07-31 14:20:40 +02:00
Jay Krell e0335d3570 Save 64 bytes from most Linux/amd64 interpreter frames. (#15926)
Probably same on all 64bit architectures. half on 32bit.

before:
```
0000000000003ce0 <interp_exec_method_full>:
    3ce0:	55                   	push   %rbp
    3ce1:	48 89 e5             	mov    %rsp,%rbp
    3ce4:	41 57                	push   %r15
    3ce6:	41 56                	push   %r14
    3ce8:	41 55                	push   %r13
    3cea:	41 54                	push   %r12
    3cec:	49 89 f4             	mov    %rsi,%r12
    3cef:	53                   	push   %rbx
    3cf0:	48 89 fb             	mov    %rdi,%rbx
    3cf3:	48 81 ec 78 01 00 00 	sub    $0x178,%rsp
```

after:
```
0000000000003ce0 <interp_exec_method_full>:
    3ce0:	55                   	push   %rbp
    3ce1:	48 89 e5             	mov    %rsp,%rbp
    3ce4:	41 57                	push   %r15
    3ce6:	41 56                	push   %r14
    3ce8:	41 55                	push   %r13
    3cea:	41 54                	push   %r12
    3cec:	49 89 f4             	mov    %rsi,%r12
    3cef:	53                   	push   %rbx
    3cf0:	48 89 fb             	mov    %rdi,%rbx
    3cf3:	48 81 ec 38 01 00 00 	sub    $0x138,%rsp
```

It was 0x158 recently.

Some frames will be larger, i.e. dynamic + pinvoke.

Classic technique:
 pointer instead of struct
 initialized to null
 alloca it when needed and not null

A more detailed analysis might enable better, such as making
the parameter optional, and/or having the callee default it to its
local storage.
2019-07-31 14:20:03 +02:00
Nikolay Sivov c8fd75734b Fix some compiler warnings for Win32/mingw64 build (#15903)
* [mono] Fix unused variable warning in mono_config_parse().

* [ligc] Fix strict prototype warning for Win32 build.

* Add some casts.
2019-07-31 14:18:53 +02:00
coypoop bed4c00e02 Remove unused libdl setting (#15884)
We test dlopen with no library and in libdl later, without the
use of this setting.

Got the hint since I'm building for a configuration which had it set wrong.
2019-07-31 14:17:34 +02:00
coypoop a65eec76be Remove unused declarations (#15907) 2019-07-31 14:16:37 +02:00
Zebediah Figura dc48d9e241 Always marshal a class as a COM interface if it has the InterfaceType attribute set. (#15671)
This fixes Game Maker Studio 2 Desktop under Wine.
2019-07-31 14:15:38 +02:00
Zebediah Figura b51c696081 [cominterop] Handle NULL pointers when marshalling native-to-managed return values. (#15906)
Fixes a bug running Rak24u with wine-mono: bugs.winehq.org/show_bug.cgi?id=47561
2019-07-31 14:14:39 +02:00
Jay Krell c37fc12ca5 Fix race condition and use less memory in mono_lookup_icall_symbol. (#14532)
It is racy because it is doing on-demand initialization, which is often racy.

The rewrite changes the result to be one pointer, atomically swapped, which implies a full barrier and data dependency, so no race.

It could also be fixed by copying and sorting an array of pairs, and one pointer to that.

However in order to save memory I instead use indirect data that is an array of uint16 pointing into the original data. That is a slight memory vs. time tradeoff.

Yes there is debugging code left, under #if, that I prefer to leave, both as evidence that I tested it, and to make it somewhat but ideally testable in future.

The code also was using an unnecessary somewhat risky but probably ok here way to compare pointers, that I fixed.

As well the result is marginally smaller and faster because it was inlining something like bubblesort, now reuses qsort.
2019-07-31 14:13:21 +02:00
Alexander Köplinger 471421023c [ci] Handle master build purging in a separate cleanup job 2019-07-31 13:32:06 +02:00
Jay Krell bd0c46a842 [Coop] Add TYPED_HANDLE_DECL. (#15909)
Extracted from #15900.
2019-07-31 12:35:59 +02:00
Bernhard Urban a6e6c72c8f [sdks] remove 32bit llvm (#15896)
[sdks] remove 32bit llvm

Also switches the MXE cross compilers for Android and WebAssembly to 64bit host.
2019-07-30 19:05:41 -04:00
Alexander Köplinger 2cfbadb0a6 [pkg] Don't keep per-commit master builds in CI
We only want the daily builds.
2019-07-31 00:34:38 +02:00
Filip Navara 60bb726eac Update CoreFX.issues.rsp (#15870)
Removed tests fixed by #13907.
Fixes #14945
Fixes #14946
Fixes #14948
2019-07-30 23:39:34 +02:00
Jay Krell ca8f155983 [Coop] Convert ves_icall_string_new_wrapper_impl. (#15901)
Static analysis prefers not even these short lived raw pointers.
Extracted from #15900.
2019-07-30 23:37:58 +02:00
Jay Krell 56082ab528 [Coop] Convert mono_string_from_byvalwstr_impl. (#15902)
Remove the raw pointer that static analysis will flag.
Extracted from #15900.
2019-07-30 23:37:09 +02:00
Zoltan Varga f8fd60b5c0 [interp] Simplify the main interpreter loop code to speed up compilation. (#15889)
* [interp] Simplify the control flow in the main loop to speed up compilation of interp.c.

Get rid of the handle_finally and main_loop labels.

* [interp] Move some rarely used locals to InterpFrame to speed up compilation and to increase the chance that the other locals end up in registers.

* Add macros for throwing exceptions to shorten the code a bit.

* Fix a previous change.
2019-07-30 16:02:27 -04:00
Larry Ewing eb68675461 Bump msbuild to xplat-master 2019-07-30 14:20:22 -05:00
Larry Ewing 11b6828420 Update nuget binary 2019-07-30 14:20:22 -05:00
Alexander Köplinger ce1653b08a [sdks] Don't keep per-commit master builds in CI
We only want the daily builds.
2019-07-30 21:14:02 +02:00
Alexander Köplinger ee636ee8d9 [sdks] Make sure python3-pip is installed for Linux Android builds 2019-07-30 20:46:11 +02:00
Alexander Köplinger 9f0d493eea [netcore] Replace dl-test-assets.py with msbuild download (#15883)
The python file was unreliable on macOS, let's see whether a simple msbuild replacement is better.

Also fixed dependencies on some of the Makefile targets so that they correctly detect when they need to run.
2019-07-30 19:27:42 +02:00
Jay Krell a8d8fb6cea Fix warnings. (#15768)
* boehm-gc.c:1923:14: warning: variable old_obj set but not used [-Wunused-but-set-variable]

* class-init.c:1720:1: warning: compare_by_interface_id defined but not used [-Wunused-function]

* class.c:1724:1: warning: compare_interface_ids defined but not used [-Wunused-function]

* icall.c:8190:13: warning: variable klass set but not used [-Wunused-but-set-variable]

* image.c:1063:20: warning: variable header set but not used [-Wunused-but-set-variable]

* metadata.c:2573:25: warning: right shift count >= width of type [-Wshift-count-overflow]

* sgen-dynarray.h:194:1: warning: dyn_array_int_is_copy defined but not used [-Wunused-function]

* sgen-mono.c:255:1: warning: emit_managed_allocater_noilgen defined but not used [-Wunused-function]
sgen-mono.c:238:1: warning: emit_nursery_check_noilgen defined but not used [-Wunused-function]

* sgen-tarjan-bridge.c:449:25: warning: right shift count >= width of type [-Wshift-count-overflow]

* threads.c:1451:11: warning: variable res set but not used [-Wunused-but-set-variable]

* w32process-unix-default.c:239:19: warning: variable min_dev set but not used [-Wunused-but-set-variable]
w32process-unix-default.c:239:10: warning: variable maj_dev set but not used [-Wunused-but-set-variable]

* w32process-unix.c:2107:25: warning: variable waitret set but not used [-Wunused-but-set-variable]

* aot-compiler.c:4180:1: warning: add_lazy_init_wrappers defined but not used [-Wunused-function]

* aot-compiler.c:7200:86: warning: format %lu expects argument of type long unsigned int, but argument 3 has type gsize {aka unsigned int} [-Wformat=]

* aot-compiler.c:10013:7: warning: unused variable index [-Wunused-variable]

* aot-compiler.c:10012:15: warning: variable method set but not used [-Wunused-but-set-variable]

* method-to-ir.c:4031:22: warning: variable tmpreg set but not used [-Wunused-but-set-variable]

* mini-runtime.c:2977:14: warning: variable invoke set but not used [-Wunused-but-set-variable]

* mono-rand.c:94:1: warning: mono_getentropy defined but not used [-Wunused-function]
2019-07-30 12:10:30 +02:00
Jay Krell f1958bacaa Remove mono_thread_get_name. (#15875)
Unused since https://github.com/mono/mono/commit/b5c0c831da328b8ab7b7283a38fa56baaa7ae380.
And not likely to come back.
2019-07-30 03:08:55 +02:00
Jay Krell 929e3135bb HANDLE_FUNCTION_RETURN_VAL must not use handles in current frame (#15873) 2019-07-30 01:39:44 +02:00
Calvin Buckley 95f8a68c56 ArgumentNullException when pointer is null for Marshal.AddRef/Release/QueryInterface (#15867)
Fixes #15853
2019-07-29 20:27:18 +02:00
dotnet-maestro[bot] a0aae297d9 Update dependencies from https://github.com/dotnet/arcade build 20190729.1 (#15881)
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19379.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19379.1
2019-07-29 20:24:40 +02:00
Jay Krell b5c0c831da Fix thread name leak in debugger. (#15858) 2019-07-29 11:31:14 +02:00
Jay Krell babcdbdcbd [Coop] Convert System.Buffer. (#15861)
And inline `mono_array_handle_length`.
And remove `inline` in .c files a little -- it only belongs in .h.
2019-07-29 11:30:16 +02:00
Zoltan Varga 967a79e5a1 [wasm] Fix the generation of emsdk_env.sh. (#15872) 2019-07-29 11:28:29 +02:00
Vlad Brezae 91b9ff3e53 [interp] Optimize ldelema (#15846)
Makes the opcode 2-3x time faster
2019-07-27 17:37:00 +03:00
Zoltan Varga 9f25324936 [WIP] [runtime] Convert WindowsIdentity.GetRoles () icall to coop. (#15849)
* [runtime] Convert WindowsIdentity.GetRoles () icall to coop.

* [runtime] Replace some Buffer icalls with Unsafe methods.

* Fix the windows build.
2019-07-27 02:27:24 -04:00
Alexander Köplinger 6a6a0f6213 [netcore] Disable new post-build.yml stage in AzDO pipeline 2019-07-27 02:54:50 +02:00
Alexander Köplinger 64be3113a3 [netcore] Add back _DotNetPublishToBlobFeed variable in AzDO pipeline 2019-07-27 02:41:41 +02:00
Alexander Köplinger e38468d0e4 [netcore] Fix condition in AzDO pipeline 2019-07-27 02:15:49 +02:00
Alexander Köplinger 4398a12286 [netcore] Fix name of empty.proj in AzDO pipeline 2019-07-27 02:07:31 +02:00
Alexander Köplinger a3fda4a3cb [netcore] Add back pushing to blob feed in AzDO pipeline 2019-07-27 02:01:20 +02:00
Alexander Köplinger 4ef62c8c46 [netcore] Set variables a bit differently 2019-07-27 00:56:25 +02:00
Alexander Köplinger 882208bbc6 [netcore] Explicitly set _PublishUsingPipelines variable to true
It shouldn't be needed when using the eng/common/templates/jobs/jobs.yml template but it is.
2019-07-27 00:34:38 +02:00
Alexander Köplinger e12a6bf5d8 [netcore] Only install Linux packages on hosted pool 2019-07-27 00:23:00 +02:00
Alexander Köplinger ae488c93e9 [netcore] Fix building of nupkg (#15855)
When https://github.com/mono/mono/commit/9010d2f290d28dc97ac276e0c0d32a2932b7550f was merged we stopped including mcs/build/config.make.
However, that file set the VERSION variable which wasn't migrated to the new netcore/config.make so the nuget package missed the version.

Additionally, System.Private.CoreLib sources were moved into the netcore folder and we now seemingly need to pass `-p:IsPackable=true` for dotnet pack to build the nuget package.
2019-07-27 00:17:31 +02:00
Alexander Köplinger dc16db91a1 [netcore] Enable nuget validation in post-build AzDO pipeline stage 2019-07-26 23:23:12 +02:00
Alexander Köplinger ccb53b88f4 [netcore] Switch back to old way of specifying poolname
Otherwise the AzDO test view contains `$(poolname)`
2019-07-26 23:18:59 +02:00
Alexander Köplinger 296d395535 [netcore] Use yaml stages in azure pipelines definition (#15838)
Arcade would like us to start using yaml stages.
2019-07-26 23:13:12 +02:00
Stephen Toub 216fad5177 Re-enable CA1802 (use consts instead of readonly statics) (#39782)
* Re-enable CA1802 (use consts instead of readonly statics)

I enabled it only for privates and internals, as we've previously violated this in public surface area, and as everything goes through API review moving forward, didn't seem worthwhile fighting it and adding suppressions in various places.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-07-26 20:08:24 +02:00
Nicholas Schell 51aa4b019a [Mono.Posix] Parameters for chown should be signed integers not unsigned (#15828)
Fixes https://github.com/mono/mono/issues/10748

The calls for chown, lchown, fchown, and fchownat (from `Mono.Unix.Native.Syscall`) are all declared with unsigned integers for the `owner` and `group` parameters. This is incorrect.

Directly from the manpage
`man 2 chown`
<pre><code>
...
       int chown(const char *pathname, uid_t owner, gid_t group);
       int fchown(int fd, uid_t owner, gid_t group);
       int lchown(const char *pathname, uid_t owner, gid_t group);

...

       int fchownat(int dirfd, const char *pathname,
                    uid_t owner, gid_t group, int flags);

...
...
       <b>If the owner or group is specified as -1</b>, then that ID is not changed.
...
</pre></code>

Emphasis added on the documentation explaining how the parameters can take the value -1, clearly not an unsigned integer. The `uid_t` struct is not an unsigned integer. I am assuming this was just an oversite on whoever re-wrote `Mono.Unix.Native.Syscall`.

The original calls from `Mono.Posix.Syscall` (now marked as Obsolete) all correctly use signed integers.
```
                [DllImport ("libc", SetLastError=true)]
                public static extern int chown (string path, int owner, int group);
                [DllImport ("libc", SetLastError=true)]
                public static extern int lchown (string path, int owner, int group);
```

Even the documentation directly from `mcs/class/Mono.Posix/Documentation/en/Mono.Unix.Native/Syscall.xml`
```
          <para>
            One of the owner or group id's
            may be left unchanged by specifying it as -1.
          </para>
```
2019-07-26 20:07:38 +02:00
Ryan Lucia 29428550c2 [netcore] Implement AssemblyLoadContext.GetLoadedAssemblies() (#15832)
* [netcore] Implement AssemblyLoadContext.GetLoadedAssemblies()

* No refonly on netcore

* Ifdef out AppDomain.GetAssemblies() on netcore

* Don't init the error

* Unnecessary ifdef
2019-07-26 18:23:43 +02:00
monojenkins 86fbf751fe [csproj] Update project files 2019-07-26 16:52:58 +02:00
Marek Safar 044982b390 [wasm] Remove assemblies we don't want to distribute 2019-07-26 16:52:58 +02:00
Bernhard Urban b4c5d68c28 [android] switch to python offset tool (#15834)
[android] switch to python offset tool

And drop C# tool.

Fixes https://github.com/mono/mono/issues/9621
2019-07-26 08:15:50 -04:00
Kenneth Pouncey ddb1f31c9e Fix http framework module that errors using SDK 2.1.801 (#15844)
- 2.1.801 seems to have a problem with:

```
						using (var uint8Buffer = Uint8Array.From(await request.Content.ReadAsByteArrayAsync ()))
```
2019-07-26 12:13:27 +02:00
Zoltan Varga 390353d48b [netcore] Add a missing make dependency. 2019-07-26 10:03:14 +02:00
unknown a809e479f3 Update dependencies from https://github.com/dotnet/core-setup build 20190725.9
- Microsoft.NETCore.App - 5.0.0-alpha1.19375.9
2019-07-26 09:10:04 +02:00
unknown 0bb362511c Update dependencies from https://github.com/dotnet/corefx build 20190725.11
- Microsoft.Private.CoreFx.NETCoreApp - 5.0.0-alpha1.19375.11
2019-07-26 09:10:04 +02:00
dotnet-maestro[bot] 08bd3e5dc1 Update dependencies from https://github.com/dotnet/arcade build 20190725.3 (#15836)
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19375.3
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19375.3
2019-07-26 02:11:51 +02:00
Alexander Köplinger ad2405761b [sdks] Report xcode11 builds as a separate GitHub status 2019-07-25 23:47:24 +02:00
Alexander Köplinger 77ac509c0b [sdks] Add special build job for building with Xcode 11 2019-07-25 23:47:24 +02:00
Aleksey Kliger (λgeek) 04517d3a39 [metadata] Fields whose types are gparams with a reference type constraint aren't blittlable. (#15761)
* [metadata] Fields whose types are gparams with a reference type constraint
aren't blittlable.
Don't try to layout the field to find out if it's blittable.
For gshared gparams, follow the blittability of the constraint.

Fixes certain recursive examples.

```
using System;

namespace TestRecursiveType
{
    class Program
    {
        static void Main(string[] args)
        {
            SubClass subC = new SubClass();
            Console.WriteLine(subC.GetTest());
        }
    }

    public struct ValueTest<U>
    {
        // When U is instantiated with T, from BaseClass, we know it'll be a
	// reference field, so we know the instantiation ValueTest<T> won't
	// be blittable.
        public readonly U value;
    }

    public abstract class BaseClass<T> where T : BaseClass<T>
    {
        public ValueTest<T> valueTest = default(ValueTest<T>);
    }

    public class SubClass : BaseClass<SubClass>
    {
        private String test = "test";

        public string GetTest()
        {
            return test;
        }
    }
}
```

Fixes https://github.com/mono/mono/issues/15760

---

The failure is happening when we are doing mono_class_setup_fields ("BaseClass<T>") which needs to decide for each field whether it is blittable or not. So therefore we are trying to decide if ValueTest<T> (that is: the ValueTest<U> field inside BaseClass<T>) is blittable or not.

So we instantiate U with T.
Now to decide whether VaueTest<T> is blittable or not, we look at every field.
So then we look at T value.
To decide if T is blittable we first check if it's a reference type.

That check is currently inadequate for generic parameters - what the PR adds is the ability to see if theres a T : class constraint or a T : C constraint - where C is some class. As soon as we know that T's constraint will force it to be a reference type we can definitely say that T won't be blittable without having to initialize C, at all.

Previously, Mono would see that T is some kind of type for which it couldn't definitively decide that it's a reference type and it would call: mono_class_setup_fields (field_class) which would then try to setup the fields of the parent class BaseClass<T>. And that would hit the recursion check.
2019-07-25 17:31:48 -04:00
Aleksey Kliger (λgeek) 090dffb9a0 [tests] Return non-zero exit code from MERP crash test on failure (#15804)
If any of the stress tests fail, return a non-zero exit code from the
supervising process.

Fixes #15781
2019-07-25 17:20:12 +02:00
Jay Krell ee34e2b122 [jit] [x86] Cleanup "_body" and finish conversion to x86_byte. (#15628) 2019-07-25 09:18:20 -04:00
Alexander Köplinger 6a196d6de8 [ci] Disable boehm on FreeBSD
It seems to cause issues and we don't want to spend time on boehm-related problems.
2019-07-25 01:38:00 +02:00
Cassio de Vargas Oliveira 8f0e9887b0 [System.ServiceModel] Add PUT, DELETE and OPTIONS handling to HttpReplyChannel (#15746)
Fixes #15740.
2019-07-25 01:09:41 +02:00
Calvin Buckley 68849282b3 Add community-supported FreeBSD CI to README (#15821) 2019-07-25 01:04:06 +02:00
Jay Krell 8cc7226834 ld configure: Add another dash to export-dynamic. (#15717)
GNU ld accept single or double dash, means the same thing.
AIX ld warns on single dash, errors on double dash, in autoconf, and then errors on single dash later.
We want the double dash so autoconf rejects it on AIX.
2019-07-25 01:02:46 +02:00
Filip Navara 30ce79f3f9 [netcore] Update CoreFX.issues.rsp (#15816)
Remove exclusions fixed by #14967
2019-07-24 22:04:56 +02:00
Aleksey Kliger (λgeek) 784b350c40 [cxx][llvm] Fix C++ build (#15807)
* [cxx] Fix LLVMModuleFlags var initialization

* [cxx] Fix C++ errors in mini-llvm.c
2019-07-24 12:37:57 -04:00
Jay Krell 9e78ce94d5 Remove unneeded auto cast with codeman. (#15792)
The uses are all void* or casted.
2019-07-24 17:31:26 +02:00
Stephen Toub 23cbdeba1b Re-enable CA1825 (zero-length array allocations) (dotnet/corefx#39708)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 17:27:08 +02:00
Vlad Brezae 4f84f21cbc [sre] Don't throw random exception when runtime shuts down (#15798)
Added here https://github.com/mono/mono/commit/89d1455a80ef`
2019-07-24 18:17:09 +03:00
Alexander Köplinger 6dff5fc813 Remove traces of w32process-win32-internals.h which was deleted by previous commit 2019-07-24 16:52:37 +02:00
Jay Krell f68c52c089 [Coop] Convert System.Diagnostics.{FileVersionInfo,Process}. (#15789) 2019-07-24 16:50:39 +02:00
Alexander Köplinger 57a6ebbf1a [netcore] Run tests only on public builds (#15812) 2019-07-24 15:26:48 +02:00
Vlad Brezae 71630cce75 [sre] Make creation of dynamic method synchronized
Before, reflection_create_dynamic_method could end up being called by two threads simultaneously. At the end of the method the ilgen field of the MonoReflectionDynamicMethod object is cleared, to enable its collection since it is no longer needed. The problem with this is that it can confuse the other thread compiling the dynamic method, making it see that there is no body.

We could either bother making this clearing work with multiple threads (and I'm not 100% convinced that the rest of the code is thread safe), disable the clearing of ilgen (which would make us use more memory) or go with the simple approach of locking on this DynamicMethod so only one thread is compiling it at one time.
2019-07-24 14:31:51 +02:00
Steve Pfister 2238c9dc5f Bump mono/corefx@eb5ba09 (#15799) 2019-07-24 07:50:52 -04:00
Alexander Köplinger 5f1da4d454 [ci] Add extra CI_TAGS for freebsd 2019-07-24 12:33:08 +02:00
Filip Navara 24bffd544a Update CoreFX.issues.rsp 2019-07-24 11:02:42 +02:00
Egor Bogatov 122494330d [netcore] Allow different enums as return type in CreateDelega… (#15787)
* Relaxed check for return type in CreateDelegate

* fix build errors
2019-07-24 09:50:29 +02:00
Marek Safar 2a01e1b125 [netcore] Update roslyn dependency 2019-07-24 09:45:03 +02:00
Jan Kotas 047a961ecd Fix build break in TimerQueue.Portable.cs
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Sung Yoon Whang 3468ef668f Fix first value of counter payload being skewed (dotnet/coreclr#25739)
* Fix issue 25709

* rename

* Fix regression test

* cleanup

* Code review feedback

* set maxincrement to 3

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub f8d61819b8 Make all event types nullable (dotnet/coreclr#25752)
Late-breaking design change: all events should be declared with nullable delegate types.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub 4ec5b8de98 Remove now unnecessary !s / TODO-NULLABLE comments (dotnet/coreclr#25749)
* Remove !s and TODO-NULLABLE comments for [DoesNotReturn]

* Remove !s and TODO-NULLABLE comments for [NotNullIfNotNull]

* Remove !s and TODO-NULLABLE comments for writes via Interlocked.CompareExchange

* Remove !s and TODO-NULLABLE comments for Debug.Assert on fields

* Update/add several TODO-NULLABLE comments

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Santiago Fernandez Madero 0a683afa9a Fix nullability warnings from compiler update
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Jan Kotas 176d523060 Delete unnecessary method (dotnet/coreclr#25661)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub ef3cbe9c56 Disable debugger evaluation of ValueTask<T>.Result (dotnet/coreclr#25727)
If the debugger evaluates a `ValueTask<T>`'s `Result`, that counts as the "you should only consume a `ValueTask<T>`" once, and ends up breaking / hanging / throwing exceptions and other bad stuff while stepping through code in the debugger.

This commit addresses that in two ways:
1. Adds `[DebuggerBrowsable(Never)]` to `Result` to prevent it from showing up in debugger views.
2. Adds a NotifyOfCrossThreadDependency call to its ToString.  This prevents the debugger from using ToString to show an implicit representation of the instance, and it forces the developer explicitly trying to access ToString (e.g. in the watch window) to click a button acknowleding the impact.

(Post 3.0, we should consider removing the `ValueTask<T>.ToString()` override altogether.)

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Jan Kotas 927d8c322d Fix use of uninitialized variables in managed decimal implementation (#25674)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub c465ad1e80 Remove stale nullable !s and pragmas (dotnet/coreclr#25640)
Also update TODO-NULLABLE comments to be more specific where appropriate.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub 089743f733 Add AggressiveInlining to Single.GetHashCode (dotnet/coreclr#25633)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub 137c333cda Fix Timer argument exception parameter names (dotnet/coreclr#25617)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Stephen Toub 9d64e1ef74 Fix regression in Timers with long timeouts (dotnet/coreclr#25604)
* Fix regression in Timers with long timeouts

Early in .NET Core 3.0, we added an important optimization to significantly reduce lock contention and CPU overheads in situations where lots of timers were firing.  The optimization worked by splitting the set of timers into those expected to fire in the very near future and then everything else.  However, for really long timers (e.g. > 24 days), due to integer overflows we can accidentally put a long timer onto the short list and cause a timer that shouldn’t fire for days to instead fire in milliseconds.  This is not the first such bug we’ve had in Timer, and there is a fair amount of complicated casting (that we sometimes get wrong, obviously) in order to keep the tick count usage within the Int32 domain.  This PR addresses the problem (and hopefully others in the future) by switching over to using Int64.  We’re already paying to get 64-bit tick counts on both Windows and Unix, so this just removes truncation that was being performed, does a little more addition/comparison with 64-bit values instead of with 32-bit, and stores an Int64 instead of Int32 in each TimerQueueTimer.  On 64-bit, this has a 0 increase in memory consumption, as it simply ends up utilizing padding space that was previously in TimerQueueTimer.  On 32-bit, it increases memory allocation when creating a Timer by 4 bytes from 80 to 84 (still 3 allocations), but I believe it’s the right trade-off for reduced complexity and bug likelihood.

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-07-24 09:45:03 +02:00
Alexander Köplinger 70c72891f2 [netcore] Update Version.Details.xml
The earlier PR didn't update this file which is used by arcade for tracking.
2019-07-24 08:39:35 +02:00
Aleksey Kliger (λgeek) 7467ce4369 [netcore] Implement AssemblyLoadContext.InternalLoad (#15753)
Fixes https://github.com/mono/mono/issues/14830
2019-07-23 19:09:27 -04:00
Filip Navara 1551fa7951 [netcore] Bump CoreFX version to 5.0.0 (#15795)
* Bump CoreLib version

* Workaround to get dotnet to find dependencies

* Compile CoreLib with nullablePublicOnly
2019-07-24 00:53:30 +02:00
Zoltan Varga 69d179cba2 Avoid creating handles in a loop. 2019-07-23 12:55:55 -05:00
Zoltan Varga 4d1a6358d0 [runtime] Convert AssemblyBuilder.basic_init to handles using a gc handle to pin the object. 2019-07-23 12:55:55 -05:00
Zoltan Varga 063c3ed7f9 [runtime] Mark a few icalls as NOHANDLES. 2019-07-23 12:55:55 -05:00
Zoltan Varga 7b384f710c [runtime] Convert Environment icalls to handles. 2019-07-23 12:55:55 -05:00
Zoltan Varga fe015de078 Fix the netcore build. 2019-07-23 12:55:55 -05:00
Zoltan Varga 5ca052fca2 [runtime] Convert ValueType icalls to handles. 2019-07-23 12:55:55 -05:00
Larry Ewing 84af927e7d [wasm] clean up the clean targets 2019-07-23 12:16:22 -04:00
Zoltan Varga a9404577e6 [runtime] Convert locale code to use handles. (#15774)
* [runtime] Convert locale code to use handles.

* Sort a list of defines.

* Remove a do-while.

* Fix c++-ism.

* Use mono_string_handle_to_utf8 (). Update comments.
2019-07-23 11:33:20 -04:00
Jo Shields ce6c02e903 Forcibly clean working directory on every AzDO build (#15801) 2019-07-23 11:05:18 -04:00
Aleksey Kliger (λgeek) 148891b515 [cxx][netcore] Fix C++ build (#15786)
assembly.c `exact_sn_match` uses bitwise-or on MonoAssemblyNameEqFlags
2019-07-23 11:42:51 +02:00
Zebediah Figura 1854e439e5 [corlib] Foward System.Type.GetTypeFromProgID() to the correct function (#15693)
Fixes a stack overflow when running Staxel under wine-mono.
2019-07-23 11:17:28 +02:00
Alexander Köplinger 3e7f130739 [ci] Make sure global variables are really global
Using "def" in groovy makes it local to the run() method.
2019-07-23 10:27:53 +02:00
Kenneth Pouncey d46fa9b1e3 Add target for the browser threads tests run 2019-07-22 20:07:34 -04:00
Kenneth Pouncey 7cd7f5f9e1 Add running the browser tests for threads. 2019-07-22 20:07:34 -04:00
Ryan Lucia beb5553398 [netcore] Make assembly name parsing and comparison case-insensitive (#15522)
* Re-enable test

* Parse assembly info case-insensitively

* Lowercase the parsed public key token

* Case-insensitive comparison of public key tokens

* Fix string copy
2019-07-22 17:23:26 +02:00
Jo Shields 548fe0e0c1 Fix build on Autoconf 2.63 (#15636)
* Fix build on Autoconf 2.63

This was broken in e7f9dbe73c, because
AS_VAR_APPEND does not exist until v2.63b (?!)

* Work around autoconf for Mac not being fit for purpose

The correct way to do this is with AC_DEFUN in a special m4 file

But this dies on Mac, because reasons.

See also https://github.com/jedisct1/libsodium/issues/217
2019-07-22 17:20:47 +02:00
Maxim Lipnin ffe5446a40 Fix ambiguous time at the end of DST period on Windows 2019-07-22 17:12:46 +02:00
Ryan Lucia 54f608d08d Miscellaneous loader-related fixes (#15765)
* Clarify contents of list

* Fix mono_assembly_get_alc

* Remove pointless ifdef

* Prepend to domain_assemblies

* Revert "Prepend to domain_assemblies"

This reverts commit dbe6e47b80470fa13d22d97d32721d18c4764597.

* Ditch another ifdef
2019-07-22 10:58:12 -04:00
Vlad Brezae 009b0dd4c3 [interp] Enable inlining of object constructors (#15665)
* [interp] Split code for obj / vt newobj instruction

* [interp] Enable inlining of object constructors

If the ctor was inlined, the newobj instruction receives a max ushort as the data item for the index, so it doesn't execute the ctor in this case. Also, when inlining, we need to push on the stack the allocated object, once as argument for the ctor and the second needs to remain on the stack as the result of the IL newobj instruction.
2019-07-22 17:03:26 +03:00
Ryan Lucia 3c2813deb7 Special-case arrays of interfaces/generic parameters in IsAssignableFrom (#15749)
* Formatting fix

* Special-case arrays of interfaces/generic parameters

* Make more generic, fix typo

* Consider class constraint and add more elucidating comment

* Fix for potential crash

* Add test

* Fix test
2019-07-22 09:56:48 -04:00
Alexander Köplinger f497893a84 Add more tests to tarball with EXTRA_DIST 2019-07-22 15:23:09 +02:00
Aleksey Kliger (λgeek) 4138f096f8 [netcore] Re-enable AssemblyGetForwardedTypesLoadFailure (#15763)
Was fixed by https://github.com/mono/mono/pull/15752
2019-07-22 12:52:17 +02:00
Kenneth Pouncey e42d338e8f Add threads support to the SDK runtime generation.
- To enable the build with threads set `<EnableMonoWasmThreads>true</EnableMonoWasmThreads>`
   - default is false.
2019-07-19 17:33:08 -04:00
Kenneth Pouncey 98d8612b6b Fix packaging of the SDK when the runtime for threads does not exist
Closes https://github.com/mono/mono/issues/15734
2019-07-19 17:33:08 -04:00
Kenneth Pouncey eddd299c72 Add Signing.props file to common directory to be referenced from a common place
- Add the `Signing.props` to the `Directory.Build.props`
2019-07-19 17:33:08 -04:00
Maxim Lipnin e8d7f7c004 [netcore] Make System.Linq.Expressions.Tests.BindTests.GlobalField Pass (#15658)
`Expression.Bind` should throw `ArgumentException` if called with `FieldInfo` for a global field. The exception is thrown from validation of the (null) `DeclaringType` (see https://github.com/dotnet/corefx/pull/15318).
Mono returns `"<Module>"` in this case and doesn't throw so the test fails. 

Fixes #14917 .
2019-07-19 23:21:22 +02:00
Alexander Köplinger d159a38c47 [ci] Clean working directory after build in package and sdks build 2019-07-19 22:59:06 +02:00
Alexander Köplinger 1ab23207b5 [ci] Don't use method which is not allowed in Groovy sandbox 2019-07-19 22:52:01 +02:00
Alexander Köplinger 0c39f1f537 [ci] Fix OSX package builds to only be timer-triggered on master 2019-07-19 22:47:05 +02:00
Alexander Köplinger 6407d27000 [packaging] Don't use dl.xamarin.com for gtksharp package anymore 2019-07-19 22:16:34 +02:00
Alexander Köplinger 7bcba4e9bc [ci] Remove -f option from genhtml during code coverage
It needs some cpan modules to be installed which is a pain on OSX.
2019-07-19 22:10:38 +02:00
Alexander Köplinger 4909cf320f [ci] Run OSX package builds on master only once per day 2019-07-19 21:10:20 +02:00
Jay Krell 3e432663cd Remove mono_arch_free_jit_tls_data. (#15662) 2019-07-19 14:40:54 -04:00
Alexander Köplinger 49ed255707 [sdks] Calculate sha1 of sdks archive and upload it
This was requested by the XA team.
2019-07-19 20:05:48 +02:00
Ryan Lucia d685b72cb2 [netcore] Implement Thread.CurrentOSThreadId (#15748)
* [netcore] Implement Thread.CurrentOSThreadId

* Add comment explaining differences with mono_native_thread_id_get

* Add haiku implementation

* Attempt to fix linux builds

* Add MonoError parameter to icall
2019-07-19 13:22:50 -04:00
Sebastien Lebreton ec3cb3eb4e Fix TypeMirror.CSharpName computation 2019-07-19 10:42:00 -04:00
Kenneth Pouncey b85c584c0f WebAssembly.Core.Array.Pop should return null if the WebAssembly.Core.Array is empty instead of error. 2019-07-19 10:32:07 -04:00
Alexander Köplinger 3ff64f1efb [metadata] Fix name of header file
The file added as part of d5b374bb51 was actually called loaded-images-internals.h
2019-07-19 15:56:12 +02:00
Alexander Köplinger 7784f20a26 [ci] Don't require master node for reportGitHubStatus method
In some jobs we use throttle() logic which would mean only one job can execute on master, blocking other jobs in the meantime.

We don't really care which node executes this so let Jenkins pick one.
2019-07-19 14:50:10 +02:00
Egor Bogatov 8b2141a741 [netcore] Fix CI failures (#15745) 2019-07-19 14:06:04 +02:00
Alexander Köplinger a543958765 [ci] Add python3-pip package to wasm sdks build 2019-07-19 13:49:07 +02:00
Bernhard Urban 3f7d33ce85 [ios] Switch to python offset tool (#15744)
The C# version doesn't work on 64bit. It would require to update its dependencies, but Zoltan already added a Python version for WebAssembly which works fine.

Some notes:
* changed the semantics of `--xcode-path` in order to match the output of `xcode-select -p`
* switched to python3/pip3
* removed hack for arm64_32, it works properly now because the Python version uses stock `clang`
* removed some leftover usages of `XCODE32_DIR`.

Manually verified by comparing output with the C# version.

This will unblock the iOS team when building on Catalina/Xcode11, as they can't consume SDK archives there. See https://github.com/xamarin/xamarin-macios/pull/6603#issuecomment-512803082

Contributes to https://github.com/mono/mono/issues/9621
2019-07-19 13:48:02 +02:00
Aleksey Kliger (λgeek) 944cc96cac [netcore] Pass ALC in second loader request in load_reference_by_aname_default_asmctx (#15752)
Fixes crash in System.Reflection.Tests netcore testsuite in
System.Reflection.Tests.AssemblyNetCoreAppTests.AssemblyGetForwardedTypesLoadFailure
2019-07-19 07:44:07 -04:00
Alexander Köplinger 6ab8aff5dd [sdks] Add z option to tar
It's required on linux.
2019-07-19 12:49:38 +02:00
Alexander Köplinger a02f3aaf4b [sdks] Add linker sources to Android linux archive 2019-07-19 11:32:38 +02:00
Alexander Köplinger 349da3387c [sdks] Add third party notices and licenses to sdks archives (#15741)
It takes all THIRD-PARTY-NOTICES.TXT and LICENSE files from the repo.
2019-07-19 11:07:39 +02:00
Vlad Brezae c449308f8b [glib] Use clock_nanosleep if available (#15443)
According to the manual, it is recommended to use clock_nanosleep instead of nanosleep when the sleep can be repeatedly interrupted by signals and resumed. The remaining sleep interval was greater than the original timespec.

Fixes random timeouts of finalizer-abort.exe
2019-07-19 10:21:21 +03:00
Aleksey Kliger (λgeek) a37ad98b47 [loader] Pass ALC to mono_domain_loaded_internal, and then to domain search hook (#15743)
* Add a temporary mono_domain_ambient_alc function

Just use this to mark places where we need to pass through an ALC

* [runtime] Mark mono_assembly_loaded_full external only.

Runtime should use mono_assembly_loaded_internal.

* [loader] Add mono_install_assembly_search_hook_v2

Use it to install mono_domain_assembly_search and
mono_domain_assembly_postload_search.

The principal difference is that we can pass an assembly load context to the
hook.  But also we cleanup the "requesting assembly" hack for
mono_domain_assembly_postload_search.

* [domain] Comments and mono_assembly_get_alc ()

* [debugger] Comment on suspicious loader lock
2019-07-19 00:01:56 -04:00
Larry Ewing 34be988e33 Merge pull request #15704 from lewing/bump-msbuild-nuget-5.2
Bump msbuild to track xplat-master
2019-07-18 20:45:02 -04:00
Andy Hanson 25e79664b9 Return HardLimitBytes from GCMemoryInfo.TotalAvailableMemoryBytes (#25437)
* Add property HardLimitBytes to GCMemoryInfo

This adds a new property HardLimitBytes.

Unlike TotalAvailableMemoryBytes,
this will reflect an explicitly set COMPLUS_GCHeapHardLimit.

It will also reflect the fraction of a container's size that we use,
where TotalAvailableMemoryBytes is the total container size.

Normally, though, it is equal to TotalAvailableMemoryBytes.

Fix #38821

* Remove HardLimitBytes; have TotalAvailableMemoryBytes take on its behavior

* Fix typos

* Separate total_physical_mem and heap_hard_limit
so we can compute highMemoryLoadThresholdBytes and memoryLoadBytes

* Do more work in gc.cpp instead of Gc.cs

* Consistently end names in "Bytes"

Signed-off-by: dotnet-bot <[email protected]>
2019-07-18 19:10:21 -04:00
Aleksey Kliger (λgeek) 3f55bfb5cc [mini] Runtime check if boxing is needed for DIM constrained calls (#15729)
This is to handle the following example.  The issue is that in Check() we have
a constrained call where the called method is IAdder`1<!!U>::PlusPlus() which
is a default interface method, and we need to determine at runtime whether to
box the this argument.

```
using System;

public interface IAdder<T> {
	int Add (int x);

	int PlusPlus () {
		return Add (1);
	}
}

interface IGen3<T> { }

struct Adder<T> : IAdder<IGen3<T>> {
	int _field;
	public int Add (int x) {
		_field = x + _field;
		return _field;
	}
}

public class P {
	public static int Check<T, U>(T t) where T : IAdder<U> {
		return t.PlusPlus () + t.PlusPlus ();
	}

	public static void Main () {
		var x = new Adder<object> ();
		int y = Check<Adder<object>, IGen3<object>> (x);
		Console.WriteLine ("expect 2, got {0}", y);
	}
}
```
2019-07-18 16:46:55 -04:00
imhameed 561b400baa [interp] Use correctly-sized writes and reads in Unsafe.AddByteOffset and Unsafe.ByteOffset (#15677)
[interp] Use correctly-sized writes and reads in Unsafe.AddByteOffset and Unsafe.ByteOffset

The truncated store to the top of the stack in`MINT_INTRINS_UNSAFE_BYTE_OFFSET` could cause the upper bytes of the address used as the first argument to `ByteOffset` to linger, yielding inaccurate (and usually excessively large) offsets on systems where `sizeof(void *) > sizeof(guint32)`; this made
`System.Text.Unicode.Utf8Utility.GetPointerToFirstInvalidByte` return nonsense results.

The truncation in `MINT_INTRINS_UNSAFE_ADD_BYTE_OFFSET` hasn't, as far as I know, caused any test failures. But `AddByteOffset` takes an `IntPtr` as its second argument, not an `int`.

Fixes #14847.
2019-07-18 14:16:46 -04:00
Maxim Lipnin ab4eca1785 [netcore] Clean up rsp-file (#15700) 2019-07-18 18:34:10 +02:00
Egor Bogatov e6d2091a24 [netcore] Revise items in CoreFX.issues.rsp (#15664) 2019-07-18 18:33:38 +02:00
Aleksey Kliger (λgeek) d5b374bb51 [loader] Add MonoAssemblyLoadContext; stop sharing MonoImages on netcore (#15527)
Implement a couple of things:

1. Add a `MonoAssemblyLoadContext` opaque struct
2. On netcore Mono, use it to keep track of  which `MonoImage`s are loaded.  (Addresses part of #13891) 

In a bit more detail:

- Add a `MonoLoadedImages` struct to hold the hash tables that we use for checking if a `MonoImage` is already loaded
- Add an opaque `MonoAssemblyLoadContext` typedef
- (Framework Mono) Add a `get_global_loaded_images()` function to get the shared `MonoLoadedImages`
- (netcore Mono) Add a `MonoAssemblyLoadContext` struct that owns a `MonoLoadedImages` and add a `default_alc` member to `MonoDomain
- Pass a `MonoAssemblyLoadContext*` to most `MonoImage`-loading functions - on Framework Mono its usually NULL and we fall back to the global loaded images, on netcore Mono it's usually `domain->default_alc->loaded_images`
- (netcore Mono) Create a default Assembly Load Context when a `MonoDomain` is created
- Add a `MonoAssemblyLoadContext*` field to `MonoAssemblyLoadRequest` and pass an ALC in a few places.

---

Remaining work:

(netcore Mono) There are still places where a `MonoAssemblyLoadRequest` or a `MonoAssemblyOpenRequest` has a `NULL` ALC - those will need to be tracked down and fixed.

(netcore Mono) Eventually `MonoAssemblyLoadContext` should keep track of the loaded `MonoAssembly`s, not just images.  One goal on netcore Mono is to ifdef out `MonoDomain:domain_assemblies`.


---



* [loader] Add a MonoLoadedImages struct to own the MonoImage sharing

   Initially, there's just a single global set of loaded images.

   Once we have AssemblyLoadContext support, each ALC will have a separate set.

   NOTE:
     - Some of the existing Mono API is difficult to support with this design
       since it assumes each MonoImage is loaded at most once.

* [loader] Add a MonoAssemblyLoadContext member to MonoImage for netcore

* [loader] Unregister image from its owner when closing

* [loader] Pass MonoLoadedImages to register_image

* [domain] Add mono_domain_default_alc and mono_alc_get_loaded_images

* [loader] Add an AssemblyLoadContext argument to mono_image_loaded_internal

* [metadata] Add mono_image_get_alc ()

* [loader] Pass MonoAssemblyLoadContext to mono_image_open_from_data_internal

* [loader] Pass MonoAssemblyLoadContext to mono_image_open_from_module_handle

* [loaded] Pass MonoLoadedImages to mono_image_open_a_lot_parameterized

* [loader] Pass MonoAssemblyLoadContext to mono_image_open_a_lot

* [runtime] Mark mono_image_open_full external only

   Runtime should use mono_image_open_a_lot.

* [runtime] Mark mono_image_loaded_by_guid external only

   Also mark mono_image_loaded_by_guid_full external only

   Runtime will need some (as yet unwritten) mechanism that takes a MonoAssemblyLoadContext as an argument.

* [domain] Create a default ALC for each MonoDomain

* [netcore] Pass default ALC in mono_core_preload_hook

   If the assembly name matches a trusted platform assembly, open it in the default ALC of the root domain

* [mini] Open main assembly in default ALC in the root domain

* [netcore] Don't define get_global_loaded_images()

   Every assembly load request should come from an assembly load context.  Each ALC has a set of loaded images.  So we should never need a global list.

   With this PR, we no longer share MonoImages across distinct ALCs.  So in principle, each ALC can open the same assembly but resolve its references in different ways.

   Fixes part of https://github.com/mono/mono/issues/13891

* [runtime] Mark mono_pe_file_open external only

* [loader] Pass MonoAssemblyLoadContext to mono_image_open_raw  and mono_image_open_metadata_only

* [loader] Pass MonoAssemblyLoadContext to do_mono_image_open and set MonoImage:alc

* [loader] Add version 2 of assembly preload hook

   Pass MonoAssemblyLoadContext through the preload hook.  This is used in mono_assembly_request_byname_nosearch which, in turn, is called by load_reference_by_aname_default_asmctx which is used to resolve references from one assembly to another.

* [netcore] Pass native ALC ptr to AssemblyLoadContext.InternalLoadFile

* [netcore] Implement AssemblyLoadContext.InternalInitializeNativeALC

   also implement mono_domain_create_individual_alc ()

* [netcore] Pass ALC to System.Reflection.Assembly.InternalLoad

   Pass the domain default ALC to mono_assembly_load ()

   Also pass MonoAssemblyLoadContext to mono_assembly_load_full_gac_base_default ()

* [loader] Implement MONO_LOADED_IMAGES_ASSEMBLY properly

   Store the MonoLoadedImages for netmodules in the MonoImage for the main image of an assembly.  This is also used by SRE - the dummy module of the AssemblyBuilder has the loaded_images that contains the ModuleBuilders.

* [loader] Use global_loaded_images for netmodules with framework Mono

   The new stuff is only for ALC support in netcore

* [loader] Drop netmodule support in MonoAssemblyLoadContext

   netmodules are not supported in netcore

* [loader] Spin out image hashing out of image.c

   Move MonoLoadedImages support code to loaded-images.c loaded-images-global.c and loaded-images-netcore.c

Also spin out assembly-load-context.c for MonoAssemblyLoadContext functions

* Move ALC icall to assembly-load-context.c

* formatting fix

* [msvc] Add new files to .filters

* update comments

* [image] Change mono_loaded_images_remove_image to decrement the refcount

Decrement the refcount and remove the image from the loaded images hashes in
one place.  Change mono_loaded_images_remove_image to return TRUE if the rest
of image unloading should proceed or not.

* [loader] Drop unused argument to mono_alc_init
2019-07-17 17:35:15 -04:00
Jay Krell e2ce0a16a6 Fix leaks in mono_lookup_internal_call_full_with_flags, for lack of destructors. (#15659)
* Fix leaks in mono_lookup_internal_call_full_with_flags, for lack of C++ destructors.

* Initialize `sigstart` also.
2019-07-17 12:09:18 -07:00
Jay Krell b08aa71948 configure/makefile To support updating zlib to 1.2.11. (#15697) 2019-07-17 12:07:56 -07:00
Ryan Lucia 86e22c4a7f Print two bytes at a time in mono_pe_file_map conversion error (#15523)
* Print two bytes at a time

* Iterate over gunichar2
2019-07-17 11:49:30 -07:00
Alexander Köplinger c62473114a Add com.apple.security.cs.allow-unsigned-executable-memory exception to hardened runtime entitlements (#15721)
This avoids the issue from https://github.com/mono/mono/issues/13445 when hardened runtime is enabled.
2019-07-17 20:48:06 +02:00
Larry Ewing 36b9e006f3 Merge pull request #15713 from lewing/sdk-disable-gac
[ios][android] disable gac in ios and android sdk builds
2019-07-17 09:09:36 -04:00
Filip Navara 9af45e09db [netcore] Clear system last error before P/Invoke calls with SetLastError=true (#15703)
Fixes #15691
2019-07-17 14:44:50 +02:00
Alexander Köplinger 0509ceccb6 Revert "[ci] Remove separate utils.groovy file" and run step on master instead
This reverts commit d732cef5fc.

The github commit status setter step requires a node context so we now just use on on master instead.
2019-07-17 14:26:24 +02:00
Egor Bogatov 4febac951e [bcl] Sync recent NS2.1 changes (#15696)
1) Copied changes from corefx for System.Data (also https://github.com/mono/corefx/pull/310)
2) New Tracing api is left empty (because the current API is empty too)
3) New AES types throw PNSE

Bump netstandard.dll AssemblyVersion to 2.1 (fixes https://github.com/mono/mono/issues/13412)
2019-07-17 13:08:29 +02:00
monojenkins b61a4368c0 [bcl] Update BCL Linked Size 2019-07-17 06:42:24 -04:00
Alexander Köplinger ba1c7da0b9 [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools profile (#15710)
* [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools profile

Requested by the XA team.

* [csproj] Update project files
2019-07-17 11:19:04 +02:00
Alexander Köplinger d732cef5fc [ci] Remove separate utils.groovy file
For some reason when using a separate file and loading the function
from there Jenkins still requires us to have a node() context when
calling the function which is not what we want.

(cherry picked from commit d410e07d8b)
2019-07-17 10:57:38 +02:00
Larry Ewing 3a075b0b1e [wasm] disable gac in wasm runtime (#15686)
* [wasm] disable gac in wasm runtime

* [wasm] disable gac wasm.mk not configure.ac
2019-07-17 04:56:42 +02:00
Zoltan Varga 553fee6f03 [runtime] Fix the error message for exceptions created from MonoError. (#15705)
Fixes https://github.com/mono/mono/issues/15692.
2019-07-16 21:09:39 -04:00
Alexander Köplinger 39472e06a3 [ci] Wait for signing outside of a node() block
We don't need to block the machine just for waiting on signing to finish.
2019-07-17 01:01:42 +02:00
Larry Ewing 6727d1a39b [ios][android] disable gac in ios and android sdk builds 2019-07-16 18:22:26 -04:00
Zoltan Varga 978653633b [interp] Make the code GC safe on wasm. (#15695)
* [interp] Pass the vtable to get_virtual_method () to reduce the number of objrefs being passed around.

* No longer a prototype.

* [interp] Make the code GC safe on wasm.

* Use frame->o only on WASM.
2019-07-16 17:49:33 -04:00
Jay Krell 43f1a26804 Fix use of freed memory. (#15660)
Fix use of freed memory.
Move the free later, on the assumption that the free itself doesn't use memory now freed before it.
Found with Valgrind.
2019-07-16 13:16:52 -07:00
Alexander Köplinger 7480830b36 [ci] Bump timeout for .pkg signing job
In preparation for notarizing our .pkg with Apple we need to bump the timeout
since it can take up to an hour for Apple's server to return a result.
2019-07-16 21:32:13 +02:00
Aleksey Kliger (λgeek) d8a679375d [netcore] Use the strict loader (#15668)
Mono historically was quite lax in how it resolved assembly references - it
would pick the first assembly with a matching simple name.  The "strict"
mode (available in framework Mono with `--assembly-loader=strict`) follows the
spec behavior: the version and public key token of the candidate assembly must
match what is requested.
2019-07-16 20:30:16 +02:00
Filip Navara d2f7851c92 [netcore] Ignore public key when matching assemblies (#15678)
Related to #15669
2019-07-16 11:05:26 -07:00
Larry Ewing ee2b1089ff Bump msbuild to track xplat-master 2019-07-16 10:43:54 -04:00
Alexander Köplinger 709ba3faa5 Revert "Update dependencies from dotnet/corefx"
This reverts commit 4e513f802a.

It broke a few netcore tests.
2019-07-16 16:31:50 +02:00
Alexander Köplinger cef723ca9c [sdks] Use Azul Zulu OpenJDK for Android Linux build, since Debian 10 has no Java 8 2019-07-15 23:25:26 +02:00
Alexis Christoforides 532605f533 [System] Keep events from firing from a disposed 'DefaultWatcher' FileSystemWatcher
Fixes https://github.com/mono/mono/issues/8747

Also try to stop other implementations from dispatching events past Dispose(), without guaranteeing it
2019-07-15 23:25:32 +03:00
Alexander Köplinger 4b6f49bb06 [sdks] Update android linux builds to Debian 10
This was suggested by XA folks to fix the weird mingw llvm issues we've been seeing.
2019-07-15 21:36:04 +02:00
Larry Ewing 11ca09b3bb [wasm][size-diff] fix typo in wasm targets (#15676) 2019-07-15 20:25:33 +02:00
Alexander Köplinger 4e513f802a Update dependencies from dotnet/corefx 2019-07-15 17:52:29 +02:00
dotnet-maestro[bot] 157728ce0d [master] Update dependencies from dotnet/corefx (#15689)
* Update dependencies from https://github.com/dotnet/corefx build 20190714.1

- Microsoft.Private.CoreFx.NETCoreApp - 4.6.0-preview8.19364.1

* Update dependencies from https://github.com/dotnet/arcade build 20190703.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19353.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19353.1

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/corefx build 20190703.3

- Microsoft.Private.CoreFx.NETCoreApp - 4.6.0-preview8.19353.3

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08
2019-07-15 17:14:49 +02:00
Thays Grazia 73128453da [debugger] Don't list finalizer gc thread if it's not executing managed code (#15618)
* The example that reproduces the bug does this:
var thread = vm.GetThreads()[0];
greetingValue = program.InvokeMethod(thread, greeting, new Value[0]) as StringMirror;

But sometimes the Thread[0] was the MainThread of the program and it works, and other times was the GC Finalizer, but when it tries to run something on GC Finalizer thread, the thread is not suspended and throws an exception.
In the fix I removed the Finalizer thread if it's not executing a managed code of the list that is returned when CMD_VM_ALL_THREADS is called.

Fix #13311
2019-07-15 11:58:04 -03:00
Alexander Köplinger d85fe2c391 [System] Remove FileSystemWatcher tests from monotouch/xammac
They were already excluded from watch and tv but they don't work on any mobile Apple profile.
2019-07-15 15:17:27 +02:00
Jay Krell ceb09980f5 Fix leak in create_runtime_invoke_info when mono_class_vtable_… (#15661) 2019-07-15 11:34:54 +02:00
Jay Krell 6cd6f65f69 Trim mono_mini_arch_lock, mono_mini_arch_unlock, mini_arch_mutex. (#15663) 2019-07-15 11:34:34 +02:00
Bernhard Urban 9a19bdb6e6 [llvm] avoid FP elimination on iOS/armv7 (#15617)
[llvm] avoid FP elimination on iOS/armv7

Instead of
```
push    {r4, r5, r6, r7, r10, r11, lr}
```

LLVM will generate
```
push    {r4, r5, r6, r7, lr}
add     r7, sp, #12
push    {r9, r10}
```

Seems like this https://github.com/mono/llvm/commit/a04e9e4a0af16f15ace258e81448b7eeca5ff599 assumes that `-disable-fp-elim` is passed for iOS targets. I wondered why this wouldn't break things for them, but when you run `clang` from Xcode with `-v`, you will discover that its driver passes `-mdisable-fp-elim`.

Together with https://github.com/mono/llvm/pull/48 this fixes crashes with FullAOT+LLVM on iOS 32bit.

Fixes https://github.com/mono/mono/issues/15058

Contributes to https://github.com/mono/mono/issues/9621
2019-07-14 16:15:55 -04:00
Aleksey Kliger (λgeek) 70e9f8c501 Add configure flag to disable GAC support; disable GAC in netcore (#15681)
* [runtime] Add configure flag to disable GAC support

* [netcore] Disable GAC support
2019-07-13 11:36:37 -04:00
Larry Ewing 016a5253bb Merge pull request #10202 from Unity-Technologies/fix-dynamic-method-marshal-info
Don't try to access metadata for dynamic method added to non-dynamic …
2019-07-12 18:22:01 -05:00
Steve Pfister 611b94cb41 [ios] Nil Check to Ensure Local TimeZone Id is Set For TimeZon… (#15682)
In runtime.m added nil check in xamarin_timezone_get_local_name. Returns Local as the Id in the case of nil.
2019-07-12 20:56:45 +02:00
Alexander Köplinger e3d5b23ff9 [ci] Upload mac-entitlements.plist to azure when uploading .pkg
So it can be used by the codesigning job.
2019-07-12 19:46:50 +02:00
monojenkins 45c82cc647 [bcl] Update BCL Linked Size 2019-07-12 20:15:06 +03:00
Jo Shields 202dca075b Add hardened runtime testing to Catalina lane (#15589)
* Permit DYLD mangling, so p/invokes work from non-system paths

* Add com.apple.security.cs.disable-library-validation

This is required for the .dylibs used in p/invokes to work.

* Use adhoc signing
2019-07-12 15:29:00 +02:00
Alexander Köplinger 02829bb914 [ci] Add back accidentally removed check 2019-07-12 15:27:39 +02:00
Kenneth Pouncey f3aaade4c7 [wasm][sdk][tests] Use the SDK to build the browser tests (#15605)
# Fix the runtime paths evaluation.
- If running from a master build the runtime builds path is different from the nuget paths.
  - From a master build the runtime builds are found in `wasm\builds` now.

# Browser Tests modifications:
- Moved all sources to a sources directory `tests\browser\src\`.
- Create a solutions project that can be built by VSfMac or VS using the SDK
- Create separate projects for each TestSuite.
- Change makefile target for the browser tests to use `dotnet` to build instead of the `packager.exe`

# SDK modifications and fixes:

Fix sdk build to run `monolinker.exe` using `dotnet`
- Run `mono` as task tool if running from `dotnet` and not running under Windows instead of executing the `monolinker.exe` directly
- Clean up WasmLinkAssemblies code.
   - Add routine to determine if build was initiated from `dotnet` instead of msbuild.
   - Add common routine to find the correct monolinker.exe to execute depending on running local or nuget.

Without the previous fixes, `monolinker` step can fail with the following message when using `dotnet build`

```
.nuget/packages/mono.webassembly.build/0.1.0-preview1/build/netstandard2.0/Mono.WebAssembly.Build.targets(76,3): error MSB6006: "monolinker.exe" exited with code 8.
```

The faling msbuild target is:

```
      _MonoWasmLink:
         /Users/Jimmy/.nuget/packages/mono.webassembly.build/0.1.0-preview1/build/tools/monolinker.exe --verbose --exclude-feature
         ... the rest of the command is cut off from here to save space

```

With the fixes command above becomes the following which execute the `monolinker` with `mono`. :

```
      _MonoWasmLink:
         /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono /Users/Jimmy/.nuget/packages/mono.webassembly.build/0.1.0-preview1/build/tools/monolinker.exe --verbose --exclude-feature remoting --exclude-feature com

         ... the rest of the command is cut off from here to save space
```
2019-07-12 14:44:40 +02:00
Jay Krell 81ed8dfaff extern "C" reduction. (#15634) 2019-07-12 12:01:35 +02:00
Marek Habersack 461ceae59b Fix offsets tool for Android NDK r20
Android NDK r20 contains a header file (`ioctl.h`) with this code:

    int ioctl(int __fd, int __request, ...);

    #if !defined(BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD)
    /* enable_if(1) just exists to break overloading ties. */
    int ioctl(int __fd, unsigned __request, ...) __overloadable __enable_if(1, "") __RENAME(ioctl);
    #endif

The `_RENAME` macro is defined as `__asm__(x)` and serves the purpose of
replacing an existing symbol with the new one, decorated with the keyword.
Unfortunately, Mono's `offsets-tool` doesn't parse the header properly as
`CppSharp` doesn't understand `__asm__`:

    Processing triple: armv7-none-linux-androideabi
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/metadata/metadata-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    Error parsing '/Users/rodo/git/xa2/external/mono/mono/mini/mini-cross-helpers.c'
    /Users/rodo/android-toolchain/ndk/sysroot/usr/include/bits/ioctl.h(60,5): error: conflicting types for 'ioctl'
    make: *** [/Users/rodo/git/xa2/external/mono/sdks/builds/android-cross-arm-release/armv7-linux-android.h] Error 1

The workaround, implemented in this commit, is to define the
`BIONIC_IOCTL_NO_SIGNEDNESS_OVERLOAD` macro thus removing the "conflicting"
declaration from the picture, which changes nothing from the offset tool's point
of view and lets us process the headers in NDK r20 properly.
2019-07-12 12:41:54 +03:00
Zoltan Varga 04034032f4 [jit] Fix the devirt of EqualityComparer<byte>. (#15644)
* [jit] Fix the devirt of EqualityComparer<byte>.

* Add a test.

* Fix the tests.
2019-07-12 01:24:52 -04:00
Steve Pfister 537fc4b60d [ios] Ensure Local TimeZone Id is Set For TimeZoneInfo.Local (#15667)
* Fixes https://github.com/mono/mono/issues/14839

The local timezone name was not known and defaulted to "Local".
This change adds an sdk function that returns the local name.
2019-07-11 23:45:03 +02:00
Egor Bogatov 792bfbff91 [netcore] Do not inline ThrowHelpers, fix SkipFrames handling. (#15607)
1) Do not inline ThrowHelpers (e.g. `static void ThrowException() => throw new Exception();`)
2) `get_trace` incorrectly handles `skip` argument
3) fix broken ci (`dl-test-assets.py` is not python2 compatible any more)

Fixes https://github.com/mono/mono/issues/15182
Fixes https://github.com/mono/mono/issues/15188
2019-07-11 21:42:14 +02:00
Egor Bogatov 972cd65365 [netcore] Fix IsAssignableFrom (#15641)
Mono used to return `true` for these three cases (see System.Reflection.Tests.TypeTests.IsAssignableFrom tests):
```csharp
using System.Reflection;
using Xunit;

namespace ConsoleApp
{
    public static class Program
    {
        static void Main(string[] args)
        {
            Assert.False(typeof(TI_Interface1[]).GetTypeInfo().IsAssignableFrom(
                         typeof(TI_StructWithInterface[]).GetTypeInfo()));

            Assert.False(typeof(int?[]).GetTypeInfo().IsAssignableFrom(
                         typeof(int[]).GetTypeInfo()));

            Assert.False(typeof(int[]).GetTypeInfo().IsAssignableFrom(
                         typeof(int?[]).GetTypeInfo()));
            
        }
    }
    
    public interface TI_Interface1 { }
    public struct TI_StructWithInterface : TI_Interface1 { }
}
```
Fixes https://github.com/mono/mono/issues/10848
Fixes https://github.com/mono/mono/issues/15080

Unrelated  System.Reflection.MetadataLoadContext.Tests tests that don't fail anymore:
Close https://github.com/mono/mono/issues/15351
Close https://github.com/mono/mono/issues/15348
Close https://github.com/mono/mono/issues/15347
Close https://github.com/mono/mono/issues/15346
Close https://github.com/mono/mono/issues/15345
Close https://github.com/mono/mono/issues/15343
Close https://github.com/mono/mono/issues/15342
Close https://github.com/mono/mono/issues/15341
Close https://github.com/mono/mono/issues/15338
Close https://github.com/mono/mono/issues/15337
Close https://github.com/mono/mono/issues/15336
2019-07-11 21:41:11 +02:00
Zebediah Figura 2f1b50bfbf [metadata] Respect unmanaged calling conventions in function signatures. (#15363)
According to ECMA-335 § II.15.3, calling conventions other than "default" and
"vararg" describe unmanaged methods. Accordingly they should be treated as
pinvokes so that that their calling conventions are respected.
2019-07-11 14:38:03 -04:00
Micah N Gorrell 1002a4de2b [Mono.Security] Allow importing certificates from authenticode assemblies (#14210)
This change allows a number of games, such as DARK, to start successfully. The games use X509Certificate.CreateFromSignedFile() to load the certificate from their assemblies in order to verify them.

* Added support for creating an AuthenticodeDeformatter for an assembly
that has been loaded into memory

* Allow importing a certificate from an assembly signed with authenticode
when using BTLS

* Allow importing a certificate from an assembly signed with authenticode
when not using BTLS

* Added a test for loading certificates from assemblies signed with
authenticode
2019-07-11 19:09:33 +02:00
Calvin Buckley 41469ee55a Misc AIX/PASE tweaks (#15651)
* Partial enablement of alternate stack for AIX/i

It turns out much like macOS, AIX doesn't like to do mprotect/valloc
for the first thread's guard pages, so skip those. It seems mostly
fine except for one or two crashes causes it to grab the wrong IAR
and deadlock dumping memory. As such, leave the code and configure
script override to disable in place, just change the comment and
add support code.

* Use Qp2getifaddrs on PASE

Not sure if proper way to implement. Reuses getifaddrs code as much
as possible, since it's merely a name change based on the docs, due
to it being namespaced in case AIX gets it or something.

I'm not sure how many of these codepaths still work properly;
one had a questionable order of ifdef.

This will eventually prepare for CoreFX NetworkInterface, so test
it here.
2019-07-11 12:42:47 -04:00
Alexander Köplinger 37f5ef91bb Bump bockbuild to fix gtk build break 2019-07-11 17:29:33 +02:00
Alexander Köplinger 439ab4fb4c Bump bockbuild to get https://github.com/mono/bockbuild/pull/105 2019-07-11 16:08:01 +02:00
Jay Krell 2950bd39e1 Remove configure cruft -- Interlocked and GetProcessId. (#15633)
They are never checked and all relevant systems have all of them.
2019-07-11 13:39:11 +02:00
Jay Krell 7bdff9065b Cleanup mono_threads_platform_get_stack_bounds. (#15632)
Fix mingw warning unused function __readfsdword.
2019-07-11 13:35:57 +02:00
Steve Pfister 625be3d1fd Bump mono/corefx@17dd540 (#15654) 2019-07-11 13:30:01 +02:00
Steve Pfister 53a0a9eaf4 Add TypeForwards for FileSystemName & CryptographicOperations (#15652)
* Add TypeForwards for:

* System.IO.Enumeration.FileSystemName
* System.Security.Cryptography.CryptographicOperations

Since these were removed in NS 2.1

* Bump API snapshot submodule
2019-07-10 23:58:59 +02:00
Thays Grazia 4917c086b6 [debugger] Add intrinsic for creating a byte array (#15591)
Fixes #15268
2019-07-10 23:29:13 +02:00
Vlad Brezae 2f2a9bcc37 [interp] Add JitHelper intrins for enum comparison (#15635)
Fixes https://github.com/mono/mono/issues/15596
2019-07-10 20:42:50 +03:00
Ryan Lucia 370e76dfc8 [netcore] Don't default to mscorlib in Assembly.GetType (#15488)
* [netcore] Don't default to mscorlib in Assembly.GetType

* Add argument for whether or not to search mscorlib

* Reorder and rename search_mscorlib argument

* Use gboolean exclusively
2019-07-10 09:21:56 -07:00
Jay Krell 7a486d8f43 Remove quadratic behavior from g_strconcat. (#15630)
Prefer strlen + memcpy over strcat.
2019-07-10 12:06:51 -04:00
Jay Krell 58fc782d86 Fix warning: unused function get_address_from_sockaddr (#15631) 2019-07-10 10:36:46 +02:00
Jay Krell 77c269721e Provide g_strnlen to replace strnlen for portability to older systems. (#15626)
See https://github.com/mistydemeo/tigerbrew/issues/605.
See https://www.gnu.org/software/gnulib/manual/html_node/strnlen.html.
2019-07-09 23:43:12 -04:00
Kenneth Pouncey 8b8fd0a820 Create package of the templates
- Create a `Mono.WebAssembly.Templates.cproj`
- Moved the existing DotNet template to a new subfolder Templates\templates
- Add `dotnet pack` of the project to the Makefile.
2019-07-09 20:36:48 -05:00
Zoltan Varga 5fb57b4fbd [wasm] Bump emscripten to 1.38.38. (#15639) 2019-07-09 20:51:54 -04:00
Zoltan Varga cba874139d [interp] Avoid declaring MonoError's in interp_exec_method_full (), it increases the native stack frame size because the optimizer can't seem to figure out live ranges. (#15613)
* [interp] Avoid declaring MonoError's in interp_exec_method_full (), it increases the native stack frame size because the optimizer can't seem to figure out live ranges.

* Remove some unused fields from InterpFrame.
2019-07-09 15:16:46 -04:00
Thays Grazia d33747a6e3 [debugger] Update client thread frames after SetIP. (#15449)
* Fixes #13408

PR of draft https://github.com/mono/mono/pull/14667

* Disabling test on Interpreter.
2019-07-09 16:07:46 -03:00
Kenneth Pouncey a6983e4a37 Add threads option
```

  --threads (enable threads)
        type: bool  default: false

```
2019-07-09 13:26:24 -05:00
Kenneth Pouncey 742c98c5b4 Update 2019-07-09 13:26:24 -05:00
Kenneth Pouncey 46025dab20 Change formatting 2019-07-09 13:26:24 -05:00
Kenneth Pouncey 607027b544 Update docs
- Include threading
2019-07-09 13:26:24 -05:00
Martin Baulig cf609799f0 [System]: Allow reading fully buffered HttpWebResponse body… (#15220)
[System]: Allow reading fully buffered `HttpWebResponse` body after close.

If we have already read the entire response body, then we should allow the
user to read it even after the request has been aborted.

* `WebConnectionStream.Read ()`: Move `Operation.ThrowIfClosedOrDisposed ()`
   call down after argument checks and call a new protected abstract method
   called `TryReadFromBufferedContent ()` prior to it.

* `WebResponseStream.TryReadFromBufferedContent ()`: Implement it here.
  If the entire response body has been buffered, then `bufferedEntireContent`
  is true and `innerStream is BufferedReadStream` and we can call the new
  `BufferedReadStream.TryReadFromBuffer ()`.

* `WebResponseStream.ReadAllAsync ()`: Set `bufferedEntireContent = true`
   after reading the entire response body.

* `BufferedReadStream.TryReadFromBuffer ()`: New internal method.

Fixes #10645.
2019-07-09 19:26:42 +02:00
Zoltan Varga 2cefbc18df [jit] Add support for valuetype arguments to gsharedvt constrained calls. (#15610) 2019-07-09 11:40:06 -04:00
dotnet-maestro[bot] 6f24998e8b [master] Update dependencies from dotnet/corefx (#15608)
* Update dependencies from https://github.com/dotnet/corefx build 20190707.5

- Microsoft.Private.CoreFx.NETCoreApp - 4.6.0-preview8.19357.5

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/corefx build 20190703.6

- Microsoft.Private.CoreFx.NETCoreApp - 4.6.0-preview8.19353.6

* Update dependencies from https://github.com/dotnet/arcade build 20190703.19

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19353.19
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19353.19

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08
2019-07-09 14:12:26 +02:00
Egor Bogatov 9dc20c9b5f [netcore] Fix netcore CI lane (#15627) 2019-07-09 13:29:19 +02:00
Alexander Köplinger b2ba8e62ca Make RemoteExecutor a no-op on all mobile profiles (#15611)
Starting another Mono process for the remote executor doesn't work on any of them.

Helps with https://github.com/mono/mono/issues/15328
2019-07-09 11:45:00 +02:00
Zoltan Varga aeacedfd38 Remove support for llvm36, its not used anymore. (#15619)
* Remove support for llvm36, its not used anymore.

* [jit] Remove support for older llvm versions.
2019-07-09 11:44:17 +02:00
Jay Krell 5a0bc3cf81 [profiler] Fix printf bug -- char* vs. MonoString* (#15602)
Gcc warned.
2019-07-09 11:24:39 +02:00
Jay Krell 369f796d8d [jit] [x86] [amd64] Cleanup with x86_byte() (#15600) 2019-07-09 11:12:52 +02:00
Jay Krell 39b52e2da8 Access mini_debug_options directly within the runtime. (#15594)
We had some inconsistency around accessing mini_debug_options in the runtime (both directly as well as through the function, mini_get_debug_options). This PR makes the access consistent through the runtime.
2019-07-09 11:12:15 +02:00
Alexander Köplinger 9cec2b0da3 [ci] Remove check-seq-points step from CI runs
The test was disabled with https://github.com/mono/mono/pull/15623
2019-07-09 11:11:04 +02:00
Jay Krell 135eaa1758 Disable check-seq-points test. (#15623)
See https://github.com/mono/mono/pull/15599#issuecomment-509444313.
See https://github.com/mono/mono/pull/15567.

Alternative to https://github.com/mono/mono/pull/15599 and part https://github.com/mono/mono/pull/15567.
2019-07-09 11:09:29 +02:00
Jan Kotas 9335f2637e Cleanup IL linker heuristic usage (dotnet/coreclr#25547)
Prep-work for using mainstream IL linker

Signed-off-by: dotnet-bot <[email protected]>
2019-07-09 08:39:16 +03:00
Alexis Christoforides c6bf6370dd The regression is the accidental removal of RegisterReportingForNativeLib from the managed side, introduced by b919ac4#diff-939ea9baecd2e8b24260bcb33f531274 2019-07-09 01:34:44 +03:00
Jay Krell e4bc8790d3 runtime.h remove extern "C" and distribute normally instead of extra. (#15558) 2019-07-08 13:59:05 -04:00
Johan Lorensson 12c962835f Align native_handle open/close for MonoThreadInfo/MonoInternalThread on Windows. (#15482)
* Align native_handle open/close for MonoThreadInfo/MonoInternalThread on Windows.

* Drop assert on DuplicateHandle failure.

* Fix native thread handle gpointer conversion.
2019-07-08 13:50:17 -04:00
Alexander Köplinger 784449a077 [sdks] Fix android sdks job, take 2
It needs to work for both debug and release configs.
2019-07-08 19:34:32 +02:00
Jay Krell 63126b72f7 Remove some unused third party code. (#15475)
i.e. C++ helpers for Boehm GC.
2019-07-08 19:30:14 +02:00
Alexander Köplinger 305b2e76e6 [sdks] Fix android sdks job
It would try to execute the .stamp-android-loader-path target in unrelated configurations.
2019-07-08 18:04:23 +02:00
Thays Grazia ebebfe263b Crashes on running the boring.il test from coreclr, the problem was that using [System.Runtime]System.String the type was not understood as the primitive type string. (#15552)
Using ilasm from coreclr and running on mono it works, because of that I decided to fix on ilasm and not on runtime.

Fixes #13923
2019-07-08 10:22:19 -03:00
Zoltan Varga cb330378cf [runtime] Implement mono_verifier_class_is_valid_generic_instantiation () etc. even if the verifier is disabled. (#15564)
Fixes https://github.com/mono/mono/issues/15551.
2019-07-08 10:36:49 +02:00
Jay Krell f19bc96343 mono_native_tls_get_value that does not change LastError. (#15568) 2019-07-08 09:44:23 +02:00
Jay Krell 963ba77863 Inline GetLastError and SetLastError. (#15560) 2019-07-08 09:42:54 +02:00
Egor Bogatov 55cc1d6d8f [netcore] Throw ICE for Array.Copy(object[], enum[]) (#15585)
* test

* undo rsp changes

* Fix ICollection_NonGeneric_CopyTo_ArrayOfEnumType

* group all "requires precise gc" issues

* Add more gc precise tests (and a flaky one)
2019-07-07 09:11:03 -04:00
Ryan Lucia 851dfdb98a Convert dl-test-assets.py to Python 3 (#15524) 2019-07-06 17:34:30 -04:00
Jay Krell d540aee31f Cleanup zlib-helper.c allocations -- move integer overflow checks into calloc. (#15505) 2019-07-06 14:59:27 -04:00
Zoltan Varga de5dca2264 [jit] Add missing float conv.u8.ovf.un conversion. (#13907)
* [jit] Add missing float conv.u8.ovf.un conversion.

* Add support for the float64 bit version as well.

* Fix interpreter support.
2019-07-06 05:47:47 -04:00
Jay Krell 1cb8e04331 Distribute sgen-dynarray.h via source, not EXTRA_DIST. (#15559) 2019-07-05 18:05:13 +02:00
Jay Krell 2b7e2d2f34 Remove extra explicit dist. (#15557)
Automake sees through the conditionals.
2019-07-05 18:04:22 +02:00
Radek Doulik 0991905c8b [sdks] Use @loader_path for mono and libmonosgen (#15580)
* [sdks] Use @loader_path for mono and libmonosgen

This way we are able to dynamically load libmonosgen and also use host
mono binary.

The library dependencies look like this after the change:
```
otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/bin/mono:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...

otool -L /Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib
/Users/rodo/git/mono/sdks/out/android-host-Darwin-release/lib/libmonosgen-2.0.dylib:
	@loader_path/libmonosgen-2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
	...
```
2019-07-05 17:38:19 +02:00
Bernhard Urban 5a5cbcc0a7 [sdks] sanity check for maximum number of open file descriptors (#15582)
Default for `ulimit -n ` seems to be 256, that isn't enough for the Android SDK build.
2019-07-05 17:19:08 +02:00
Alexander Köplinger 3ffe317138 [netcore] Only do CI builds on master
We don't care about the 2019-xx branches yet.
2019-07-05 17:15:12 +02:00
Alexander Köplinger bc269dc39e [netcore] Read NETCORETESTS_VERSION and NETCOREAPP_VERSION fro… (#15584)
* [netcore] Simplify CI build invocation

We now have build.sh as a one-stop entry to the build system.

* [netcore] Read NETCORETESTS_VERSION and NETCOREAPP_VERSION from eng/Versions.props

The Versions.props is auto-updated by Arcade so we automatically get new versions.
2019-07-05 17:03:49 +02:00
dotnet-maestro[bot] 3b3829e33c [master] Update dependencies from dotnet/arcade (#15550)
* Update dependencies from https://github.com/dotnet/arcade build 20190703.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19353.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19353.1

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/corefx build 20190703.3

- Microsoft.Private.CoreFx.NETCoreApp - 4.6.0-preview8.19353.3

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.05

- Microsoft.NETCore.App - 3.0.0-preview8-27903-05

* Update dependencies from https://github.com/dotnet/core-setup build 20190703.08

- Microsoft.NETCore.App - 3.0.0-preview8-27903-08
2019-07-05 17:03:09 +02:00
Egor Bogatov 3a7afab91c [netcore] CI work, add rsp for Windows (#15549) 2019-07-05 17:02:44 +02:00
Jay Krell e7f9dbe73c Link with -z now or -bind_at_load. (#14562)
Advantages:
 Dynamic linker gets out of the way of random function calls,
 in terms of timing, in terms of locking, and in term of success,
 and possibly in terms of security (readonly function pointers).
 If you successfully load, then you can make function calls with impunity,
 without regard as to if they are in the same .so as the caller, well,
 assuming they are to a leaf like libc, or to something itself built with -z now.

 The "success" is significant, as otherwise loaded LLVM has unresolved symbols lurking, depending on which code paths you send it down.

Disadvantage: Slower startup.

Mixed:
 Platform consistency, since this is how Windows behaves usually (unless you say link /delayload)

This will combine well with an upcoming revision of https://github.com/mono/mono/pull/14524.

Difficult alternative: Force the binding along sensitive paths by calling their callees earlier.
Typical alternative: Ignore the matter.

Android: It is believed this switch produces a warning, which is ignored, there is no affect, and it is unclear if the behavior is actually "now" or "lazy". Logs could be checked and a test run, but neither.
2019-07-05 15:54:16 +02:00
Jay Krell 4cf527f6d9 Readonly codemanager callbacks. (#15538) 2019-07-05 15:46:10 +02:00
Jay Krell 28ec3644ef Make thread locals just always extern. (#15563)
Making them static for C is not worth the ifdef.
2019-07-05 15:38:15 +02:00
Jay Krell b84b85688b Win32 zlib support. (#15509)
Fixes https://github.com/mono/mono/issues/15503

Always statically link on Windows (msvc and mingw).
This is a somewhat old zlib, and will be updated shortly (https://github.com/mono/mono/pull/15480)

Note that zlib is not valid C++, so always compile as C.
2019-07-05 14:38:56 +02:00
Steve Pfister 063985e04d Bump mono/corefx@9354dfd (#15545) 2019-07-05 14:35:14 +02:00
Bernhard Urban 808592925b [android] switch cross compilers host from 32bit to 64bit (#15553)
Untested, but we need to do it anyway soon. This will help us to understand the situation on Xamarin.Android while finishing the work for Xamarin.iOS.

Contributes to https://github.com/mono/mono/issues/9621
2019-07-05 13:58:38 +02:00
Johan Lorensson b1b61d7cb7 Interpreter reads last error to late after pinvoke. (#15571)
* Interpreter reads last error to late after pinvoke.

This could lead to clobbered last error since interpreter loop needs to
run two additional IL instructions in order to read out the value. This works
on JIT since the lowering saving the last error will happen right after
pinvoke.

This fix makes sure interpreter will get needed information to read out
last error right after pinvoke (if requested).

It adds a new IL prefix issued in managed->native wrapper that will tell
codegen (and interpreter) that upcoming calli instruction needs to save
last error, meaning that it could be done right after issuing the call,
no need to run any additional IL instructions to do that work, closing
the gap between the return of pinvoke call and save of last error.

Should fix, https://github.com/mono/mono/issues/15541.

* Optimize emit_get_last_error.
2019-07-05 10:30:17 +02:00
Kenneth Pouncey a90b735cf8 Update ,gitignore (#15561)
- emsdk_env.sh
- builds/threads
2019-07-05 07:29:52 +02:00
Kenneth Pouncey d8f2ebe384 Add a package.json to the tests/runtime directory. (#15573)
- Installs the required packages to run
- Modify make target `run-threads-sample` to use npm to install and run the test.
- There are two targets in the package.json file one for `test` and another 'start`
2019-07-05 05:43:50 +02:00
Larry Ewing 6530ba9861 [wasm] Add BlazingPizza.Client to size tesets 2019-07-04 19:48:34 -05:00
Jan Kotas a288cb1aae Delete some dead code (dotnet/coreclr#25546)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-04 19:18:35 +03:00
Jay Krell 5637cbd42d Do not compile metadata support/compress.c. (#15535) 2019-07-04 16:05:17 +02:00
Maxim Lipnin e0da61d691 [netcore] Make System.Reflection.Tests.TypeTests.GetEnumUnderlyingType Pass (#15510)
* [netcore] Make System.Reflection.Tests.TypeTests.GetEnumUnderlyingType Pass

* Remove recursive name parsing in case of "+" character; remove if condition for nested names
2019-07-04 09:10:18 -04:00
Alexander Köplinger 16750e4fd9 [netcore] Trigger Azure Pipelines builds only for master and release branches
Also batch changes so the system waits until a build is completed,
then queues another build of all changes that have not yet been built.
2019-07-04 09:16:56 +02:00
Larry Ewing 03d5525757 [wasm] allow overriding the build command invocation 2019-07-03 23:37:58 -05:00
Kenneth Pouncey 1a08a840c2 Fix the directory paths after threads commit
- Add threads build directories to the Runtime package.
2019-07-03 17:24:52 -05:00
Jay Krell 167709be8c Make sure TLS (thread local storage) is always TLS and not FLS (fiber local storage).
The Windows SDKs are inconsistent (and clearly documented as so), and the JIT assumes TLS.
2019-07-03 15:45:47 -05:00
Thays Grazia fb49d2d631 [debugger][wasm] Display content of "this" (#15528)
* Implementing watch of variable "this"
Fixes #15446

* Fixing indentation.
2019-07-03 12:15:18 -03:00
Vlad Brezae 6136d51ae9 [interp] Simplify MINT_NEWOBJ_ARRAY 2019-07-03 09:08:20 -05:00
Vlad Brezae aa7b713715 [interp] Make newarr lockfree
Before this commit, we were creating the array class from the element class during run time, also taking a lock in the meantime. This makes array allocation 2x faster. (on multithread much faster)
2019-07-03 09:08:20 -05:00
Johan Lorensson e30184752b Abort async suspend request on failure doing preemptive suspend. (#15486)
When a preemptive suspend fails, state machine will get into an invalid
state and the thread is in an unknown state (running/self-suspended). In case of
failure in mono_threads_suspend_begin_async_suspend, a new method should be
called, mono_threads_transition_abort_async_suspend making sure state machine
reflects correct thread state as well as reporting correct result to initial
caller of mono_threads_suspend_begin_async_suspend.
2019-07-03 13:08:05 +02:00
Jay Krell 08feb423c1 HAVE_DL_LOADER is never used so remove it. (#15500) 2019-07-03 11:01:59 +02:00
Jay Krell 33b4d2695e AC_DEFINE instead of AC_SUBST MONO_DL_NEED_USCORE. (#15499)
i.e. so it can be used in C instead of make.
i.e. so it can actually work.

The lone use is here:
```
C:\s\mono2\mono\utils\mono-dl.c(271):#if MONO_DL_NEED_USCORE
```
2019-07-03 11:01:19 +02:00
Jay Krell 177ae36c7a Do not compile unused support/uncompr.c. (#15516) 2019-07-03 10:59:38 +02:00
Egor Bogatov 57f362207c [netcore] Clean up CoreFX.issues.rsp (#15526) 2019-07-03 10:58:50 +02:00
Zoltan Varga c533ad99ce [wasm] Ongoing work. (#15504)
* [wasm] Avoid AOTing mscorlib if the input file haven't changed.

* [wasm] Store the wasm builds in a builds/ subdir.

* [wasm] Add a run-chrome- target to run test suites under chrome using a puppeteer script.

* [wasm] Link with -O0 in debug mode.

* [wasm] Add a runtime-threads make target.

* [wasm] Compile driver.o etc. with -Oz. Make the build-native -> packager.exe dependency order only.

* [wasm] Run the threads sample using chrome.

* Bump emscripten to 1.38.37.

* Fix the build.

* Fix the tests.

* Fix the tests.
2019-07-03 08:48:51 +02:00
Rich Lander 8fe4109ead Add entry for GCC (#15473) 2019-07-02 17:57:38 +02:00
Michal Strehovský 29e1d630b4 Use Array.MaxArrayLength in ArrayList (dotnet/coreclr#25530)
Now that `ArrayList` is in CoreLib, we don't need the copy.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-02 17:26:31 +02:00
Jay Krell 4b074e842b Correct MONO_DL_NEED_USCORE for C++, and memcpy is faster than strcpy. (#15498) 2019-07-02 11:09:32 -04:00
Alexander Köplinger e843b52098 [netcore] Add DARC subscriptions for Microsoft.NETCore.App 2019-07-02 15:11:32 +02:00
Johan Lorensson 5fee1d3eea Add support for TLS callbacks on Windows. (#15423)
* Add support for TLS callbacks on Windows.

Mono runtime depends on DllMain to be called by OS in order to correctly
detach threads from the runtime. If this doesn't happen,
threads from native thread pools (not owned by runtime) and attached
using native->managed callback, won't detach resulting in attached
threads no longer running. Since threads are still attached
to runtime, next GC will try to suspend the thread and that will fail
and that in turn will put the state machine in an incorrect state,
bringing down the runtime at next GC (when the thread state is revisited).

This problem is currently handled by DllMain, but since DllMain only exist
in DLL's, statically link Mono runtime will get us into the above scenario.

This commit add support to hook up a TLS callback using the same mechanism
used by MSVC linker and c-runtime (also available under MINGW). The
callback will be included in a section/segment of an object file and included
in final image (DLL or EXE) by linker and called by OS loader, solving the
problem with static linked Mono runtime not being able to detach terminating
threads.

The commit keeps current DllMain method when building a DLL but
includes a mechanism making sure we only use one of the callback techniques
at runtime, if both have been included in final image. This will simplify the build
of libmini since we can always build the object including the callback, regardless
how the library will be consumed.
2019-07-02 14:06:57 +02:00
Alexander Köplinger b62878973d [netcore] Add DARC subscriptions for Microsoft.Private.CoreFx.NETCoreApp
It contains the corefx test assets.
2019-07-02 13:56:09 +02:00
Bernhard Urban 6f7da4ce49 [stats] distinguish between mini and LLVM AOT method (#15483)
[stats] distinguish between mini and LLVM AOT method

Also print the stats on shutdown when configured with `--with-core=only`
2019-07-02 04:55:50 -04:00
Zoltan Varga adbb7df7ce [netcore] Remove some unused SRE icalls. (#15484) 2019-07-02 04:27:48 -04:00
Jay Krell d956cf687c Fix const warning. (#15478) 2019-07-02 09:08:05 +02:00
Jay Krell caecad895a Make codeman not public (#15415)
Seems a bit obscure to be public, the header is not public.
2019-07-02 09:07:18 +02:00
Zoltan Varga 4473e761c5 Fix make package. 2019-07-01 22:15:21 -05:00
Zoltan Varga 5767de0213 Fix some targets. 2019-07-01 22:15:21 -05:00
Zoltan Varga 708d1948e4 Make a local copy of emsdk_env.sh for the interpreter builds. 2019-07-01 22:15:21 -05:00
Zoltan Varga 6304c094db Make a local copy of emsdk_env.sh during the ninja build. 2019-07-01 22:15:21 -05:00
Zoltan Varga e84e9e786d Avoid calling emsdk_env.sh, it calls emsdk update_env which is racy. 2019-07-01 22:15:21 -05:00
Zoltan VargaandLarry Ewing aacd83ad39 Fix formatting.
Co-Authored-By: Larry Ewing <[email protected]>
2019-07-01 22:15:21 -05:00
Zoltan VargaandLarry Ewing f01cb9c9d2 Fix formatting
Co-Authored-By: Larry Ewing <[email protected]>
2019-07-01 22:15:21 -05:00
Zoltan Varga beea086ec9 Enable threads builds on CI. 2019-07-01 22:15:21 -05:00
Zoltan Varga 3970c6112a Add back some stubs for the non-threads build. 2019-07-01 22:15:21 -05:00
Zoltan Varga 648bef053e Set Module.mainScriptUrlOrBlob so workers can load it. 2019-07-01 22:15:21 -05:00
Zoltan Varga 16ed192da3 Add missing file. 2019-07-01 22:15:21 -05:00
Zoltan Varga b40a624898 [wasm] Add threads support to the packager. 2019-07-01 22:15:21 -05:00
Zoltan Varga 7c3adaf873 [wasm] Implement pthreads support in the runtime. 2019-07-01 22:15:21 -05:00
Zoltan Varga e21bed53a0 [wasm] Allow overwriting EMSCRIPTEN_SDK_DIR. Add a runtime build with threads support if ENABLE_WASM_THREADS is defined in Make.config. 2019-07-01 22:15:21 -05:00
Zoltan Varga 7fa884b721 [wasm] Add a threads sample. 2019-07-01 22:15:21 -05:00
Zoltan Varga bc787a8a01 [runtime] Disable a a AC_TRY_RUN when cross-compiling. 2019-07-01 22:15:21 -05:00
Zoltan Varga 20a8edacb8 [wasm] Add a make define for wasm runtime builds. 2019-07-01 22:15:21 -05:00
Steve Pfister d39c84d45f [netcore] Remove IsByRef check on Reflection.Emit.DynamicMethod
Fixes https://github.com/mono/mono/issues/15324

Removed check on the return type being by ref.
2019-07-01 21:12:06 -04:00
Martin Baulig 3508cd8026 [System.Net.Http]: Delete broken Send_Transfer_Encoding_Custom() test. (#15470)
This test was checking a broken Mono-specific behavior, which has been
removed after we switched to the new corefx-based handler. It also does
not make much sense to use a `Transfer-Encoding` header with a GET request.
2019-07-02 01:27:42 +02:00
Johan Lorensson 51a11b68c5 Disable zlib on mingw Windows build to avoid dependency on zlib (#15493)
#4484 broke Windows mingw build due to new dependency, zlib1.dll not available in PATH on build bots. Disabling for now until we know how this should be handled (and also fixed for MSVC build runtime).
2019-07-02 01:15:37 +02:00
Alexander Köplinger 1a1e921efe Bump bockbuild to bring in fix for mono/mono#14957 2019-07-02 00:30:58 +02:00
Marek Safar 71d1b10c3e [netcore] Bump Roslyn 2019-07-01 23:01:13 +02:00
Michal Strehovský 68872c4a60 Use AssemblyLoadContext from shared partition (dotnet/corert#7570)
We were missing methods that got added for 3.0. The only LoadContext that works is the default one though.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Stephen Toub d0d2096fc7 Fix ActivityTracker ES_BUILD_STANDALONE build
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Koundinya Veluri 306e8c3cd6 Fix an assertion failure in the thread pool's QueueUserWorkItemCallbackBase (#25482)
Fixes https://github.com/dotnet/coreclr/issues/25242
- The issue could be that there is no memory barrier (or in the wrong place) on the thread reading the value, and an old value could be cached and read

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Santiago Fernandez Madero 4dfd7437a7 React to new compiler nullability warnings
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Jan Kotas c88c0dd873 Fix typo & make the code leaner (dotnet/coreclr#25442)
Enum.HasFlag generates bigger IL and depends on complex JIT optimization for a good code.
The classic bit check is strongly preferred accross CoreCLR and CoreFX.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Stephen Toub 45e5bb8ec0 Tweak nullable attribute per API review (dotnet/coreclr#25443)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Levi Broderick 83a9ee16fb Fix EncoderNLS / DecoderNLS regression in reporting error index (#25397)
Also fixes incorrect asserts in the Encoding type

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Ben Adams aeb8b0fa8f Don't show AggressiveInlining items in exception stacks (#25408)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Tanner Gooding c754857c0e Reverting the ShouldRoundUp logic to not change for custom numeric format strings. (#25400)
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Jan Kotas 674f8876cc Delete DeserializationBlockedException (dotnet/coreclr#25393)
Contributes to dotnet/corefx#36723

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Stephen Toub 1543276813 React to compiler respecting nullable attributes on properties
Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Tarek Mahmoud Sayed 0890f588e9 Map C Locale to Invariant (dotnet/coreclr#25373)
ICU map the C locale to en-US_POSIX which has a non desired collation behavior as it doesn't support case insensitive comparison.
This change is to force mapping C locale to Invariant instead.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Jan Kotas cc779bd7e3 Fix StackChange value for OpCodes.Stelem (dotnet/coreclr#25382)
Fixes dotnet/corefx#37955

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Santiago Fernandez Madero 4da8b52e28 Address last round of nullable feedback - S.Resources.ResourceManager, System.Reflection.Emit and ILGeneration (#25318)
* Address nullable feedback for System.Resources.ResourceManager

* Address nullable feedback for System.Reflection.Emit.ILGeneration

* Address nullable feedback for System.Reflection.Emit

* PR Feedback and clean-up redundant casts

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Tanner Gooding c471996c8b Updating Dragon4 to ensure the number buffer always provides a significant digit if one exists. (#25096)
* Updating Dragon4 to ensure the number buffer always provides a significant digit if one exists.

* Changing System.Number.RoundNumber to not round up floating-point numbers.

* Re-enabling the RealFormatterTestsBase CoreFX tests

* Updating Number.RoundNumber to take a isCorrectlyRounded parameter and to use IEEE compliant rounding for floating-point numbers.

* Change SinglePrecisionCustomFormat to 7, ensuring it matches the value used in netcoreapp2.1

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Ganbarukamo41 52814eb6bb Make Vector*.ToString() Culture invariant (dotnet/coreclr#25343)
* Remove IFormattable
* Hardcode characters
* Add tests

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 23:01:13 +02:00
Egor Bogatov d20fe8ec37 [netcore] Fix Array.Copy* tests. Bump NETCORETESTS_VERSION ver… (#15428)
* Fix Array.Copy* tests

* simplify code

* Add comments

* Address feedback

* Update CoreFX.issues.rsp

* Fix SRE tests

* Add comment

* bump versions again
2019-07-01 20:20:25 +02:00
Maxim Lipnin 15668a6292 Bump corefx (#15463)
This PR will bring mono/corefx#303
2019-07-01 19:01:23 +02:00
Miguel de Icaza de18c594f0 [metadata] Rename, null-initialize, and properly free *args variables (#13855) 2019-07-01 12:11:02 -04:00
Rich Lander 241d2d1809 Create THIRD-PARTY-NOTICES.TXT
Use: https://github.com/mono/mono/blob/6a9950c5d22a1d748a2163ce6bdbc7fa6ffd8e77/mcs/tools/sqlmetal/src/DbLinq/Test/Providers/ReadTest_Complex.cs#L571
2019-07-01 17:16:51 +02:00
dotnet-maestro[bot] 36b0c02463 [master] Update dependencies from dotnet/arcade (#15477)
* Update dependencies from https://github.com/dotnet/arcade build 20190630.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19330.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19330.1
2019-07-01 17:10:47 +02:00
Michal Strehovský 75dfa5c298 Remove AggressiveOptimization flag from System.Text (dotnet/coreclr#25356)
Based on conversation with Levi, I don't think this flag is needed.

The flag currently controls two things:
* Disqualifies the method from precompilation
* Disables Tier-0 JIT for the method

Removing the attribute allows us to precompile some of the methods (not all, because of `Vector<T>`). It measurably helps improve startup time because many of the attributed methods are complex and take a while to compile.

Once tier-1 JIT kicks in, the generated code should be identical to what we were getting before.

Signed-off-by: dotnet-bot <[email protected]>
2019-07-01 17:07:00 +02:00
Jay Krell 8c20d5c67a Remove unused AMD64 JIT CALLCONV_IS_STDCALL. (#15416) 2019-07-01 09:53:01 +02:00
Larry Ewing df74f5227f [wasm][debugger] remove args parsing hack 2019-06-30 12:25:40 -05:00
Larry Ewing 3b337e31c3 [wasm] use the router for all requests 2019-06-30 12:25:40 -05:00
Larry Ewing 18b031870e [wasm] remove TabMapper and just filter values key by key
proxy the version response through our logic so that we match
the actual protocol versions. Refactor the mapping logic so we
just rewrite specific keys that we understand and pass the rest
along.
2019-06-30 12:25:40 -05:00
Larry Ewing f293fd7174 [wasm] make the host url a uri 2019-06-30 12:25:40 -05:00
Larry Ewing 9455030364 [wasm] use options directly in other places 2019-06-30 12:25:40 -05:00
Larry Ewing 0065b50c93 [wasm] Start reworking the server logic for the Proxy and Debugger
Split some of the configuration logic out into options so that
we can start moving away from a fixed configuration.
2019-06-30 12:25:40 -05:00
Larry Ewing d7cfc6994a [wasm] make extract_conn_url return null on unexpected data
We want to avoid throwing inside ErrorDataReceived on unexpected
input because tends to hang the process in a bad state.  Instead
we return null from extract_conn_url and let the normal cleanup
process happen.
2019-06-30 12:25:10 -05:00
Rich Lander 40ebd4e815 Add entry for Silicon Graphics and HP
For https://github.com/mono/mono/blob/d58dfa185af1d48fb93efdef1d046762f214f0a7/libgc/include/gc_allocator.h
2019-06-29 20:54:42 +02:00
lateralusX be77d03c22 Fix 32-bit build error when using C++ compiler. 2019-06-29 13:48:34 -05:00
Katelyn Gadd c239906a01 Disable compiler server on darwin. Add force-enable option for compiler server. (#15450) 2019-06-28 18:24:09 -04:00
Filip Navara c788634b54 Add tests 2019-06-28 14:56:47 -05:00
Filip Navara cbe365b659 Fix marshalling bugs for StringBuilder parameters 2019-06-28 14:56:47 -05:00
Jo Shields 4421ebc1f2 Correct env vars used for Android directory override (#15469)
This fixes https://github.com/mono/mono/pull/15453

The issue is the env var names differ vs. Xamarin.Android
2019-06-28 15:29:37 -04:00
Jo Shields 714da9f5be Use /mnt/scratch for Android SDK, to match https://github.com/… (#15453) 2019-06-28 19:20:41 +02:00
Maxim Lipnin efa0a0bb3a Fix time zone transition out of DST (#15401)
When a datetime appears in range of [end_of_DST_period - Daylight_delta; end_of_DST_period] it's not DST and should return base offset.

Fixes #9664.
2019-06-28 19:04:21 +02:00
Kenneth Pouncey dfb9cdd072 Fix generation of wrong version. 2019-06-28 10:55:35 -05:00
Ryan Lucia c879bc4691 [netcore] Fix Assembly.GetForwardedTypes() (#15452)
* [netcore] Make GetForwardedTypes return nested types

* [netcore] Throw proper error in GetTopLevelForwardedTypes

* Remove unnecessary function

* Formatting

* Re-enable relevant tests

* Switch to REFERENCE_MISSING

* Use local error
2019-06-28 11:54:27 -04:00
dotnet-maestro 9bf0809439 Update dependencies from https://github.com/dotnet/arcade build 20190627.39
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19327.39
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19327.39
2019-06-28 17:27:39 +02:00
dotnet-maestro 5da074c43d Update dependencies from https://github.com/dotnet/arcade build 20190626.44
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19326.44
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19326.44
2019-06-28 17:27:39 +02:00
Zoltan Varga 112297ef5c [runtime] Implement support for embedded ppdb, when the ppdb blob is compressed and embedded into the pe image. 2019-06-28 15:19:43 +02:00
Jay Krell 19681ad2d4 Fix year+ old MIPS compile break. (#15459)
Fixes https://github.com/mono/mono/issues/15441
2019-06-28 11:28:05 +02:00
Alexander Köplinger 4e022f7f07 Replace two more MONO_DEBUG=clr-memory-model env vars with commandline switches
Follow up to https://github.com/mono/mono/pull/15447
2019-06-28 10:19:16 +02:00
Kenneth Pouncey 6b5113220c Simplify the file generation and remove the Encoding="Unicode" option 2019-06-28 01:01:58 -05:00
Kenneth Pouncey c692828f49 Add a target to set the version of the Templates's sdk version in the global.json.
- The `_WriteDotNetTemplateVersion` is called from the sdk build.

```
"Mono.WebAssembly.Sdk": "0.1.0-preview1"
```
2019-06-28 01:01:58 -05:00
Kenneth Pouncey 4c1cf2c148 Trying BeforeTargets="_GenerateRestoreProjectSpec" 2019-06-28 01:01:58 -05:00
Kenneth Pouncey 7e2433d9f4 Modify _CreatePackagesDir
- Another try
2019-06-28 01:01:58 -05:00
Kenneth Pouncey 1a2f43d4cb Address review comments separate versions for each sdk package
- Update to use only the SDK package version.
2019-06-28 01:01:58 -05:00
Kenneth Pouncey 6e6580beba Rename target to prefix it with an underscore to indicate that users should not call it directly. 2019-06-28 01:01:58 -05:00
Kenneth Pouncey 99e64a970f Add target to create packages directory if it does not exist. 2019-06-28 01:01:58 -05:00
Kenneth Pouncey 5ca42e24fb Remove the WebAssembly.Net.Http.xml file that was inadvertently commited 2019-06-28 01:01:58 -05:00
Kenneth Pouncey c9db6a08f1 Add versioning to the sdk and framework project files.
- Add a central versioning file `common\Versions`
- Modify projects to use the central version file
- Auto generate a versions file for the SDK versions so that they can be updated automatically.
2019-06-28 01:01:58 -05:00
Zoltan Varga a29cbc429d [netcore] Avoid calling mono_runtime_install_appctx_properties () in aot mode. (#15448) 2019-06-27 23:16:47 -04:00
Thays Grazia c011456f3f [dim] Emit error when calling abstract methods (#15433)
* Emitting bad image exception when a non virtual call is calling an abstract method.

* Fixing side-effect.
Implementing the same thing on interpreter.
Adding unit test.
2019-06-27 22:02:39 -03:00
Zoltan Varga 064d367ede Pass the --clr-memory-model flag on the command line instead of MONO_DEBUG so its not inherited by child processes. (#15447) 2019-06-27 20:44:39 -04:00
lateralusX c5ef80bf86 FIx id formatting to align with rest of code. 2019-06-27 14:59:59 -05:00
lateralusX 56d38ca430 Fix incorrect thread state on suspend thread failure.
According to documentation of mono_threads_suspend_begin_async_suspend:

"If begin suspend fails the thread must be left uninterrupted and resumed."

Current implementation on Windows didn't resume thread in case GetThreadContext
fails, leaving a thread in suspended state on failure.

Fix is to resume thread if GetThreadContext fails.
2019-06-27 14:59:59 -05:00
lateralusX 973f430d45 Fix use THREAD_SUSPEND_DEBUG macros on Windows.
Several errors in current implementation, both compile time errors as
well as warnings.
2019-06-27 14:59:59 -05:00
dotnet-maestro 6ae04f597d Update dependencies from https://github.com/dotnet/arcade build 20190626.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19326.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19326.2
2019-06-27 14:21:36 +02:00
Steve Pfister 814d20db91 [netcore] Review and document netcore test failures that are skipped (#15359)
* Went through the majority of corefx tests that were being skipped in CoreFX.issues.rsp.

For each failure, a github issue was logged and if possible, a more expanded
failure reason was specified.

Part of the https://github.com/mono/mono/issues/14787 epic work

* Cleaned up header that no longer had tests in it

* Pesky -method still around.  It'll ignore now
2019-06-27 14:14:02 +02:00
Ryan Lucia 8e4d1eb06f Disable flaky test HttpClientTest.CancelRequestViaProxy 2019-06-26 21:01:58 -05:00
Ryan Lucia 19bb89805a CreateNLSocket and CloseNLSocket should return gpointer (#15408)
In managed, these functions are used as `static external IntPtr`. This means that previously on arm64 the top bits of the return value were garbage. A comparison with 64-bit -1 like in LinuxNetworkChange.EnsureSocket would never be true, which was causing us to hit other assertions in the runtime.
Co-authored-by: Aleksey Kliger <[email protected]>
2019-06-26 15:41:11 -04:00
Steve Pfister 452fa3b50c Update CODEOWNERS (#15429) 2019-06-26 15:14:07 -04:00
Johan Lorensson b70694828d Fix unused-function warning in main.c. (#15424) 2019-06-26 19:16:11 +02:00
Aleksey Kliger (λgeek) 927c608184 [sdks] dynamically link libmonosgen to mono-sgen for monodroid host builds (#15410)
* [configure] Cosmetic AC_MSG_WARN fix

* [runtime] Mark mono_file_map_error as MONO_API, always

Fixed the `--with-static_mono=no` build on non-Win32

* [sdks] dynamically link libmonosgen to mono-sgen for monodroid host builds
2019-06-26 11:10:03 -04:00
Jay Krell cad60fce03 Remove mono_tls_get_tls_getter. 2019-06-26 07:14:30 -05:00
Jay Krell 8d6feabf0d Replace MONO_PATCH_INFO_GET_TLS_TRAMP with MONO_PATCH_INFO_JIT_ICALL_ADDR_NOCALL. 2019-06-26 07:14:30 -05:00
Vlad Brezae 6359520633 [sgen] Fix crashes in sgen due to xdomain refs
CADMessages are passed directly between appdomains. When using contained data, we need to make sure we don't create any cross domain references.

When domains are unloaded, all threads executing code in that appdomain are aborted and the entire heap is traversed and objects in that appdomain are cleared. If any objects in other domains are directly referencing objects in the unloaded domain, they will reference junk. If any, cross domain refs should only be allowed when the referenced object is in the root domain which is never unloaded.
2019-06-26 07:13:07 -05:00
Martin Baulig c56b37e00d [test-helpers]: Don't use named Mutexes on mobile. 2019-06-26 07:11:08 -05:00
Vlad Brezae aa05d4f43c [interp] Reduce computation under calc_section mutex
Calc_section mutex is meant to be a lower level mutex used only for synchronization on InterpMethod contents. Before this commit we were taking the loader mutex which was leading to deadlocks.
2019-06-26 07:10:37 -05:00
Larry Ewing f185358d96 [wasm] update the debugger instructions 2019-06-26 07:10:00 -05:00
dotnet-maestro d99c1e631b Update dependencies from https://github.com/dotnet/arcade build 20190624.24
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19324.24
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19324.24
2019-06-26 11:57:51 +02:00
Egor Bogatov 78b9c66501 Add xunit-summary.py (#15373) 2019-06-26 11:49:24 +02:00
Alexander Köplinger 6307f0156c [profiler] Add helper.h to tarball 2019-06-26 10:41:38 +02:00
Ryan Lucia a2a28b94c1 [netcore] Fix build.sh (#15380)
* [netcore] Revert changes to build.sh

It appears to have accidentally been changed in #15215, and the script no longer works correctly as a result, so revert

* Strip call to make
2019-06-25 23:58:41 -04:00
Larry Ewing 73dd9ac01e [wasm] make packager.exe depend on release/ and debug/ because it may use either 2019-06-25 21:44:06 -05:00
Alexis Christoforides 43f504a43e Bump bockbuild for fix to https://github.com/mono/mono/issues/14957 2019-06-25 20:59:50 -05:00
Larry Ewing 191854ded8 Merge branch 'master' into fix-dynamic-method-marshal-info 2019-06-25 20:18:14 -05:00
Zoltan Varga dc321b06a2 [aot] Save the number of locals into the aot debug info so the method header doesn't have to be decoded when its loaded. 2019-06-25 20:03:35 -05:00
Kenneth Pouncey 4699ff275d Add signing of the framework assemblies to the projects using the Open key. (#15303) 2019-06-25 17:45:13 -05:00
Thays Grazia 2bdf49cae1 [debugger][wasm] Local vars (#15385)
* Fixing side effect of #15357.

* Fixing side effect of #15357.
2019-06-25 17:36:26 -05:00
Alexander Köplinger 41767754d5 [System] Fix monotouch_watch xunit tests build
Forgot to exclude one more file.
2019-06-25 22:21:26 +02:00
Jay Krell 1c09e4de62 Fix tarball build and refresh monolite due to memory model change (#15370)
* Revise monolite due to memory model change.

* Fix tarball builds wrt netcore/config.make.

* Remove System.Private.CoreLib from mcs/class/Makefile, it was moved to netcore/

* Bump min_mono_version in basic-profile-check

The new MONO_DEBUG=clr-memory-model runtime option isn't available in earlier Mono versions.
2019-06-25 22:14:41 +02:00
Jay Krell 3e270ffe20 [threads] Define interlocked_mutex as non-recursive and on 32bit platforms only (#15393) 2019-06-25 14:56:37 -04:00
Radek Doulik 3efc0959fe [profiler] Add helper.c to static sources where needed (#15400)
This should fix broken `OS X x64 iOS SDK` build.
2019-06-25 20:04:16 +02:00
Alexander Köplinger bc1d080ba9 Exclude a few tests on monotouch_watch/xammac (#15397)
* [System.Core] Exclude AnonymousPipe tests on monotouch/xammac profiles

They should only be run on xammac_net_4_5.

Fixes https://github.com/mono/mono/issues/15329

* [System] Exclude WebClient and WebSocket tests on monotouch_watch

Fixes https://github.com/mono/mono/issues/15265.
2019-06-25 20:01:17 +02:00
Zoltan Varga ba242226f4 [wasm] Bump emscripten to 1.38.36. 2019-06-25 11:43:12 -05:00
Johan Lorensson 5ca92bdbe8 Add cooperative suspend aware threads for hybrid suspend. (#15204)
* Add cooperative suspend aware threads for hybrid suspend.

When running in hybrid suspend and having threads in blocking mode, GC Safe
mode, threads will still be preemptive suspended. This is done to make sure
compatibility is kept with embedders since current embedding API is not coop
aware. If, however the external attached threads promise to follow the coop
rules for a thread in Safe mode, it could still be possible to have them
running, making sure they won't be preempted when running STW. Unlocking this
possibility could have large performance wins if embedding threads runs a lot
of native code on threads attached to runtime.

This commit adds support for a new pair of API's where an external thread can
mark itself as cooperative suspend aware. Doing that guarantees the runtime
that it will follow all rules as dictated by cooperative suspend when a thread
is in safe mode. The change only goes one way since this is a lifetime commitment
for a thread. Once’s set, thread will be kept running when in safe mode.
2019-06-25 17:40:39 +02:00
Kevin Pilch d36d2dd02d Remove a stale comment (#15378)
Pushing https://github.com/aspnet/AspNetCore/pull/10561 upstream, to fix
AzDO Bug #890301 (https://dev.azure.com/devdiv/DevDiv/_workitems/edit/890301/)
2019-06-25 17:25:34 +02:00
Zoltan Varga 9ace0a3b9c [runtime] Refactor mono-debug.c. (#15381)
* [debug] Refactor the code in mono-debug.c.

Store the domain->debug info table mapping in MonoDomain to avoid a hashtable lookup.
Rename MonoDebugDataTable to DebugDomainInfo to better reflect its meaning.

* Remove some mono_ prefixes from static functions, along with the full/internal prefixes.
2019-06-25 09:35:21 -04:00
Alexander Köplinger b2ac112d37 [bcl] Move MONO_DEBUG env variable after Q_MCS
Otherwise we'd get "/bin/sh: @echo: command not found" errors.
2019-06-25 14:38:37 +02:00
Radek Doulik 858b2b8405 [aot profiler] Listen for commands on given port (#14936)
* [aot profiler] Listen for commands on given port

Extended the aot profiler so, that it can receive commands over the network.

Added new option `port=PORT`, which enables command server and specifies
the port the server should listen at.

We have 2 commands so far.

    save ... save the profile data to socket and close it
    quit ... quit the profiler. it also saves the profile to the file
             in case the output option was specified.

It makes the aot profiling much easier to use on Android as we don't need to
save the profile to a file on the device. That means we don't need to set
`<application android:debuggable="true"/>` flag, which changes the Android
app behavior and thus influence the profile itself. It is also convenient way
for IDEs to retrieve the aot profiles.

Example use for Xamarin.Android. Save and receive the profile data:

	adb shell setprop debug.mono.profile aot:verbose,port=9999
	adb forward tcp:9999 tcp:9999
	nc -v 127.0.0.1 9999 > profile.aotprof
	found 0 associations
	found 1 connections:
	     1:	flags=82<CONNECTED,PREFERRED>
		outif lo0
		src 127.0.0.1 port 50398
		dst 127.0.0.1 port 9999
		rank info not available
		TCP aux info available

	Connection to 127.0.0.1 port 9999 [tcp/distinct] succeeded!
	save

Check what we received:

	aotprofile-tool -sd profile.aotprof
	Modules:
		132475FF-85AA-430D-913C-7642CBD8587E mscorlib
		F7BDCAB5-83D3-4A0A-A33D-BBE94284667D Mono.Android
		22897151-27C9-4064-B001-8A04F5847812 System
		AA3FC224-D486-4304-A8C9-EBF316556B26 Java.Interop
		00D98403-051B-4168-A391-F3F601637CBD System.Core
		FD06519A-C321-426C-BDD8-C97777452057 xat_latests_and_greatest_vsmac_8._1
		9ECB003E-D449-438B-BE35-58810DCBC917 Xamarin.Android.Support.v7.AppCompat
		11659962-EFAC-400D-9F8D-AF925B26AE41 Xamarin.Android.Support.Fragment
		48C436F7-611D-470D-9949-AC13FFDCF613 Xamarin.Essentials
		2792D10D-7E13-46E6-BB4A-933C25635F48 Xamarin.Android.Support.Design
	Summary:
		Modules:         10
		Types:          221
		Methods:        932

* Try to fix WIN32 compilation

* Add helper.c to mono-profiler-log.vcxproj

* Add helper.c to mono-profiler-log.vcxproj.filters as well
2019-06-25 12:17:45 +02:00
Johan Lorensson f8fbbcf280 Increase max size of localloc and localloc_imm on amd64. (#15374)
We hit the following assert when doing full AOT of some assemblies:

wrong maximal instruction length of instruction localloc (expected 96, got 99)
Assertion: should not be reached at mono\mono\mini\mini-amd64.c:6801

Turns out that that size could be hit when a couple of things happens as part
of lowering OP_LOCALLOC and OP_LOCALLOC_IMM:

sreg1 is not RAX, RCX or RDI
dreg is not RAX, RCX or RDI
cfg->param_area is bigger than 127

Fix is to increase max size of the instruction, from 96 -> 120 (same as x86).
2019-06-25 10:46:33 +02:00
Johan Lorensson b181dbc50d Add support for Windows x64 Full AOT + LLVM + Interpreter on CI. (#15276) 2019-06-25 08:20:25 +02:00
Nikita Voronchev c30a10bc3e [System.Windows.Forms] Fix crash of DataGridView in case of assigning a custom Column Header (#15135) 2019-06-24 21:51:47 -04:00
Johan Lorensson b2124dab01 Add missing CI test script when running full AOT interpreter on Windows. (#15281) 2019-06-24 17:00:01 -04:00
Marek Safar 7066c05c39 [System.Private.CoreLib] Add new shared string resource 2019-06-24 21:12:12 +02:00
Andrii Siriak 0abee2baf5 Switch from IndexOf(string) to Contains(string) for better readability and performance (#38757)
* Fix performance IndexOf(string) -> Contains(string)

* Make comparison ordinal

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Stephen Toub eeb377842e Force secondary await continuations to run asynchronously (#25280)
* Force secondary await continuations to run asynchronously

For performance reasons, await continuations have been invoked synchronously, meaning they're invoked as part of the antecedent task's completion (as long as that task allows it, as long as there's sufficient stack space, etc.)  This generally works out well in the case where there's a single await continuation, which is far and away the common case.  However, it can cause problems in situations where there are multiple await continuations, as those continuations will end up being serialized, which can lead to slowdowns and deadlocks in niche situations.  To address that, this commit backs off a bit.  The first await continuation is still invoked synchronously, but subsequent await continuations are invoked asynchronously, such that they are not blocked by a previously registered await continuation.

* Fix nits

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Marco Rossignoli 3af0d87397 MemoryExtensions ToUpper / ToLower throws for overlapping buffer (#25327)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Roman Marusyk 96481aedf0 Change exception throwing in Path.GetRelativePath (dotnet/coreclr#25047)
* Fix Path.GetRelativePath throws ArgumentNullException when relativeTo is empty string

* Fix ArgumentException for relativeTo

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Herman Eldering a555a5b5b3 ArgumentException: use system-supplied message if message is null (#25115)
Enable using a system-supplied message while also providing an argument/parameter name by passing in a null message. Made to match the behavior of FileNotFoundException.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Santiago Fernandez Madero 9ac62ea31a Address nullable feedback for System.Diagnostics.Tracing and StackTrace (#25303)
* Address nullable feedback for System.Diagnostics.Tracing and StackTrace

* Disable corefx tests that need to be updated

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Stephen Toub 7d4ce9cd8c Remove several unnecessary string allocations (dotnet/coreclr#25305)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Stephen Toub 366c80aed8 Reduce string allocation in DiagnosticCounter.GetMetadataString (#25219)
When the _metadata dictionary contains a single key/value pair, the method ends up allocating an unnecessary StringBuilder, char[] beneath it, and input string.

When the _metadata dictionary contains more key/value pairs, the method ends up allocating an unnecessary string per key/value pair.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-24 21:12:12 +02:00
Zoltan Varga 1150d4a3e9 [netcore] Disable more runtime functionality. (#15369) 2019-06-24 14:47:00 -04:00
Alexander Köplinger 80db1ef59f [sdks] Do the WATCHOS5_VERSION -> WATCHOS64_32_VERSION rename in run-jenkins.sh too
It was missed in https://github.com/mono/mono/pull/15202
2019-06-24 19:14:54 +02:00
dotnet-maestro 683db00b2a Update dependencies from https://github.com/dotnet/arcade build 20190623.4
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19323.4
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19323.4
2019-06-24 17:25:34 +02:00
Thays Grazia 057f51d833 [wasm][debugger] Adding support to debug local vars in async methods (#15357)
- Improving performance calling only once mono_wasm_get_var_info.
- Adding support to debug local vars in async methods. They are special in the way that local variables can be lifted to generated class fields.
2019-06-24 10:34:42 -03:00
Johan Lorensson de90561fbd Fix race related to final event signaling in RegisteredWaitHandle. (#15280)
We have seen sporadic failures in thread pool tests on CI related to
test case:

System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
          at System.Threading.ThreadPools.Tests.ThreadPoolTests.QueueRegisterPositiveAndFlowTest () [0x00143] in D:\j\workspace\z3\label\w64\external\corefx\src\System.Threading.ThreadPool\tests\ThreadPoolTests.cs:337
          at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
          at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in D:\j\workspace\z3\label\w64\mcs\class\corlib\System.Reflection\RuntimeMethodInfo.cs:395

Finally managed to track down the issue to a race when using the same final
event in sequential calls to RegisterWaitForSingleObject. The event could, depending on
timing, get signaled multiple times for the same RegisterWaitHandle instance
causing a thread waiting for completion on another RegisterWaitHandle using same event
to be signaled before that callback was completed. That will cause the race seen in the
failing test above.

Fix make sure we only signal the final event one time per RegisterWaitHandle
instance.
2019-06-24 10:56:21 +02:00
Bernhard Urban 7d5daa2f77 [arm] set exception handling model to dwarf for iOS
We recently switched the AOT compiler for iOS armv7 from 32bit host to 64bit host. At the same time we _also_ switched from LLVM 3.6 to the newer LLVM 6.0 fork we support.

In a Xamarin.iOS Release build for iOS armv7 exception handling would crash. Specifically the problem was that the exception object wasn't properly injected into the handler block. The way it works is, that the exception handler machinery passes the pointer to the exception object via `r0`.

With the newer LLVM the generated code of the exception handling block
looks like this:
```
LBB1_15:                                @ %EH_CLAUSE0_BB3
Ltmp8:
	ldr	r0, [sp, #24]
	str	r0, [sp, #4]
	ldr	r0, [sp, #28]
	ldr	r0, [sp, #12]
	ldr	r0, [r0]
	cmp	r0, #0
	beq	LBB1_17
```
thus overwriting `r0`, so that the pointer to the exception object is lost.  With the older LLVM 3.6 the generated code looks like this:
```
LBB1_8:                                 @ %EH_CLAUSE0_BB3
Ltmp8:
	str	r0, [sp, #4]
	ldr	r0, [r6]
	cmp	r0, #0
	beq	LBB1_10
```
correctly storing the exception object into a stack slot for later usage.

After some time I figured out that there are _different_ exception handling models. Depending on the model, the exception pointer is passed in `r0` or not:
https://github.com/mono/llvm/blob/2bd2f1db1803f7b36687e5abf912c69baa848305/lib/Target/ARM/ARMISelLowering.cpp#L14416-L14428

`SjLj` stands for "SetJump / LongJump" and  means that exception handling is implemented with this. On iOS this is the default model that is used. We want `dwarf` instead, so we tell this `llc` now and the generated code handles it correctly.

So why was it working before? In our older LLVM 3.6 fork we hardcoded it:
https://github.com/mono/llvm/blob/f80899cb3eb75f7f5640b4519e83bd96991bffb8/lib/Target/ARM/ARMISelLowering.cpp#L756-L762

The `-exception-model=` option is also not available in LLVM 3.6.

Contributes to https://github.com/mono/mono/issues/15058 and https://github.com/mono/mono/issues/9621
2019-06-24 09:48:45 +02:00
dotnet-maestro d027cbbc06 Update dependencies from https://github.com/dotnet/arcade build 20190622.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19322.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19322.2
2019-06-24 09:12:13 +02:00
dotnet-maestro 33a4a35826 Update dependencies from https://github.com/dotnet/arcade build 20190621.75
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19321.75
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19321.75
2019-06-24 09:12:13 +02:00
Zoltan Varga bbfce00878 [netcore] Remove unused icalls. 2019-06-23 10:05:22 +02:00
Zebediah Figura 9f49cc85af [mini] Support tracing pointer return values. (#15356) 2019-06-22 12:57:49 -04:00
Zoltan Varga c3161826e3 [aot] Fix a deadlock in init_plt () by calling create_trampoline () outside the lock. (#15332)
Fixes https://github.com/mono/mono/issues/15330.
2019-06-21 23:35:57 -04:00
Zoltan Varga f42fcc0459 [sdks] Avoid building everything on make all. (#15296)
* [sdks] Avoid building everything on make all.

* Add a dummy all target.
2019-06-21 23:35:42 -04:00
Vlad Brezae 0cf363c4db [arm64] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-21 15:47:44 -05:00
Vlad Brezae a7a884e19e [arm] Add location of this and rgctx arg
Used to fetch the value from async suspend stack frame, during exception handling.
2019-06-21 15:47:44 -05:00
Vlad Brezae e1e26b87b5 [amd64] Fix location saving for *this*
With static methods we were saving location for irrelevant argument.
2019-06-21 15:47:44 -05:00
Jay Krell ab16b39a65 Remove three seemingly unused functions:
mono_aot_init_gshared_method_this
  mono_aot_init_gshared_method_mrgctx
  mono_aot_init_gshared_method_vtable

Update comments that refer to them, to refer to the renames.
 mini_llvm_init instead of mono_aot_init.

See 80bc0069e4 and 04d65f2986.

I'm here because I broke something nearby in a work-in-progress PR
and the existance of these functions made me look into them.
2019-06-21 15:46:24 -05:00
Ankit Jain 81f529689c Bump msbuild to track xplat-master (#15295)
* Bump msbuild to track xplat-master

* [msbuild] Bump to track xplat-master

.. which effectively enables use of the shared compiler.
2019-06-21 21:36:43 +02:00
Filip Navara 42eb919d52 Add basic implementation of Environment.FailFast (#15279)
* Add basic implementation of Environment.FailFast

* Add comment about __failfast on Windows 8+

* Re-add some test exclusions and clarify the comment around them

* Update icall.c
2019-06-21 18:52:12 +02:00
Aleksey Kliger (λgeek) 71ceefccf6 [api] Add a mono/metadata/profiler-legacy.h header (#15098)
* [api] Add mono/metadata/profiler-legacy.h header

Add a header with the legacy Mono profiler API.  New code should use the API
from mono/metadata/profiler.h.

* [profiler] Mark the legacy profiler API with MONO_DEPRECATED

* [runtime] Don't install the profiler-legacy.h header

Make the decision later whether these should be in the set of public API headers
2019-06-21 18:47:26 +02:00
monojenkins 3d48d85472 [ci] One more workaround for Git 2.22.0 (#15326)
Homebrew Git 2.22.0 misparses the submodule command and passes arguments like --hard and -xdff to git-submodule instead of to git-reset or git-clean.  Passing the entire command as a single argument seems to help.

Git mailing list bug report: https://public-inbox.org/git/[email protected]/
2019-06-21 10:57:38 -04:00
dotnet-maestro c6b8e90230 Update dependencies from https://github.com/dotnet/arcade build 20190620.1
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19320.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19320.1
2019-06-21 16:26:01 +02:00
monojenkins 304899de1b [bcl] Update BCL Linked Size 2019-06-21 13:56:27 +02:00
Egor Bogatov 9eb1f19909 Update Makefile 2019-06-21 13:55:33 +02:00
EgorBo 44cba4e442 fix build-test-corefx command 2019-06-21 13:55:33 +02:00
Kenneth Pouncey b32b5ceba2 Make sure that bindings are initialized in mono_wasm_new. (#15274)
- Fixes `this.mono_string_get_utf8 is not a function`.
2019-06-21 13:38:50 +02:00
Dmitry Kalyanov 480f636808 TransactionScope now actually uses IsolationLevel that was specified in TransactionOptions
Previously, using TransactionScope would always produce transactions
that use serializable isolation level even when user specified another
level (like ReadCommited).

fixes #13945
2019-06-21 11:27:59 +02:00
Stephen Toub a8b1b2ed20 Fix visibility on Exception.InnerExceptionPrefix (dotnet/coreclr#25278)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-21 08:34:24 +02:00
Santiago Fernandez Madero 93fdf6489c Address nullable feedback for S.T.Overlapped, S.Memory, S.R.InteropServices (#25214)
* Address nullable feedback for System.Threading.Overlapped

* Address nullable feedback for System.Memory

* Address nullable for System.Runtime.InteropServices

* Update xml comments and make CultureInfo nullable to match String.ToLower/Upper feedback

* PR feedback: revert accepting null comparer in BinarySearch

* Disable test that needs to be updated in corefx

Signed-off-by: dotnet-bot <[email protected]>
2019-06-21 08:34:24 +02:00
Dan Moseley 5dee50f30c Exception format cleanup subset (dotnet/coreclr#25185)
* Change ArgumentException message to one line

* Disable 2 tests

* Remove duplication in AggregateException.ToString()

* Inner exceptions on new lines

* Caps letter

* Typo

Signed-off-by: dotnet-bot <[email protected]>
2019-06-21 08:34:24 +02:00
lateralusX ad90ebd3ab Make copy_stack_data support configurable.
By default when called through macros like MONO_ENTER_GC_UNSAFE
code will always dump current registers into stack space on hybrid/coop.
The registers will be scanned but only when hitting a GC and when running
in GC unsafe, that will only happen when we block in the enter or hit a safe
point at a later point in time. There are currently a couple of frequent
scenarios where we go into GC safe when thread won't be suspended and in
that case the copy of the registers will only reflect the state when
entering GC unsafe.

Commits add option to enable/disable copy_stack_data support in configure step:
--enable-copy-stack-data, --disable-copy-stack-data.

copy_stack_data is enabled by default on all platforms except desktop platforms
macos, linux and windows (both 32/64-bit), since these platforms already support
full context as well as capabilities to scan the stack of a running thread.
2019-06-20 23:34:56 -05:00
Larry Ewing 7af0e36db3 Merge pull request #15293 from lewing/wasm-clean
[wasm] Add some missing directories to the clean target
2019-06-20 22:21:08 -05:00
Larry Ewing efa06b3ecd Merge pull request #15291 from lewing/doublecheck-290
[wasm] Add an explicit test for https://github.com/dotnet/try/issues/290
2019-06-20 20:46:39 -05:00
Aleksey Kliger (λgeek) 18ce655751 [loader] remap only System.Threading.Tasks.Dataflow older than 4.0.0.0 (#15266)
[loader] remap only System.Threading.Tasks.Dataflow older than 4.0.0.0

Allow through requests for newer versions unchanged.
2019-06-20 19:25:43 -04:00
Thays Grazia 405d521149 [debugger] Debugger doesn't break on unhandled exceptions (#15234)
* When the catch found is in a MONO_WRAPPER_RUNTIME_INVOKE the exception should be treated as an UNHANDLED by the debugger.

* Adding unit test.

* trying to fix side effect.

* I've tested the MonoTests.DebuggerTests.UnhandledException_2 test on VsWin using .net Framework and I think the test is wrong because the first break is in throw and the next break will be on breakpoint.
In our implementation we were ignoring the unhandled exception and stopping only on breakpoint.
2019-06-20 18:48:13 -04:00
Larry Ewing b20ef4cc36 [wasm] Add some missing directories to the clean target 2019-06-20 14:28:20 -05:00
Vlad Brezae b4e43abbcc [jit] Make alias analysis properly handle alias cloberring (#15255)
Alias analysis attempts to remove indirection when using OP_LDADDR (which fetches the address of a variable). For example :

	ldaddr r1 <- r0
	load_mem r2 <- r1

is equivalent to

	mov r2 <- r0

The remembered r1 alias needs to be forgotten in case r1 is changed. We were handling this for direct assignment with MOV, but not for the more subtle case where an alias to r1 would be passed to a call, which could lead to changing r1.

https://github.com/mono/mono/issues/14872
2019-06-20 21:35:18 +03:00
Larry Ewing 653a12ad3c Add an explicit test for https://github.com/dotnet/try/issues/290 2019-06-20 13:25:29 -05:00
dotnet-maestro 6023f36749 Update dependencies from https://github.com/dotnet/arcade build 20190619.25
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19319.25
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19319.25
2019-06-20 18:29:47 +02:00
Vlad Brezae 1120e82312 [jit] Add debug option for using clr memory model (#15019)
* [jit] Move value storing in a new write barrier emitting method

Brings together ref storing and the write barrier generation.

* [jit] Add debug option to attempt using the CLR memory model

Which prevents reordering of stores. Part of the way this is done on CoreCLR is by adding memory barriers to the write barrier, so we attempt a similar behavior.

This is not a complete implementation.

* [build] Add memory model flag to csc when building the bcl

This should prevent random crashes during the build

* [build] Add memory model flag to compiler server

* [build] Add memory model flag to test building
2019-06-20 19:18:26 +03:00
Jay Krell fb6ce0df91 [Coop] Establish handle frame in reflection_setup_internal_class. (#15248)
It is called in a loop by ves_icall_TypeBuilder_create_runtime_class.
See https://github.com/mono/mono/issues/15247.
2019-06-20 11:00:36 -04:00
Jay Krell 2104d1d957 Write to the same union member that is later read. (#15269)
It should not matter in this case, but it is the usual practise,
and not strictly valid otherwise.
2019-06-20 10:57:04 -04:00
Jay Krell e24f1ed945 [interp] Optimize CEE_MONO_TLS due their being only one live producer. (#15245)
There is a dead producer in Boehm. (see https://github.com/mono/mono/pull/15241)
This combined with https://github.com/mono/mono/pull/15226
will leave no remaining uses of mono_tls_get_tls_getter.

If there were really multiple values of the enum, consider
doing the switch in transform instead of interp.
2019-06-20 16:39:20 +03:00
Radek Doulik 688d7f0938 [Mono.Profiler.Aot] Add profile writer
It writes the profile data to given stream. It comes from
https://github.com/radekdoulik/aotprofile-tool
2019-06-20 14:13:34 +02:00
Egor Bogatov 5a21eb4288 Fix arcade stuff 2019-06-20 14:07:13 +02:00
Johan Lorensson fb35615c56 Interpreter PR builds run to many test suites. (#15251)
Fix makes sure we don't run full test suite on PR builds or when targeting
arm architectures or Windows.
2019-06-20 13:30:47 +02:00
monojenkins 3b53959f1e [csproj] Update project files 2019-06-20 12:04:07 +02:00
Marek Safar 7373266eb5 Bump linker & cecil 2019-06-20 12:04:07 +02:00
Johan Lorensson dfb46dad7d Add Windows x64 Full AOT Interpreter support on CI. (#15127)
Add support to run full interpreter test suite on CI for Windows x64 Full AOT.
Since Windows x64 Full AOT uses a different full AOT profile WinAOT a separate
testing profile was setup for the interpreter testing. NOTE, this profile is
pure for testing, inline with the other testing profiles. It is also reusing
most of its sources from other sources files (mainly WinAOT) to reduce maintenance
of the profile.

Commit also includes some smaller adjustments needed in order to get full
pass rate for Windows x64 Full AOT + Interpreter on CI.
2019-06-20 09:23:39 +02:00
Aleksey Kliger b2a860c949 [coop] Mark a few functions with MONO_PROFILER_API 2019-06-20 00:22:01 +02:00
Aleksey Kliger cf3bb20575 [runtime] Don't do GC Unsafe transition in mono_gchandle_free
Xamarin.Mac can call back here from a native thread that isn't attached to the
runtime.  SGen gchandle code is lockfree, and the profiler event can fire
without involving Mono.
2019-06-20 00:22:01 +02:00
Aleksey Kliger ee74ee90b3 [profiler] Don't switch to GC Safe when locking if thread isn't attached to the
runtime.

Some profiler events (for example gc_handle_deleted) could be raised on threads
that aren't attached to the runtime.  Since the thread won't participate in
Mono's stop the world (since the runtime can't see it), there's no need to do
the transition to GC Safe while the thread is trying to grab the profiler
buffer lock.
2019-06-20 00:22:01 +02:00
Jay Krell 09869a9c8b Fix compiler warnings. (#15196)
* Fix compiler warnings.
 unused variable: save_sigint
 unused variable: mono_environ
 uprototyed function: mono_marshal_xdomain_copy_value

* add static
2019-06-19 18:18:31 -04:00
Jay Krell 6e1cc91d8d Remove unused MONO_PATCH_INFO_SET_TLS_TRAMP, corresponding unused JIT icalls, etc. (#15201) 2019-06-19 16:25:34 -04:00
Marek Safar a4ef5c3590 Update CODEOWNERS 2019-06-19 20:39:03 +02:00
dotnet-maestro 7390b78b2d Update dependencies from https://github.com/dotnet/arcade build 20190619.1
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19319.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19319.1
2019-06-19 19:10:45 +02:00
Ryan Lucia 2a82614f98 Correctly set ReflectionType with inherited generic methods (#15233)
* Correctly set ReflectionType with inherited generic methods

* Allow ValueType

* Re-enable relevant test

* Switch to mono_class_from_mono_type_internal
2019-06-19 13:08:39 -04:00
Johan Lorensson 7ac0287a7b Increase timeout for interpreter fullaotmixed regression. (#15225)
On Windows (partly due to slow shell execution) this test take longer to
complete than current given timeout.
2019-06-19 17:45:10 +02:00
Zoltan Varga 7341c9f636 [jit] Avoid calling mini_jit_info_table_find () in mini_add_method_trampoline () unless needed. (#15239) 2019-06-19 10:52:27 -04:00
Filip Navara c1068f9db5 [netcore] Update the checks for ByRef return type in reflection invocation (#14967)
* Update the checks for ByRef return type in reflection invocation to also check for IsByRefLike types. This was missed as part of #13901 and noted in the description.

Fixes #14959, #14960, #14961, #14963

* Modify the reflection checks for IsByRef/IsByRefLike to check for same conditions as CoreCLR

* Handle reflection invocation with ref void return type

* Update test-ref-07 to have Main method that is valid to invoke through reflection (as the test runner does)

* Update expected IL
2019-06-19 08:51:43 +02:00
Zoltan Varga 611ba9a018 [interp] Implement tracing of return values. Improve printing of arrays. (#15217) 2019-06-18 23:03:13 -04:00
Alexis Christoforides 7c4300aa7f [filewatcher] Fall back to default if a new instance cannot be created 2019-06-19 05:26:57 +03:00
Alexis Christoforides cc6a2d4606 Improve robustness of post-install script (#15213)
We have several issues (e.g. https://github.com/mono/mono/issues/14181) where the installer occasionally fails in the postinstall script for unknown causes, on some hosts.

This seems to be sometimes the culpable section and this patch should help trace the issue or alleviate the common symptoms of it (see https://github.com/mono/mono/issues/14181)
2019-06-19 02:21:38 +03:00
Bernhard Urban e3f8596156 [arm] fix some more host/target pointer size issues (#15223)
[arm] fix some more host/target pointer size issues

As suggested by Jay in https://github.com/mono/mono/issues/15058#issuecomment-501995400

I haven't seen fixing it any specific issue, but we should still backport it to `2019-06` in order to avoid hard-to-debug regressions.

Contributes to https://github.com/mono/mono/issues/9621
2019-06-18 17:45:59 -04:00
Bernhard Urban fe7027935b [mini] fix OP_SEQ_POINT of other backends regarding target pointer size (#15222)
See https://github.com/mono/mono/pull/15205#issuecomment-502925368

Theoretically fixes cross compilation on a 32bit host to a 64bit target.
2019-06-19 00:06:54 +03:00
monojenkins 95259f9947 Bump API snapshot submodule 2019-06-18 20:53:25 +02:00
Steve Pfister 2d94e79f5b Bump mono/corefx to fix https://github.com/mono/mono/issues/14864 2019-06-18 20:53:25 +02:00
Ryan Lucia 7a1605cb07 [netcore] Don't default to an empty array for a null publicKey 2019-06-18 19:44:51 +02:00
dotnet-maestro d3d499f34e Update dependencies from https://github.com/dotnet/arcade build 20190618.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19318.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19318.2
2019-06-18 19:35:04 +02:00
Ryan Lucia 6c5bbbd757 [eglib] Don't escape chars in argv parsing enclosed in single quotes (#15047)
Fixes #14724

This may need to be reverted if customers are relying on the existing behavior, but we can cross that bridge when we come to it.

The larger issue of incompatibility with .NET Core should be addressed later.
2019-06-18 13:30:31 -04:00
Marek Safar 3428cec003 Mirror changes from mono/corefx,corert,coreclr (#15221)
Mirror changes from mono/corefx,corert,coreclr
2019-06-18 12:44:36 +02:00
Marek Safar 7052e60600 [netcore] Excluded tests update 2019-06-18 11:25:51 +02:00
Marek Safar d44da802a9 [System.Private.CoreLib] Merge fixes 2019-06-18 11:10:59 +02:00
Marek Safar e29f79e68e Merge remote-tracking branch 'upstream/master' into mirror-merge-10235436 2019-06-18 10:54:06 +02:00
Marek Safar 3d311876d2 [netcore] Reflect shared partition changes 2019-06-18 10:52:05 +02:00
Marek Safar 879c8501c9 [System.Private.CoreLib] Align on CA naming for easier sharing 2019-06-18 10:10:16 +02:00
naricc ac2894920a Implement GC.GetAllocatedBytes (#14779) 2019-06-18 09:08:21 +02:00
Bernhard Urban d5a35798c6 [arm] use target_mgreg_t instead of plain pointer type in sdb breakpoint (#15205)
[arm] use target_mgreg_t instead of plain pointer type in sdb breakpoint

Fixes Debug mode for iOS 32bit.

Contributes to https://github.com/mono/mono/issues/15058 & https://github.com/mono/mono/issues/9621

Diff before (left old cross32, right new cross32 running on amd64 host _before_ this fix):
<img width="1257" alt="Screenshot 2019-06-17 at 14 09 51" src="https://user-images.githubusercontent.com/75403/59603413-a9bd4f00-9109-11e9-8f18-aa7b4ca5817f.png">

/cc @mandel-macaque
2019-06-18 02:46:00 -04:00
Jay Krell fc74334e20 Allocate MonoJumpInfo instead of MonoAotCompile. 2019-06-17 23:41:26 -05:00
Kenneth Pouncey 67b4d55d91 Simplify header item fetch using the Headers entries iterator.
- Use the Headers `entries{}` iterator object.
- This replaces the `foreach` delegate call where the object returned in the delegate was being disposed of too early.
- The foreach delegate takes three objects name, value, other.  The `other` being disposed was causing a problem when calling back into the delegate later.
- Removes the lamda expression called by the delegate without the worry of a memory leak if the finally is removed.
2019-06-17 21:56:47 -05:00
Kenneth Pouncey f7bba7172e Add tests to the IssuesTestSuite to catch regressions
- See issue https://github.com/mono/mono/issues/14940
2019-06-17 21:56:47 -05:00
Kenneth Pouncey 8f864e4d92 Fix problem with fetch response header iteration prematurely disposing of the header object.
- Use the iterator of the headers object do iteration over the fetch response headers.
2019-06-17 21:56:47 -05:00
Jay Krell a97408ac1e Revert "threads.c:6403:6: warning: 'current_idx' may be used uninitialized in this function [-Wmaybe-uninitialized]"
This reverts commit bf2f1abe01978102c22d76b4a10996e8d8b50b0f.

The logic is not clear as to if there is a bug, so leave the warning.
There are two uses of the possibly uninitialized variable.
One is guarded and not a bug.
The other is not clear, and don't want to blindly use 0.
2019-06-17 21:53:08 -05:00
Jay Krell af2c235f31 PR: Restore nearby unusual formatting. 2019-06-17 21:53:08 -05:00
Jay Krell 9c70b53cc3 intrinsics.c: In function 'mini_emit_inst_for_method':
intrinsics.c:1580:11: warning: 'result' may be used uninitialized in this function [-Wmaybe-uninitialized]
     *dest = result;
     ~~~~~~^~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 6e19637797 debugger-engine.c: In function 'set_bp_in_method':
debugger-engine.c:354:11: warning: variable 'code' set but not used [-Wunused-but-set-variable]
  gpointer code;
           ^~~~
2019-06-17 21:53:08 -05:00
Jay Krell 496cb4cfce tramp-arm64-gsharedvt.c: In function 'mono_arch_get_gsharedvt_trampoline':
tramp-arm64-gsharedvt.c:217:54: warning: variable 'callee_stack_area_offset' set but not used [-Wunused-but-set-variable]
  int caller_reg_area_offset, callee_reg_area_offset, callee_stack_area_offset;
                                                      ^~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 179b5e85cf mini-arm64-gsharedvt.c: In function 'mono_arch_get_gsharedvt_call_info':
mini-arm64-gsharedvt.c:142:29: warning: variable 'gsig' set but not used [-Wunused-but-set-variable]
  MonoMethodSignature *sig, *gsig;
                             ^~~~
2019-06-17 21:53:08 -05:00
Jay Krell b504d7e6d8 interp/mintops.c: In function 'mono_interp_dis_mintop':
interp/mintops.c:87:37: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'long int' [-Wformat=]
   g_string_append_printf (str, " %lld", (gint64)READ64 (ip + 1));
2019-06-17 21:53:08 -05:00
Jay Krell 452209414e In file included from object.c:40:0:
object.c: In function 'mono_class_proxy_vtable':
../../mono/metadata/profiler-private.h:191:4: warning: 'pvt' may be used uninitialized in this function [-Wmaybe-uninitialized]
    mono_profiler_raise_ ## name args; \
    ^~~~~~~~~~~~~~~~~~~~
object.c:2338:19: note: 'pvt' was declared here
  MonoVTable *vt, *pvt;
                   ^~~
2019-06-17 21:53:08 -05:00
Jay Krell bdb20ebd80 w32file-unix.c: In function 'mono_w32file_get_file_size':
w32file-unix.c:4994:19: warning: 'length_hi' may be used uninitialized in this function [-Wmaybe-uninitialized]
  return length | ((gint64)length_hi << 32);
                   ^~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 457d0f02c5 interp/interp.c: In function 'interp_entry.constprop':
interp/interp.c:1847:3: warning: 'orig_domain' may be used uninitialized in this function [-Wmaybe-uninitialized]
   mono_threads_detach_coop (orig_domain, &attach_cookie);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 3357a1a611 image.c:1351:20: warning: variable 'header' set but not used [-Wunused-but-set-variable]
MonoDotNetHeader *header;
                    ^~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 00570db6a4 exception.c:747:1: warning: 'mono_new_exception_argument_out_of_range' defined but not used [-Wunused-function]
mono_new_exception_argument_out_of_range (const char *arg, MonoError *error)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell bc5d1a48d8 os-event-unix.c: In function 'mono_os_event_wait_multiple':
os-event-unix.c:195:12: warning: 'start' may be used uninitialized in this function [-Wmaybe-uninitialized]
    elapsed = mono_msec_ticks () - start;
    ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 96129ba873 In file included from ../../mono/eglib/glib.h:45:0,
from os-event.h:9,
                 from os-event-unix.c:11:
../../mono/eglib/eglib-remap.h:146:25: warning: 'data' may be used uninitialized in this function [-Wmaybe-uninitialized]
 #define g_ptr_array_add monoeg_g_ptr_array_add
                         ^~~~~~~~~~~~~~~~~~~~~~
os-event-unix.c:121:19: note: 'data' was declared here
  OSEventWaitData *data;
                   ^~~~
2019-06-17 21:53:08 -05:00
Jay Krell faadb64774 marshal.c: In function 'mono_marshal_get_runtime_invoke_full':
marshal.c:2662:8: warning: 'sig_cache' may be used uninitialized in this function [-Wmaybe-uninitialized]
    res = (MonoMethod *)g_hash_table_lookup (sig_cache, sig_key);
    ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 64c321293b sgen-mono.c:260:1: warning: 'install_noilgen' defined but not used [-Wunused-function]
install_noilgen (void)
2019-06-17 21:53:08 -05:00
Jay Krell 0b5b42f6c7 In file included from sgen-new-bridge.c:31:0:
sgen-dynarray.h:331:1: warning: 'dyn_array_ptr_set_all' defined but not used [-Wunused-function]
 dyn_array_ptr_set_all (DynPtrArray *dst, DynPtrArray *src)
 ^~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell e94887a18c threads.c:6403:6: warning: 'current_idx' may be used uninitialized in this function [-Wmaybe-uninitialized]
if (i == current_idx)
      ^
threads.c:6549:6: note: 'current_idx' was declared here
  int current_idx;
      ^~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 67680a2e81 sgen-dynarray.h:248:1: warning: 'dyn_array_ptr_set' defined but not used [-Wunused-function]
dyn_array_ptr_set (DynPtrArray *da, int x, void *ptr)
 ^~~~~~~~~~~~~~~~~

sgen-dynarray.h:202:1: warning: 'dyn_array_ptr_init' defined but not used [-Wunused-function]
 dyn_array_ptr_init (DynPtrArray *da)
 ^~~~~~~~~~~~~~~~~~

sgen-dynarray.h:194:1: warning: 'dyn_array_int_is_copy' defined but not used [-Wunused-function]
 dyn_array_int_is_copy (DynIntArray *da)
 ^~~~~~~~~~~~~~~~~~~~~

sgen-dynarray.h:188:1: warning: 'dyn_array_int_copy' defined but not used [-Wunused-function]
 dyn_array_int_copy (DynIntArray *dst, DynIntArray *src)
 ^~~~~~~~~~~~~~~~~~

sgen-dynarray.h:182:1: warning: 'dyn_array_int_ensure_independent' defined but not used [-Wunused-function]
 dyn_array_int_ensure_independent (DynIntArray *da)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sgen-dynarray.h:168:1: warning: 'dyn_array_int_get' defined but not used [-Wunused-function]
dyn_array_int_get (DynIntArray *da, int x)
 ^~~~~~~~~~~~~~~~~

sgen-dynarray.h:161:1: warning: 'dyn_array_int_add' defined but not used [-Wunused-function]
 dyn_array_int_add (DynIntArray *da, int x)
 ^~~~~~~~~~~~~~~~~

sgen-dynarray.h:147:1: warning: 'dyn_array_int_size' defined but not used [-Wunused-function]
 dyn_array_int_size (DynIntArray *da)
 ^~~~~~~~~~~~~~~~~~
sgen-dynarray.h:141:1: warning: 'dyn_array_int_uninit' defined but not used [-Wunused-function]
 dyn_array_int_uninit (DynIntArray *da)
 ^~~~~~~~~~~~~~~~~~~~
sgen-dynarray.h:135:1: warning: 'dyn_array_int_init' defined but not used [-Wunused-function]
 dyn_array_int_init (DynIntArray *da)
 ^~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell ae357bebbd In file included from sre.c:30:0:
../../mono/metadata/reflection-cache.h:43:1: warning: 'free_reflected_entry' defined but not used [-Wunused-function]
 free_reflected_entry (ReflectedEntry *entry)
 ^~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell cc60fe04d0 sre.c:2067:1: warning: 'is_sr_mono_field' defined but not used [-Wunused-function]
is_sr_mono_field (MonoClass *klass)
 ^~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 54366ee658 w32handle.c: In function 'mono_w32handle_wait_multiple':
w32handle.c:1146:13: warning: 'start' may be used uninitialized in this function [-Wmaybe-uninitialized]
     elapsed = mono_msec_ticks () - start;
     ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell f2f5c45d9d custom-attrs.c:269:1: warning: 'load_cattr_type_object' defined but not used [-Wunused-function]
load_cattr_type_object (MonoImage *image, MonoType *t, gboolean header, const char *p, const char *boundp, const char **end, MonoError *error, guint32 *slen)
 ^~~~~~~~~~~~~~~~~~~~~~
2019-06-17 21:53:08 -05:00
Jay Krell 267a75f365 sre-save.c:114:1: warning: 'swap_with_size' defined but not used [-Wunused-function]
swap_with_size (char *dest, const char* val, int len, int nelem) {
2019-06-17 21:53:08 -05:00
Jay Krell 404a03b87e mono-mmap.c: In function 'mono_file_map_error':
mono-mmap.c:401:82: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 5 has type 'guint64 {aka long unsigned int}' [-Wformat=]
    *error_message = g_strdup_printf ("%s failed file:%s length:0x%zX offset:0x%lluX error:%s(0x%X)\n",
2019-06-17 21:53:08 -05:00
Jay Krell a37f71a4ea threadpool.c: In function 'mono_threadpool_remove_domain_jobs':
threadpool.c:571:91: warning: 'end' may be used uninitialized in this function [-Wmaybe-uninitialized]
    res = mono_coop_cond_timedwait (&tpdomain->cleanup_cond, &threadpool.domains_lock, end - now);
2019-06-17 21:53:08 -05:00
Sung Yoon Whang e734fca321 make DiagnosticCounter and its children lock 'this' instead of an arbitrary String (#25173)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:13 +00:00
Ben Adams 0ca0d1dfd9 IndexOfAny(values.Length == 0) == -1 (dotnet/coreclr#25075)
* IndexOfAny(values.Length == 0) == -1

* Skip tests

Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:12 +00:00
Adam Sitnik b58c351daf don't use var
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:10 +00:00
Adam SitnikandJan Kotas 62f597ea23 Apply suggestions from code review
Co-Authored-By: Jan Kotas <[email protected]>

Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:09 +00:00
Adam Sitnik 2597806f5f set the charset to Ansi
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:08 +00:00
Adam Sitnik a22d66c98c simplify the code
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:07 +00:00
Adam Sitnik 0dcea57a81 lock the dictionary
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:07 +00:00
Adam Sitnik fd52429be9 move the sort key logic to a dedicated helper type to avoid static field initialization order issues
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:06 +00:00
Adam Sitnik 3a45ebfa72 introduce cache for sort handles to prevent from memory leak in certain scenarios
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:05 +00:00
Adam Sitnik d7506847fe use IntPtr instead of CriticalHandle to avoid resurrection issues. It's ok to never free the handle
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:05 +00:00
Adam Sitnik ba42c41536 seal the CriticalSortHandle class
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:04 +00:00
Adam Sitnik 67e2d3ff62 change SafeSortHandle to be CriticalSortHandle to avoid expensive ref counting
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:03 +00:00
Marek Safar 3e3925ff93 Move parts of RuntimeType to shared partition (dotnet/coreclr#25176)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:02 +00:00
Sung Yoon Whang b723b841c0 Make DisplayUnits string.Empty by default instead of null (#25104)
* Make DisplayUnits string.Empty by default instead of null

* PR feedback on DisplayName and setter validation

Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:01 +00:00
Tarek Mahmoud Sayed 321b02ce2c Fix French Decimal number parsing (dotnet/coreclr#25154)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:01 +00:00
Jan Kotas 18a4099fbd Optimize Activator.CreateInstance (dotnet/coreclr#25145)
* Optimize Activator.CreateInstance
- Short-circuit Activator.CreateInstance<T>() for value types without default constructor
- Cache default constructor delegate on RuntimeType instead of fixed-size singleton cache

Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:37:00 +00:00
Stephen Toub 4a29e94abb Update Corelib to adapt to compiler nullability updates
Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:36:59 +00:00
Viktor Hofer ee64a8364f Delete dead uapaot code (dotnet/corefx#38330)
* Delete dead uapaot code

Signed-off-by: dotnet-bot <[email protected]>
2019-06-17 22:36:58 +00:00
Egor Bogatov c04dc84a9f fix RuntimeType.cs 2019-06-17 21:14:06 +02:00
Bernhard Urban 1e5ca1479c [watch] consistent naming for watchOS min version (#15202)
[watch] consistent naming for watchOS min version

See https://github.com/xamarin/xamarin-macios/pull/6307
2019-06-17 13:26:09 -04:00
Jay Krell 7d824dc2ae Fix race conditions in mono_lazy_initialize,
which is meant to be all about handling race conditions.

1. Less severe:
Add missing read barrier to mono_lazy_initialize().

This is unfortunate.

The general pattern of:

if initialized
  use the data
else:
  initialize the data, possibly with lots of locks and barriers
  mark initialized

is racy, because "use the data" can be scheduled ahead of "if initialized".

"Barriers come in pairs" generally, and this was missing one.

It depends somewhat. If the data is all pointers, initialized in the else path,
from null to non-null, and use includes dereferencing, which is a common
but not universal case, then it is ok on all non-Alpha processors, due
to "data dependency".

But if the data includes reading globals, then there is a race.

This is a bit of a slow down on fast paths on arm, and possibly
other architectures.

There are barrier-free ways to solve this, involving a thread local,
that should seriously be considered, and applied throughout the runtime.

The runtime has a lot of on-demand initialization and a lot of looks racy.

See www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2660.htm.
Which would have to be adopted to be coop-friendly which should not be difficult.

Notice that the existing "lazy" mechanism is also not coop-friendly.
That is not changed by this PR.

2. Change `status` to `*lazy_init` to fix severe race condition.
2019-06-17 15:31:35 +02:00
dotnet-maestro 6bd68bc2a4 Update dependencies from https://github.com/dotnet/arcade build 20190615.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19315.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19315.2
2019-06-17 15:30:58 +02:00
Jay Krell 8966528c21 Replace MONO_PATCH_INFO_TRAMPOLINE_FUNC_ADDR with MONO_PATCH_INFO_JIT_ICALL_ADDR. (#15053)
* Replace MONO_PATCH_INFO_TRAMPOLINE_FUNC_ADDR with MONO_PATCH_INFO_JIT_ICALL_ADDR.

* PR: Abstract the TrampolineType / jit_icall_id translation.
2019-06-17 01:25:53 -04:00
Vlad Brezae fd089b6e4d [interp] Prefer interp_in wrappers over trampolines (#14684)
[interp] Prefer interp_in wrappers over trampolines
2019-06-15 17:25:06 -04:00
Katelyn Gadd 59a0d1e70f Add the compiler server log to the gitignore 2019-06-15 16:16:12 -05:00
dotnet-maestro 7269db058e Update dependencies from https://github.com/dotnet/arcade build 20190614.27
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19314.27
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19314.27
2019-06-15 16:08:10 +02:00
Aleksey Kliger 8a0495ea3b [debugger-agent] Ignore thread_end event if thread is detached
The thread_stopped profiler event can be raised by the thread_info_key_dtor tls
key destructor when the thread is already doesn't have a domain set.  In that
case, don't call process_profiler_event since it cannot handle a thread with
null TLS values.

Addresses https://github.com/xamarin/xamarin-android/issues/2920
with the following stack trace

```
* thread #20, name = 'Filter', stop reason = signal SIGSEGV: invalid address (fault address: 0xbc)
  * frame #0: libmonosgen-2.0.so`mono_class_vtable_checked(domain=0x0000000000000000, klass=0x0000007200230648, error=0x00000071e92f9178) at object.c:1890
    frame #1: libmonosgen-2.0.so`get_current_thread_ptr_for_domain(domain=0x0000000000000000, thread=0x00000071ebfec508) at threads.c:595
    frame #2: libmonosgen-2.0.so`mono_thread_current at threads.c:1939
    frame #3: libmonosgen-2.0.so`process_event(event=<unavailable>, arg=<unavailable>, il_offset=<unavailable>, ctx=<unavailable>, events=<unavailable>, suspend_policy=<unavailable>) at debugger-agent.c:3715
    frame #4: libmonosgen-2.0.so`thread_end [inlined] process_profiler_event(event=EVENT_KIND_THREAD_DEATH, arg=0x00000071ebfec508) at debugger-agent.c:3875
    frame #5: libmonosgen-2.0.so`thread_end(prof=<unavailable>, tid=<unavailable>) at debugger-agent.c:3991
    frame #6: libmonosgen-2.0.so`mono_profiler_raise_thread_stopped(tid=<unavailable>) at profiler-events.h:105
    frame #7: libmonosgen-2.0.so`mono_thread_detach_internal(thread=<unavailable>) at threads.c:979
    frame #8: libmonosgen-2.0.so`thread_detach(info=0x00000071e949a000) at threads.c:3215
    frame #9: libmonosgen-2.0.so`unregister_thread(arg=<unavailable>) at mono-threads.c:544
    frame #10: libmonosgen-2.0.so`thread_info_key_dtor(arg=0x00000071e949a000) at mono-threads.c:774
    frame #11: 0x00000072899c58e8 libc.so`pthread_key_clean_all() + 124
    frame #12: 0x00000072899c5374 libc.so`pthread_exit + 76
    frame #13: 0x00000072899c5264 libc.so`__pthread_start(void*) + 44
    frame #14: 0x000000728996617c libc.so`__start_thread + 72
```
2019-06-15 09:31:05 +02:00
Marek Safar b5b58d67cf [System.Private.CoreLib] Extract RuntimeType for easier refactoring 2019-06-15 09:28:47 +02:00
Egor Bogatov 26f0222284 [netcore] Fix CI build script (#15130) 2019-06-15 09:27:15 +02:00
Jay Krell eafcbfd6ed Read only interp callbacks, option 4, without ordering sensitivity. 2019-06-14 18:51:56 -05:00
Jay Krell 1fe336691e Cleanup AIX code a little. 2019-06-14 18:43:36 -05:00
Aleksey Kliger (λgeek) 94725cb85e [llvm] bump to mono/llvm@2bd2f1db18 (#15049)
Emit .fnend directives on arm even for functions which don't have a corresponding mono method.

Fixes https://github.com/mono/mono/issues/15048
2019-06-14 16:17:07 -04:00
Marek Safar ef4c8bbbb0 [netcore] Arcade does not yet have CLSCompliant support, revert to manual settings 2019-06-14 22:00:40 +02:00
Thays Grazia a81cc67327 Implementing reabstraction of default interface methods. (#14790)
Implementing reabstraction of default interface methods.
Fix #14495
2019-06-14 16:15:38 -03:00
Marek Safar 49dc99be58 [netcore] Add more Arcade magic for signing 2019-06-14 21:10:20 +02:00
Alexander Kyte 877a2b6548 [metadata] Move problematic image out-param usage (#15131) 2019-06-14 14:38:09 -04:00
dotnet-maestro 377030eaab Update dependencies from https://github.com/dotnet/arcade build 20190613.18
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19313.18
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19313.18
2019-06-14 18:41:59 +02:00
Aleksey Kliger (λgeek) 0e0f1b9be0 [mini] Move some MONO_API functions to public header (#15105)
* [mini] Move some MONO_API functions to public header

And mark them all MONO_RT_EXTERNAL_ONLY.

mono_set_use_llvm is new - Xamarin.Android previously assigned to mono_use_llvm
directly.
2019-06-14 11:33:11 -04:00
Johan Lorensson b33c379057 Enable aotcheck and test-aot CI tests on Windows x64. (#15062) 2019-06-14 15:56:24 +02:00
Egor Bogatov 4212270b23 Fix csproj 2019-06-14 13:16:46 +02:00
Johan Lorensson a382edc9e1 Deadlock in loader when using bundling. (#15061)
We can hit the following deadlock when loading assemblies (using reflection)
together with bundling:

One thread calls mono_assembly_open_from_bundle, that will take assemblies lock
and then try to load assembly. If this race with a different load of the
same assemblies, one will lose, and that method will call mono_image_close that
in turn will call unload hook (mono_class_unregister_image_generic_subclasses)
that will take loader lock.

If we at the same time have a different thread that calls mono_class_create_from_typedef
, it will take loader lock and then it might end up calling mono_assembly_load_reference
that will take assemblies lock, but since that thread takes the locks
in different order compare to first thread, they could deadlock.

Looking into the use of assemblies lock in mono_assembly_open_from_bundle it
uses a static variable, bundles, that is not protected in other scenarios
and only set by mono_register_bundled_assemblies, normally called during
boot. The method called in the loop mono_image_open_from_data_internal is called
at several places without locks (and looks internal using image lock), so that
doesn't need to be protected. The rest is local data access and the only thing the
lock could do is serialize the whole loop between threads (but that shouldn't be needed).

Fix will remove the locking and also eliminating the incorrect lock order
causing the deadlock.
2019-06-14 11:20:28 +02:00
Marek Safar 9010d2f290 [netcore] Clean up netcore build 2019-06-14 11:03:35 +02:00
Ben Adams 12b737d980 Intrinsicify SpanHelpers.IndexOf(char) (dotnet/coreclr#22505)
* Helpers to support Intrinsics in SpanHelpers.Char

* Intrinsicify SpanHelpers.IndexOf(char)

* Feedback

* fix

* Fix assert

* Improve comment warning

* fix

* fix

* Fix

* Fix

Signed-off-by: dotnet-bot <[email protected]>
2019-06-14 09:37:52 +02:00
Jay Krell 467fd6516b Fix S390x compile since #14870 and a warning. (#15056) 2019-06-14 00:37:06 -05:00
Larry Ewing 56e27ca309 [wasm] fix most cases of numeric debugging for wasm
Fix the signededness of numeric types and use double
for the display of everthing since we are stuck with that
on the js side.
2019-06-13 22:25:23 -05:00
imhameed e6e9d5f6eb Don't create a cycle when translating IL break to IR. (#14980)
Don't create a cycle when translating IL `break` to IR.

`mono_emit_jit_icall` adds the instruction it generates to the current
basic block, and the redundant use of `MONO_ADD_INS` on an instruction
that has already been added to a BB will create a cycle in the `prev`
link chain.

Fixes #9706.
2019-06-13 13:56:07 -04:00
Jo Shields 1e0890f61f Ensure start-compiler-server.sh is in tarballs (#15071) 2019-06-13 13:10:38 -04:00
Jo Shields 7c99f6f249 [netcore] Add native ARM64 build to --with-core=only matrix (#14536)
* [netcore] Add native ARM64 build to `--with-core=only` matrix
2019-06-13 13:06:56 -04:00
dotnet-maestro 8affbbe98d Update dependencies from https://github.com/dotnet/arcade build 20190612.21
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19312.21
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19312.21
2019-06-13 17:32:22 +02:00
Ryan Lucia 20d83c24d6 Propagate error in mono_unicode_to_external (#14879)
* Add internal version of mono_unicode_to_external to propagate error

* Migrate things to _error version where sensible

* Remove redundant goto

* Print input string when conversion fails

* Remove todos

* Switch everything to MonoError

* Naming: err -> error

* Actually print out *filename correctly
2019-06-13 17:03:34 +02:00
Kenneth Pouncey e795253dc7 Fix generated warning of driver.c (#15036)
The following warning is generated when building master for interpreter as well as during AOT build of driver.c

```
driver.c:346:31: warning: initializing 'const MonoBundledAssembly **' with an expression of type 'MonoBundledAssembly **' discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
                const MonoBundledAssembly **bundle_array = g_new0 (MonoBundledAssembly*, assembly_count + 1);
                                            ^              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2019-06-13 15:15:30 +02:00
Marek Safar ffe5166a9b [System.Private.CoreLib] Add licence headers 2019-06-13 09:45:50 +02:00
Marek Safar f2c74837d2 [netcore] Bump roslyn version 2019-06-13 09:44:12 +02:00
Stephen Toub 5eedd4f813 Fix nullable annotations on Binder.ReorderArgumentArray
Signed-off-by: dotnet-bot <[email protected]>
2019-06-13 09:44:12 +02:00
Stephen Toub 001844340a Remove defunct !s and update some TODO-NULLABLE comments
Signed-off-by: dotnet-bot <[email protected]>
2019-06-13 09:44:12 +02:00
dotnet-maestro[bot] 8eec14a2f1 Update dependencies from dotnet/arcade (dotnet/corefx#38474)
* Update dependencies from https://github.com/dotnet/arcade build 20190611.2

- Microsoft.DotNet.XUnitExtensions - 2.4.1-beta.19311.2
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19311.2
- Microsoft.DotNet.VersionTools.Tasks - 1.0.0-beta.19311.2
- Microsoft.DotNet.ApiCompat - 1.0.0-beta.19311.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19311.2
- Microsoft.DotNet.Build.Tasks.Configuration - 1.0.0-beta.19311.2
- Microsoft.DotNet.Build.Tasks.Feed - 2.2.0-beta.19311.2
- Microsoft.DotNet.Build.Tasks.Packaging - 1.0.0-beta.19311.2
- Microsoft.DotNet.CodeAnalysis - 1.0.0-beta.19311.2
- Microsoft.DotNet.CoreFxTesting - 1.0.0-beta.19311.2
- Microsoft.DotNet.GenAPI - 1.0.0-beta.19311.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19311.2
- Microsoft.DotNet.RemoteExecutor - 1.0.0-beta.19311.2
- Microsoft.DotNet.GenFacades - 1.0.0-beta.19311.2

* Update `where T : object` constraint to be `where T : notnull`

* Update several TODO-NULLABLE comments

Signed-off-by: dotnet-bot <[email protected]>
2019-06-13 09:44:12 +02:00
Alexander Kyte 4ba594ad25 [crash] Print native module name when crash privacy disabled (#14986)
This makes it so that disabling crash privacy ensures we get the full native symbols printed regardless of allow and deny lists.

Before: https://gist.github.com/alexanderkyte/2b1f38a880c61ea15aa4c851e44af578

After: https://gist.github.com/alexanderkyte/ce80788a51761d91890d812874fa2da2
2019-06-13 10:07:10 +03:00
Marek Safar ad3150a5c5 Convert PermissionSet NIE to PlatformNotSupportedException (#25114)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-13 08:49:27 +02:00
Alexis Christoforides 167564efc6 Another speculative fix for 'pkg-config' sometimes getting built as a 32-bit package regardless of arch configuration (#14955)
Mono issue: https://github.com/mono/mono/issues/13804

The hypothesis is that at reconfigure using the configure cache file, the lack of explicit setting of the host somehow breaks the build process.
2019-06-13 06:34:03 +03:00
Zoltan Varga f87af4bfef [wasm] Simplify AOT profiling initialization. (#15038)
Current steps:
- Call MONO.mono_wasm_init_aot_profiler (null) before calling MONO.mono_load_runtime_and_bcl ().
- Call WebAssembly.Runtime.StopProfile () from the c# code to stop profiling.
- The profile data will be in Module.aot_profile_data.
2019-06-12 18:07:24 -04:00
Kenneth Pouncey 7eadeb3747 Add DataView test cases to BindingsTestSuite. 2019-06-12 16:40:46 -05:00
Kenneth Pouncey 98092c280b Add a private routine UnBoxValue to fix cast exceptions. 2019-06-12 16:40:46 -05:00
Kenneth Pouncey ce1d92568c [wasm][bindings] Add binding support for DataView 2019-06-12 16:40:46 -05:00
Jo Shields 607bd23f2e Install bc as a Linux prerequisite (#15046)
Our run-step logic uses `bc` to calculate step times, so `bc` is
needed to avoid ugly things on the console like:
```
../scripts/ci/./run-step.sh: line 59: echo: write error: Broken pipe
```
2019-06-12 15:45:22 -04:00
Jo Shields 698aed3946 Increase timeout on --core=only CI to 90 minutes, for ARM's benefit (#15044) 2019-06-12 15:27:00 -04:00
dotnet-maestro cf87874290 Update dependencies from https://github.com/dotnet/arcade build 20190611.2
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19311.2
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19311.2
2019-06-12 19:00:44 +02:00
Stephen Toub 6a2f0d94d1 Add missing TraceOperationCreation log event in GetStateMachineBox (#25097)
This was removed accidentally as part of overhauling the async infrastructure in 2.1. VS depends on it for some task tracking logic.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-12 18:59:56 +02:00
monojenkins 4471332fd9 Bump API snapshot submodule 2019-06-12 18:57:17 +02:00
Zoltan Varga 918d7b0aaa [corlib] Add missing Unsafe.Unbox () method.
Fixes https://github.com/mono/mono/issues/14871.
2019-06-12 18:57:17 +02:00
Egor Bogatov 3a0191194b [netcore] Enable CoreFX tests on CI for Linux (#15015)
* Enable CoreFX tests on CI for Linux

* implement NativeLibrary.Unix.cs

* add missing libc

* add few -nonamespace

* more flaky tests
2019-06-12 18:55:44 +02:00
Larry Ewing bd7cb7e4d4 Enable the compiler server by default 2019-06-12 10:58:25 -05:00
Marek Safar 225890a180 [System.Private.CoreLib] Clean up few not implemented exceptions 2019-06-12 15:35:29 +02:00
Zoltan Varga a9949782d6 [runtime] Fix gshared support for BeginInvoke ()/EndInvoke () wrappers. (#14985)
Fixes https://github.com/mono/mono/issues/14972.
2019-06-12 09:18:14 -04:00
Vlad Brezae 9c265ace06 [jit] Fix interface cast (#14978)
* [jit] Throw exception if casting fails for special array interface

After the standard interface check (mini_emit_iface_cast), for special interfaces we need to do additional checks. We were skipping these checks if the object was not an array, without throwing a cast exception. This code path is reached for configurations where remoting is disabled.

Fixes https://github.com/mono/mono/issues/14729

* [tests] Add test for interface casting case

This test would have failed on fullaot lane for example.
2019-06-12 15:42:46 +03:00
Marius Ungureanu 669629330f Plug leak of ppdb_filename
mono_image_open_metadata_only and its callees all dupe the filename string, so free the string after mono_image_open_metadata_only is done with it:

```
      1 (128 bytes) ROOT LEAK: 0x7fb07f086c80 [128]  length: 105  "/Users/therzok/Work/md/md-addins/external-addins/MonoDevelop.MonoDroid/build/Xamarin.Installer.Common.pdb"
```
2019-06-12 14:22:24 +02:00
Maxim Lipnin 578b91eb07 Fix some warning related to nullability of reference types 2019-06-12 14:20:09 +02:00
Aleksey Kliger cd8b760220 [runtime] Switch to GC Unsafe for all GC external API functions
Fixes https://github.com/mono/mono/issues/14975
2019-06-12 14:11:01 +02:00
Martin Baulig a44db72a1c [System.Net.Http]: Add hack to the legacy HttpClient to pass down Timeout property. #12577. (#14970)
* [System.Net.Http]: Add hack to the legacy HttpClient to pass down Timeout property. #12577.

* `HttpClientHandler`, `IMonoHttpClientHandler`: add new internal `MonoSetTimeout (TimeSpan)` function.

* `HttpClient.SendAsyncWorker()`: if `handler` is `HttpClientHandler`, call the new internal
  `MonoSetTimeout()` function to pass down the timeout.

* `MonoWebRequestHandler.SendAsync()`: this is only used here to pass the timeout to the `HttpWebRequest`.

* Address feedback.

* Update HttpMessageInvoker.cs

* Make it build.

* Really make it build (forgot monotouch_watch).
2019-06-12 09:42:20 +02:00
Koundinya Veluri 2c7c7398bb Track timer count and add Timer.ActiveCount (dotnet/coreclr#25061)
* Track timer count and add Timer.Count

Part of https://github.com/dotnet/corefx/issues/38422

* Use lock to prevent tearing

* Add assert

* Rename Count to ActiveCount

* Fix build

Signed-off-by: dotnet-bot <[email protected]>
2019-06-12 09:41:15 +02:00
Egor Bogatov 56398813ae [netcore] Run CoreFX tests on CI (macOS) (#14849) 2019-06-12 09:40:15 +02:00
Zoltan Varga dd0f7eb2e1 [corlib] Preserve System.RuntimeType. 2019-06-12 08:50:06 +02:00
Aleksey Kliger bf61c7c973 [.github] Update CODEOWNERS 2019-06-12 08:46:16 +02:00
Zoltan Varga bcd14601c2 [aot] Open AOT profile files using "rb" so it works on windows.
Fixes https://github.com/xamarin/xamarin-android/issues/3168.
2019-06-12 08:31:05 +02:00
Stephen Toub d2e9be7d08 Update IEquatable/IComparable<T> for nullability (dotnet/coreclr#25053)
We need to revise our use of nullable for `IComparable<T>` and `IEquatable<T>`:
- For reference types that implement `IComparable<T>`, the implementation should use `IComparable<T?>`, e.g. `String : IComparable<String?>`, so that the type can be used in methods with a `where T : IComparable<T>` constraint: `IComparable<T>`'s contravariance then allows both `T` and `T?` to work as generic type arguments.
- For reference types that implement `IEquatable<T>`, we can't take advantage of such contravariance, as by design `IEquatable<T>` is invariant (it would be a breaking change to make it contravariant, and even if it wasn't doing so is of questionable design given that equality and inheritance don't mix well).  Given the current language design, we have little option but to explicitly make any reference type implementations of `IEquatable<T>` be oblivious, and also make all `where T : IEquatable<T>` constraints be oblivious.  Otherwise, regardless nullable annotations, a developer will get a warning for using either or both of `T`/`T?` with any given method employing such a constraint.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-11 20:10:51 +02:00
Acid Chicken (硫酸鶏) c31180acc5 Fix Japan's new era fallback isn't reflected in Mono (#14896)
* Fix Japan's new era fallback isn't reflected in Mono

refs: <https://github.com/dotnet/coreclr/pull/23614>

* Fix array length

* Fix MonoTests.System.Globalization.CalendarTest

* Fix MonoTests.System.Globalization.EastAsianLunisolarCalendarTest

* Fix era index

refs: 1715e1d0e738ce95382ba372c24502967002a2f0

* Amend fixing

refs: 05af6e6f75fcea76e3d9e9f2f95f0caedb753aa7
2019-06-11 19:16:56 +02:00
Maryam Ariyan 9c542c174d Make Debug.Print(string) behavior consistent with .NET Framework (#25036)
*     Debug.Print should call Debug.WriteLine
    - to behave similar to .NET Framework

* temporarily disable tests
- Waiting for coreclr/fx changes to propagate

Signed-off-by: dotnet-bot <[email protected]>
2019-06-11 18:56:15 +02:00
Zoltan Varga 23ab664623 Revert some changes which don't have proper dependencies. 2019-06-11 11:51:10 -05:00
Zoltan Varga 1f289c9e44 Add a rule to create /aot-in. 2019-06-11 11:51:10 -05:00
Zoltan Varga 16c1682494 Make the creation of the dummy assemblies deterministic. 2019-06-11 11:51:10 -05:00
Zoltan Varga 23573b5348 [wasm] Avoid rebuilding the wasm files for the aot images if the aot image hasn't changed. 2019-06-11 11:51:10 -05:00
Zoltan Varga 0af5b236f9 [wasm] Make the cross compiler emit a depfile, not used yet. 2019-06-11 11:51:10 -05:00
Zoltan Varga 6b5779a956 [wasm] Fix dependencies for mscorlib. 2019-06-11 11:51:10 -05:00
Zoltan Varga d010cf5048 [aot] Avoid recompiling the dedup image if it hadn't changed. 2019-06-11 11:51:10 -05:00
Zoltan Varga d22e537c4e [wasm] Make the linker output deterministic. Avoid AOTing mscorlib if it havent' changed. 2019-06-11 11:51:10 -05:00
Zoltan Varga 8326c11779 [wasm] Compile the dedup image to wasm as well. Fix the dependencies of the dedup image. Fix a typo. 2019-06-11 11:51:10 -05:00
Alexander Kyte e4d3752684 [llvm] Fix disabling direct calls and inlining (#14880)
* [llvm] Enable disabling inlining on llvm

* [llvm] Fix disabling direct calls on llvm

* [llvm] Rename mono_inited even when doing self_init (generated init functions refer to it)
2019-06-11 12:30:04 -04:00
dotnet-maestro 22887567e1 Update dependencies from https://github.com/dotnet/arcade build 20190610.24
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19310.24
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19310.24
2019-06-11 17:17:07 +02:00
Aleksey Kliger (λgeek) 5440491ae8 [install] Include .exe extension when symlinking the mono binary (#14958)
Prevent the SDKs from making dangling symlinks when packaging the MXE
cross-compilers: should hav

    mono.exe -> mono-sgen.exe

rather than

    mono -> mono-sgen (dangling).
2019-06-11 09:42:24 -04:00
Johan Lorensson 3a193b7ae5 Merge pull request #14811 from lateralusX/lateralusX/fix-lmf-with-ctx-windows-msvc
Fix interpreter failure in install_eh_callback.exe under Windows x64 MSVC.
2019-06-11 13:22:18 +02:00
Jay Krell c398e19d35 Remove JIT icall hashing by address. (#14870)
* Work toward removing JIT icall hashing by address.

* Remove JIT icall hashing by address.

* Cleanup amd64 emit_call use.

* Move MonoJumpInfoTarget to among types, not functions.
If mini.h includes mini-runtime.h, then can move it nearer MonoJumpInfo.
Or if returned by out parameter (pointer) instead of struct by value.
Or pair of out parameters instead of struct.

* Closer to historical semantics, at least to avoid assertion failure, will try to refine in a later PR.

* Remove new comments about partial barriers, I think the comments are wrong and the code is correct (except that lock/unlock probably need to be full barriers, if they are not.)

* PR: Don't worry about size of MonoCallInst.jit_icall_id. Let it be 32 or 64 bits instead of 9.
2019-06-11 07:40:39 +03:00
dotnet-maestro 3dd5240c72 Update dependencies from https://github.com/dotnet/arcade build 20190609.1
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19309.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19309.1
2019-06-10 17:34:18 +02:00
dotnet-maestro 3ec86e017f Update dependencies from https://github.com/dotnet/arcade build 20190608.1
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19308.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19308.1
2019-06-10 13:36:50 +02:00
Marek Safar 7f2d519b0d [System.Private.CoreLib] Update public APIs 2019-06-09 13:01:50 +02:00
Zoltan Varga 23a928826c [wasm] Prototype for collecting aot profile data. (#14865)
* [runtime] Add some options to the aot profiler:
- write-at-method=<METHOD> Write the data when <METHOD> is compiled.
- send-to-method=<METHOD> Call <METHOD> with the collected data.

* [wasm] Fix profiler support in the packager.

* [wasm] Add prototype for generating aot profile data.

* Fix indentation.
2019-06-08 23:16:18 -04:00
dotnet-maestro bcf03bc442 Update dependencies from https://github.com/dotnet/arcade build 20190607.62
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19307.62
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19307.62
2019-06-08 22:51:37 +02:00
dotnet-maestro 4469cdaffc Update dependencies from https://github.com/dotnet/arcade build 20190606.12
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19306.12
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19306.12
2019-06-08 22:51:37 +02:00
Michal Strehovský b2768ac469 Move Marshal class to S.P.CoreLib (dotnet/corert#7502)
After this change, CoreRT only uses S.P.Interop to host the interop hashtable readers. Those can't move to CoreLib because they call into the type loader.

I didn't touch ProjectN's version of Marshal and I left the `PInvokeMarshal` class in place to help with that. I guess we'll need to make a call at some point on what to do with S.P.Interop since without MCG the code in it is kind of useless.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-08 22:50:24 +02:00
Jay Krell 440764a82e g_new/g_new0 in wasm/driver.c (#14890) 2019-06-08 13:38:20 -04:00
Jay Krell 63f17d2f55 Cleanup amd64 emit_call / emit_call_body -- last parameter always FALSE. (#14897)
And it is reasonable to expect that to remain the case, as C compilers
do not generate code as would occur with the last parameter = TRUE.

They allocate the room for their maximum outgoing parameters at function entry,
and keep a constant stack, except for functions that call _alloca,
and this code risks but not necessarily breaks ABI compliance.

In particular, the rsp adjustment either needs unwind codes, or needs
to only occur in a function with a frame pointer other than RSP.

And the uwind codes for adding to RSP are very tricky to impossible.
Normally you only ever subtract from RSP. There is an unwind code for that.
But there isn't really for add.
You might be able to simulate it with chained unwind, making these
regions look kinda like end of function.
But this is all most unlikely.
2019-06-08 13:38:00 -04:00
monojenkins 609878dcb5 [bcl] Update BCL Linked Size 2019-06-08 09:35:38 +02:00
Steve MacLean 11761ebd7f Fix AppDomain issues (dotnet/coreclr#25019)
Populate AppDomain.CurrentDomain when invoking events
Make AppDomain method implemented in AssemblyLoadContext internal

Signed-off-by: dotnet-bot <[email protected]>
2019-06-08 09:34:45 +02:00
Carlos Sanchez Lopez 132969ee3e Fix issue in NamedPipeServerStream.GetImpersonationUserName() (#37676)
Fix for issue where NamedPipeServerStream.GetImpersonationUserName() throws unexpected InvalidationOperationException("The operation completed successfully") when impersonation is not allowed, when in fact it should simply return an empty string.

The fix will call GetNamedPipeHandleStateW once. If it fails, it will try to load sspicli.dll if it was not loaded, and will reattempt the call to GetNamedPipeHandleStateW.

This fix is only for Win32: UAP does not allow calling LoadLibraryEx, and Windows Nano says denies access to GetNamedPipeHandleStateW even after attempting to load sspicli.dll.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-08 08:52:42 +02:00
Zoltan Varga 16532397c3 [wasm] Switch to the LLVM wasm backend. (#14734)
* [wasm] Switch to the llvm wasm backend.

* [wasm] Packager changes for the wasm backend.

* -s EMULATED_FUNCTION_POINTERS=1 no longer needed.
* Run wasm-strip on the final binary.
* Run opt during AOT.
* Disable the dynamic build for now, it doesn't seem to work with the wasm backend.

* [wasm] Use STACK_MAX instead of STACK_BASE for the bottom limit of the stack.

* [aot] Handle llvmopts= in llvmonly mode as well.

* [wasm] Disable an assertion which is no longer true with the wasm backend.

* [wasm] Disable computed gotos in the interpreter, the wasm backend doesn't support it.

Fix llvm target layout so its compatible with clang compiled code.

* [wasm] Pass llvm-path to the cross compiler.

* [wasm] Fix some problems with the dynamic build, still doesn't work. Fix packaging.

* [wasm] Pass -s DISABLE_EXCEPTION_CATCHING=0 to the runtime build, otherwise catch clauses are not generated in llvm-runtime.cpp.

* [aot] Allow llvmopts= argument multiple times.

* Bump emscripten version to 1.38.34.

* [wasm] Add https://github.com/emscripten-core/emscripten/pull/8457 as a patch.

* [wasm] Compile the aot .bc files to .o in a separate step using emcc, this will help with incremental compilation later.

* [wasm] Source emsdk_set_env.sh instead of emsdk_env.sh the latter calls emsdk construct_env to construct emsdk_set_env.sh which is racy.
2019-06-08 00:46:56 -04:00
Jerome Laban f5cfc67c8e [Wasm] Fix memory override in mono_wasm_add_assembly 2019-06-07 22:00:31 -05:00
Jeremy Kuhne 4102c7fd90 Fix redist
Signed-off-by: dotnet-bot <[email protected]>
2019-06-07 23:17:51 +02:00
Santiago Fernandez Madero 355d611e94 Address nullable feedback for System.Runtime and System.Runtime.Extensions (#25017)
* Address last nullable feedback from System.Runtime

* Address nullable feedback for System.Runtime.Extensions

* PR Feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-06-07 23:17:51 +02:00
Stephen Toub 3f1bca5a34 Add [NotNull] to syncLock parameter of EnsureInitialized (#25012)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-07 23:17:51 +02:00
Sung Yoon Whang cf93ce4fce Add DisplayUnits property to DiagnosticCounter (dotnet/coreclr#24981)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-07 23:17:51 +02:00
Ankit Jain 46108fd7de Bump msbuild to track xplat-master (#14814)
Picks up changes in msbuild after:
	[mono] Update SDKs to track cli branch `release/2.1.8xx`

(https://github.com/mono/msbuild/pull/110)
2019-06-07 21:54:03 +02:00
Alexander Kyte d54a6544b3 [crash] Move /proc/self/maps crash collection to created JSON blob (#14704)
* [runtime] Don't print /proc/self/maps on Android

* [runtime] Add proc mapping to crash JSON output

* [runtime] Zero initialize locals in threading macros

* [crash] Filter out illegal characters from process_maps

* [crash] Prevent concurrent reads from /proc/self/maps

* [crash] Clean up memset in g_async_safe_fgets

* [crash] Remove stack staging array in async_safe_fgets

* Revert "[crash] Prevent concurrent reads from /proc/self/maps"

This reverts commit f65166559ad24ad65c530f7abbc096e4d4634878.
2019-06-07 15:18:57 -04:00
Calvin Buckley 18e0ebfe89 AIX/PASE integration improvements (#14652)
* Allow using SysV-style sonames on AIX/PASE

This uses a far saner naming convention with libtool, and is
consistent with the official PASE RPM repository. However, some
loader changes are required to make it "just work" and workaround
some real dumb behaviour on AIX's part.

Note that while Mono works when built without the different soname
tweak, it won't when installed, because the convention is strange
and the libtool .la archives won't be installed. It's recommended
as such that installation of Mono keeps the libtool files or just
uses SVR4 sonames like the rest of the world, even if they're made
strange like AIX.

* Specify a specific version of unixODBC, make an include

A workaround against PASE RPMs of unixODBC not including the just
"libodbc.so" except in the development version. If a platform needs
a specific version (the macOS reference not changed in the DllMap)
then it can be specified in the configure script.

AIX doesn't need a special name because both AIX Toolbox and Perzl
include non-archive libraries in /opt/freeware/lib - unexpected.

* Don't try to use shm_open on PASE due to it not being implemented

All this will do is cause a coredump if running the script on PASE,
and write an entry to the SLIC diagnostic logs. This test could be
re-enabled if i 7.4 supports it properly.

* fix ODBC include

* Check for what library to use for libintl when dlopenning on AIX

These checks are for the benefit of the DllMap. The construct used
is kinda hacky, IMHO, but it resolves the concerns of hardcoding.
In the event that this doesn't work (it does for me), it falls back
to what the IBM RPMs seem to use for linking gettext et al.

Not sure of the utility for other platforms, since they usually
use sane soname conventions and the one-liner here depends heavily
on an AIX developer tool and its output format.

The use of cut/awk should work with the stock AIX utils, no GNU
necessarily needed.
2019-06-07 20:02:43 +02:00
Martin Baulig 96125c693c Bump CoreFX to pickup https://github.com/mono/corefx/pull/298. (#14851) 2019-06-07 13:14:21 -04:00
Martin Baulig ac908dd334 Implement RSACertificateExtensions.CopyWithPrivateKey(). #14152. (#14860)
* Implement RSACertificateExtensions.CopyWithPrivateKey().  #14152.

* RSACertificateExtensions.CopyWithPrivateKey(): implement.

Since `RSACertificateExtensions` lives in `System.Core` and only `mscorlib`
internals (but not `System` internals) are visible to it, we need to use a
little trick here:

* System.Security.Cryptography.X509Certificates.X509CertificateImpl:
  add new abstract methods CopyWithPrivateKey() and CreateCertificate().

* System.Security.Cryptography.X509Certificates.X509Certificate2Impl:
  sealed implement them here.

The `httpcfg` tool, only checks whether the key is valid, so we don't really
need to call `CopyWithPrivateKey()` (which requires us to take a dependency
on `System.Core`).

* Do the same thing in HttpListener.
2019-06-07 13:13:48 -04:00
Aleksey Kliger (λgeek) 277bbdaada [runtime] Make cross-appdomain wrappers throw NIE for methods with ref struct args (#14863)
* [AppDomain] Cross-appdomain wrapper throws NIE for methods with ref struct arguments

If we have a ref struct and a method that takes it as an argument:

```
[Serializable]
public ref struct R {
  public int i;
}

public interface I {
  public void M1 (R r, SomeClass o);
}

public class M : MarshalByRefObject, I {
  public void M1 (R r, SomeClass o) { ... }
}
```

If we create an instance of M in another domain and try to call it:
```
  I obj = (I)otherDomain.CreateInstanceAndUnwrap (typeof(M).Assembly.FullName,
  nameof(M));
  R r;
  r.i = 42;
  SomeClass o = ...;
  obj.M1 (r, o);
```

Mono will need to create a wrapper that can invoke M.M1 in the other domain.
The way the wrapper works in `mono_marshal_get_xappdomain_invoke` is by
creating an array of all the arguments that need to be serialized (see
`mono_get_xdomain_marshal_type`) and then serializing the whole array (by
calling out to `System.Runtime.Remoting.RemotingServices.SerializeCallData`).
In the example it would make a two element array and try to serialize `r` and
`o` into it. For valuetypes it normally does this by boxing the argument.
However since `R` is a ref-struct, boxing it is not allowed.

This works on .NET Framework.

However for Mono we would need to serilize `R` without boxing it, which seems
challenging with our current setup.

So for now we generate a wrapper that just throws a NotImplementedException if
it is ever called.

This fixes https://github.com/mono/mono/issues/14809 (in that marshalling a
TextWriter across domains works again) but in an unsatisfactory way (because
you can't call any of the ReadOnlySpan<char> overloads on the transparent proxy
object).

* [corlib] Add back Serializable attribute on some reflection classes

Tests that use https://www.nuget.org/packages/NUnit.ApplicationDomain/
need to serialize MethodInfo (and hence MethodBase).

* [corlib] Add test for transparent proxies for classes with methods that take
ref struct args

Regression test for https://github.com/mono/mono/issues/14809

* Bump API snapshot submodule
2019-06-07 13:06:32 +02:00
Ryan Lucia 65608fe3f4 Remove exclusions for resolved issue 2019-06-07 09:31:43 +02:00
Tanner Gooding e0471729b0 Marking Vector128<T>.Count and Vector256<T>.Count as [Intrinsic] (#24991)
* Marking Vector128<T>.Count and Vector256<T>.Count as [Intrinsic]

* Fixing NI_Vector128_Count and NI_Vector256_Count to use clsHnd when getting the simdSize and baseType

* Applying the formatting patch.

* Changing some comments to just be "vector element count".

* Fixing impBaseIntrinsic to set the baseType so Vector128_Count and Vector256_Count don't return nullptr

Signed-off-by: dotnet-bot <[email protected]>
2019-06-07 07:10:14 +02:00
Zoltan Varga bd1ad0089f Switch Hashtables to Dictionary in the encoding classes so Hashtable can be linked out. (#14774) 2019-06-06 23:03:22 -04:00
Bernhard Urban 03c6855d05 [ios] switch cross32/crosswatch to be executed on x86_64 and use newer LLVM (#14852)
Contributes to https://github.com/mono/mono/issues/9621
Fixes https://github.com/mono/mono/issues/14841

Note that it breaks iOS 32bit on devices. The Apple Watch _might_ still works (only tested on `2019-02`).  It unblocks some build related issues on the Xamarin.iOS `2019-06` integration, and we have to do it anyway. I'll come back to it to fix device related issues.
2019-06-07 00:01:00 +02:00
Ryan Lucia a5a26a2f92 Add logic to object array typecheck to handle arrays of unmanaged pointers (#14733)
Fixes #14555 

The NULL check alone is not enough, because interfaces are also parented to NULL for some reason, and so I've moved `class_kind` out from the bitfield section to make it accessible from IR. The beginning of the struct is going to be padded to 32 bits anyway at the least, so this move should save space if anything.

Our type checking overall is not very robust, and in particular the supertype comparison doesn't work well for anything that isn't an object, so it's possible there are some other weird corner cases. As far as I can tell looking through the array of objects stuff though, we should have reasonable coverage with the existing tests + the stuff I added.
2019-06-06 15:38:57 -04:00
Ryan Lucia 6d2c77fb33 Switch supported runtimes storage from an array to a list (#14777)
Fixes #12141 

Switched to a list because in theory any number of tags can be present. Old behavior was to put them into an array of size 3 and zero the byte following them, which was causing the crash with more than two tags. There's no null check for the default runtime because that should never happen, and the list is reversed to preserve the order from the XML. Changing the signature of `get_runtimes_from_exe` should be fine because it's static and only called in the one place.
2019-06-06 15:02:57 -04:00
Jan Kotas 4d5807110f Delete unnecessary security transparency attributes (dotnet/coreclr#25007)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 20:57:11 +02:00
Bernhard Urban 812357ec51 [mach] cast objc_msgSend{,Super} usages (#14836)
Fixes #14792
2019-06-06 20:14:59 +02:00
Martin Baulig 88c05a8910 Bump CoreFX to pickup https://github.com/mono/corefx/pull/298. 2019-06-06 13:45:29 -04:00
Bernhard Urban e8367ce3d7 [llvm] emit call to mini_llvmonly_throw_nullref_exception without exceptions in mind (#14840)
Fixes https://github.com/mono/mono/issues/14243
2019-06-06 19:24:20 +02:00
Alexander Kyte 364a8423e3 [runtime] Gracefully skip AOTing problematic assemblies (#14232) 2019-06-06 12:07:31 -04:00
Egor Bogatov b861cacd48 [netcore] Improve netcore/build.sh script (#14835)
* Improve netcore/build.sh

* bump corefx tests version

* ignore StringTests failures

* Ignore more tests for xtestall

* Implement Environment.TickCount64

* cleanup

* Add two debug-* rules
2019-06-06 17:43:57 +02:00
dotnet-maestro 636e06334f Update dependencies from https://github.com/dotnet/arcade build 20190605.13
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19305.13
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19305.13
2019-06-06 17:28:03 +02:00
Thays Grazia 2da7707373 [Mono.Debugger.Soft] Properly close connections
This PR fixes #7377
2019-06-06 11:12:47 -03:00
dotnet-maestro 2e4b6b34ac Update dependencies from https://github.com/dotnet/arcade build 20190604.23
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19304.23
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19304.23
2019-06-06 13:47:26 +02:00
dotnet-maestro f02d09f7f6 Update dependencies from https://github.com/dotnet/arcade build 20190604.1
- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19304.1
- Microsoft.DotNet.Helix.Sdk - 2.0.0-beta.19304.1
2019-06-06 13:47:26 +02:00
Martin Baulig 3998dc01c9 Make CalendarData more linker friendly.
See https://github.com/mono/corefx/pull/298 for a description of the
technique that we're using here to make it more linker friendly.
2019-06-06 13:40:53 +02:00
Alexander Köplinger 51202617a1 [sdks] Use 7z format for the android/ios/mac archives (#14837)
It has way better compression performance.
Kept WebAssembly on .zip for now since external users took dependencies on it [1].

Fixes https://github.com/mono/mono/issues/14215

[1] https://github.com/nventive/Uno.Wasm.Bootstrap/blob/e2bf8946e162642ca1bdf4f003f8de4a94954f73/Readme.md#updating-the-mono-sdk-build
2019-06-06 12:24:56 +02:00
Santiago Fernandez Madero 504a1a30fa More PR Feedback
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:32:52 +02:00
Santiago Fernandez Madero 420a9da24a PR Feedback
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:32:52 +02:00
Santiago Fernandez Madero eab7775571 PR Feedback
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:32:52 +02:00
Santiago Fernandez Madero c2796feb97 Fix some nullable annotations from API Review
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:32:52 +02:00
Santiago Fernandez Madero a8fd49434f Remove stale Task async extension methods from TaskExtensions (#24958)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:07:27 +02:00
Santiago Fernandez Madero 992eacfac9 Remove Collection<T> range APIs (dotnet/coreclr#24938)
Signed-off-by: dotnet-bot <[email protected]>
2019-06-06 10:07:27 +02:00
Jay Krell 515234a128 [thread] Print function name when asserting MonoThreadInfo is non-NULL (#13814)
https://github.com/mono/mono/issues/13813.
2019-06-06 10:40:39 +03:00
monojenkins bf78c04740 [bcl] Update BCL Linked Size 2019-06-06 09:31:40 +02:00
Kenneth Pouncey 0562fde077 [wasm][bcl] Use zlib for DeflateStream on WASM profile (#14308)
* [wasm][bcl] Use zlib for DeflateStream on WASM profile

- Add `-s USE-ZLIB=1` flag for the emscripten build of `mono.js`
- Add new `zlib-helper` module build that is included in the `mono` wasm build.
- This adds roughly 300 kb to the size.

* Add zlib calls to `icall` for `WASM` profile

* Address comments use `TARGET_WASM` instead of `HOST_WASM`

* Hack around JS exception: RuntimeError: indirect call signature mismatch

- Instead of using the function pointer from DeflateStream (Marshal.GetFunctionPointerForDelegate) obtain the callback functions from the object itself.

* Update error messages and add exception parameter to `mono_runtime_invoke`.

* Add `mono/support` zlib include sources files to make `package` target.

* Add `zlib-helper` build to packager.

- Note: Right now only supports zlib build from mono tree build.
- Add `zlibtest.cs` scenario to compress and uncompress using DeflateStream and GZip.
- Add test to make file to build and run aot version of ziplibtest.

* Add copy of zlib header support files to `package-wasm-runtime`

- The support library is not built for wasm but the zlib include files are needed to build support during wasm build.
- See mono.js build and the use of the `-s USE_ZLIB=1` flag.
- The include files will be used in packager.exe to build zlib support for wasm.

* Use the zlib includes from support to build the zlib support.

* Setup base tests for testing System.IO.Compression integration

* Formatting

* Add more ziparchive tests

* Formatting and indentation

* Add more Zip tests

* Add more tests.

- Add more tests for ZipArchive.
- Add tests for Deflate - small, large and very large strings.
- Add tests for GZipStream - small, large and very large strings.

* Fix merge of EMCC_FLAGS

* Add new option to packager to enable the use of zlib

-   --enable-zlib (enable the use of zlib for System.IO.Compression support)
        type: bool  default: false

- update packager doc

* Address review comments

- Change `enable-zlib` to `zlib`
2019-06-06 05:07:56 +02:00
Martin Baulig 2b3a23c511 Bump CoreFX to pickup the fix for #14214. (#14821)
HttpClient.SendAsync() should not attempt to read response body on a HEAD request.

* CoreFX PR https://github.com/dotnet/corefx/pull/38129.

* Backported in https://github.com/mono/corefx/pull/297.

* Fixes https://github.com/mono/mono/issues/14214.
2019-06-05 22:14:14 -04:00
Egor Bogatov 181a602ace Fix build on macOS 10.15 Catalina + xcode 11 (#14820)
1) CC_MD5_Init, CC_MD5_Update and CC_MD5_Final are deprecated.

```c
mono-md5.c:45:2: error: 'CC_MD5_Init' is deprecated: first deprecated in macOS 10.15 - This function is
      cryptographically broken and should not be used in security contexts. Clients should migrate to SHA256
      (or stronger). [-Werror,-Wdeprecated-declarations]
        CC_MD5_Init (ctx);
        ^
```

2) objc_msgSend is deprecated
```
mono-threads-mach-helper.c:95:17: error: too many arguments to function call, expected 0, have 2
                objc_msgSend (value, release);
                ~~~~~~~~~~~~  ^~~~~~~~~~~~~~
```
XCode has a special switch `Enable Strict Checking of objc_msgSend Calls` which is now `true` by default. It's fixed by `-DOBJC_OLD_DISPATCH_PROTOTYPES=1` flag to clang or the corresponding define in the source file.

3) Fixes https://github.com/mono/mono/issues/14793

It worked on pre 10.15 without `-framework GSS`, because it got pulled in through the dependencies. You can see it by running DYLD_PRINT_LIBRARIES=1 csharp and then trace it back. On Mojave /System/Library/Frameworks/GSS.framework/Versions/A/GSS got loaded as dependency of /usr/lib/libcups.2.dylib and so on. Unlike Windows, macOS will resolve the symbols to whatever is already available so it was just happy that someone loaded GSS before libmono-native[-compat/-unified].
2019-06-06 00:39:04 +02:00
Vlad Brezae e548a7d231 [interp] Inline more methods (#14755)
* [interp] Add support for inlining methods with locals

For each local in the inlined method we allocate a new local in the parent method.

* [interp] Enable inlining of methods using ldarga opcode

* [interp] Don't inline methods that do localloc

Inlining localloc in a loop leads to stack overflows. JIT has same behavior.
2019-06-05 22:05:30 +02:00
Alexander Köplinger a83f700546 [tests] Exclude more nunit categories on ios/mac (#14807)
A few more that we missed adding to the exclude file in the sdks archive.

(cherry picked from commit 3d9513efd4)
2019-06-05 21:14:53 +02:00
Egor Bogatov c5ed9bfc0d Reflect recent NS2.1 changes (#14679)
Due to few recent updates to dotnet/standard (NETStandard 2.1) we had 40 apicompat issues.
Mostly copy-pasted from dotnet/corefx.
Didn't implement Cryptography stuff yet.
Checked all profiles.

Implements #14609
2019-06-05 20:44:46 +02:00
Jerome Laban f7042d48a5 Enforce C++ exception support for the release-dynamic configuration 2019-06-05 11:19:21 -05:00
imhameed 993d484285 [jit] Treat ternary operations as non-constant during constant evaluation and propagation (#14782)
* Treat ternary operations as non-constant... (#13873)

...during constant evaluation and propagation.

* Move ternary SSA cfold test from basic.cs to objects.cs.
2019-06-05 17:42:18 +02:00
Brendan Zagaeski 888a8d44f6 [arm64] Correct exception filter stack frame size (#14757)
* [arm64] Correct exception filter stack frame size

Fixes: https://github.com/mono/mono/issues/14170
Fixes: https://github.com/xamarin/xamarin-android/issues/3112

The `call_filter()` function generated by `mono_arch_get_call_filter()`
was overwriting a part of the previous stack frame because it was not
creating a large enough new stack frame for itself.  This had been
working by luck before the switch to building the runtime with Android
NDK r19.

I used a local build of [xamarin/xamarin-android/d16-1@87a80b][0] to
verify that this change resolved both of the issues linked above.  I
also confirmed that I was able to reintroduce the issues in my local
environment by removing the change and rebuilding.

After this change, the generated `call_filter()` function now starts by
reserving sufficient space on the stack to hold a 64-bit value at the
`ctx_offset` location.  The 64-bit `ctx` value is saved to the
`ctx_offset` location (offset 344 in the new stack frame) shortly
afterwards:

    stp	x29, x30, [sp,#-352]!
    mov	x29, sp
    str	x0, [x29,#344]

As expected, the invocation of `call_filter()` now no longer modifies
the top of the previous stack frame.

Top of the stack before `call_filter()`:

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

Top of the stack after `call_filter()` (unchanged):

    (gdb) x/8x $sp
    0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
    0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

Additional background information
=================================

The original lucky, "good" behavior worked as follows for
[mono/mono/2018-08@725ba2a built with Android NDK r14][1]:

 1. The `resume` parameter for `mono_handle_exception_internal()` is
    held in register `w23`.

 2. That register is saved into the current stack frame at offset 20 by
    a `str` instruction:

           0x00000000000bc1bc <+3012>:	str	w23, [sp,#20]

 3. `handle_exception_first_pass()` invokes `call_filter()` via a `blr`
    instruction:

        2279							filtered = call_filter (ctx, ei->data.filter);
           0x00000000000bc60c <+4116>:	add	x9, x22, x24, lsl #6
           0x00000000000bc610 <+4120>:	ldr	x8, [x8,#3120]
           0x00000000000bc614 <+4124>:	ldr	x1, [x9,#112]
           0x00000000000bc618 <+4128>:	add	x0, sp, #0x110
           0x00000000000bc61c <+4132>:	blr	x8

    Before the `blr` instruction, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x0a8b31d8	0x00000071
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

 4. `call_filter()` runs.  This function is generated by
    `mono_arch_get_call_filter()` and starts with:

           stp	x29, x30, [sp,#-336]!
           mov	x29, sp
           str	x0, [x29,#344]

    Note in particular how the first line subtracts 336 from `sp` to
    start a new stack frame, but the third line writes to a position
    that is 344 bytes beyond that, back within the *previous* stack
    frame.

 5. After the invocations of `call_filter()` and
    `handle_exception_first_pass()`, `w23` is restored from the stack:

           0x00000000000bc820 <+4648>:	ldr	w23, [sp,#20]

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x00000000	0x00000000	0x1ef51980	0x00000071

    Notice how `call_filter()` has overwritten bytes 8 through 15 of the
    stack frame.  In this original lucky scenario, this does not affect
    the value restored into register `w23` because that value starts at
    byte 20.

 6. `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000bb960 <+872>:	cbz	w23, 0xbb9c4 <mono_handle_exception_internal+972>

    Since `w23` is `0`, `mono_handle_exception_internal()` correctly
    continues into the `else` branch.

The bad behavior for
[mono/mono/2018-08@725ba2a built with Android NDK r19][2]
works just slightly differently:

 1. As before, the local `resume` parameter starts in register `w23`.
 2. This time, the register is saved into the stack frame at offset 12
    (instead of 20):

           0x00000000000bed7c <+3200>:	str	w23, [sp,#12]

 3. As before, `handle_exception_first_pass()` invokes `call_filter()`
    via a `blr` instruction.

    At this step, the top of the stack looks like this:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0x2724b700	0x00000000
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

 4. `call_filter()` runs as before.  And the first few instructions of
    that function are the same as before:

        stp	x29, x30, [sp,#-336]!
        mov	x29, sp
        str	x0, [x29,#344]

 5. `w23` is again restored from the stack, but this time from offset 12:

           0x00000000000bf2c0 <+4548>:	ldr	w23, [sp,#12]

    The problem is that `call_filter()` has again overwritten bytes 8
    through 15 of the stack frame:

        (gdb) x/8x $sp
        0x7fcd2774a0:	0x7144d250	0x00000000	0xcd2775b0	0x0000007f
        0x7fcd2774b0:	0x1ee19ff0	0x00000071	0x0cc00300	0x00000071

    So after the `ldr` instruction, `w23` has an incorrect value of
    `0x7f` rather than the correct value `0`.

 6. As before, `mono_handle_exception_internal()` tests the value of `w23` to
    decide how to set the `ji` local variable:

        2574			if (resume) {
           0x00000000000be430 <+820>:	cbz	w23, 0xbe834 <mono_handle_exception_internal+1848>

    But this time because `w23` is *not* zero, execution continues
    *incorrectly* into the `if` branch rather than the `else` branch.
    The result is that `ji` is set to `0` rather than a valid
    `(MonoJitInfo *)` value.  This incorrect value for `ji` leads to a
    null pointer dereference, as seen in the bug reports.

[0]: https://github.com/xamarin/xamarin-android/tree/xamarin-android-9.3.0.22
[1]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1432/Azure/
[2]: https://jenkins.xamarin.com/view/Xamarin.Android/job/xamarin-android-freestyle/1436/Azure/

* reduce frame size
2019-06-05 17:38:06 +02:00
lateralusX ae84befc0f Fix interpreter failure in install_eh_callback.exe under Windows MSVC.
INTERP_PUSH_LMF_WITH_CTX_BODY is not fully supported under Windows MSVC
since that compiler doesn't support taking address from a label, needed
by current implementation to correctly resume after unwinding exceptions
happening in native frames as tested by install_eh_callback.exe.

This fix add support to restore to a known location and use a flag
indicating if we have been resumed to the location or just passing by
for the first time. If we resume it will transfer control to specified label.
This will mimic the behavior of taking the address of a label and use that
address as the restore point and works like how setjmp/longjmp work.

I also considered switching to setjmp/longjmp but that would have had a bigger
impact on all platforms and needed changes into the resume context
machinery as well.

While doing the implementation I also found an interesting issue with
MONO_CONTEXT_GET_CURRENT under Windows x64. On most other platforms it is
implemented as inline assembler, but on MSVC x64 inline assembler is not
supported so it is implemented as a function. Problem with that is how IP
was resolved, since it won't be resolved to the last instruction in the
expanded macro, but to the IP within the mono_context_get_current. This has
not had any visible effects in the past since it will just do the ret and get
back to the call site, but when this was used in INTERP_PUSH_LMF_WITH_CTX_BODY
it caused some issues. This was fixed to make sure rsp and rip reflect location
in caller frame and not within mono_context_get_current, similar to how
it works on all platforms with inline assembler support.
2019-06-05 17:32:02 +02:00
Kenneth Pouncey 7a74dd65ff [wasm][sdk] Clean the managed directory files on /t:Clean build target (#14756)
* Clean the `managed` directory files on `/t:Clean` build target

* Modify the `CleanManaged` target to `_CleanManagedOutputDir`.

- Address comments by Mikayla.
2019-06-05 15:53:35 +02:00
Bijan Tabatabai a3de0304a1 Align mcs's tuple deconstruction behavior with csc
Before the code
(a, b) = (b, a);
was the equivalent of
a = b;
b = a;

now, it evaluates it to so something like
tmp1 = a;
tmp2 = b;
a = tmp2;
b = tmp1;

Signed-off-by: Bijan Tabatabai <[email protected]>
2019-06-05 13:08:59 +02:00
Johan Lorensson 644bfc93d4 Merge pull request #14760 from lateralusX/lateralusX/increase-windows-main-thread-stack-size
Increase main thread stack size on Windows.
2019-06-05 12:26:50 +02:00
Egor Bogatov d3101fbaeb [netcore] Download and use local .NET Core SDK (#14696)
* Download and use local .NET Core SDK

* depend on 'all' rule in nupkg

* fix build error

* Windows support

* use InitializeToolset function

* ouput full path of dotnet in init-tools.sh

* remove init-tools

* Add init-tools.ps1 (Windows)

* fix / issue on cygwin

* address Jo's feedback

* add powershell to PATH for make nupkg

* add netcore/build.sh

* handle pack and test commands

* Update tools.ps1

* Update tools.ps1
2019-06-05 11:52:00 +02:00
Youssef1313 9e5c2c1e52 Being consistent (dotnet/coreclr#24927)
Being consistent using all if statement with curly braces.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-05 10:31:21 +02:00
Adam Sitnik 9fb74fb894 follow the ICU User Guide recommendation to optimize the perf of InvariantCultureIgnoreCase on Linux (#24889)
* follow the ICU User Guide recommendation to optimize the perf of InvariantCultureIgnoreCase on Linux:

1. try to guess the max size and call ucol_getSortKey just once
2. if the buffer is not big enough, call the method again providing the actual sort key length

* handle 0 case

* handle integer overflow

* shorten the time the buffers are pinned

* use the cheapest pinning

* code review fixes: don't use variable length stackalloc, don't copy text from docs (licensing) + don't try to go with the fast path when it would require allocating more managed memory for big strings

* simplify the condition

Signed-off-by: dotnet-bot <[email protected]>
2019-06-05 10:31:21 +02:00
Stephen Toub 5e1dd8d78d Remove unnecessary Shared<T> class from Task (dotnet/coreclr#24931)
It's just StrongBox<T>.

Signed-off-by: dotnet-bot <[email protected]>
2019-06-05 10:31:21 +02:00
Stephen Toub b4aa38f0c2 Adjust several where T : class? constraints (dotnet/coreclr#24894)
* Adjust several `where T : class?` constraints

* Address PR feedback

Signed-off-by: dotnet-bot <[email protected]>
2019-06-05 10:31:21 +02:00
Michal Strehovsky 76526cf1c9 Work around ProjectN compiler bug
[tfs-changeset: 1738641]

Signed-off-by: dotnet-bot <[email protected]>
2019-06-05 10:31:21 +02:00
Aleksey Kliger be92baf12d [runtime] Autoconf check for pthread_mutexattr_setprotocol
Android doesn't always declare that function.
2019-06-05 09:55:34 +02:00
Johan Lorensson a76fc055cf Merge pull request #14694 from lateralusX/lateralusX/fix-llvm-build-path-msvc
Fix LLVM build break on Windows MSVC.
2019-06-04 11:16:49 +02:00
lateralusX 29a2383333 Increase main thread stack size on Windows.
On Windows default thread stack size has been 1MB for a long time
(both 32/64-bit). When starting to use interpreter some of the tests
include deep recursions that will hit stackoverflow on Windows
(ackermann.exe as one example). This commit adjust the default reserved
stack size to 8MB for debug builds and 1.5MB (same as coreclr), for main
thread only. All other threads will still default to 1MB stack size
(both debug/release).
2019-06-04 09:37:39 +02:00
Tanner Gooding 4b81fcbcd7 Removing the TestAllOnes, TestAllZeros, and TestMixOnesZeros methods from the Sse41 class (#24886)
* Removing the TestAllOnes, TestAllZeros, and TestMixOnesZeros methods from the Sse41 class

* Removing JIT support for the SSE41_TestAllOnes, SSE41_TestAllZeros, and SSE41_TestMixOnesZeros intrinsics

* Removing tests for the Sse41.TestAllOnes, Sse41.TestAllZeros, and Sse41.TestMixOnesZeros methods

Signed-off-by: dotnet-bot <[email protected]>
2019-06-04 09:20:29 +02:00
Rich Lander 697c1402ea Update license to match directory 2019-06-04 09:06:40 +02:00
Rich Lander f0ef7aa4df Update license to match directory 2019-06-04 09:06:40 +02:00
Johan Lorensson 0fa59f11ea Merge pull request #14766 from lateralusX/lateralusX/fix-win-32-createprocess-error
Change in process_complete_path exposed uninitialized variable error.
2019-06-04 08:49:02 +02:00
monojenkins deb0227a78 Bump roslyn to release 3.1.0 (#14778) 2019-06-03 22:41:42 +02:00
lateralusX 85bdaee7a5 Change in process_complete_path exposed uninitialized variable error.
https://github.com/mono/mono/commit/5d088cf0de7f3e50e3547dba361af4401e938dd4
exposed a problem with an uninitialized variable, only exposed on some
build configurations, 32-bit Windows release builds.
2019-06-03 16:04:32 +02:00
lateralusX 0a6b13f83a Fix LLVM build break on Windows MSVC.
https://github.com/mono/mono/pull/14607 broke LLVM build on Windows MSVC.
This is just a quick fix falling back to old behavior on Windows MSVC. Since
all LLVM tools will be available in the same folder as mono runtime binary
there is no specific need to point out other build folders hosting LLVM tooling.
2019-05-29 12:09:59 +02:00
Jonathan Chambers 9de58a07c0 Don't try to access metadata for dynamic method added to non-dynamic image. Fixes issue #10201. 2018-08-20 16:51:18 -04:00
3524 changed files with 176178 additions and 164585 deletions

No files matched your search

+7 -1
View File
@@ -1,3 +1,6 @@
# diff settings
*.cs diff=csharp
# ensure LF endings on all checkouts
configure.ac crlf=input
config.rpath crlf=input
@@ -7,6 +10,7 @@ mkinstalldirs crlf=input
*.sources crlf=input
.gitattributes crlf=input
*akefile* crlf=input
m4/mono-output.m4 crlf=input
# ensure native line endings on checkout
*.c crlf
@@ -16,8 +20,10 @@ mkinstalldirs crlf=input
*.sln crlf
*.*proj* crlf
*.bat text eol=crlf
*.cmd text eol=crlf
# don't do anything to line-endings. Let CRLFs/LFs go into the repo, and CRLF/LFs on checkout
*.bat -crlf
*.xml -crlf
# CRLF Handling
+42 -32
View File
@@ -15,7 +15,7 @@
# Order is important. The last matching pattern has the most precedence.
/acceptance-tests @akoeplinger @kg
/acceptance-tests/profiler-stress* @alexrp @akoeplinger
/acceptance-tests/profiler-stress* @BrzVlad @akoeplinger
/data @directhex @akoeplinger
/data/gdb @lewurm @vargaz
@@ -23,7 +23,7 @@
/docs @marek-safar @akoeplinger
/docs/riscv.md @alexrp @lewurm
/docs/sources/mono-api-profiler.html @alexrp
/docs/sources/mono-api-profiler.html @BrzVlad
/external @akoeplinger @marek-safar
/external/api-snapshot @marek-safar @akoeplinger
@@ -31,39 +31,39 @@
/external/bockbuild @akoeplinger @alexischr
/external/boringssl @baulig
/external/cecil @marek-safar @akoeplinger
/external/corefx @akoeplinger @marek-safar @egorbo @alexischr
/external/corert @akoeplinger @marek-safar @egorbo @alexischr
/external/corefx @akoeplinger @marek-safar @egorbo @steveisok
/external/corert @akoeplinger @marek-safar @egorbo @steveisok
/external/linker @marek-safar
/external/llvm @alexanderkyte @vargaz
/external/llvm-project @SamMonoRT @vargaz @imhameed
/external/nuget-buildtasks @radical @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger
/external/roslyn-binaries @kg @marek-safar @akoeplinger @lewing
/libgc @BrzVlad @vargaz
/libgc @BrzVlad @vargaz @naricc
/llvm @vargaz @alexanderkyte
/llvm @vargaz @SamMonoRT @imhameed
/man @marek-safar @miguel
/man/*prof* @alexrp
/man/*prof* @BrzVlad
/mcs @marek-safar @akoeplinger @egorbo
/mcs/build @marek-safar @akoeplinger
/mcs/class @marek-safar @akoeplinger
/mcs/class @steveisok @akoeplinger
/mcs/class/corlib/System.Reflection*/ @vargaz @lambdageek
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/Mono.Btls.Interface @bauligg @marek-safar
/mcs/class/corlib/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/Mono.Btls.Interface @baulig @steveisok
/mcs/class/Mono.Data.Tds @egorbo
/mcs/class/Mono.Debugger.Soft @vargaz @thaystg
/mcs/class/Mono.Options @jonpryor @marek-safar
/mcs/class/Mono.Profiler.Log @lambdageek
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @marek-safar
/mcs/class/Mono.Security/Mono.Security/Interface @baulig @steveisok
/mcs/class/System/Mono.AppleTls @baulig @rolf
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @marek-safar
/mcs/class/System/Mono.Security.Interface @baulig @marek-safar
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @marek-safar
/mcs/class/System.Data* @egorbo @marek-safar
/mcs/class/System/Mono.Btls @baulig @marek-safar
/mcs/class/System/Mono.Net.Security @baulig @steveisok
/mcs/class/System/Mono.Security.Interface @baulig @steveisok
/mcs/class/System/System.Security.Cryptography.X509Certificates @baulig @steveisok
/mcs/class/System.Data* @egorbo @steveisok
/mcs/errors @marek-safar @lewurm
/mcs/ilasm @marek-safar @lewurm
@@ -71,10 +71,10 @@
/mcs/mcs @marek-safar @lewurm
/mcs/tests @marek-safar @lewurm
/mcs/tools @marek-safar @akoeplinger
/mcs/tools @marek-safar @akoeplinger @steveisok
/mcs/tools/gensources @kg @akoeplinger
/mcs/tools/resx2sr @baulig
/mcs/tools/upload-to-sentry @alexanderkyte @akoeplinger
/mcs/tools/upload-to-sentry @directhex @akoeplinger
/mono/arch @vargaz @lewurm
@@ -82,44 +82,55 @@
/mono/dis @lambdageek @vargaz
/mono/eglib @vargaz @lambdageek
/mono/benchmark @alexanderkyte @lewurm
/mono/benchmark @SamMonoRT @lewurm
/mono/metadata @vargaz @lambdageek @thaystg
/mono/metadata/*profiler* @lambdageek
/mono/metadata/handle* @lambdageek @vargaz
/mono/metadata/monitor* @brzvlad @vargaz
/mono/metadata/sgen* @brzvlad @vargaz
/mono/metadata/sgen* @brzvlad @vargaz @naricc
/mono/metadata/thread* @lateralusX @lambdageek
/mono/metadata/w32* @lateralusX @lambdageek
/mono/metadata/*-win* @lateralusX @lambdageek
/mono/metadata/appdomain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/assembly* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/domain* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/loader* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/native-library* @vargaz @lambdageek @CoffeeFlux
/mono/metadata/*-unix* @lambdageek @CoffeeFlux
/mono/mini @vargaz @lewurm @lambdageek
/mono/mini @vargaz @lewurm @lambdageek @SamMonoRT
/mono/mini/*cfgdump* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz
/mono/mini/*exceptions* @lewurm @vargaz @BrzVlad
/mono/mini/*ppc* @lewurm @vargaz
/mono/mini/*riscv* @alexrp @lewurm
/mono/mini/*type-check* @lewurm @lambdageek
/mono/mini/*llvm* @vargaz @imhameed
/mono/mini/interp/* @lewurm @BrzVlad
/mono/mini/*profiler* @alexrp @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas
/mono/mini/*profiler* @BrzVlad @lambdageek
/mono/mini/debugger-agent.c @vargaz @thaystg @DavidKarlas @lambdageek
/mono/native @baulig @egorbo @alexischr @marek-safar
/mono/profiler @alexrp @lambdageek
/mono/profiler @BrzVlad @lambdageek
/mono/sgen @BrzVlad @lambdageek
/mono/tools/offsets-tool @lewurm @rolfbjarne
/mono/utils @vargaz @lewurm @lambdageek
/mono/utils/atomic* @lewurm @vargaz
/mono/utils/mono-hwcap* @lewurm @vargaz
/mono/utils/mono-mem* @lewurm @vargaz
/mono/utils/mono-merp* @alexischr @lambdageek
/mono/utils/mono-state* @alexischr @lambdageek
/mono/utils/mono-threads* @lambdageek @vargaz
/mono/utils/*-win* @lateralusX @lambdageek
/msvc @lateralusX @kg @akoeplinger
/msvc/scripts @akoeplinger @kg
/netcore @marek-safar @akoeplinger @egorbo
/netcore @marek-safar @akoeplinger @egorbo @vargaz @steveisok
/packaging @alexischr @akoeplinger @directhex
/packaging/MacSDK/msbuild.py @radical @akoeplinger
@@ -127,13 +138,13 @@
/runtime @akoeplinger @marek-safar
/samples @lambdageek @lewurm
/samples/profiler @alexrp @lambdageek
/samples/profiler @BrzVlad @lambdageek
/samples/size @alexrp @lambdageek
/scripts @akoeplinger @directhex
/scripts/ci @akoeplinger @directhex
/sdks @marek-safar @lewurm @vargaz
/sdks @marek-safar @lewurm @vargaz @lambdageek
/sdks/builds/android* @alexischr @jonpryor
/sdks/builds/mxe* @alexischr @jonpryor
/sdks/builds/ios* @vargaz @rolfbjarne
@@ -147,6 +158,5 @@
/tools @lambdageek @vargaz
/tools/locale-builder @marek-safar @egorbo
/tools/nuget-hash-extractor @marek-safar @radical @akoeplinger
/tools/offsets-tool @lewurm @rolfbjarne
/tools/pedump @lambdageek @lewurm
/tools/sgen @brzvlad @vargaz
/tools/sgen @brzvlad @vargaz @naricc
+2 -2
View File
@@ -138,7 +138,7 @@ llvm/build/
llvm/usr/
sdks/builds/llvm-*/
sdks/builds/android-*/
mcs/build/compiler-server.log
##############################################################################
# Arcade output directories
@@ -146,4 +146,4 @@ sdks/builds/android-*/
.packages
artifacts
.dotnet
.dotnet
+7 -4
View File
@@ -58,13 +58,16 @@
[submodule "external/api-snapshot"]
path = external/api-snapshot
url = git://github.com/mono/api-snapshot.git
[submodule "external/llvm"]
path = external/llvm
url = git://github.com/mono/llvm.git
branch = release_60
[submodule "external/helix-binaries"]
path = external/helix-binaries
url = git://github.com/mono/helix-binaries.git
[submodule "external/illinker-test-assets"]
path = external/illinker-test-assets
url = git://github.com/mono/illinker-test-assets.git
[submodule "external/llvm-project"]
path = external/llvm-project
url = git://github.com/dotnet/llvm-project.git
branch = release/6.x
[submodule "external/bdwgc"]
path = external/bdwgc
url = git://github.com/Unity-Technologies/bdwgc.git
+15 -50
View File
@@ -2,32 +2,11 @@ ACLOCAL_AMFLAGS = -I m4
AM_CFLAGS = $(WERROR_CFLAGS)
MONOTOUCH_SUBDIRS = $(libgc_dir) mono
SUBDIRS = @MONO_SUBDIRS@
noinst_SUBDIRS = @MONO_NOINST_SUBDIRS@
DIST_SUBDIRS = $(SUBDIRS) m4 netcore
# Some tools might not build when cross-compiling
if CROSS_COMPILING
tools_dir =
else
tools_dir = tools
endif
if ENABLE_MSVC
build_with_msvc = msvc
build_without_msvc =
else
build_with_msvc =
build_without_msvc = msvc
endif
if ENABLE_NETCORE
update_submodules =
else
update_submodules = update_submodules
endif
SUBDIRS = $(build_with_msvc) mk po $(libgc_dir) llvm mono $(ikvm_native_dir) support data runtime scripts man samples $(tools_dir) $(build_without_msvc) $(docs_dir) acceptance-tests
# Keep in sync with SUBDIRS
DIST_SUBDIRS = $(build_with_msvc) m4 mk po $(libgc_dir) llvm mono ikvm-native support data runtime scripts man samples tools $(build_without_msvc) docs acceptance-tests netcore
if !ENABLE_NETCORE
all: $(update_submodules)
@@ -45,9 +24,10 @@ EXTRA_DIST= \
winconfig.h \
code_of_conduct.md \
external \
netcore/Makefile \
mcs/class/referencesource
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_MCS=false EXTERNAL_RUNTIME=false
DISTCHECK_CONFIGURE_FLAGS = EXTERNAL_RUNTIME=false
# Distribute the 'mcs' tree too
GIT_DIR ?= $(srcdir)/.git
@@ -65,6 +45,9 @@ dist-hook:
rm -f `find $(top_distdir)/external -path '*\.exe' -not -path '*/roslyn-binaries/*'`
rm -f `find $(top_distdir)/external -path '*\.dll' -not -path '*/binary-reference-assemblies/*' -not -path '*/roslyn-binaries/*' -not -path '*/helix-binaries/*'`
rm -rf "$(top_distdir)/external/linker/test"
rm -rf "$(top_distdir)/external/llvm-project/lldb/test"
rm -rf "$(top_distdir)/external/llvm-project/libcxx/test"
rm -rf "$(top_distdir)/external/llvm-project/clang/test"
pkgconfigdir = $(libdir)/pkgconfig
noinst_DATA = mono-uninstalled.pc
@@ -111,30 +94,10 @@ mcs-do-compiler-tests:
bootstrap-world: compiler-tests
$(MAKE) install
if INSTALL_MONOTOUCH
monotouch-do-build: config.h
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-build";; \
* ) target="all";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-build)
monotouch-do-clean:
@list='$(MONOTOUCH_SUBDIRS)'; for subdir in $$list; do \
case "x$$subdir" in \
xmono ) target="monotouch-do-clean";; \
* ) target="clean";; \
esac; \
echo "Making $$target in $$subdir"; \
(cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target); \
done;
(cd runtime && $(MAKE) $(AM_MAKEFLAGS) monotouch-do-clean)
endif
install:
for mydir in $(filter-out $(noinst_SUBDIRS),$(SUBDIRS)); do \
(cd $${mydir} && ${MAKE} install) \
done
update-csproj:
-rm msvc/scripts/order
@@ -178,3 +141,5 @@ update-solution-files:
update-solution-files-with-tests:
$(MAKE) "GENPROJ_ARGS=2012 true true" update-solution-files
endif
+1 -1
View File
@@ -2,7 +2,7 @@
<configuration>
<packageSources>
<clear />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
+4 -6
View File
@@ -30,7 +30,8 @@ The Mono project is part of the [.NET Foundation](https://www.dotnetfoundation.o
| Windows | i386 | [![windows-i386][17]][18] |
| CentOS | s390x (cs) | [![centos-s390x][19]][20] |
| Debian 9 | ppc64el (cs) | [![debian-9-ppc64el][21]][22]|
| AIX | ppc64 (cs) | [![aix-ppc64][23]][24] |
| AIX 6.1 | ppc64 (cs) | [![aix-ppc64][23]][24] |
| FreeBSD 12 | amd64 (cs) | [![freebsd-amd64][25]][26] |
_(cs) = community supported architecture_
@@ -58,6 +59,8 @@ _(cs) = community supported architecture_
[22]: https://jenkins.mono-project.com/job/test-mono-mainline-community-chroot/label=debian-9-ppc64el
[23]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64/badge/icon
[24]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=aix-ppc64
[25]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64/badge/icon
[26]: https://jenkins.mono-project.com/job/test-mono-mainline-community/label=freebsd-12-amd64
Compilation and Installation
============================
@@ -431,7 +434,6 @@ Disables compilation for the SSA optimization
framework, and the various SSA-based optimizations.
* `--enable-llvm`
* `--enable-loadedllvm`
* This enables the use of LLVM as a code generation engine
for Mono. The LLVM code generator and optimizer will be
@@ -444,10 +446,6 @@ full details and up-to-date information on this feature.
* You will need to have an LLVM built that Mono can link
against.
* The `--enable-loadedllvm` variant will make the LLVM backend
into a runtime-loadable module instead of linking it directly
into the main mono binary.
* `--enable-big-arrays` - Enable use of arrays with indexes larger
than Int32.MaxValue.
+125
View File
@@ -58,3 +58,128 @@ License notice for Apple
-------------------------
Content: https://github.com/mono/mono/blob/435f968102d0820ca65d63e1e741e4130b4d1303/support/ios/net/route.h
License: https://opensource.apple.com/apsl/
License for GCC
---------------
License: http://gcc.gnu.org/onlinedocs/gcc-4.6.4/libstdc++/manual/manual/license.html
Content: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini
Example: https://github.com/mono/mono/blob/c398e19d3515c6bbb6046ed014b2095594a15229/mono/mini/decompose.c#L967
GCC RUNTIME LIBRARY EXCEPTION
Version 3.1, 31 March 2009
Copyright (C) 2009 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
This GCC Runtime Library Exception ("Exception") is an additional
permission under section 7 of the GNU General Public License, version
3 ("GPLv3"). It applies to a given file (the "Runtime Library") that
bears a notice placed by the copyright holder of the file stating that
the file is governed by GPLv3 along with this Exception.
When you use GCC to compile a program, GCC may combine portions of
certain GCC header files and runtime libraries with the compiled
program. The purpose of this Exception is to allow compilation of
non-GPL (including proprietary) programs to use, in this way, the
header files and runtime libraries covered by this Exception.
0. Definitions.
A file is an "Independent Module" if it either requires the Runtime
Library for execution after a Compilation Process, or makes use of an
interface provided by the Runtime Library, but is not otherwise based
on the Runtime Library.
"GCC" means a version of the GNU Compiler Collection, with or without
modifications, governed by version 3 (or a specified later version) of
the GNU General Public License (GPL) with the option of using any
subsequent versions published by the FSF.
"GPL-compatible Software" is software whose conditions of propagation,
modification and use would permit combination with GCC in accord with
the license of GCC.
"Target Code" refers to output from any compiler for a real or virtual
target processor architecture, in executable form or suitable for
input to an assembler, loader, linker and/or execution
phase. Notwithstanding that, Target Code does not include data in any
format that is used as a compiler intermediate representation, or used
for producing a compiler intermediate representation.
The "Compilation Process" transforms code entirely represented in
non-intermediate languages designed for human-written code, and/or in
Java Virtual Machine byte code, into Target Code. Thus, for example,
use of source code generators and preprocessors need not be considered
part of the Compilation Process, since the Compilation Process can be
understood as starting with the output of the generators or
preprocessors.
A Compilation Process is "Eligible" if it is done using GCC, alone or
with other GPL-compatible software, or if it is done without using any
work based on GCC. For example, using non-GPL-compatible Software to
optimize any GCC intermediate representations would not qualify as an
Eligible Compilation Process.
1. Grant of Additional Permission.
You have permission to propagate a work of Target Code formed by
combining the Runtime Library with Independent Modules, even if such
propagation would otherwise violate the terms of GPLv3, provided that
all Target Code was generated by Eligible Compilation Processes. You
may then convey such a combination under terms of your choice,
consistent with the licensing of the Independent Modules.
2. No Weakening of GCC Copyleft.
The availability of this Exception does not imply any general
presumption that third-party software is unaffected by the copyleft
requirements of the license of GCC.
Hopefully that text is self-explanatory. If it isn't, you need to speak to your lawyer, or the Free Software Foundation.
License notice for Silicon Graphics and HP
------------------------------------------
/*
* Copyright (c) 1996-1997
* Silicon Graphics Computer Systems, Inc.
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Silicon Graphics makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*
* Copyright (c) 2002
* Hewlett-Packard Company
*
* Permission to use, copy, modify, distribute and sell this software
* and its documentation for any purpose is hereby granted without fee,
* provided that the above copyright notice appear in all copies and
* that both that copyright notice and this permission notice appear
* in supporting documentation. Hewlett-Packard Company makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied warranty.
*/
/*
* This implements standard-conforming allocators that interact with
* the garbage collector. Gc_alloctor<T> allocates garbage-collectable
* objects of type T. Traceable_allocator<T> allocates objects that
* are not temselves garbage collected, but are scanned by the
* collector for pointers to collectable objects. Traceable_alloc
* should be used for explicitly managed STL containers that may
* point to collectable objects.
*
* This code was derived from an earlier version of the GNU C++ standard
* library, which itself was derived from the SGI STL implementation.
*/
Content: https://github.com/mono/mono/blob/d58dfa185af1d48fb93efdef1d046762f214f0a7/libgc/include/gc_allocator.h
+1 -1
View File
@@ -2,7 +2,7 @@
{
"name": "roslyn",
"url": "git://github.com/mono/roslyn.git",
"rev": "7a840bd5697ef6ea41f554a09d837636fba9d88d",
"rev": "6bfcd4e770609a38b8523e83ce1ec943162f30bb",
"remote-branch": "origin/mono-testing",
"branch": "mono-testing",
"directory": "roslyn"
+30 -9
View File
@@ -104,12 +104,10 @@ if grep "^AM_PROG_LIBTOOL" configure.ac >/dev/null; then
fi
fi
#
# Plug in the extension module
#
# Parse parameters
has_ext_mod=false
ext_mod_args=''
has_disable_boehm=false
for PARAM; do
if [[ $PARAM =~ "--enable-extension-module" ]] ; then
has_ext_mod=true
@@ -117,8 +115,14 @@ for PARAM; do
ext_mod_args=`echo $PARAM | cut -d= -f2`
fi
fi
if [[ $PARAM =~ "--disable-boehm" ]] ; then
has_disable_boehm=true
fi
done
#
# Plug in the extension module
#
if test x$has_ext_mod = xtrue; then
pushd $top_srcdir../mono-extensions/scripts
sh ./prepare-repo.sh $ext_mod_args || exit 1
@@ -150,17 +154,34 @@ automake $am_opt ||
echo "Running autoconf ..."
autoconf || { echo "**Error**: autoconf failed."; exit 1; }
if test -d $srcdir/libgc; then
echo Running libgc/autogen.sh ...
(cd $srcdir/libgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running libgc/autogen.sh ...
# Update all submodules recursively to ensure everything is checked out
if test -e $srcdir/scripts/update_submodules.sh; then
(cd $srcdir && scripts/update_submodules.sh)
fi
if test x$has_disable_boehm = xfalse -a -d $srcdir/external/bdwgc; then
echo Running external/bdwgc/autogen.sh ...
(cd $srcdir/external/bdwgc ; NOCONFIGURE=1 ./autogen.sh "$@")
echo Done running external/bdwgc/autogen.sh ...
fi
if test x$MONO_EXTRA_CONFIGURE_FLAGS != x; then
echo "MONO_EXTRA_CONFIGURE_FLAGS is $MONO_EXTRA_CONFIGURE_FLAGS"
fi
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings" #--enable-iso-c
host_conf_flag=
build_uname_all=`(uname -a) 2>/dev/null`
case "$build_uname_all" in
CYGWIN*)
if [[ "$@" != *"--host="* ]]; then
echo "Missing --host parameter, configure using ./configure --host=i686-w64-mingw32 or --host=x86_64-w64-mingw32"
echo "Falling back using --host=x86_64-w64-mingw32 as default."
host_conf_flag="--host=x86_64-w64-mingw32"
fi
;;
esac
conf_flags="$MONO_EXTRA_CONFIGURE_FLAGS --enable-maintainer-mode --enable-compile-warnings $host_conf_flag" #--enable-iso-c
if test x$NOCONFIGURE = x; then
echo Running $srcdir/configure $conf_flags "$@" ...
+94 -86
View File
@@ -574,6 +574,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ilasm", "mcs/ilasm/ilasm.cs
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "al", "mcs/tools/al/al.csproj", "{0A20671F-CCA9-457D-886C-0828C7D64128}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "aprofutil", "mcs/tools/aprofutil/aprofutil.csproj", "{658B566B-CED1-4C42-9926-C1AA930BA368}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "monolinker", "mcs/tools/linker/monolinker.csproj", "{D60432F1-3C12-4235-A1CC-4CA5575403BD}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "culevel", "mcs/tools/culevel/culevel.csproj", "{E8E246BD-CD0C-4734-A3C2-7F44796EC47B}"
@@ -1534,10 +1536,8 @@ Global
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|unreal.Build.0 = Debug|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.ActiveCfg = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|unreal.Build.0 = Release|unreal
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.Build.0 = Debug|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.Build.0 = Release|wasm
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|wasm.ActiveCfg = Release|net_4_x
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.ActiveCfg = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Debug|winaot.Build.0 = Debug|winaot
{2281525D-7F2E-41C1-982F-E612AF3ECF61}.Release|winaot.ActiveCfg = Release|winaot
@@ -1662,10 +1662,8 @@ Global
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|unreal.Build.0 = Debug|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.ActiveCfg = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|unreal.Build.0 = Release|unreal
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.Build.0 = Debug|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.Build.0 = Release|wasm
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|wasm.ActiveCfg = Release|net_4_x
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.ActiveCfg = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Debug|winaot.Build.0 = Debug|winaot
{7395C26D-7C2A-4F7F-AB7D-CE0B50BDFF38}.Release|winaot.ActiveCfg = Release|winaot
@@ -2212,10 +2210,8 @@ Global
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|unreal.Build.0 = Debug|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.ActiveCfg = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|unreal.Build.0 = Release|unreal
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.Build.0 = Debug|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.Build.0 = Release|wasm
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|wasm.ActiveCfg = Debug|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|wasm.ActiveCfg = Release|net_4_x
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.ActiveCfg = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Debug|winaot.Build.0 = Debug|winaot
{2401CC34-7A55-4EC4-B8BF-6E846245F5FE}.Release|winaot.ActiveCfg = Release|winaot
@@ -2274,10 +2270,8 @@ Global
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|unreal.Build.0 = Debug|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.ActiveCfg = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|unreal.Build.0 = Release|unreal
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.Build.0 = Debug|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.Build.0 = Release|wasm
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|wasm.ActiveCfg = Release|net_4_x
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.ActiveCfg = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Debug|winaot.Build.0 = Debug|winaot
{89CDEBFC-EF90-4437-9D6B-439277F2C7E5}.Release|winaot.ActiveCfg = Release|winaot
@@ -2380,10 +2374,8 @@ Global
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|unreal.Build.0 = Debug|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.ActiveCfg = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|unreal.Build.0 = Release|unreal
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.Build.0 = Debug|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.Build.0 = Release|wasm
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|wasm.ActiveCfg = Release|net_4_x
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.ActiveCfg = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Debug|winaot.Build.0 = Debug|winaot
{182E5AD7-2F59-48E5-B338-C515733AE79B}.Release|winaot.ActiveCfg = Release|winaot
@@ -2574,10 +2566,8 @@ Global
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|unreal.Build.0 = Debug|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.ActiveCfg = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|unreal.Build.0 = Release|unreal
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.Build.0 = Debug|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.Build.0 = Release|wasm
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|wasm.ActiveCfg = Release|net_4_x
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.ActiveCfg = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Debug|winaot.Build.0 = Debug|winaot
{B6102416-28B0-44C3-88A6-84B8C6B5E7EF}.Release|winaot.ActiveCfg = Release|winaot
@@ -3642,10 +3632,8 @@ Global
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|unreal.Build.0 = Debug|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.ActiveCfg = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|unreal.Build.0 = Release|unreal
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.Build.0 = Debug|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.Build.0 = Release|wasm
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|wasm.ActiveCfg = Debug|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|wasm.ActiveCfg = Release|net_4_x
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.ActiveCfg = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Debug|winaot.Build.0 = Debug|winaot
{49AC287E-29E1-4683-98B8-6E65DA581854}.Release|winaot.ActiveCfg = Release|winaot
@@ -3968,10 +3956,8 @@ Global
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|unreal.Build.0 = Debug|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.ActiveCfg = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|unreal.Build.0 = Release|unreal
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.Build.0 = Debug|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.Build.0 = Release|wasm
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|wasm.ActiveCfg = Debug|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|wasm.ActiveCfg = Release|net_4_x
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.ActiveCfg = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Debug|winaot.Build.0 = Debug|winaot
{B949A52E-3240-4BD3-B57B-89C78D9EB9D4}.Release|winaot.ActiveCfg = Release|winaot
@@ -4028,24 +4014,42 @@ Global
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|net_4_x.Build.0 = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|net_4_x.Build.0 = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|net_4_x
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.ActiveCfg = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monodroid.Build.0 = Debug|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.ActiveCfg = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monodroid.Build.0 = Release|monodroid
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.ActiveCfg = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch.Build.0 = Debug|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.ActiveCfg = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch.Build.0 = Release|monotouch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.ActiveCfg = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_tv.Build.0 = Debug|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.ActiveCfg = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_tv.Build.0 = Release|monotouch_tv
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.ActiveCfg = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|monotouch_watch.Build.0 = Debug|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.ActiveCfg = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|monotouch_watch.Build.0 = Release|monotouch_watch
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.ActiveCfg = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|orbis.Build.0 = Debug|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.ActiveCfg = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|orbis.Build.0 = Release|orbis
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.ActiveCfg = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|unreal.Build.0 = Debug|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.ActiveCfg = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|unreal.Build.0 = Release|unreal
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.ActiveCfg = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|wasm.Build.0 = Debug|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.ActiveCfg = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|wasm.Build.0 = Release|wasm
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.ActiveCfg = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|winaot.Build.0 = Debug|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.ActiveCfg = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|winaot.Build.0 = Release|winaot
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.ActiveCfg = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Debug|xammac.Build.0 = Debug|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.ActiveCfg = Release|xammac
{AC3AC4DA-7C10-4AE9-A131-6DBBE61432E3}.Release|xammac.Build.0 = Release|xammac
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|net_4_x.Build.0 = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|net_4_x.ActiveCfg = Release|net_4_x
@@ -4074,10 +4078,8 @@ Global
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|unreal.Build.0 = Debug|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.ActiveCfg = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|unreal.Build.0 = Release|unreal
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.Build.0 = Debug|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.Build.0 = Release|wasm
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|wasm.ActiveCfg = Release|net_4_x
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.ActiveCfg = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Debug|winaot.Build.0 = Debug|winaot
{7C660E7A-C89B-4ACA-9458-492B6D14B168}.Release|winaot.ActiveCfg = Release|winaot
@@ -4198,10 +4200,8 @@ Global
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|unreal.Build.0 = Debug|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.ActiveCfg = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|unreal.Build.0 = Release|unreal
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.Build.0 = Debug|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.Build.0 = Release|wasm
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|wasm.ActiveCfg = Release|net_4_x
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.ActiveCfg = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Debug|winaot.Build.0 = Debug|winaot
{7B3EB110-1E57-4CA0-B65B-DB51B7F95D4D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4238,10 +4238,8 @@ Global
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|unreal.Build.0 = Debug|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.ActiveCfg = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|unreal.Build.0 = Release|unreal
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.Build.0 = Debug|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.Build.0 = Release|wasm
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|wasm.ActiveCfg = Debug|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|wasm.ActiveCfg = Release|net_4_x
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.ActiveCfg = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Debug|winaot.Build.0 = Debug|winaot
{E161C076-8631-4D5D-96BD-2B2FB2CEE76D}.Release|winaot.ActiveCfg = Release|winaot
@@ -4498,10 +4496,8 @@ Global
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|unreal.Build.0 = Debug|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.ActiveCfg = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|unreal.Build.0 = Release|unreal
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.Build.0 = Debug|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.Build.0 = Release|wasm
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|wasm.ActiveCfg = Debug|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|wasm.ActiveCfg = Release|net_4_x
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.ActiveCfg = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Debug|winaot.Build.0 = Debug|winaot
{0F4B1E4B-86CA-43CE-B773-BD942E71B990}.Release|winaot.ActiveCfg = Release|winaot
@@ -6016,10 +6012,8 @@ Global
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|unreal.Build.0 = Debug|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.ActiveCfg = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|unreal.Build.0 = Release|unreal
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.Build.0 = Debug|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.Build.0 = Release|wasm
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|wasm.ActiveCfg = Release|net_4_x
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.ActiveCfg = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Debug|winaot.Build.0 = Debug|winaot
{7D51E040-A2D0-4F86-9B1F-C010AB99396B}.Release|winaot.ActiveCfg = Release|winaot
@@ -6056,10 +6050,8 @@ Global
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|unreal.Build.0 = Debug|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.ActiveCfg = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|unreal.Build.0 = Release|unreal
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.Build.0 = Debug|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.Build.0 = Release|wasm
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|wasm.ActiveCfg = Debug|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|wasm.ActiveCfg = Release|net_4_x
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.ActiveCfg = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Debug|winaot.Build.0 = Debug|winaot
{6E922C6D-E914-45AD-A5E0-4DCC6CB8C469}.Release|winaot.ActiveCfg = Release|winaot
@@ -6536,10 +6528,8 @@ Global
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|unreal.Build.0 = Debug|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.ActiveCfg = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|unreal.Build.0 = Release|unreal
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.Build.0 = Debug|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.Build.0 = Release|wasm
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|wasm.ActiveCfg = Release|net_4_x
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.ActiveCfg = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Debug|winaot.Build.0 = Debug|winaot
{7EC5DC98-75BE-44FB-9D99-056044EC18BD}.Release|winaot.ActiveCfg = Release|winaot
@@ -6576,10 +6566,8 @@ Global
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|unreal.Build.0 = Debug|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.ActiveCfg = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|unreal.Build.0 = Release|unreal
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.Build.0 = Debug|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.Build.0 = Release|wasm
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|wasm.ActiveCfg = Debug|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|wasm.ActiveCfg = Release|net_4_x
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.ActiveCfg = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Debug|winaot.Build.0 = Debug|winaot
{7B87655B-F9BB-48FB-88FF-CB92582C4F89}.Release|winaot.ActiveCfg = Release|winaot
@@ -9856,10 +9844,8 @@ Global
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|unreal.Build.0 = Debug|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.ActiveCfg = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|unreal.Build.0 = Release|unreal
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.Build.0 = Debug|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.Build.0 = Release|wasm
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|wasm.ActiveCfg = Debug|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|wasm.ActiveCfg = Release|net_4_x
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.ActiveCfg = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Debug|winaot.Build.0 = Debug|winaot
{4F62F70F-FD2B-45E5-8A62-9ABFB6A29FC5}.Release|winaot.ActiveCfg = Release|winaot
@@ -10134,6 +10120,28 @@ Global
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|winaot.ActiveCfg = Release|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Debug|xammac.ActiveCfg = Debug|net_4_x
{0A20671F-CCA9-457D-886C-0828C7D64128}.Release|xammac.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|net_4_x.Build.0 = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|net_4_x.Build.0 = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monodroid.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monodroid.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_tv.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_tv.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|monotouch_watch.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|monotouch_watch.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|orbis.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|orbis.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|unreal.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|unreal.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|wasm.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|wasm.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|winaot.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|winaot.ActiveCfg = Release|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Debug|xammac.ActiveCfg = Debug|net_4_x
{658B566B-CED1-4C42-9926-C1AA930BA368}.Release|xammac.ActiveCfg = Release|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.ActiveCfg = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Debug|net_4_x.Build.0 = Debug|net_4_x
{D60432F1-3C12-4235-A1CC-4CA5575403BD}.Release|net_4_x.ActiveCfg = Release|net_4_x
+787 -467
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -6,7 +6,7 @@
<dllmap dll="libintl" name="bind_textdomain_codeset" target="@LIBC@" os="solaris"/>
<dllmap dll="libintl" target="@INTL@" os="!windows"/>
<dllmap dll="i:libxslt.dll" target="libxslt@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libodbc@libsuffix@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="@ODBC@" os="!windows"/>
<dllmap dll="i:odbc32.dll" target="libiodbc.dylib" os="osx"/>
<dllmap dll="oci" target="libclntsh@libsuffix@" os="!windows"/>
<dllmap dll="db2cli" target="libdb2_36@libsuffix@" os="!windows"/>
+13 -13
View File
@@ -1,8 +1,9 @@
import lldb
import traceback
def print_frames(thread, num_frames, current_thread):
# TODO: Make output header similar to bt.
print '%c thread #%i' % ('*' if current_thread else ' ', thread.idx)
print('%c thread #%i' % ('*' if current_thread else ' ', thread.idx))
if current_thread:
selected_frame = thread.GetSelectedFrame()
@@ -14,17 +15,16 @@ def print_frames(thread, num_frames, current_thread):
if function_name == "interp_exec_method_full":
try:
s = 'frame->imethod->method'
klassname = frame.EvaluateExpression('(char*) ' + s + '->klass->name').summary[1:-1]
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
methoddesc = frame.EvaluateExpression('(char*) mono_method_full_name (' + s + ', 1)').summary[1:-1]
ipoffset = frame.EvaluateExpression('ip').GetValueAsUnsigned()
insn = ''
if ipoffset != 0:
ipoffset -= frame.EvaluateExpression('imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname [*ip]').summary[1:-1] + "\""
var = '%s::%s @ %d %s || %s' % (klassname, methodname, ipoffset, insn, frame)
ipoffset -= frame.EvaluateExpression('frame->imethod->code').GetValueAsUnsigned()
insn = "\"" + frame.EvaluateExpression('mono_interp_opname (*ip)').summary[1:-1] + "\""
var = '%s @ %d %s || %s' % (methoddesc, ipoffset, insn, frame)
except Exception as e:
print "DBG: execfail:" + str(e)
traceback.print_exc()
elif function_name == "mono_interp_transform_method":
try:
s = 'runtime_method->method'
@@ -32,7 +32,7 @@ def print_frames(thread, num_frames, current_thread):
methodname = frame.EvaluateExpression('(char*) ' + s + '->name').summary[1:-1]
var = 'transforming %s::%s || %s' % (klassname, methodname, frame)
except Exception as e:
print "DBG: transformfail:" + str(e)
print("DBG: transformfail:" + str(e))
elif pc[0] == '0':
try:
framestr = frame.EvaluateExpression('(char*)mono_pmip((void*)%s)' % pc).summary[1:-1]
@@ -40,7 +40,7 @@ def print_frames(thread, num_frames, current_thread):
except:
pass
print ' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var)
print(' %c %s' % ('*' if current_thread and frame.idx == selected_frame.idx else ' ', var))
def monobt(debugger, command, result, dict):
opts = {'all_bt': False, 'num_frames': None}
@@ -50,20 +50,20 @@ def monobt(debugger, command, result, dict):
elif command.isdigit():
opts['num_frames'] = int(command)
elif command != '':
print 'error: monobt [<number>|all]'
print('error: monobt [<number>|all]')
return
target = debugger.GetSelectedTarget()
process = target.process
if not process.IsValid():
print 'error: invalid process'
print('error: invalid process')
return
if opts['all_bt']:
for thread in process.threads:
print_frames(thread, len(thread), process.selected_thread == thread)
print ''
print('')
else:
thread = process.selected_thread
num_frames = len(thread) if opts['num_frames'] is None else opts['num_frames']
@@ -76,4 +76,4 @@ def __lldb_init_module (debugger, dict):
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f monobt.monobt monobt')
debugger.HandleCommand('command alias mbt monobt')
print '"monobt" command installed'
print('"monobt" command installed')
@@ -242,7 +242,7 @@ namespace HtmlAgilityPack
}
/// <summary>
/// Checks for existance of attribute with given name
/// Checks for existence of attribute with given name
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
+23
View File
@@ -18,6 +18,27 @@ convert.exe: $(srcdir)/convert.cs AgilityPack.dll
AgilityPack.dll:
$(CSCOMPILE) -r:$(topdir)/class/lib/$(PROFILE)/mscorlib.dll -r:$(topdir)/class/lib/$(PROFILE)/System.dll -r:$(topdir)/class/lib/$(PROFILE)/System.Xml.dll -target:library -out:$@ $(srcdir)/HtmlAgilityPack/*.cs
ifdef MCS_MODE
monoapi.zip:
touch $@
monoapi.tree: monoapi.zip
touch $@
mono-tools.zip:
touch $@
mono-tools.tree: mono-tools.zip
touch $@
mono-file-formats.zip:
touch $@
mono-file-formats.tree: mono-file-formats.zip
touch $@
else
monoapi.zip: monoapi.tree
@test -f $@ || { rm -f $< && $(MAKE) $<; }
@@ -36,5 +57,7 @@ mono-file-formats.zip: mono-file-formats.tree
mono-file-formats.tree: $(srcdir)/mono-file-formats.config $(srcdir)/docs.make
$(MDOC) assemble -o mono-file-formats -f man $<
endif
.doc-stamp:
+61 -39
View File
@@ -2,36 +2,58 @@
These are some useful links and notes pertaining to the RISC-V port of Mono.
## Useful RISC-V documents
## Useful RISC-V documentation
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications/)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa/)
* [RISC-V Debug Specification Standard](https://github.com/riscv/riscv-debug-spec/blob/master/riscv-debug-spec.pdf)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual/blob/master/riscv-asm.md)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md)
* [RISC-V User-Level ISA Specification](https://riscv.org/specifications)
* [RISC-V Privileged ISA Specification](https://riscv.org/specifications/privileged-isa)
* [RISC-V Debug Specification](https://riscv.org/specifications/debug-specification)
* [RISC-V ELF psABI Specification](https://github.com/riscv/riscv-elf-psabi-doc)
* [RISC-V C API Specification](https://github.com/riscv/riscv-c-api-doc)
* [RISC-V Toolchain Conventions](https://github.com/riscv/riscv-toolchain-conventions)
* [RISC-V Assembly Programmer's Manual](https://github.com/riscv/riscv-asm-manual)
## Useful RISC-V repositories
* [RISC-V Organization](https://github.com/riscv)
* [RISC-V Linux](https://github.com/riscv/riscv-linux)
* [RISC-V LLD](https://github.com/riscv/riscv-lld)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V Tools](https://github.com/riscv/riscv-tools)
* [RISC-V GNU Toolchain](https://github.com/riscv/riscv-gnu-toolchain)
* [RISC-V Binutils](https://github.com/riscv/riscv-binutils-gdb)
* [RISC-V GCC](https://github.com/riscv/riscv-gcc)
* [RISC-V Glibc](https://github.com/riscv/riscv-glibc)
* [RISC-V Newlib](https://github.com/riscv/riscv-newlib)
* [RISC-V QEMU](https://github.com/riscv/riscv-qemu)
* [RISC-V Opcodes](https://github.com/riscv/riscv-opcodes)
* [RISC-V Tests](https://github.com/riscv/riscv-tests)
* [lowRISC Organization](https://github.com/lowrisc)
* [RISC-V LLVM](https://github.com/lowrisc/riscv-llvm)
## Setting up a cross environment
Setting up a cross environment with a Linux toolchain and QEMU is quite easy.
### Debian/Ubuntu packages
First, add these to your `.bashrc` (or some other script that you run):
This is the most painless way of getting a functional toolchain installed. Note
that these packages may not be available on older distributions; this is tested
on Ubuntu 19.04.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=/usr/riscv64-linux-gnu
```
Next, install the toolchain packages like so:
```bash
# apt install autoconf automake binutils-riscv64-linux-gnu build-essential gcc-riscv64-linux-gnu gdb-multiarch g++-riscv64-linux-gnu libtool qemu qemu-system-misc qemu-user qemu-user-static
```
You will now have all the toolchain binaries in `/usr/bin`.
### Building manually
This approach may be somewhat unstable since you will be using the latest
versions of the various parts of the toolchain.
First, add this to your `$HOME/.bashrc` (or some other script that you run):
```bash
export RISCV=$HOME/riscv
@@ -39,47 +61,42 @@ export PATH=$RISCV/bin:$PATH
export QEMU_LD_PREFIX=$RISCV/sysroot
```
Install some dependencies needed to build the toolchain:
Next, install some dependencies needed to build the toolchain:
```console
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev libusb-1.0-0-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev device-tree-compiler pkg-config libexpat-dev libglib2.0-dev libpixman-1-dev
# apt install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
```
Now you can build the toolchain:
Finally, build the toolchain:
```console
$ git clone --recursive [email protected]:riscv/riscv-tools.git
$ cd riscv-tools
$ ./build.sh
$ git clone --recursive [email protected]:riscv/riscv-gnu-toolchain.git
$ cd riscv-gnu-toolchain
$ ./configure --prefix=$RISCV --enable-multilib
$ ./configure --prefix=$RISCV --enable-linux --enable-multilib --with-arch=rv64imafdc --with-abi=lp64d
$ make linux
$ cd ../..
$ git clone --recursive [email protected]:riscv/riscv-qemu.git
$ cd riscv-qemu
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --disable-werror
$ make
$ make install
```
This will install the built toolchain binaries in `$RISCV/bin`.
## Building Mono with a cross toolchain
Building Mono is quite straightforward:
```console
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
$ make
$ make install
```
(Note: You may need to use `--host=riscv64-unknown-linux-gnu` instead if you
built the toolchain manually.)
You can set `CFLAGS` as appropriate to change the RISC-V options, such as which
standard extensions and ABI to use. For example, to use the 64-bit soft float
ABI:
```console
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-unknown-linux-gnu
$ CFLAGS="-mabi=lp64" ./autogen.sh --prefix=$RISCV/sysroot --host=riscv64-linux-gnu
```
Note that, since this is a cross build, the `mcs` directory won't be built. You
@@ -89,19 +106,24 @@ Mono and copy them into `$RISCV/sysroot`.
You can run Mono with QEMU like this:
```console
$ qemu-riscv64 $RISCV/sysroot/bin/mono hello.exe
$ qemu-riscv64 ./mono/mini/mono hello.exe
```
## Debugging
Debugging with GDB currently requires a manually built toolchain.
It can be done like so:
```console
$ qemu-riscv64 -g 12345 ./mono/mini/mono-sgen --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono-sgen
$ qemu-riscv64 -g 12345 ./mono/mini/mono --interp basic.exe &
$ riscv64-unknown-elf-gdb -ex 'target remote localhost:12345' -ex 'b main' -ex 'c' ./mono/mini/mono
```
## Things to be done
In no particular order:
Things that need to be done beyond the basic 64-bit port, in no particular
order:
* Complete the soft float port.
* Complete the 32-bit port.
+6
View File
@@ -109,6 +109,12 @@ typedef struct {
<h3>Write Barriers</h3>
<p>SGen is a concurrent and generational GC, features which require
tracking changes to the state of the heap. This is achieved through
write barriers. Whenever native code is changing the state of the
heap by storing references into another managed object, it needs to
do it using this write barrier API.
<h4><a name="api:mono_gc_wbarrier_arrayref_copy">mono_gc_wbarrier_arrayref_copy</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_nostore">mono_gc_wbarrier_generic_nostore</a></h4>
<h4><a name="api:mono_gc_wbarrier_generic_store">mono_gc_wbarrier_generic_store</a></h4>
+16 -4
View File
@@ -1,15 +1,27 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.Private.CoreFx.NETCoreApp" Version="5.0.0-alpha.1.19563.3">
<Uri>https://github.com/dotnet/corefx</Uri>
<Sha>5cee7c97d602f294e27c582d4dab81ec388f1d7b</Sha>
</Dependency>
<Dependency Name="Microsoft.NETCore.App" Version="5.0.0-alpha.1.19564.1">
<Uri>https://github.com/dotnet/core-setup</Uri>
<Sha>c77948d92a2f950140f09384f057cb893ec3955a</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers" Version="3.3.1-beta4-19476-06">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>39d545dc14bc4d5a546e5e2f58d1ebaca72fcd61</Sha>
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19302.2">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.20171.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>e6a5d5f970bb872451c6310ae34eda31041fb552</Sha>
<Sha>7dbc907fa03eacf4c57f827cb4235d77b7ed4fcd</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19302.2">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.20171.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>e6a5d5f970bb872451c6310ae34eda31041fb552</Sha>
<Sha>7dbc907fa03eacf4c57f827cb4235d77b7ed4fcd</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
+8 -1
View File
@@ -1,7 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<VersionPrefix>1.0.0</VersionPrefix>
<VersionPrefix>5.0.0</VersionPrefix>
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
<!-- Package versions -->
<!-- corefx -->
<MicrosoftPrivateCoreFxNETCoreAppVersion>5.0.0-alpha.1.19563.3</MicrosoftPrivateCoreFxNETCoreAppVersion>
<!-- core-setup -->
<MicrosoftNETCoreAppVersion>5.0.0-alpha.1.19564.1</MicrosoftNETCoreAppVersion>
<!-- roslyn -->
<MicrosoftNetCompilersVersion>3.3.1-beta4-19476-06</MicrosoftNetCompilersVersion>
</PropertyGroup>
</Project>
+11
View File
@@ -0,0 +1,11 @@
@{
IncludeRules=@('PSAvoidUsingCmdletAliases',
'PSAvoidUsingWMICmdlet',
'PSAvoidUsingPositionalParameters',
'PSAvoidUsingInvokeExpression',
'PSUseDeclaredVarsMoreThanAssignments',
'PSUseCmdletCorrectly',
'PSStandardDSCFunctionsInResource',
'PSUseIdenticalMandatoryParametersForDSC',
'PSUseIdenticalParametersForDSC')
}
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PushToStaticFeed task that will read the build asset
manifest and publish the assets described in the manifest to
informed target feeds.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(MSBuildThisFileDirectory)DefaultVersions.props" Condition="Exists('$(MSBuildThisFileDirectory)DefaultVersions.props')" />
<!--
This won't be necessary once we solve this issue:
https://github.com/dotnet/arcade/issues/2266
-->
<Import Project="$(MSBuildThisFileDirectory)ArtifactsCategory.props" Condition="Exists('$(MSBuildThisFileDirectory)ArtifactsCategory.props')" />
<Import Project="$(NuGetPackageRoot)microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<Target Name="PublishToFeed">
<Error Condition="'$(ArtifactsCategory)' == ''" Text="ArtifactsCategory: The artifacts' category produced by the build wasn't provided." />
<Error Condition="'$(AccountKeyToStaticFeed)' == ''" Text="AccountKeyToStaticFeed: Account key for target feed wasn't provided." />
<Error Condition="'$(ManifestsBasePath)' == ''" Text="Full path to asset manifests directory wasn't provided." />
<Error Condition="'$(BlobBasePath)' == '' AND '$(PackageBasePath)' == ''" Text="A valid full path to BlobBasePath of PackageBasePath is required." />
<ItemGroup>
<!-- Include all manifests found in the manifest folder. -->
<ManifestFiles Include="$(ManifestsBasePath)*.xml" />
</ItemGroup>
<Error Condition="'@(ManifestFiles)' == ''" Text="No manifest file was found in the provided path: $(ManifestsBasePath)" />
<!--
For now the type of packages being published will be informed for the whole build.
Eventually this will be specified on a per package basis:
TODO: https://github.com/dotnet/arcade/issues/2266
-->
<PropertyGroup>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCORE'">https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == '.NETCOREVALIDATION'">https://dotnetfeed.blob.core.windows.net/arcade-validation/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETCORETOOLING'">https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ENTITYFRAMEWORKCORE'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETEXTENSIONS'">https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORECLR'">https://dotnetfeed.blob.core.windows.net/dotnet-coreclr/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'CORESDK'">https://dotnetfeed.blob.core.windows.net/dotnet-sdk/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSINTERNAL'">https://dotnetfeed.blob.core.windows.net/dotnet-tools-internal/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'TOOLSET'">https://dotnetfeed.blob.core.windows.net/dotnet-toolset/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'WINDOWSDESKTOP'">https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'NUGETCLIENT'">https://dotnetfeed.blob.core.windows.net/nuget-nugetclient/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETENTITYFRAMEWORK6'">https://dotnetfeed.blob.core.windows.net/aspnet-entityframework6/index.json</TargetStaticFeed>
<TargetStaticFeed Condition="'$(ArtifactsCategory.ToUpper())' == 'ASPNETBLAZOR'">https://dotnetfeed.blob.core.windows.net/aspnet-blazor/index.json</TargetStaticFeed>
</PropertyGroup>
<Error
Condition="'$(TargetStaticFeed)' == ''"
Text="'$(ArtifactsCategory)' wasn't recognized as a valid artifact category. Valid categories are: '.NetCore' and '.NetCoreValidation'" />
<!-- Iterate publishing assets from each manifest file. -->
<PushArtifactsInManifestToFeed
ExpectedFeedUrl="$(TargetStaticFeed)"
AccountKey="$(AccountKeyToStaticFeed)"
BARBuildId="$(BARBuildId)"
MaestroApiEndpoint="$(MaestroApiEndpoint)"
BuildAssetRegistryToken="$(BuildAssetRegistryToken)"
Overwrite="$(OverrideAssetsWithSameName)"
PassIfExistingItemIdentical="$(PassIfExistingItemIdentical)"
MaxClients="$(MaxParallelUploads)"
UploadTimeoutInMinutes="$(MaxUploadTimeoutInMinutes)"
AssetManifestPath="%(ManifestFiles.Identity)"
BlobAssetsBasePath="$(BlobBasePath)"
PackageAssetsBasePath="$(PackageBasePath)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
</ItemGroup>
</Project>
-82
View File
@@ -1,82 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the PublishSymbols tasks to publish symbols to MSDL and SymWeb.
Parameters:
- PDBArtifactsDirectory : Full path to directory containing PDB files to be published.
- BlobBasePath : Full path containing *.symbols.nupkg packages to be published.
- DotNetSymbolServerTokenMsdl : PAT to access MSDL.
- DotNetSymbolServerTokenSymWeb : PAT to access SymWeb.
- DotNetSymbolExpirationInDays : Expiration days for published packages. Default is 3650.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<Import Project="$(NuGetPackageRoot)microsoft.symboluploader.build.task\$(SymbolUploaderVersion)\build\PublishSymbols.targets" />
<Target Name="PublishSymbols">
<ItemGroup>
<FilesToPublishToSymbolServer Include="$(PDBArtifactsDirectory)\*.pdb"/>
<PackagesToPublishToSymbolServer Include="$(BlobBasePath)\*.symbols.nupkg"/>
<!--
These packages from Arcade-Services include some native libraries that
our current symbol uploader can't handle. Below is a workaround until
we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
-->
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Darc.*" />
<PackagesToPublishToSymbolServer Remove="$(BlobBasePath)\Microsoft.DotNet.Maestro.Tasks.*" />
</ItemGroup>
<PropertyGroup>
<DotNetSymbolExpirationInDays Condition="'$(DotNetSymbolExpirationInDays)' == ''">3650</DotNetSymbolExpirationInDays>
<PublishToSymbolServer>true</PublishToSymbolServer>
<PublishToSymbolServer Condition="'@(FilesToPublishToSymbolServer)' == '' and '@(PackagesToPublishToSymbolServer)' == ''">false</PublishToSymbolServer>
</PropertyGroup>
<Message
Importance="High"
Text="No symbol package(s) were found to publish."
Condition="$(PublishToSymbolServer) == false" />
<!-- Symbol Uploader: MSDL -->
<Message Importance="High" Text="Publishing symbol packages to MSDL ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenMsdl)"
SymbolServerPath="https://microsoftpublicsymbols.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
<!--
Symbol Uploader: SymWeb
Watson, VS insertion testings and the typical internal dev usage require SymWeb.
Currently we need to call the task twice (https://github.com/dotnet/core-eng/issues/3489).
-->
<Message Importance="High" Text="Publishing symbol packages to SymWeb ..." Condition="$(PublishToSymbolServer)" />
<PublishSymbols PackagesToPublish="@(PackagesToPublishToSymbolServer)"
FilesToPublish="@(FilesToPublishToSymbolServer)"
PersonalAccessToken="$(DotNetSymbolServerTokenSymWeb)"
SymbolServerPath="https://microsoft.artifacts.visualstudio.com/DefaultCollection"
ExpirationInDays="$(DotNetSymbolExpirationInDays)"
VerboseLogging="true"
DryRun="false"
ConvertPortablePdbsToWindowsPdbs="false"
PdbConversionTreatAsWarning=""
Condition="$(PublishToSymbolServer)"/>
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.SymbolUploader.Build.Task" Version="$(SymbolUploaderVersion)" />
</ItemGroup>
</Project>
+141
View File
@@ -0,0 +1,141 @@
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: PowerShell@2
# displayName: Setup Private Feeds Credentials
# condition: eq(variables['Agent.OS'], 'Windows_NT')
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.ps1
# arguments: -ConfigFile $(Build.SourcesDirectory)/NuGet.config -Password $Env:Token
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)][string]$ConfigFile,
[Parameter(Mandatory = $true)][string]$Password
)
$ErrorActionPreference = "Stop"
Set-StrictMode -Version 2.0
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
. $PSScriptRoot\tools.ps1
# Add source entry to PackageSources
function AddPackageSource($sources, $SourceName, $SourceEndPoint, $creds, $Username, $Password) {
$packageSource = $sources.SelectSingleNode("add[@key='$SourceName']")
if ($packageSource -eq $null)
{
$packageSource = $doc.CreateElement("add")
$packageSource.SetAttribute("key", $SourceName)
$packageSource.SetAttribute("value", $SourceEndPoint)
$sources.AppendChild($packageSource) | Out-Null
}
else {
Write-Host "Package source $SourceName already present."
}
AddCredential -Creds $creds -Source $SourceName -Username $Username -Password $Password
}
# Add a credential node for the specified source
function AddCredential($creds, $source, $username, $password) {
# Looks for credential configuration for the given SourceName. Create it if none is found.
$sourceElement = $creds.SelectSingleNode($Source)
if ($sourceElement -eq $null)
{
$sourceElement = $doc.CreateElement($Source)
$creds.AppendChild($sourceElement) | Out-Null
}
# Add the <Username> node to the credential if none is found.
$usernameElement = $sourceElement.SelectSingleNode("add[@key='Username']")
if ($usernameElement -eq $null)
{
$usernameElement = $doc.CreateElement("add")
$usernameElement.SetAttribute("key", "Username")
$sourceElement.AppendChild($usernameElement) | Out-Null
}
$usernameElement.SetAttribute("value", $Username)
# Add the <ClearTextPassword> to the credential if none is found.
# Add it as a clear text because there is no support for encrypted ones in non-windows .Net SDKs.
# -> https://github.com/NuGet/Home/issues/5526
$passwordElement = $sourceElement.SelectSingleNode("add[@key='ClearTextPassword']")
if ($passwordElement -eq $null)
{
$passwordElement = $doc.CreateElement("add")
$passwordElement.SetAttribute("key", "ClearTextPassword")
$sourceElement.AppendChild($passwordElement) | Out-Null
}
$passwordElement.SetAttribute("value", $Password)
}
function InsertMaestroPrivateFeedCredentials($Sources, $Creds, $Password) {
$maestroPrivateSources = $Sources.SelectNodes("add[contains(@key,'darc-int')]")
Write-Host "Inserting credentials for $($maestroPrivateSources.Count) Maestro's private feeds."
ForEach ($PackageSource in $maestroPrivateSources) {
Write-Host "`tInserting credential for Maestro's feed:" $PackageSource.Key
AddCredential -Creds $creds -Source $PackageSource.Key -Username $Username -Password $Password
}
}
if (!(Test-Path $ConfigFile -PathType Leaf)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
}
if (!$Password) {
Write-PipelineTelemetryError -Category 'Build' -Message 'Eng/common/SetupNugetSources.ps1 returned a non-zero exit code. Please supply a valid PAT'
ExitWithExitCode 1
}
# Load NuGet.config
$doc = New-Object System.Xml.XmlDocument
$filename = (Get-Item $ConfigFile).FullName
$doc.Load($filename)
# Get reference to <PackageSources> or create one if none exist already
$sources = $doc.DocumentElement.SelectSingleNode("packageSources")
if ($sources -eq $null) {
$sources = $doc.CreateElement("packageSources")
$doc.DocumentElement.AppendChild($sources) | Out-Null
}
# Looks for a <PackageSourceCredentials> node. Create it if none is found.
$creds = $doc.DocumentElement.SelectSingleNode("packageSourceCredentials")
if ($creds -eq $null) {
$creds = $doc.CreateElement("packageSourceCredentials")
$doc.DocumentElement.AppendChild($creds) | Out-Null
}
# Insert credential nodes for Maestro's private feeds
InsertMaestroPrivateFeedCredentials -Sources $sources -Creds $creds -Password $Password
$dotnet3Source = $sources.SelectSingleNode("add[@key='dotnet3']")
if ($dotnet3Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$dotnet31Source = $sources.SelectSingleNode("add[@key='dotnet3.1']")
if ($dotnet31Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2" -Creds $creds -Username "dn-bot" -Password $Password
}
$doc.Save($filename)
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# This file is a temporary workaround for internal builds to be able to restore from private AzDO feeds.
# This file should be removed as part of this issue: https://github.com/dotnet/arcade/issues/4080
#
# What the script does is iterate over all package sources in the pointed NuGet.config and add a credential entry
# under <packageSourceCredentials> for each Maestro's managed private feed. Two additional credential
# entries are also added for the two private static internal feeds: dotnet3-internal and dotnet3-internal-transport.
#
# This script needs to be called in every job that will restore packages and which the base repo has
# private AzDO feeds in the NuGet.config.
#
# See example YAML call for this script below. Note the use of the variable `$(dn-bot-dnceng-artifact-feeds-rw)`
# from the AzureDevOps-Artifact-Feeds-Pats variable group.
#
# - task: Bash@3
# displayName: Setup Private Feeds Credentials
# inputs:
# filePath: $(Build.SourcesDirectory)/eng/common/SetupNugetSources.sh
# arguments: $(Build.SourcesDirectory)/NuGet.config $Token
# condition: ne(variables['Agent.OS'], 'Windows_NT')
# env:
# Token: $(dn-bot-dnceng-artifact-feeds-rw)
ConfigFile=$1
CredToken=$2
NL='\n'
TB=' '
source="${BASH_SOURCE[0]}"
# resolve $source until the file is no longer a symlink
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ ! -f "$ConfigFile" ]; then
Write-PipelineTelemetryError -Category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Couldn't find the NuGet config file: $ConfigFile"
ExitWithExitCode 1
fi
if [ -z "$CredToken" ]; then
Write-PipelineTelemetryError -category 'Build' "Error: Eng/common/SetupNugetSources.sh returned a non-zero exit code. Please supply a valid PAT"
ExitWithExitCode 1
fi
if [[ `uname -s` == "Darwin" ]]; then
NL=$'\\\n'
TB=''
fi
# Ensure there is a <packageSources>...</packageSources> section.
grep -i "<packageSources>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSources>...</packageSources> section."
ConfigNodeHeader="<configuration>"
PackageSourcesTemplate="${TB}<packageSources>${NL}${TB}</packageSources>"
sed -i.bak "s|$ConfigNodeHeader|$ConfigNodeHeader${NL}$PackageSourcesTemplate|" NuGet.config
fi
# Ensure there is a <packageSourceCredentials>...</packageSourceCredentials> section.
grep -i "<packageSourceCredentials>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding <packageSourceCredentials>...</packageSourceCredentials> section."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceCredentialsTemplate="${TB}<packageSourceCredentials>${NL}${TB}</packageSourceCredentials>"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourcesNodeFooter${NL}$PackageSourceCredentialsTemplate|" NuGet.config
fi
PackageSources=()
# Ensure dotnet3-internal and dotnet3-internal-transport are in the packageSources if the public dotnet3 feeds are present
grep -i "<add key=\"dotnet3\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3-internal\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal')
grep -i "<add key=\"dotnet3-internal-transport\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3-internal-transport')
fi
# Ensure dotnet3.1-internal and dotnet3.1-internal-transport are in the packageSources if the public dotnet3.1 feeds are present
grep -i "<add key=\"dotnet3.1\"" $ConfigFile
if [ "$?" == "0" ]; then
grep -i "<add key=\"dotnet3.1-internal\"" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal')
grep -i "<add key=\"dotnet3.1-internal-transport\">" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding dotnet3.1-internal-transport to the packageSources."
PackageSourcesNodeFooter="</packageSources>"
PackageSourceTemplate="${TB}<add key=\"dotnet3.1-internal-transport\" value=\"https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2\" />"
sed -i.bak "s|$PackageSourcesNodeFooter|$PackageSourceTemplate${NL}$PackageSourcesNodeFooter|" $ConfigFile
fi
PackageSources+=('dotnet3.1-internal-transport')
fi
# I want things split line by line
PrevIFS=$IFS
IFS=$'\n'
PackageSources+="$IFS"
PackageSources+=$(grep -oh '"darc-int-[^"]*"' $ConfigFile | tr -d '"')
IFS=$PrevIFS
for FeedName in ${PackageSources[@]} ; do
# Check if there is no existing credential for this FeedName
grep -i "<$FeedName>" $ConfigFile
if [ "$?" != "0" ]; then
echo "Adding credentials for $FeedName."
PackageSourceCredentialsNodeFooter="</packageSourceCredentials>"
NewCredential="${TB}${TB}<$FeedName>${NL}<add key=\"Username\" value=\"dn-bot\" />${NL}<add key=\"ClearTextPassword\" value=\"$CredToken\" />${NL}</$FeedName>"
sed -i.bak "s|$PackageSourceCredentialsNodeFooter|$NewCredential${NL}$PackageSourceCredentialsNodeFooter|" $ConfigFile
fi
done
-83
View File
@@ -1,83 +0,0 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
<Project Sdk="Microsoft.NET.Sdk">
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
- PackageBasePath : Directory containing all files that need to be validated.
- SignCheckVersion : Version of SignCheck package to be used.
- SignValidationExclusionList : ItemGroup containing exclusion list to be forwarded to SignCheck.
- EnableJarSigningCheck : Whether .jar files should be validated.
- EnableStrongNameCheck : Whether strong name check should be performed.
-->
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<!--
From 'Signing.props' we import $(SignValidationExclusionList)
-->
<Import Project="$(MSBuildThisFileDirectory)Signing.props" Condition="Exists('$(MSBuildThisFileDirectory)Signing.props')" />
<Target Name="ValidateSigning">
<PropertyGroup>
<SignCheckToolPath>$(NuGetPackageRoot)Microsoft.DotNet.SignCheck\$(SignCheckVersion)\tools\Microsoft.DotNet.SignCheck.exe</SignCheckToolPath>
<SignCheckInputDir>$(PackageBasePath)</SignCheckInputDir>
<SignCheckLog>signcheck.log</SignCheckLog>
<SignCheckErrorLog>signcheck.errors.log</SignCheckErrorLog>
<SignCheckExclusionsFile>signcheck.exclusions.txt</SignCheckExclusionsFile>
</PropertyGroup>
<ItemGroup>
<!--
Documentation for these arguments is available here:
https://github.com/dotnet/arcade/tree/master/src/SignCheck
-->
<SignCheckArgs Include="--recursive" />
<SignCheckArgs Include="--traverse-subfolders" />
<SignCheckArgs Include="--file-status AllFiles" />
<SignCheckArgs Include="--log-file $(SignCheckLog)" />
<SignCheckArgs Include="--error-log-file $(SignCheckErrorLog)" />
<SignCheckArgs Include="--input-files $(SignCheckInputDir)" />
<SignCheckArgs Include="--exclusions-file $(SignCheckExclusionsFile)" Condition="'@(SignValidationExclusionList)' != ''" />
<SignCheckArgs Include="--verify-jar" Condition="'$(EnableJarSigningCheck)' == 'true'" />
<SignCheckArgs Include="--verify-strongname" Condition="'$(EnableStrongNameCheck)' == 'true'" />
</ItemGroup>
<WriteLinesToFile
File="$(SignCheckExclusionsFile)"
Lines="@(SignValidationExclusionList)"
Condition="'@(SignValidationExclusionList)' != ''"
Overwrite="true"
Encoding="Unicode"/>
<!--
IgnoreExitCode='true' because the tool doesn't return '0' on success.
-->
<Exec
Command="&quot;$(SignCheckToolPath)&quot; @(SignCheckArgs, ' ')"
IgnoreExitCode='true'
ConsoleToMsBuild="false"
StandardErrorImportance="high" />
<Error
Text="Signing validation failed. Check $(SignCheckErrorLog) for more information."
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
<Message
Text="##vso[artifact.upload containerfolder=LogFiles;artifactname=LogFiles]{SignCheckErrorLog}"
Condition="Exists($(SignCheckErrorLog)) and '$([System.IO.File]::ReadAllText($(SignCheckErrorLog)))' != ''" />
</Target>
<ItemGroup>
<PackageReference Include="Microsoft.DotNet.SignCheck" Version="$(SignCheckVersion)" />
</ItemGroup>
</Project>
-184
View File
@@ -1,184 +0,0 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SourceLinkToolPath, # Full path to directory where dotnet SourceLink CLI was installed
[Parameter(Mandatory=$true)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$true)][string] $GHCommit # GitHub commit SHA used to build the packages
)
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @(".dll", ".exe", ".pdb")
Write-Host -NoNewLine "Validating" ([System.IO.Path]::GetFileName($PackagePath)) "... "
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith(".resources.dll")) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
# Makes easier to reference `sourcelink cli`
Push-Location $using:SourceLinkToolPath
$SourceLinkInfos = .\sourcelink.exe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = -Join("https://raw.githubusercontent.com/", $using:GHRepoName, "/", $using:GHCommit, "/")
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and $Uri.Host -match "github") {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
Pop-Location
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
$zip.Dispose()
if ($FailedFiles -eq 0) {
Write-Host "Passed."
}
}
function ValidateSourceLinkLinks {
if (!($GHRepoName -Match "^[^\s\/]+/[^\s\/]+$")) {
Write-Host "GHRepoName should be in the format <org>/<repo>"
$global:LASTEXITCODE = 1
return
}
if (!($GHCommit -Match "^[0-9a-fA-F]{40}$")) {
Write-Host "GHCommit should be a 40 chars hexadecimal string"
$global:LASTEXITCODE = 1
return
}
$RepoTreeURL = -Join("https://api.github.com/repos/", $GHRepoName, "/git/trees/", $GHCommit, "?recursive=1")
$CodeExtensions = @(".cs", ".vb", ".fs", ".fsi", ".fsx", ".fsscript")
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL"
$global:LASTEXITCODE = 1
return
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
$Jobs = @()
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
Measure-Command { ValidateSourceLinkLinks }
+60 -48
View File
@@ -18,6 +18,7 @@ Param(
[switch] $sign,
[switch] $pack,
[switch] $publish,
[switch] $clean,
[switch][Alias('bl')]$binaryLog,
[switch] $ci,
[switch] $prepareMachine,
@@ -25,49 +26,55 @@ Param(
[Parameter(ValueFromRemainingArguments=$true)][String[]]$properties
)
. $PSScriptRoot\tools.ps1
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
# Unset 'Platform' environment variable to avoid unwanted collision in InstallDotNetCore.targets file
# some computer has this env var defined (e.g. Some HP)
if($env:Platform) {
$env:Platform=""
}
function Print-Usage() {
Write-Host "Common settings:"
Write-Host " -configuration <value> Build configuration: 'Debug' or 'Release' (short: -c)"
Write-Host " -platform <value> Platform configuration: 'x86', 'x64' or any valid Platform value to pass to msbuild"
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit"
Write-Host ""
Write-Host "Actions:"
Write-Host " -restore Restore dependencies (short: -r)"
Write-Host " -build Build solution (short: -b)"
Write-Host " -rebuild Rebuild solution"
Write-Host " -deploy Deploy built VSIXes"
Write-Host " -deployDeps Deploy dependencies (e.g. VSIXes for integration tests)"
Write-Host " -test Run all unit tests in the solution (short: -t)"
Write-Host " -integrationTest Run all integration tests in the solution"
Write-Host " -performanceTest Run all performance tests in the solution"
Write-Host " -pack Package build outputs into NuGet packages and Willow components"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host ""
Write-Host "Advanced settings:"
Write-Host " -projects <value> Semi-colon delimited list of sln/proj's to build. Globbing is supported (*.sln)"
Write-Host " -ci Set when running on CI server"
Write-Host " -prepareMachine Prepare machine for CI run, clean up processes after build"
Write-Host " -warnAsError <value> Sets warnaserror msbuild parameter ('true' or 'false')"
Write-Host " -msbuildEngine <value> Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -co for configuration, -t for test, etc.)."
}
. $PSScriptRoot\tools.ps1
function InitializeCustomToolset {
if (-not $restore) {
return
}
$script = Join-Path $EngRoot "restore-toolset.ps1"
$script = Join-Path $EngRoot 'restore-toolset.ps1'
if (Test-Path $script) {
. $script
@@ -78,13 +85,17 @@ function Build {
$toolsetBuildProj = InitializeToolset
InitializeCustomToolset
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { "" }
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'Build.binlog') } else { '' }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { '' }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')
$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
@@ -109,7 +120,15 @@ function Build {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($clean) {
if (Test-Path $ArtifactsDir) {
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host 'Artifacts directory deleted.'
}
exit 0
}
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
@@ -119,14 +138,7 @@ try {
$nodeReuse = $false
}
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
$configureToolsetScript = Join-Path $EngRoot "configure-toolset.ps1"
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
}
if (($restore) -and ($null -eq $env:DisableNativeToolsetInstalls)) {
if ($restore) {
InitializeNativeTools
}
@@ -134,7 +146,7 @@ try {
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTaskError -Message $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+12 -12
View File
@@ -26,6 +26,7 @@ usage()
echo " --pack Package build outputs into NuGet packages and Willow components"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo ""
echo "Advanced settings:"
@@ -62,6 +63,7 @@ publish=false
sign=false
public=false
ci=false
clean=false
warn_as_error=true
node_reuse=true
@@ -82,6 +84,9 @@ while [[ $# > 0 ]]; do
usage
exit 0
;;
-clean)
clean=true
;;
-configuration|-c)
configuration=$2
shift
@@ -196,20 +201,15 @@ function Build {
ExitWithExitCode 0
}
# Import custom tools configuration, if present in the repo.
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
exit 0
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
if [[ "$restore" == true && -z ${DisableNativeToolsetInstalls:-} ]]; then
if [[ "$restore" == true ]]; then
InitializeNativeTools
fi
@@ -1,41 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR arm)
## Specify the toolchain
set(TOOLCHAIN "arm-linux-androideabi")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
add_compile_options(-mfloat-abi=soft)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/)
include_directories(SYSTEM ${CROSS_NDK_TOOLCHAIN}/include/c++/4.9.x/arm-linux-androideabi/)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
@@ -1,42 +0,0 @@
set(CROSS_NDK_TOOLCHAIN $ENV{ROOTFS_DIR}/../)
set(CROSS_ROOTFS ${CROSS_NDK_TOOLCHAIN}/sysroot)
set(CLR_CMAKE_PLATFORM_ANDROID "Android")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
## Specify the toolchain
set(TOOLCHAIN "aarch64-linux-android")
set(CMAKE_PREFIX_PATH ${CROSS_NDK_TOOLCHAIN})
set(TOOLCHAIN_PREFIX ${TOOLCHAIN}-)
find_program(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}clang++)
find_program(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}clang)
find_program(CMAKE_AR ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_LD ${TOOLCHAIN_PREFIX}ar)
find_program(CMAKE_OBJCOPY ${TOOLCHAIN_PREFIX}objcopy)
find_program(CMAKE_OBJDUMP ${TOOLCHAIN_PREFIX}objdump)
add_compile_options(--sysroot=${CROSS_ROOTFS})
add_compile_options(-fPIE)
## Needed for Android or bionic specific conditionals
add_compile_options(-D__ANDROID__)
add_compile_options(-D__BIONIC__)
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -B ${CROSS_ROOTFS}/usr/lib/gcc/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -L${CROSS_ROOTFS}/lib/${TOOLCHAIN}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} --sysroot=${CROSS_ROOTFS}")
set(CROSS_LINK_FLAGS "${CROSS_LINK_FLAGS} -fPIE -pie")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${CROSS_LINK_FLAGS}" CACHE STRING "" FORCE)
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+3 -3
View File
@@ -157,15 +157,15 @@ fetch_tizen_pkgs()
Inform "Initialize arm base"
fetch_tizen_pkgs_init standard base
Inform "fetch common packages"
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel
fetch_tizen_pkgs armv7l gcc glibc glibc-devel libicu libicu-devel libatomic
fetch_tizen_pkgs noarch linux-glibc-devel
Inform "fetch coreclr packages"
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel tizen-release lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
fetch_tizen_pkgs armv7l lldb lldb-devel libgcc libstdc++ libstdc++-devel libunwind libunwind-devel lttng-ust-devel lttng-ust userspace-rcu-devel userspace-rcu
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l libcom_err libcom_err-devel zlib zlib-devel libopenssl libopenssl-devel krb5 krb5-devel libcurl libcurl-devel
Inform "Initialize standard unified"
fetch_tizen_pkgs_init standard unified
Inform "fetch corefx packages"
fetch_tizen_pkgs armv7l gssdp gssdp-devel
fetch_tizen_pkgs armv7l gssdp gssdp-devel tizen-release
+45 -51
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
set -e
__NDK_Version=r14
__NDK_Version=r21
usage()
{
@@ -16,11 +16,11 @@ usage()
echo.
echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"
echo "you can set the NDK_DIR environment variable to have this script use that installation of the NDK."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.21-arm64. This file is to replace '/etc/os-release', which is not available for Android."
echo "By default, this script will generate a file, android_platform, in the root of the ROOTFS_DIR directory that contains the RID for the supported and tested Android build: android.28-arm64. This file is to replace '/etc/os-release', which is not available for Android."
exit 1
}
__ApiLevel=21 # The minimum platform for arm64 is API level 21
__ApiLevel=28 # The minimum platform for arm64 is API level 21 but the minimum version that support glob(3) is 28. See $ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/glob.h
__BuildArch=arm64
__AndroidArch=aarch64
__AndroidToolchain=aarch64-linux-android
@@ -53,13 +53,20 @@ for i in "$@"
done
# Obtain the location of the bash script to figure out where the root of the repo is.
__CrossDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__ScriptBaseDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
__Android_Cross_Dir="$__CrossDir/android-rootfs"
__NDK_Dir="$__Android_Cross_Dir/android-ndk-$__NDK_Version"
__libunwind_Dir="$__Android_Cross_Dir/libunwind"
__lldb_Dir="$__Android_Cross_Dir/lldb"
__ToolchainDir="$__Android_Cross_Dir/toolchain/$__BuildArch"
__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"
if [[ ! -f "$__CrossDir" ]]; then
mkdir -p "$__CrossDir"
fi
# Resolve absolute path to avoid `../` in build logs
__CrossDir="$( cd "$__CrossDir" && pwd )"
__NDK_Dir="$__CrossDir/android-ndk-$__NDK_Version"
__lldb_Dir="$__CrossDir/lldb"
__ToolchainDir="$__CrossDir/android-ndk-$__NDK_Version"
if [[ -n "$TOOLCHAIN_DIR" ]]; then
__ToolchainDir=$TOOLCHAIN_DIR
@@ -78,60 +85,47 @@ echo "Target Toolchain location: $__ToolchainDir"
if [ ! -d $__NDK_Dir ]; then
echo Downloading the NDK into $__NDK_Dir
mkdir -p $__NDK_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__Android_Cross_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/android-ndk-$__NDK_Version-linux-x86_64.zip -O $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip
unzip -q $__CrossDir/android-ndk-$__NDK_Version-linux-x86_64.zip -d $__CrossDir
fi
if [ ! -d $__lldb_Dir ]; then
mkdir -p $__lldb_Dir
echo Downloading LLDB into $__lldb_Dir
wget -nv -nc --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__Android_Cross_Dir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
wget -q --progress=bar:force:noscroll --show-progress https://dl.google.com/android/repository/lldb-2.3.3614996-linux-x86_64.zip -O $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip
unzip -q $__CrossDir/lldb-2.3.3614996-linux-x86_64.zip -d $__lldb_Dir
fi
# Create the RootFS for both arm64 as well as aarch
rm -rf $__Android_Cross_Dir/toolchain
echo "Download dependencies..."
__TmpDir=$__CrossDir/tmp/$__BuildArch/
mkdir -p "$__TmpDir"
echo Generating the $__BuildArch toolchain
$__NDK_Dir/build/tools/make_standalone_toolchain.py --arch $__BuildArch --api $__ApiLevel --install-dir $__ToolchainDir
# combined dependencies for coreclr, installer and libraries
__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"
# Install the required packages into the toolchain
# TODO: Add logic to get latest pkg version instead of specific version number
rm -rf $__Android_Cross_Dir/deb/
rm -rf $__Android_Cross_Dir/tmp
for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\
grep -A15 "Package: \(${__AndroidPackages// /\\|}\)" | grep -v "static\|tool" | grep Filename); do
mkdir -p $__Android_Cross_Dir/deb/
mkdir -p $__Android_Cross_Dir/tmp/$arch/
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libicu-dev_60.2_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb
if [[ "$path" != "Filename:" ]]; then
echo "Working on: $path"
wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"
fi
done
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob-dev_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-glob_0.4_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support-dev_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libandroid-support_22_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma-dev_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/liblzma_5.2.3_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind-dev_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb
wget -nv -nc http://termux.net/dists/stable/main/binary-$__AndroidArch/libunwind_1.2.20170304_$__AndroidArch.deb -O $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb
echo Unpacking Termux packages
dpkg -x $__Android_Cross_Dir/deb/libicu_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libicu-dev_60.2_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob-dev_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-glob_0.4_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support-dev_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libandroid-support_22_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma-dev_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/liblzma_5.2.3_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind-dev_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
dpkg -x $__Android_Cross_Dir/deb/libunwind_1.2.20170304_$__AndroidArch.deb $__Android_Cross_Dir/tmp/$__AndroidArch/
cp -R $__Android_Cross_Dir/tmp/$__AndroidArch/data/data/com.termux/files/usr/* $__ToolchainDir/sysroot/usr/
cp -R "$__TmpDir/data/data/com.termux/files/usr/"* "$__ToolchainDir/sysroot/usr/"
# Generate platform file for build.sh script to assign to __DistroRid
echo "Generating platform file..."
echo "RID=android.${__ApiLevel}-${__BuildArch}" > $__ToolchainDir/sysroot/android_platform
echo "RID=android.21-arm64" > $__ToolchainDir/sysroot/android_platform
echo Now run:
echo CONFIG_DIR=\`realpath cross/android/$__BuildArch\` ROOTFS_DIR=\`realpath $__ToolchainDir/sysroot\` ./build.sh cross $__BuildArch skipgenerateversion skipnuget cmakeargs -DENABLE_LLDBPLUGIN=0
echo "Now to build coreclr, libraries and installers; run:"
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory coreclr
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory libraries
echo ROOTFS_DIR=\$\(realpath $__ToolchainDir/sysroot\) ./build.sh --cross --arch $__BuildArch \
--subsetCategory installer
+17 -5
View File
@@ -25,8 +25,9 @@ __UbuntuPackages="build-essential"
__AlpinePackages="alpine-base"
__AlpinePackages+=" build-base"
__AlpinePackages+=" linux-headers"
__AlpinePackages+=" lldb-dev"
__AlpinePackages+=" llvm-dev"
__AlpinePackagesEdgeTesting=" lldb-dev"
__AlpinePackagesEdgeMain=" llvm9-libs"
__AlpinePackagesEdgeMain+=" python3"
# symlinks fixer
__UbuntuPackages+=" symlinks"
@@ -181,7 +182,7 @@ if [ -z "$__RootfsDir" ] && [ ! -z "$ROOTFS_DIR" ]; then
fi
if [ -z "$__RootfsDir" ]; then
__RootfsDir="$__CrossDir/rootfs/$__BuildArch"
__RootfsDir="$__CrossDir/../../../.tools/rootfs/$__BuildArch"
fi
if [ -d "$__RootfsDir" ]; then
@@ -193,18 +194,29 @@ fi
if [[ "$__LinuxCodeName" == "alpine" ]]; then
__ApkToolsVersion=2.9.1
__AlpineVersion=3.7
__AlpineVersion=3.9
__ApkToolsDir=$(mktemp -d)
wget https://github.com/alpinelinux/apk-tools/releases/download/v$__ApkToolsVersion/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -P $__ApkToolsDir
tar -xf $__ApkToolsDir/apk-tools-$__ApkToolsVersion-x86_64-linux.tar.gz -C $__ApkToolsDir
mkdir -p $__RootfsDir/usr/bin
cp -v /usr/bin/qemu-$__QEMUArch-static $__RootfsDir/usr/bin
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/main \
-X http://dl-cdn.alpinelinux.org/alpine/v$__AlpineVersion/community \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackages
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/main \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeMain
$__ApkToolsDir/apk-tools-$__ApkToolsVersion/apk \
-X http://dl-cdn.alpinelinux.org/alpine/edge/testing \
-U --allow-untrusted --root $__RootfsDir --arch $__AlpineArch --initdb \
add $__AlpinePackagesEdgeTesting
rm -r $__ApkToolsDir
elif [[ -n $__LinuxCodeName ]]; then
qemu-debootstrap --arch $__UbuntuArch $__LinuxCodeName $__RootfsDir $__UbuntuRepo
+46 -35
View File
@@ -31,6 +31,10 @@ else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64 and x86 are supported!")
endif()
if(DEFINED ENV{TOOLCHAIN})
set(TOOLCHAIN $ENV{TOOLCHAIN})
endif()
# Specify include paths
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN)
@@ -39,50 +43,47 @@ if(TARGET_ARCH_NAME STREQUAL "armel")
endif()
endif()
# add_compile_param - adds only new options without duplicates.
# arg0 - list with result options, arg1 - list with new options.
# arg2 - optional argument, quick summary string for optional using CACHE FORCE mode.
macro(add_compile_param)
if(NOT ${ARGC} MATCHES "^(2|3)$")
message(FATAL_ERROR "Wrong using add_compile_param! Two or three parameters must be given! See add_compile_param description.")
endif()
foreach(OPTION ${ARGV1})
if(NOT ${ARGV0} MATCHES "${OPTION}($| )")
set(${ARGV0} "${${ARGV0}} ${OPTION}")
if(${ARGC} EQUAL "3") # CACHE FORCE mode
set(${ARGV0} "${${ARGV0}}" CACHE STRING "${ARGV2}" FORCE)
endif()
if("$ENV{__DistroRid}" MATCHES "android.*")
if(TARGET_ARCH_NAME STREQUAL "arm")
set(ANDROID_ABI armeabi-v7a)
elseif(TARGET_ARCH_NAME STREQUAL "arm64")
set(ANDROID_ABI arm64-v8a)
endif()
endforeach()
endmacro()
# extract platform number required by the NDK's toolchain
string(REGEX REPLACE ".*\\.([0-9]+)-.*" "\\1" ANDROID_PLATFORM "$ENV{__DistroRid}")
set(ANDROID_TOOLCHAIN clang)
set(FEATURE_EVENT_TRACE 0) # disable event trace as there is no lttng-ust package in termux repository
set(CMAKE_SYSTEM_LIBRARY_PATH "${CROSS_ROOTFS}/usr/lib")
set(CMAKE_SYSTEM_INCLUDE_PATH "${CROSS_ROOTFS}/usr/include")
# include official NDK toolchain script
include(${CROSS_ROOTFS}/../build/cmake/android.toolchain.cmake)
else()
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
set(CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
set(CMAKE_ASM_COMPILER_EXTERNAL_TOOLCHAIN "${CROSS_ROOTFS}/usr")
endif()
# Specify link flags
add_compile_param(CROSS_LINK_FLAGS "--sysroot=${CROSS_ROOTFS}")
add_compile_param(CROSS_LINK_FLAGS "--gcc-toolchain=${CROSS_ROOTFS}/usr")
add_compile_param(CROSS_LINK_FLAGS "--target=${TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-fuse-ld=gold")
if(TARGET_ARCH_NAME STREQUAL "armel")
if(DEFINED TIZEN_TOOLCHAIN) # For Tizen only
add_compile_param(CROSS_LINK_FLAGS "-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib")
add_compile_param(CROSS_LINK_FLAGS "-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-B${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
add_link_options("-L${CROSS_ROOTFS}/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib")
add_link_options("-L${CROSS_ROOTFS}/usr/lib/gcc/${TIZEN_TOOLCHAIN}")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_param(CROSS_LINK_FLAGS "-m32")
add_link_options(-m32)
endif()
add_compile_param(CMAKE_EXE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_SHARED_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
add_compile_param(CMAKE_MODULE_LINKER_FLAGS "${CROSS_LINK_FLAGS}" "TOOLCHAIN_EXE_LINKER_FLAGS")
# Specify compile options
add_compile_options("--sysroot=${CROSS_ROOTFS}")
add_compile_options("--target=${TOOLCHAIN}")
add_compile_options("--gcc-toolchain=${CROSS_ROOTFS}/usr")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$")
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64)$" AND NOT "$ENV{__DistroRid}" MATCHES "android.*")
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
@@ -90,7 +91,17 @@ endif()
if(TARGET_ARCH_NAME MATCHES "^(arm|armel)$")
add_compile_options(-mthumb)
add_compile_options(-mfpu=vfpv3)
if (NOT DEFINED CLR_ARM_FPU_TYPE)
set (CLR_ARM_FPU_TYPE vfpv3)
endif (NOT DEFINED CLR_ARM_FPU_TYPE)
add_compile_options (-mfpu=${CLR_ARM_FPU_TYPE})
if (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
set (CLR_ARM_FPU_CAPABILITY 0x7)
endif (NOT DEFINED CLR_ARM_FPU_CAPABILITY)
add_definitions (-DCLR_ARM_FPU_CAPABILITY=${CLR_ARM_FPU_CAPABILITY})
if(TARGET_ARCH_NAME STREQUAL "armel")
add_compile_options(-mfloat-abi=softfp)
if(DEFINED TIZEN_TOOLCHAIN)
@@ -103,7 +114,7 @@ elseif(TARGET_ARCH_NAME STREQUAL "x86")
add_compile_options(-Wno-error=unused-command-line-argument)
endif()
# Set LLDB include and library paths
# Set LLDB include and library paths for builds that need lldb.
if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
if(TARGET_ARCH_NAME STREQUAL "x86")
set(LLVM_CROSS_DIR "$ENV{LLVM_CROSS_HOME}")
@@ -131,7 +142,7 @@ if(TARGET_ARCH_NAME MATCHES "^(arm|armel|x86)$")
endif()
endif()
set(CMAKE_FIND_ROOT_PATH "${CROSS_ROOTFS}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+25 -11
View File
@@ -1,20 +1,21 @@
param (
$darcVersion = $null,
$versionEndpoint = "https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
$versionEndpoint = 'https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16',
$verbosity = 'minimal',
$toolpath = $null
)
$verbosity = "m"
. $PSScriptRoot\tools.ps1
function InstallDarcCli ($darcVersion) {
$darcCliPackageName = "microsoft.dotnet.darc"
function InstallDarcCli ($darcVersion, $toolpath) {
$darcCliPackageName = 'microsoft.dotnet.darc'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = Invoke-Expression "& `"$dotnet`" tool list -g"
$toolList = & "$dotnet" tool list -g
if ($toolList -like "*$darcCliPackageName*") {
Invoke-Expression "& `"$dotnet`" tool uninstall $darcCliPackageName -g"
& "$dotnet" tool uninstall $darcCliPackageName -g
}
# If the user didn't explicitly specify the darc version,
@@ -22,12 +23,25 @@ function InstallDarcCli ($darcVersion) {
if (-not $darcVersion) {
$darcVersion = $(Invoke-WebRequest -Uri $versionEndpoint -UseBasicParsing).Content
}
$arcadeServicesSource = 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'
$arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
Write-Host "Installing Darc CLI version $darcVersion..."
Write-Host "You may need to restart your command window if this is the first dotnet tool you have installed."
Invoke-Expression "& `"$dotnet`" tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
if (-not $toolpath) {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity -g"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g
}else {
Write-Host "'$dotnet' tool install $darcCliPackageName --version $darcVersion --add-source '$arcadeServicesSource' -v $verbosity --tool-path '$toolpath'"
& "$dotnet" tool install $darcCliPackageName --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath"
}
}
InstallDarcCli $darcVersion
try {
InstallDarcCli $darcVersion $toolpath
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Darc' -Message $_
ExitWithExitCode 1
}
+26 -8
View File
@@ -2,7 +2,8 @@
source="${BASH_SOURCE[0]}"
darcVersion=''
versionEndpoint="https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16"
versionEndpoint='https://maestro-prod.westus2.cloudapp.azure.com/api/assets/darc-version?api-version=2019-01-16'
verbosity='minimal'
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
@@ -15,6 +16,14 @@ while [[ $# > 0 ]]; do
versionEndpoint=$2
shift
;;
--verbosity)
verbosity=$2
shift
;;
--toolpath)
toolpath=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
@@ -34,7 +43,6 @@ while [[ -h "$source" ]]; do
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
verbosity=m
. "$scriptroot/tools.sh"
@@ -48,17 +56,27 @@ function InstallDarcCli {
InitializeDotNetCli
local dotnet_root=$_InitializeDotNetCli
local uninstall_command=`$dotnet_root/dotnet tool uninstall $darc_cli_package_name -g`
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
if [ -z "$toolpath" ]; then
local tool_list=$($dotnet_root/dotnet tool list -g)
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name -g)
fi
else
local tool_list=$($dotnet_root/dotnet tool list --tool-path "$toolpath")
if [[ $tool_list = *$darc_cli_package_name* ]]; then
echo $($dotnet_root/dotnet tool uninstall $darc_cli_package_name --tool-path "$toolpath")
fi
fi
local arcadeServicesSource="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json"
local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
if [ -z "$toolpath" ]; then
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity -g)
else
echo $($dotnet_root/dotnet tool install $darc_cli_package_name --version $darcVersion --add-source "$arcadeServicesSource" -v $verbosity --tool-path "$toolpath")
fi
}
InstallDarcCli
+12 -11
View File
@@ -1,26 +1,27 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $architecture = "",
[string] $version = "Latest",
[string] $runtime = "dotnet"
[string] $verbosity = 'minimal',
[string] $architecture = '',
[string] $version = 'Latest',
[string] $runtime = 'dotnet',
[string] $RuntimeSourceFeed = '',
[string] $RuntimeSourceFeedKey = ''
)
. $PSScriptRoot\tools.ps1
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
$installdir = $dotnetRoot
try {
if ($architecture -and $architecture.Trim() -eq "x86") {
$installdir = Join-Path $installdir "x86"
if ($architecture -and $architecture.Trim() -eq 'x86') {
$installdir = Join-Path $installdir 'x86'
}
InstallDotNet $installdir $version $architecture $runtime $true
}
InstallDotNet $installdir $version $architecture $runtime $true -RuntimeSourceFeed $RuntimeSourceFeed -RuntimeSourceFeedKey $RuntimeSourceFeedKey
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
+45 -5
View File
@@ -11,9 +11,13 @@ while [[ -h "$source" ]]; do
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
version='Latest'
architecture=''
runtime='dotnet'
runtimeSourceFeed=''
runtimeSourceFeedKey=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
@@ -29,20 +33,56 @@ while [[ $# > 0 ]]; do
shift
runtime="$1"
;;
-runtimesourcefeed)
shift
runtimeSourceFeed="$1"
;;
-runtimesourcefeedkey)
shift
runtimeSourceFeedKey="$1"
;;
*)
echo "Invalid argument: $1"
usage
Write-PipelineTelemetryError -Category 'Build' -Message "Invalid argument: $1"
exit 1
;;
esac
shift
done
. "$scriptroot/tools.sh"
# Use uname to determine what the CPU is.
cpuname=$(uname -p)
# Some Linux platforms report unknown for platform, but the arch for machine.
if [[ "$cpuname" == "unknown" ]]; then
cpuname=$(uname -m)
fi
case $cpuname in
aarch64)
buildarch=arm64
;;
amd64|x86_64)
buildarch=x64
;;
armv*l)
buildarch=arm
;;
i686)
buildarch=x86
;;
*)
echo "Unknown CPU $cpuname detected, treating it as x64"
buildarch=x64
;;
esac
dotnetRoot="$repo_root/.dotnet"
InstallDotNet $dotnetRoot $version "$architecture" $runtime true || {
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
InstallDotNet $dotnetRoot $version "$architecture" $runtime true $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
echo "dotnet-install.sh failed (exit code '$exit_code')." >&2
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "dotnet-install.sh failed (exit code '$exit_code')." >&2
ExitWithExitCode $exit_code
}
@@ -0,0 +1,13 @@
param(
[string] $token
)
. $PSScriptRoot\pipeline-logging-functions.ps1
# Write-PipelineSetVariable will no-op if a variable named $ci is not defined
# Since this script is only ever called in AzDO builds, just universally set it
$ci = $true
Write-PipelineSetVariable -Name 'VSS_NUGET_ACCESSTOKEN' -Value $token -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'VSS_NUGET_URI_PREFIXES' -Value 'https://dnceng.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/dnceng/;https://devdiv.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/devdiv/' -IsMultiJobVariable $false
+33 -34
View File
@@ -3,46 +3,46 @@ Param(
[Parameter(Mandatory=$true)][string] $gitHubPat, # GitHub personal access token from https://github.com/settings/tokens (no auth scopes needed)
[Parameter(Mandatory=$true)][string] $azdoPat, # Azure Dev Ops tokens from https://dev.azure.com/dnceng/_details/security/tokens (code read scope needed)
[Parameter(Mandatory=$true)][string] $outputFolder, # Where the graphviz.txt file will be created
[string] $darcVersion = '1.1.0-beta.19175.6', # darc's version
[string] $darcVersion, # darc's version
[string] $graphvizVersion = '2.38', # GraphViz version
[switch] $includeToolset # Whether the graph should include toolset dependencies or not. i.e. arcade, optimization. For more about
# toolset dependencies see https://github.com/dotnet/arcade/blob/master/Documentation/Darc.md#toolset-vs-product-dependencies
)
$ErrorActionPreference = "Stop"
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
function CheckExitCode ([string]$stage)
{
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
Write-Host "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
Write-PipelineTelemetryError -Category 'Arcade' -Message "Something failed in stage: '$stage'. Check for errors above. Exiting now..."
ExitWithExitCode $exitCode
}
}
try {
$ErrorActionPreference = 'Stop'
. $PSScriptRoot\tools.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
Push-Location $PSScriptRoot
Write-Host "Installing darc..."
Write-Host 'Installing darc...'
. .\darc-init.ps1 -darcVersion $darcVersion
CheckExitCode "Running darc-init"
CheckExitCode 'Running darc-init'
$engCommonBaseDir = Join-Path $PSScriptRoot "native\"
$engCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$graphvizInstallDir = CommonLibrary\Get-NativeInstallDirectory
$nativeToolBaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external"
$installBin = Join-Path $graphvizInstallDir "bin"
$nativeToolBaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external'
$installBin = Join-Path $graphvizInstallDir 'bin'
Write-Host "Installing dot..."
Write-Host 'Installing dot...'
.\native\install-tool.ps1 -ToolName graphviz -InstallPath $installBin -BaseUri $nativeToolBaseUri -CommonLibraryDirectory $engCommonBaseDir -Version $graphvizVersion -Verbose
$darcExe = "$env:USERPROFILE\.dotnet\tools"
$darcExe = Resolve-Path "$darcExe\darc.exe"
Create-Directory $outputFolder
# Generate 3 graph descriptions:
# 1. Flat with coherency information
# 2. Graphviz (dot) file
@@ -51,37 +51,36 @@ try {
$graphVizImageFilePath = "$outputFolder\graph.png"
$normalGraphFilePath = "$outputFolder\graph-full.txt"
$flatGraphFilePath = "$outputFolder\graph-flat.txt"
$baseOptions = "get-dependency-graph --github-pat $gitHubPat --azdev-pat $azdoPat --password $barToken"
$baseOptions = @( '--github-pat', "$gitHubPat", '--azdev-pat', "$azdoPat", '--password', "$barToken" )
if ($includeToolset) {
Write-Host "Toolsets will be included in the graph..."
$baseOptions += " --include-toolset"
Write-Host 'Toolsets will be included in the graph...'
$baseOptions += @( '--include-toolset' )
}
Write-Host "Generating standard dependency graph..."
Invoke-Expression "& `"$darcExe`" $baseOptions --output-file $normalGraphFilePath"
CheckExitCode "Generating normal dependency graph"
Write-Host 'Generating standard dependency graph...'
& "$darcExe" get-dependency-graph @baseOptions --output-file $normalGraphFilePath
CheckExitCode 'Generating normal dependency graph'
Write-Host "Generating flat dependency graph and graphviz file..."
Invoke-Expression "& `"$darcExe`" $baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath"
CheckExitCode "Generating flat and graphviz dependency graph"
Write-Host 'Generating flat dependency graph and graphviz file...'
& "$darcExe" get-dependency-graph @baseOptions --flat --coherency --graphviz $graphVizFilePath --output-file $flatGraphFilePath
CheckExitCode 'Generating flat and graphviz dependency graph'
Write-Host "Generating graph image $graphVizFilePath"
$dotFilePath = Join-Path $installBin "graphviz\$graphvizVersion\release\bin\dot.exe"
Invoke-Expression "& `"$dotFilePath`" -Tpng -o'$graphVizImageFilePath' `"$graphVizFilePath`""
CheckExitCode "Generating graphviz image"
& "$dotFilePath" -Tpng -o"$graphVizImageFilePath" "$graphVizFilePath"
CheckExitCode 'Generating graphviz image'
Write-Host "'$graphVizFilePath', '$flatGraphFilePath', '$normalGraphFilePath' and '$graphVizImageFilePath' created!"
}
catch {
if (!$includeToolset) {
Write-Host "This might be a toolset repo which includes only toolset dependencies. " -NoNewline -ForegroundColor Yellow
Write-Host "Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again..." -ForegroundColor Yellow
Write-Host 'This might be a toolset repo which includes only toolset dependencies. ' -NoNewline -ForegroundColor Yellow
Write-Host 'Since -includeToolset is not set there is no graph to create. Include -includeToolset and try again...' -ForegroundColor Yellow
}
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
} finally {
Pop-Location
Pop-Location
}
+38 -28
View File
@@ -35,7 +35,7 @@ File path to global.json file
#>
[CmdletBinding(PositionalBinding=$false)]
Param (
[string] $BaseUri = "https://netcorenativeassets.blob.core.windows.net/resource-packages/external",
[string] $BaseUri = 'https://netcorenativeassets.blob.core.windows.net/resource-packages/external',
[string] $InstallDirectory,
[switch] $Clean = $False,
[switch] $Force = $False,
@@ -45,26 +45,27 @@ Param (
)
if (!$GlobalJsonFile) {
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName "global.json"
$GlobalJsonFile = Join-Path (Get-Item $PSScriptRoot).Parent.Parent.FullName 'global.json'
}
Set-StrictMode -version 2.0
$ErrorActionPreference="Stop"
$ErrorActionPreference='Stop'
Import-Module -Name (Join-Path $PSScriptRoot "native\CommonLibrary.psm1")
. $PSScriptRoot\pipeline-logging-functions.ps1
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1')
try {
# Define verbose switch if undefined
$Verbose = $VerbosePreference -Eq "Continue"
$Verbose = $VerbosePreference -Eq 'Continue'
$EngCommonBaseDir = Join-Path $PSScriptRoot "native\"
$EngCommonBaseDir = Join-Path $PSScriptRoot 'native\'
$NativeBaseDir = $InstallDirectory
if (!$NativeBaseDir) {
$NativeBaseDir = CommonLibrary\Get-NativeInstallDirectory
}
$Env:CommonLibrary_NativeInstallDir = $NativeBaseDir
$InstallBin = Join-Path $NativeBaseDir "bin"
$InstallerPath = Join-Path $EngCommonBaseDir "install-tool.ps1"
$InstallBin = Join-Path $NativeBaseDir 'bin'
$InstallerPath = Join-Path $EngCommonBaseDir 'install-tool.ps1'
# Process tools list
Write-Host "Processing $GlobalJsonFile"
@@ -74,51 +75,59 @@ try {
}
$NativeTools = Get-Content($GlobalJsonFile) -Raw |
ConvertFrom-Json |
Select-Object -Expand "native-tools" -ErrorAction SilentlyContinue
Select-Object -Expand 'native-tools' -ErrorAction SilentlyContinue
if ($NativeTools) {
$NativeTools.PSObject.Properties | ForEach-Object {
$ToolName = $_.Name
$ToolVersion = $_.Value
$LocalInstallerCommand = $InstallerPath
$LocalInstallerCommand += " -ToolName $ToolName"
$LocalInstallerCommand += " -InstallPath $InstallBin"
$LocalInstallerCommand += " -BaseUri $BaseUri"
$LocalInstallerCommand += " -CommonLibraryDirectory $EngCommonBaseDir"
$LocalInstallerCommand += " -Version $ToolVersion"
$LocalInstallerArguments = @{ ToolName = "$ToolName" }
$LocalInstallerArguments += @{ InstallPath = "$InstallBin" }
$LocalInstallerArguments += @{ BaseUri = "$BaseUri" }
$LocalInstallerArguments += @{ CommonLibraryDirectory = "$EngCommonBaseDir" }
$LocalInstallerArguments += @{ Version = "$ToolVersion" }
if ($Verbose) {
$LocalInstallerCommand += " -Verbose"
$LocalInstallerArguments += @{ Verbose = $True }
}
if (Get-Variable 'Force' -ErrorAction 'SilentlyContinue') {
if($Force) {
$LocalInstallerCommand += " -Force"
$LocalInstallerArguments += @{ Force = $True }
}
}
if ($Clean) {
$LocalInstallerCommand += " -Clean"
$LocalInstallerArguments += @{ Clean = $True }
}
Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$LocalInstallerCommand'"
Invoke-Expression "$LocalInstallerCommand"
Write-Verbose "Executing '$InstallerPath $($LocalInstallerArguments.Keys.ForEach({"-$_ '$($LocalInstallerArguments.$_)'"}) -join ' ')'"
& $InstallerPath @LocalInstallerArguments
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
Write-Warning $errMsg
$showNativeToolsWarning = $true
if ((Get-Variable 'DoNotDisplayNativeToolsInstallationWarnings' -ErrorAction 'SilentlyContinue') -and $DoNotDisplayNativeToolsInstallationWarnings) {
$showNativeToolsWarning = $false
}
if ($showNativeToolsWarning) {
Write-Warning $errMsg
}
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host $errMsg
exit 1
}
}
}
if ((Get-Variable 'toolInstallationFailure' -ErrorAction 'SilentlyContinue') -and $toolInstallationFailure) {
# We cannot change this to Write-PipelineTelemetryError because of https://github.com/dotnet/arcade/issues/4482
Write-Host 'Native tools bootstrap failed'
exit 1
}
}
else {
Write-Host "No native tools defined in global.json"
Write-Host 'No native tools defined in global.json'
exit 0
}
@@ -126,17 +135,18 @@ try {
exit 0
}
if (Test-Path $InstallBin) {
Write-Host "Native tools are available from" (Convert-Path -Path $InstallBin)
Write-Host 'Native tools are available from ' (Convert-Path -Path $InstallBin)
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
Write-Error "Native tools install directory does not exist, installation failed"
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message 'Native tools install directory does not exist, installation failed'
exit 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
exit 1
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message $_
ExitWithExitCode 1
}
+43 -11
View File
@@ -12,6 +12,7 @@ retry_wait_time_seconds=30
global_json_file="$(dirname "$(dirname "${scriptroot}")")/global.json"
declare -A native_assets
. $scriptroot/pipeline-logging-functions.sh
. $scriptroot/native/common-library.sh
while (($# > 0)); do
@@ -33,6 +34,14 @@ while (($# > 0)); do
force=true
shift 1
;;
--donotabortonfailure)
donotabortonfailure=true
shift 1
;;
--donotdisplaywarnings)
donotdisplaywarnings=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
@@ -51,6 +60,8 @@ while (($# > 0)); do
echo " - (default) %USERPROFILE%/.netcoreeng/native"
echo ""
echo " --clean Switch specifying not to install anything, but cleanup native asset folders"
echo " --donotabortonfailure Switch specifiying whether to abort native tools installation on failure"
echo " --donotdisplaywarnings Switch specifiying whether to display warnings during native tools installation on failure"
echo " --force Clean and then install tools"
echo " --help Print help and exit"
echo ""
@@ -70,8 +81,7 @@ function ReadGlobalJsonNativeTools {
# Only extract the contents of the object.
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[:space:]')"
native_tools_list=$( echo "$native_tools_list" | sed 's/\s//g' | sed 's/,/\n/g' )
local old_IFS=$IFS
while read -r line; do
@@ -92,6 +102,7 @@ if [[ -z $install_directory ]]; then
fi
install_bin="${native_base_dir}/bin"
installed_any=false
ReadGlobalJsonNativeTools
@@ -103,11 +114,12 @@ else
for tool in "${!native_assets[@]}"
do
tool_version=${native_assets[$tool]}
installer_name="install-$tool.sh"
installer_command="$native_installer_dir/$installer_name"
installer_path="$native_installer_dir/install-$tool.sh"
installer_command="$installer_path"
installer_command+=" --baseuri $base_uri"
installer_command+=" --installpath $install_bin"
installer_command+=" --version $tool_version"
echo $installer_command
if [[ $force = true ]]; then
installer_command+=" --force"
@@ -117,11 +129,29 @@ else
installer_command+=" --clean"
fi
$installer_command
if [[ $? != 0 ]]; then
echo "Execution Failed" >&2
exit 1
if [[ -a $installer_path ]]; then
$installer_command
if [[ $? != 0 ]]; then
if [[ $donotabortonfailure = true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed"
exit 1
fi
else
$installed_any = true
fi
else
if [[ $donotabortonfailure == true ]]; then
if [[ $donotdisplaywarnings != true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
fi
else
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Execution Failed: no install script"
exit 1
fi
fi
done
fi
@@ -134,8 +164,10 @@ if [[ -d $install_bin ]]; then
echo "Native tools are available from $install_bin"
echo "##vso[task.prependpath]$install_bin"
else
echo "Native tools install directory does not exist, installation failed" >&2
exit 1
if [[ $installed_any = true ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Native tools install directory does not exist, installation failed"
exit 1
fi
fi
exit 0
+133
View File
@@ -0,0 +1,133 @@
param(
[Parameter(Mandatory=$true)][string] $Operation,
[string] $AuthToken,
[string] $CommitSha,
[string] $RepoName,
[switch] $IsFeedPrivate
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
. $PSScriptRoot\tools.ps1
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables. This should ONLY be called from identified
# internal builds
function SetupCredProvider {
param(
[string] $AuthToken
)
# Install the Cred Provider NuGet plugin
Write-Host 'Setting up Cred Provider NuGet plugin in the agent...'
Write-Host "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
$url = 'https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1'
Write-Host "Writing the contents of 'installcredprovider.ps1' locally..."
Invoke-WebRequest $url -OutFile installcredprovider.ps1
Write-Host 'Installing plugin...'
.\installcredprovider.ps1 -Force
Write-Host "Deleting local copy of 'installcredprovider.ps1'..."
Remove-Item .\installcredprovider.ps1
if (-Not("$env:USERPROFILE\.nuget\plugins\netcore")) {
Write-PipelineTelemetryError -Category 'Arcade' -Message 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
}
else {
Write-Host 'CredProvider plugin was installed correctly!'
}
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
$nugetConfigPath = "$RepoRoot\NuGet.config"
if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'NuGet.config file not found in repo root!'
ExitWithExitCode 1
}
$endpoints = New-Object System.Collections.ArrayList
$nugetConfigPackageSources = Select-Xml -Path $nugetConfigPath -XPath "//packageSources/add[contains(@key, 'darc-int-')]/@value" | foreach{$_.Node.Value}
if (($nugetConfigPackageSources | Measure-Object).Count -gt 0 ) {
foreach ($stableRestoreResource in $nugetConfigPackageSources) {
$trimmedResource = ([string]$stableRestoreResource).Trim()
[void]$endpoints.Add(@{endpoint="$trimmedResource"; password="$AuthToken"})
}
}
if (($endpoints | Measure-Object).Count -gt 0) {
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
$endpointCredentials = @{endpointCredentials=$endpoints} | ConvertTo-Json -Compress
# Create the environment variables the AzDo way
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $endpointCredentials -Properties @{
'variable' = 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS'
'issecret' = 'false'
}
# We don't want sessions cached since we will be updating the endpoints quite frequently
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data 'False' -Properties @{
'variable' = 'NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED'
'issecret' = 'false'
}
}
else
{
Write-Host 'No internal endpoints found in NuGet.config'
}
}
#Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
$dotnetTempDir = "$RepoRoot\dotnet"
$dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
$dotnet = "$dotnetTempDir\dotnet.exe"
$restoreProjPath = "$PSScriptRoot\restore.proj"
Write-Host "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Out-File "$restoreProjPath"
& $dotnet restore $restoreProjPath
Write-Host 'Arcade SDK restored!'
if (Test-Path -Path $restoreProjPath) {
Remove-Item $restoreProjPath
}
if (Test-Path -Path $dotnetTempDir) {
Remove-Item $dotnetTempDir -Recurse
}
}
try {
Push-Location $PSScriptRoot
if ($Operation -like 'setup') {
SetupCredProvider $AuthToken
}
elseif ($Operation -like 'install-restore') {
InstallDotNetSdkAndRestoreArcade
}
else {
Write-PipelineTelemetryError -Category 'Arcade' -Message "Unknown operation '$Operation'!"
ExitWithExitCode 1
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Arcade' -Message $_
ExitWithExitCode 1
}
finally {
Pop-Location
}
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env bash
set -e
# Sets VSS_NUGET_EXTERNAL_FEED_ENDPOINTS based on the "darc-int-*" feeds defined in NuGet.config. This is needed
# in build agents by CredProvider to authenticate the restore requests to internal feeds as specified in
# https://github.com/microsoft/artifacts-credprovider/blob/0f53327cd12fd893d8627d7b08a2171bf5852a41/README.md#environment-variables.
# This should ONLY be called from identified internal builds
function SetupCredProvider {
local authToken=$1
# Install the Cred Provider NuGet plugin
echo "Setting up Cred Provider NuGet plugin in the agent..."...
echo "Getting 'installcredprovider.ps1' from 'https://github.com/microsoft/artifacts-credprovider'..."
local url="https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh"
echo "Writing the contents of 'installcredprovider.ps1' locally..."
local installcredproviderPath="installcredprovider.sh"
if command -v curl > /dev/null; then
curl $url > "$installcredproviderPath"
else
wget -q -O "$installcredproviderPath" "$url"
fi
echo "Installing plugin..."
. "$installcredproviderPath"
echo "Deleting local copy of 'installcredprovider.sh'..."
rm installcredprovider.sh
if [ ! -d "$HOME/.nuget/plugins" ]; then
Write-PipelineTelemetryError -category 'Build' 'CredProvider plugin was not installed correctly!'
ExitWithExitCode 1
else
echo "CredProvider plugin was installed correctly!"
fi
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully
local nugetConfigPath="$repo_root/NuGet.config"
if [ ! "$nugetConfigPath" ]; then
Write-PipelineTelemetryError -category 'Build' "NuGet.config file not found in repo's root!"
ExitWithExitCode 1
fi
local endpoints='['
local nugetConfigPackageValues=`cat "$nugetConfigPath" | grep "key=\"darc-int-"`
local pattern="value=\"(.*)\""
for value in $nugetConfigPackageValues
do
if [[ $value =~ $pattern ]]; then
local endpoint="${BASH_REMATCH[1]}"
endpoints+="{\"endpoint\": \"$endpoint\", \"password\": \"$authToken\"},"
fi
done
endpoints=${endpoints%?}
endpoints+=']'
if [ ${#endpoints} -gt 2 ]; then
# Create the JSON object. It should look like '{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}'
local endpointCredentials="{\"endpointCredentials\": "$endpoints"}"
echo "##vso[task.setvariable variable=VSS_NUGET_EXTERNAL_FEED_ENDPOINTS]$endpointCredentials"
echo "##vso[task.setvariable variable=NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED]False"
else
echo "No internal endpoints found in NuGet.config"
fi
}
# Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
local dotnetTempDir="$repo_root/dotnet"
local dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
local restoreProjPath="$repo_root/eng/common/restore.proj"
echo "Installing dotnet SDK version $dotnetSdkVersion to restore Arcade SDK..."
echo "<Project Sdk=\"Microsoft.DotNet.Arcade.Sdk\"/>" > "$restoreProjPath"
InstallDotNetSdk "$dotnetTempDir" "$dotnetSdkVersion"
local res=`$dotnetTempDir/dotnet restore $restoreProjPath`
echo "Arcade SDK restored!"
# Cleanup
if [ "$restoreProjPath" ]; then
rm "$restoreProjPath"
fi
if [ "$dotnetTempDir" ]; then
rm -r $dotnetTempDir
fi
}
source="${BASH_SOURCE[0]}"
operation=''
authToken=''
repoName=''
while [[ $# > 0 ]]; do
opt="$(echo "$1" | awk '{print tolower($0)}')"
case "$opt" in
--operation)
operation=$2
shift
;;
--authtoken)
authToken=$2
shift
;;
*)
echo "Invalid argument: $1"
usage
exit 1
;;
esac
shift
done
while [[ -h "$source" ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
source="$(readlink "$source")"
# if $source was a relative symlink, we need to resolve it relative to the path where the
# symlink file was located
[[ $source != /* ]] && source="$scriptroot/$source"
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. "$scriptroot/tools.sh"
if [ "$operation" = "setup" ]; then
SetupCredProvider $authToken
elif [ "$operation" = "install-restore" ]; then
InstallDotNetSdkAndRestoreArcade
else
echo "Unknown operation '$operation'!"
fi
+2 -3
View File
@@ -1,6 +1,6 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
[switch] $ci,
@@ -18,9 +18,8 @@ try {
MSBuild @extraArgs
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+34 -3
View File
@@ -59,9 +59,38 @@ function DownloadAndExtract {
-Verbose:$Verbose
if ($UnzipStatus -Eq $False) {
Write-Error "Unzip failed"
return $False
# Retry Download one more time with Force=true
$DownloadRetryStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries 1 `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$True `
-Verbose:$Verbose
if ($DownloadRetryStatus -Eq $False) {
Write-Error "Last attempt of download failed as well"
return $False
}
# Retry unzip again one more time with Force=true
$UnzipRetryStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$True `
-Verbose:$Verbose
if ($UnzipRetryStatus -Eq $False)
{
Write-Error "Last attempt of unzip failed as well"
# Clean up partial zips and extracts
if (Test-Path $TempToolPath) {
Remove-Item $TempToolPath -Force
}
if (Test-Path $InstallDirectory) {
Remove-Item $InstallDirectory -Force -Recurse
}
return $False
}
}
return $True
}
@@ -123,6 +152,8 @@ function Get-File {
}
else {
Write-Verbose "Downloading $Uri"
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
while($Attempt -Lt $DownloadRetries)
{
try {
@@ -209,7 +240,7 @@ function New-ScriptShim {
Remove-Item (Join-Path $ShimDirectory "$ShimName.exe")
}
Invoke-Expression "$ShimDirectory\WinShimmer\winshimmer.exe $ShimName $ToolFilePath $ShimDirectory"
& "$ShimDirectory\WinShimmer\winshimmer.exe" $ShimName $ToolFilePath $ShimDirectory
return $True
}
catch {
+5 -5
View File
@@ -34,7 +34,7 @@ function ExpandZip {
echo "'Force flag enabled, but '$output_directory' exists. Removing directory"
rm -rf $output_directory
if [[ $? != 0 ]]; then
echo Unable to remove '$output_directory'>&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to remove '$output_directory'"
return 1
fi
fi
@@ -45,7 +45,7 @@ function ExpandZip {
echo "Extracting archive"
tar -xf $zip_path -C $output_directory
if [[ $? != 0 ]]; then
echo "Unable to extract '$zip_path'" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Unable to extract '$zip_path'"
return 1
fi
@@ -117,7 +117,7 @@ function DownloadAndExtract {
# Download file
GetFile "$uri" "$temp_tool_path" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to download '$uri' to '$temp_tool_path'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to download '$uri' to '$temp_tool_path'."
return 1
fi
@@ -125,7 +125,7 @@ function DownloadAndExtract {
echo "extracting from $temp_tool_path to $installDir"
ExpandZip "$temp_tool_path" "$installDir" $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Failed to extract '$temp_tool_path' to '$installDir'." >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Failed to extract '$temp_tool_path' to '$installDir'."
return 1
fi
@@ -148,7 +148,7 @@ function NewScriptShim {
fi
if [[ ! -f $tool_file_path ]]; then
echo "Specified tool file path:'$tool_file_path' does not exist" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' "Specified tool file path:'$tool_file_path' does not exist"
return 1
fi
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
source="${BASH_SOURCE[0]}"
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
. $scriptroot/common-library.sh
base_uri=
install_path=
version=
clean=false
force=false
download_retries=5
retry_wait_time_seconds=30
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--baseuri)
base_uri=$2
shift 2
;;
--installpath)
install_path=$2
shift 2
;;
--version)
version=$2
shift 2
;;
--clean)
clean=true
shift 1
;;
--force)
force=true
shift 1
;;
--downloadretries)
download_retries=$2
shift 2
;;
--retrywaittimeseconds)
retry_wait_time_seconds=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --baseuri <value> Base file directory or Url wrom which to acquire tool archives"
echo " --installpath <value> Base directory to install native tool to"
echo " --clean Don't install the tool, just clean up the current install of the tool"
echo " --force Force install of tools even if they previously exist"
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --downloadretries Total number of retry attempts"
echo " --retrywaittimeseconds Wait time between retry attempts in seconds"
echo ""
exit 0
;;
esac
done
tool_name="cmake-test"
tool_os=$(GetCurrentOS)
tool_folder=$(echo $tool_os | awk '{print tolower($0)}')
tool_arch="x86_64"
tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
echo "Cleaning $tool_install_directory"
if [[ -d $tool_install_directory ]]; then
rm -rf $tool_install_directory
fi
echo "Cleaning $shim_path"
if [[ -f $shim_path ]]; then
rm -rf $shim_path
fi
tool_temp_path=$(GetTempPathFileName $uri)
echo "Cleaning $tool_temp_path"
if [[ -f $tool_temp_path ]]; then
rm -rf $tool_temp_path
fi
exit 0
fi
# Install tool
if [[ -f $tool_file_path ]] && [[ $force = false ]]; then
echo "$tool_name ($version) already exists, skipping install"
exit 0
fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
# Generate Shim
# Always rewrite shims so that we are referencing the expected version
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
exit 0
+3 -3
View File
@@ -69,7 +69,7 @@ tool_name_moniker="$tool_name-$version-$tool_os-$tool_arch"
tool_install_directory="$install_path/$tool_name/$version"
tool_file_path="$tool_install_directory/$tool_name_moniker/bin/$tool_name"
shim_path="$install_path/$tool_name.sh"
uri="${base_uri}/$tool_folder/cmake/$tool_name_moniker.tar.gz"
uri="${base_uri}/$tool_folder/$tool_name/$tool_name_moniker.tar.gz"
# Clean up tool and installers
if [[ $clean = true ]]; then
@@ -101,7 +101,7 @@ fi
DownloadAndExtract $uri $tool_install_directory $force $download_retries $retry_wait_time_seconds
if [[ $? != 0 ]]; then
echo "Installation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Installation failed'
exit 1
fi
@@ -110,7 +110,7 @@ fi
NewScriptShim $shim_path $tool_file_path true
if [[ $? != 0 ]]; then
echo "Shim generation failed" >&2
Write-PipelineTelemetryError -category 'NativeToolsBootstrap' 'Shim generation failed'
exit 1
fi
+7 -5
View File
@@ -46,6 +46,8 @@ Param (
[int] $RetryWaitTimeInSeconds = 30
)
. $PSScriptRoot\..\pipeline-logging-functions.ps1
# Import common library modules
Import-Module -Name (Join-Path $CommonLibraryDirectory "CommonLibrary.psm1")
@@ -93,7 +95,7 @@ try {
-Verbose:$Verbose
if ($InstallStatus -Eq $False) {
Write-Error "Installation failed"
Write-PipelineTelemetryError "Installation failed" -Category "NativeToolsetBootstrapping"
exit 1
}
}
@@ -103,7 +105,7 @@ try {
Write-Error "There are multiple copies of $ToolName in $($ToolInstallDirectory): `n$(@($ToolFilePath | out-string))"
exit 1
} elseif (@($ToolFilePath).Length -Lt 1) {
Write-Error "$ToolName was not found in $ToolFilePath."
Write-Host "$ToolName was not found in $ToolFilePath."
exit 1
}
@@ -117,14 +119,14 @@ try {
-Verbose:$Verbose
if ($GenerateShimStatus -Eq $False) {
Write-Error "Generate shim failed"
Write-PipelineTelemetryError "Generate shim failed" -Category "NativeToolsetBootstrapping"
return 1
}
exit 0
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category "NativeToolsetBootstrapping" -Message $_
exit 1
}
@@ -0,0 +1,121 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
<PropertyGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<WorkItemCommand>%HELIX_CORRELATION_PAYLOAD%\performance\scripts\benchmarks_ci.py --csproj %HELIX_CORRELATION_PAYLOAD%\performance\$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<BaselineCoreRun>%HELIX_CORRELATION_PAYLOAD%\Baseline_Core_Root\CoreRun.exe</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd;set PYTHONPATH=%HELIX_WORKITEM_PAYLOAD%\scripts%3B%HELIX_WORKITEM_PAYLOAD%</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>%HELIX_CORRELATION_PAYLOAD%\performance\src\tools\ResultsComparer\ResultsComparer.csproj</ResultsComparer>
<DotnetExe>%HELIX_CORRELATION_PAYLOAD%\performance\tools\dotnet\$(Architecture)\dotnet.exe</DotnetExe>
<Percent>%25%25</Percent>
<XMLResults>%HELIX_WORKITEM_ROOT%\testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
<BaseDirectory>$HELIX_CORRELATION_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)/performance</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'true'">
<BaseDirectory>$HELIX_WORKITEM_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)</PerformanceDirectory>
</PropertyGroup>
<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT'">
<WorkItemCommand>$(PerformanceDirectory)/scripts/benchmarks_ci.py --csproj $(PerformanceDirectory)/$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<BaselineCoreRun>$(BaseDirectory)/Baseline_Core_Root/corerun</BaselineCoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
<BaselineArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts_Baseline</BaselineArtifactsDirectory>
<ResultsComparer>$(PerformanceDirectory)/src/tools/ResultsComparer/ResultsComparer.csproj</ResultsComparer>
<DotnetExe>$(PerformanceDirectory)/tools/dotnet/$(Architecture)/dotnet</DotnetExe>
<Percent>%25</Percent>
<XMLResults>$HELIX_WORKITEM_ROOT/testResults.xml</XMLResults>
</PropertyGroup>
<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(UseBaselineCoreRun)' == 'true'">
<BaselineCoreRunArgument>--corerun $(BaselineCoreRun)</BaselineCoreRunArgument>
</PropertyGroup>
<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>
<PropertyGroup Condition="'$(_Framework)' != 'net461'">
<WorkItemCommand>$(WorkItemCommand) $(CliArguments)</WorkItemCommand>
</PropertyGroup>
<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>
<PropertyGroup>
<PartitionCount>5</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="$(BuildConfig).Partition0" Index="0" />
<Partition Include="$(BuildConfig).Partition1" Index="1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>
<PropertyGroup Condition="'$(Compare)' == 'true'">
<FailOnTestFailure>false</FailOnTestFailure>
</PropertyGroup>
<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults);$(FinalCommand)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands Condition="'$(Compare)' == 'true'">$(WorkItemCommand) --bdn-artifacts $(BaselineArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(BaselineCoreRunArgument)"</PreCommands>
<Command>$(WorkItemCommand) --bdn-artifacts $(ArtifactsDirectory) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument)"</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(DotnetExe) run -f $(_Framework) -p $(ResultsComparer) --base $(BaselineArtifactsDirectory) --diff $(ArtifactsDirectory) --threshold 2$(Percent) --xml $(XMLResults)</PostCommands>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<HelixWorkItem Include="Crossgen System.Private.Xml.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Private.Xml.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen System.Linq.Expressions.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name System.Linq.Expressions.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.VisualBasic.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.VisualBasic.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
<HelixWorkItem Include="Crossgen Microsoft.CodeAnalysis.CSharp.dll">
<PayloadDirectory>$(WorkItemDirectory)\ScenarioCorrelation</PayloadDirectory>
<Command>$(Python) %HELIX_CORRELATION_PAYLOAD%\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.CSharp.dll --core-root %HELIX_CORRELATION_PAYLOAD%\Core_Root</Command>
</HelixWorkItem>
</ItemGroup>
</Project>
@@ -0,0 +1,107 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $BaselineCoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp5.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="Libraries Runtime",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[switch] $Compare,
[string] $Configurations="CompilationMode=$CompilationMode RunKind=$Kind"
)
$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)
$UseBaselineCoreRun = ($BaselineCoreRootDirectory -ne [string]::Empty)
$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
$WorkItemDirectory = (Join-Path $SourceDirectory "workitem")
$ExtraBenchmarkDotNetArguments = "--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
$Creator = $env:BUILD_DEFINITIONNAME
$PerfLabArguments = ""
$HelixSourcePrefix = "pr"
$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"
if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS5.Open"
}
if ($Compare) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf.Open"
$PerfLabArguments = ""
$ExtraBenchmarkDotNetArguments = ""
}
if ($Internal) {
$Queue = "Windows.10.Amd64.19H1.Tiger.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}
# FIX ME: This is a workaround until we get this from the actual pipeline
$CommonSetupArguments="--channel master --queue $Queue --build-number $BuildNumber --build-configs $Configurations --architecture $Architecture"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"
if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"
robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}
if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}
if ($UseBaselineCoreRun) {
$NewBaselineCoreRoot = (Join-Path $PayloadDirectory "Baseline_Core_Root")
Move-Item -Path $BaselineCoreRootDirectory -Destination $NewBaselineCoreRoot
}
$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory
# Set variables that we will need to have in future steps
$ci = $true
. "$PSScriptRoot\..\pipeline-logging-functions.ps1"
# Directories
Write-PipelineSetVariable -Name 'PayloadDirectory' -Value "$PayloadDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerformanceDirectory' -Value "$PerformanceDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'WorkItemDirectory' -Value "$WorkItemDirectory" -IsMultiJobVariable $false
# Script Arguments
Write-PipelineSetVariable -Name 'Python' -Value "py -3" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'ExtraBenchmarkDotNetArguments' -Value "$ExtraBenchmarkDotNetArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'SetupArguments' -Value "$SetupArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerfLabArguments' -Value "$PerfLabArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'BDNCategories' -Value "$RunCategories" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseBaselineCoreRun' -Value "$UseBaselineCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Compare' -Value "$Compare" -IsMultiJobVariable $false
# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Queue' -Value "$Queue" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'HelixSourcePrefix' -Value "$HelixSourcePrefix" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name '_BuildConfig' -Value "$Architecture.$Kind.$Framework" -IsMultiJobVariable $false
exit 0
+216
View File
@@ -0,0 +1,216 @@
#!/usr/bin/env bash
source_directory=$BUILD_SOURCESDIRECTORY
core_root_directory=
baseline_core_root_directory=
architecture=x64
framework=netcoreapp5.0
compilation_mode=tiered
repository=$BUILD_REPOSITORY_NAME
branch=$BUILD_SOURCEBRANCH
commit_sha=$BUILD_SOURCEVERSION
build_number=$BUILD_BUILDNUMBER
internal=false
compare=false
kind="micro"
run_categories="Libraries Runtime"
csproj="src\benchmarks\micro\MicroBenchmarks.csproj"
configurations="CompliationMode=$compilation_mode RunKind=$kind"
run_from_perf_repo=false
use_core_run=true
use_baseline_core_run=true
while (($# > 0)); do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
case $lowerI in
--sourcedirectory)
source_directory=$2
shift 2
;;
--corerootdirectory)
core_root_directory=$2
shift 2
;;
--baselinecorerootdirectory)
baseline_core_root_directory=$2
shift 2
;;
--architecture)
architecture=$2
shift 2
;;
--framework)
framework=$2
shift 2
;;
--compilationmode)
compilation_mode=$2
shift 2
;;
--repository)
repository=$2
shift 2
;;
--branch)
branch=$2
shift 2
;;
--commitsha)
commit_sha=$2
shift 2
;;
--buildnumber)
build_number=$2
shift 2
;;
--kind)
kind=$2
shift 2
;;
--runcategories)
run_categories=$2
shift 2
;;
--csproj)
csproj=$2
shift 2
;;
--internal)
internal=true
shift 1
;;
--compare)
compare=true
shift 1
;;
--configurations)
configurations=$2
shift 2
;;
--help)
echo "Common settings:"
echo " --corerootdirectory <value> Directory where Core_Root exists, if running perf testing with --corerun"
echo " --architecture <value> Architecture of the testing being run"
echo " --configurations <value> List of key=value pairs that will be passed to perf testing infrastructure."
echo " ex: --configurations \"CompilationMode=Tiered OptimzationLevel=PGO\""
echo " --help Print help and exit"
echo ""
echo "Advanced settings:"
echo " --framework <value> The framework to run, if not running in master"
echo " --compliationmode <value> The compilation mode if not passing --configurations"
echo " --sourcedirectory <value> The directory of the sources. Defaults to env:BUILD_SOURCESDIRECTORY"
echo " --repository <value> The name of the repository in the <owner>/<repository name> format. Defaults to env:BUILD_REPOSITORY_NAME"
echo " --branch <value> The name of the branch. Defaults to env:BUILD_SOURCEBRANCH"
echo " --commitsha <value> The commit sha1 to run against. Defaults to env:BUILD_SOURCEVERSION"
echo " --buildnumber <value> The build number currently running. Defaults to env:BUILD_BUILDNUMBER"
echo " --csproj The relative path to the benchmark csproj whose tests should be run. Defaults to src\benchmarks\micro\MicroBenchmarks.csproj"
echo " --kind <value> Related to csproj. The kind of benchmarks that should be run. Defaults to micro"
echo " --runcategories <value> Related to csproj. Categories of benchmarks to run. Defaults to \"coreclr corefx\""
echo " --internal If the benchmarks are running as an official job."
echo ""
exit 0
;;
esac
done
if [ "$repository" == "dotnet/performance" ] || [ "$repository" == "dotnet-performance" ]; then
run_from_perf_repo=true
fi
if [ -z "$configurations" ]; then
configurations="CompliationMode=$compilation_mode"
fi
if [ -z "$core_root_directory" ]; then
use_core_run=false
fi
if [ -z "$baseline_core_root_directory" ]; then
use_baseline_core_run=false
fi
payload_directory=$source_directory/Payload
performance_directory=$payload_directory/performance
workitem_directory=$source_directory/workitem
extra_benchmark_dotnet_arguments="--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
perflab_arguments=
queue=Ubuntu.1804.Amd64.Open
creator=$BUILD_DEFINITIONNAME
helix_source_prefix="pr"
if [[ "$compare" == true ]]; then
extra_benchmark_dotnet_arguments=
perflab_arguments=
# No open queues for arm64
if [[ "$architecture" = "arm64" ]]; then
echo "Compare not available for arm64"
exit 1
fi
queue=Ubuntu.1804.Amd64.Tiger.Perf.Open
fi
if [[ "$internal" == true ]]; then
perflab_arguments="--upload-to-perflab-container"
helix_source_prefix="official"
creator=
extra_benchmark_dotnet_arguments=
if [[ "$architecture" = "arm64" ]]; then
queue=Ubuntu.1804.Arm64.Perf
else
queue=Ubuntu.1804.Amd64.Tiger.Perf
fi
fi
common_setup_arguments="--channel master --queue $queue --build-number $build_number --build-configs $configurations --architecture $architecture"
setup_arguments="--repository https://github.com/$repository --branch $branch --get-perf-hash --commit-sha $commit_sha $common_setup_arguments"
if [[ "$run_from_perf_repo" = true ]]; then
payload_directory=
workitem_directory=$source_directory
performance_directory=$workitem_directory
setup_arguments="--perf-hash $commit_sha $common_setup_arguments"
else
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $performance_directory
docs_directory=$performance_directory/docs
mv $docs_directory $workitem_directory
fi
if [[ "$use_core_run" = true ]]; then
new_core_root=$payload_directory/Core_Root
mv $core_root_directory $new_core_root
fi
if [[ "$use_baseline_core_run" = true ]]; then
new_baseline_core_root=$payload_directory/Baseline_Core_Root
mv $baseline_core_root_directory $new_baseline_core_root
fi
ci=true
_script_dir=$(pwd)/eng/common
. "$_script_dir/pipeline-logging-functions.sh"
# Make sure all of our variables are available for future steps
Write-PipelineSetVariable -name "UseCoreRun" -value "$use_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "UseBaselineCoreRun" -value "$use_baseline_core_run" -is_multi_job_variable false
Write-PipelineSetVariable -name "Architecture" -value "$architecture" -is_multi_job_variable false
Write-PipelineSetVariable -name "PayloadDirectory" -value "$payload_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerformanceDirectory" -value "$performance_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "WorkItemDirectory" -value "$workitem_directory" -is_multi_job_variable false
Write-PipelineSetVariable -name "Queue" -value "$queue" -is_multi_job_variable false
Write-PipelineSetVariable -name "SetupArguments" -value "$setup_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "Python" -value "$python3" -is_multi_job_variable false
Write-PipelineSetVariable -name "PerfLabArguments" -value "$perflab_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "ExtraBenchmarkDotNetArguments" -value "$extra_benchmark_dotnet_arguments" -is_multi_job_variable false
Write-PipelineSetVariable -name "BDNCategories" -value "$run_categories" -is_multi_job_variable false
Write-PipelineSetVariable -name "TargetCsproj" -value "$csproj" -is_multi_job_variable false
Write-PipelineSetVariable -name "RunFromPerfRepo" -value "$run_from_perf_repo" -is_multi_job_variable false
Write-PipelineSetVariable -name "Creator" -value "$creator" -is_multi_job_variable false
Write-PipelineSetVariable -name "HelixSourcePrefix" -value "$helix_source_prefix" -is_multi_job_variable false
Write-PipelineSetVariable -name "Kind" -value "$kind" -is_multi_job_variable false
Write-PipelineSetVariable -name "_BuildConfig" -value "$architecture.$kind.$framework" -is_multi_job_variable false
Write-PipelineSetVariable -name "Compare" -value "$compare" -is_multi_job_variable false
@@ -1,4 +1,4 @@
# Source for this file was taken from https://github.com/microsoft/azure-pipelines-task-lib/blob/11c9439d4af17e6475d9fe058e6b2e03914d17e6/powershell/VstsTaskSdk/LoggingCommandFunctions.ps1
# Source for this file was taken from https://github.com/microsoft/azure-pipelines-task-lib/blob/11c9439d4af17e6475d9fe058e6b2e03914d17e6/powershell/VstsTaskSdk/LoggingCommandFunctions.ps1 and modified.
# NOTE: You should not be calling these method directly as they are likely to change. Instead you should be calling the Write-Pipeline* functions defined in tools.ps1
@@ -12,6 +12,102 @@ $script:loggingCommandEscapeMappings = @( # TODO: WHAT ABOUT "="? WHAT ABOUT "%"
# TODO: BUG: Escape % ???
# TODO: Add test to verify don't need to escape "=".
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTelemetryError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Category,
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force)
$PSBoundParameters.Remove('Category') | Out-Null
if($Force -Or ((Test-Path variable:ci) -And $ci)) {
$Message = "(NETCORE_ENGINEERING_TELEMETRY=$Category) $Message"
}
$PSBoundParameters.Remove('Message') | Out-Null
$PSBoundParameters.Add('Message', $Message)
Write-PipelineTaskError @PSBoundParameters
}
# Specify "-Force" to force pipeline formatted output even if "$ci" is false or not set
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput,
[switch]$Force
)
if(!$Force -And (-Not (Test-Path variable:ci) -Or !$ci)) {
if($Type -eq 'error') {
Write-Host $Message -ForegroundColor Red
return
}
elseif ($Type -eq 'warning') {
Write-Host $Message -ForegroundColor Yellow
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
$PSBoundParameters.Remove('Force') | Out-Null
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Name,
[string]$Value,
[switch]$Secret,
[switch]$AsOutput,
[bool]$IsMultiJobVariable=$true)
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'isSecret' = $Secret
'isOutput' = $IsMultiJobVariable
} -AsOutput:$AsOutput
}
}
function Write-PipelinePrependPath {
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if((Test-Path variable:ci) -And $ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
<########################################
# Private functions.
########################################>
@@ -143,4 +239,4 @@ function Write-LogIssue {
}
Write-Host $command -ForegroundColor $foregroundColor -BackgroundColor $backgroundColor
}
}
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env bash
function Write-PipelineTelemetryError {
local telemetry_category=''
local force=false
local function_args=()
local message=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-category|-c)
telemetry_category=$2
shift
;;
-force|-f)
force=true
;;
-*)
function_args+=("$1 $2")
shift
;;
*)
message=$*
;;
esac
shift
done
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$message" >&2
return
fi
message="(NETCORE_ENGINEERING_TELEMETRY=$telemetry_category) $message"
function_args+=("$message")
if [[ $force == true ]]; then
function_args+=("-force")
fi
Write-PipelineTaskError $function_args
}
function Write-PipelineTaskError {
if [[ $force != true ]] && [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
local message_type="error"
local sourcepath=''
local linenumber=''
local columnnumber=''
local error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-ln)
linenumber=$2
shift
;;
-columnnumber|-cn)
columnnumber=$2
shift
;;
-errcode|-e)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
local message="##vso[task.logissue"
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
function Write-PipelineSetVariable {
if [[ "$ci" != true ]]; then
return
fi
local name=''
local value=''
local secret=false
local as_output=false
local is_multi_job_variable=true
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-name|-n)
name=$2
shift
;;
-value|-v)
value=$2
shift
;;
-secret|-s)
secret=true
;;
-as_output|-a)
as_output=true
;;
-is_multi_job_variable|-i)
is_multi_job_variable=$2
shift
;;
esac
shift
done
value=${value/;/%3B}
value=${value/\\r/%0D}
value=${value/\\n/%0A}
value=${value/]/%5D}
local message="##vso[task.setvariable variable=$name;isSecret=$secret;isOutput=$is_multi_job_variable]$value"
if [[ "$as_output" == true ]]; then
$message
else
echo "$message"
fi
}
function Write-PipelinePrependPath {
local prepend_path=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-path|-p)
prepend_path=$2
shift
;;
esac
shift
done
export PATH="$prepend_path:$PATH"
if [[ "$ci" == true ]]; then
echo "##vso[task.prependpath]$prepend_path"
fi
}
@@ -0,0 +1,48 @@
param(
[Parameter(Mandatory=$true)][int] $BuildId,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that the channel we are going to promote the build to exist
$channelInfo = Get-MaestroChannel -ChannelId $ChannelId
if (!$channelInfo) {
Write-PipelineTelemetryCategory -Category 'PromoteBuild' -Message "Channel with BAR ID $ChannelId was not found in BAR!"
ExitWithExitCode 1
}
# Get info about which channel(s) the build has already been promoted to
$buildInfo = Get-MaestroBuild -BuildId $BuildId
if (!$buildInfo) {
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "Build with BAR ID $BuildId was not found in BAR!"
ExitWithExitCode 1
}
# Find whether the build is already assigned to the channel or not
if ($buildInfo.channels) {
foreach ($channel in $buildInfo.channels) {
if ($channel.Id -eq $ChannelId) {
Write-Host "The build with BAR ID $BuildId is already on channel $ChannelId!"
ExitWithExitCode 0
}
}
}
Write-Host "Promoting build '$BuildId' to channel '$ChannelId'."
Assign-BuildToChannel -BuildId $BuildId -ChannelId $ChannelId
Write-Host 'done.'
}
catch {
Write-Host $_
Write-PipelineTelemetryError -Category 'PromoteBuild' -Message "There was an error while trying to promote build '$BuildId' to channel '$ChannelId'"
ExitWithExitCode 1
}
@@ -0,0 +1,25 @@
param(
[Parameter(Mandatory=$true)][string] $PromoteToChannels, # List of channels that the build should be promoted to
[Parameter(Mandatory=$true)][array] $AvailableChannelIds # List of channel IDs available in the YAML implementation
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Check that every channel that Maestro told to promote the build to
# is available in YAML
$PromoteToChannelsIds = $PromoteToChannels -split "\D" | Where-Object { $_ }
foreach ($id in $PromoteToChannelsIds) {
if (($id -ne 0) -and ($id -notin $AvailableChannelIds)) {
Write-PipelineTaskError -Type 'warning' -Message "Channel $id is not present in the post-build YAML configuration!"
}
}
Write-Host 'done.'
}
catch {
Write-Host $_
Write-PipelineTelemetryError -Category 'CheckChannelConsistency' -Message "There was an error while trying to check consistency of Maestro default channels for the build and post-build YAML configuration."
ExitWithExitCode 1
}
@@ -0,0 +1,24 @@
# This script validates NuGet package metadata information using this
# tool: https://github.com/NuGet/NuGetGallery/tree/jver-verify/src/VerifyMicrosoftPackage
param(
[Parameter(Mandatory=$true)][string] $PackagesPath, # Path to where the packages to be validated are
[Parameter(Mandatory=$true)][string] $ToolDestinationPath # Where the validation tool should be downloaded to
)
try {
. $PSScriptRoot\post-build-utils.ps1
$url = 'https://raw.githubusercontent.com/NuGet/NuGetGallery/3e25ad135146676bcab0050a516939d9958bfa5d/src/VerifyMicrosoftPackage/verify.ps1'
New-Item -ItemType 'directory' -Path ${ToolDestinationPath} -Force
Invoke-WebRequest $url -OutFile ${ToolDestinationPath}\verify.ps1
& ${ToolDestinationPath}\verify.ps1 ${PackagesPath}\*.nupkg
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'NuGetValidation' -Message $_
ExitWithExitCode 1
}
@@ -0,0 +1,91 @@
# Most of the functions in this file require the variables `MaestroApiEndPoint`,
# `MaestroApiVersion` and `MaestroApiAccessToken` to be globally available.
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
. $PSScriptRoot\..\tools.ps1
function Create-MaestroApiRequestHeaders([string]$ContentType = 'application/json') {
Validate-MaestroVars
$headers = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$headers.Add('Accept', $ContentType)
$headers.Add('Authorization',"Bearer $MaestroApiAccessToken")
return $headers
}
function Get-MaestroChannel([int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}?api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroBuild([int]$BuildId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/builds/${BuildId}?api-version=$MaestroApiVersion"
$result = try { return Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Get-MaestroSubscriptions([string]$SourceRepository, [int]$ChannelId) {
Validate-MaestroVars
$SourceRepository = [System.Web.HttpUtility]::UrlEncode($SourceRepository)
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions?sourceRepository=$SourceRepository&channelId=$ChannelId&api-version=$MaestroApiVersion"
$result = try { Invoke-WebRequest -Method Get -Uri $apiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
return $result
}
function Assign-BuildToChannel([int]$BuildId, [int]$ChannelId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/channels/${ChannelId}/builds/${BuildId}?api-version=$MaestroApiVersion"
Invoke-WebRequest -Method Post -Uri $apiEndpoint -Headers $apiHeaders | Out-Null
}
function Trigger-Subscription([string]$SubscriptionId) {
Validate-MaestroVars
$apiHeaders = Create-MaestroApiRequestHeaders -AuthToken $MaestroApiAccessToken
$apiEndpoint = "$MaestroApiEndPoint/api/subscriptions/$SubscriptionId/trigger?api-version=$MaestroApiVersion"
Invoke-WebRequest -Uri $apiEndpoint -Headers $apiHeaders -Method Post | Out-Null
}
function Validate-MaestroVars {
try {
Get-Variable MaestroApiEndPoint -Scope Global | Out-Null
Get-Variable MaestroApiVersion -Scope Global | Out-Null
Get-Variable MaestroApiAccessToken -Scope Global | Out-Null
if (!($MaestroApiEndPoint -Match '^http[s]?://maestro-(int|prod).westus2.cloudapp.azure.com$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiEndPoint is not a valid Maestro URL. '$MaestroApiEndPoint'"
ExitWithExitCode 1
}
if (!($MaestroApiVersion -Match '^[0-9]{4}-[0-9]{2}-[0-9]{2}$')) {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message "MaestroApiVersion does not match a version string in the format yyyy-MM-DD. '$MaestroApiVersion'"
ExitWithExitCode 1
}
}
catch {
Write-PipelineTelemetryError -Category 'MaestroVars' -Message 'Error: Variables `MaestroApiEndPoint`, `MaestroApiVersion` and `MaestroApiAccessToken` are required while using this script.'
Write-Host $_
ExitWithExitCode 1
}
}
@@ -0,0 +1,257 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where Symbols.NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$false)][string] $GHRepoName, # GitHub name of the repo including the Org. E.g., dotnet/arcade
[Parameter(Mandatory=$false)][string] $GHCommit, # GitHub commit SHA used to build the packages
[Parameter(Mandatory=$true)][string] $SourcelinkCliVersion # Version of SourceLink CLI to use
)
. $PSScriptRoot\post-build-utils.ps1
# Cache/HashMap (File -> Exist flag) used to consult whether a file exist
# in the repository at a specific commit point. This is populated by inserting
# all files present in the repo at a specific commit point.
$global:RepoFiles = @{}
# Maximum number of jobs to run in parallel
$MaxParallelJobs = 6
# Wait time between check for system load
$SecondsBetweenLoadChecks = 10
$ValidatePackage = {
param(
[string] $PackagePath # Full path to a Symbols.NuGet package
)
. $using:PSScriptRoot\..\tools.ps1
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
Write-Host "Input file does not exist: $PackagePath"
return 1
}
# Extensions for which we'll look for SourceLink information
# For now we'll only care about Portable & Embedded PDBs
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Validating ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
$FailedFiles = 0
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath) | Out-Null
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$FileName = $_.FullName
$Extension = [System.IO.Path]::GetExtension($_.Name)
$FakeName = -Join((New-Guid), $Extension)
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $FakeName
# We ignore resource DLLs
if ($FileName.EndsWith('.resources.dll')) {
return
}
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
$ValidateFile = {
param(
[string] $FullPath, # Full path to the module that has to be checked
[string] $RealPath,
[ref] $FailedFiles
)
$sourcelinkExe = "$env:USERPROFILE\.dotnet\tools"
$sourcelinkExe = Resolve-Path "$sourcelinkExe\sourcelink.exe"
$SourceLinkInfos = & $sourcelinkExe print-urls $FullPath | Out-String
if ($LASTEXITCODE -eq 0 -and -not ([string]::IsNullOrEmpty($SourceLinkInfos))) {
$NumFailedLinks = 0
# We only care about Http addresses
$Matches = (Select-String '(http[s]?)(:\/\/)([^\s,]+)' -Input $SourceLinkInfos -AllMatches).Matches
if ($Matches.Count -ne 0) {
$Matches.Value |
ForEach-Object {
$Link = $_
$CommitUrl = "https://raw.githubusercontent.com/${using:GHRepoName}/${using:GHCommit}/"
$FilePath = $Link.Replace($CommitUrl, "")
$Status = 200
$Cache = $using:RepoFiles
if ( !($Cache.ContainsKey($FilePath)) ) {
try {
$Uri = $Link -as [System.URI]
# Only GitHub links are valid
if ($Uri.AbsoluteURI -ne $null -and ($Uri.Host -match 'github' -or $Uri.Host -match 'githubusercontent')) {
$Status = (Invoke-WebRequest -Uri $Link -UseBasicParsing -Method HEAD -TimeoutSec 5).StatusCode
}
else {
$Status = 0
}
}
catch {
write-host $_
$Status = 0
}
}
if ($Status -ne 200) {
if ($NumFailedLinks -eq 0) {
if ($FailedFiles.Value -eq 0) {
Write-Host
}
Write-Host "`tFile $RealPath has broken links:"
}
Write-Host "`t`tFailed to retrieve $Link"
$NumFailedLinks++
}
}
}
if ($NumFailedLinks -ne 0) {
$FailedFiles.value++
$global:LASTEXITCODE = 1
}
}
}
&$ValidateFile $TargetFile $FileName ([ref]$FailedFiles)
}
}
catch {
}
finally {
$zip.Dispose()
}
if ($FailedFiles -eq 0) {
Write-Host 'Passed.'
return 0
}
else {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$PackagePath has broken SourceLink links."
return 1
}
}
function ValidateSourceLinkLinks {
if ($GHRepoName -ne '' -and !($GHRepoName -Match '^[^\s\/]+/[^\s\/]+$')) {
if (!($GHRepoName -Match '^[^\s-]+-[^\s]+$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHRepoName should be in the format <org>/<repo> or <org>-<repo>. '$GHRepoName'"
ExitWithExitCode 1
}
else {
$GHRepoName = $GHRepoName -replace '^([^\s-]+)-([^\s]+)$', '$1/$2';
}
}
if ($GHCommit -ne '' -and !($GHCommit -Match '^[0-9a-fA-F]{40}$')) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "GHCommit should be a 40 chars hexadecimal string. '$GHCommit'"
ExitWithExitCode 1
}
if ($GHRepoName -ne '' -and $GHCommit -ne '') {
$RepoTreeURL = -Join('http://api.github.com/repos/', $GHRepoName, '/git/trees/', $GHCommit, '?recursive=1')
$CodeExtensions = @('.cs', '.vb', '.fs', '.fsi', '.fsx', '.fsscript')
try {
# Retrieve the list of files in the repo at that particular commit point and store them in the RepoFiles hash
$Data = Invoke-WebRequest $RepoTreeURL -UseBasicParsing | ConvertFrom-Json | Select-Object -ExpandProperty tree
foreach ($file in $Data) {
$Extension = [System.IO.Path]::GetExtension($file.path)
if ($CodeExtensions.Contains($Extension)) {
$RepoFiles[$file.path] = 1
}
}
}
catch {
Write-Host "Problems downloading the list of files from the repo. Url used: $RepoTreeURL . Execution will proceed without caching."
}
}
elseif ($GHRepoName -ne '' -or $GHCommit -ne '') {
Write-Host 'For using the http caching mechanism both GHRepoName and GHCommit should be informed.'
}
if (Test-Path $ExtractPath) {
Remove-Item $ExtractPath -Force -Recurse -ErrorAction SilentlyContinue
}
# Process each NuGet package in parallel
Get-ChildItem "$InputPath\*.symbols.nupkg" |
ForEach-Object {
Start-Job -ScriptBlock $ValidatePackage -ArgumentList $_.FullName | Out-Null
$NumJobs = @(Get-Job -State 'Running').Count
while ($NumJobs -ge $MaxParallelJobs) {
Write-Host "There are $NumJobs validation jobs running right now. Waiting $SecondsBetweenLoadChecks seconds to check again."
sleep $SecondsBetweenLoadChecks
$NumJobs = @(Get-Job -State 'Running').Count
}
foreach ($Job in @(Get-Job -State 'Completed')) {
Receive-Job -Id $Job.Id
Remove-Job -Id $Job.Id
}
}
$ValidationFailures = 0
foreach ($Job in @(Get-Job)) {
$jobResult = Wait-Job -Id $Job.Id | Receive-Job
if ($jobResult -ne '0') {
$ValidationFailures++
}
}
if ($ValidationFailures -gt 0) {
Write-PipelineTelemetryError -Category 'SourceLink' -Message "$ValidationFailures package(s) failed validation."
ExitWithExitCode 1
}
}
function InstallSourcelinkCli {
$sourcelinkCliPackageName = 'sourcelink'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$sourcelinkCliPackageName*") -and ($toolList -like "*$sourcelinkCliVersion*")) {
Write-Host "SourceLink CLI version $sourcelinkCliVersion is already installed."
}
else {
Write-Host "Installing SourceLink CLI version $sourcelinkCliVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $sourcelinkCliPackageName --version $sourcelinkCliVersion --verbosity "minimal" --global
}
}
try {
InstallSourcelinkCli
ValidateSourceLinkLinks
}
catch {
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'SourceLink' -Message $_
ExitWithExitCode 1
}
@@ -1,11 +1,9 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $SymbolToolPath # Full path to directory where dotnet symbol-tool was installed
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where NuGet packages to be checked are stored
[Parameter(Mandatory=$true)][string] $ExtractPath, # Full path to directory where the packages will be extracted during validation
[Parameter(Mandatory=$true)][string] $DotnetSymbolVersion # Version of dotnet symbol to use
)
Add-Type -AssemblyName System.IO.Compression.FileSystem
function FirstMatchingSymbolDescriptionOrDefault {
param(
[string] $FullPath, # Full path to the module that has to be checked
@@ -21,36 +19,39 @@ function FirstMatchingSymbolDescriptionOrDefault {
# checking and which type of file was uploaded.
# The file itself is returned
$SymbolPath = $SymbolsPath + "\" + $FileName
$SymbolPath = $SymbolsPath + '\' + $FileName
# PDB file for the module
$PdbPath = $SymbolPath.Replace($Extension, ".pdb")
$PdbPath = $SymbolPath.Replace($Extension, '.pdb')
# PDB file for R2R module (created by crossgen)
$NGenPdb = $SymbolPath.Replace($Extension, ".ni.pdb")
$NGenPdb = $SymbolPath.Replace($Extension, '.ni.pdb')
# DBG file for a .so library
$SODbg = $SymbolPath.Replace($Extension, ".so.dbg")
$SODbg = $SymbolPath.Replace($Extension, '.so.dbg')
# DWARF file for a .dylib
$DylibDwarf = $SymbolPath.Replace($Extension, ".dylib.dwarf")
$DylibDwarf = $SymbolPath.Replace($Extension, '.dylib.dwarf')
.\dotnet-symbol.exe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
$dotnetSymbolExe = "$env:USERPROFILE\.dotnet\tools"
$dotnetSymbolExe = Resolve-Path "$dotnetSymbolExe\dotnet-symbol.exe"
& $dotnetSymbolExe --symbols --modules --windows-pdbs $TargetServerParam $FullPath -o $SymbolsPath | Out-Null
if (Test-Path $PdbPath) {
return "PDB"
return 'PDB'
}
elseif (Test-Path $NGenPdb) {
return "NGen PDB"
return 'NGen PDB'
}
elseif (Test-Path $SODbg) {
return "DBG for SO"
return 'DBG for SO'
}
elseif (Test-Path $DylibDwarf) {
return "Dwarf for Dylib"
return 'Dwarf for Dylib'
}
elseif (Test-Path $SymbolPath) {
return "Module"
return 'Module'
}
else {
return $null
@@ -64,11 +65,12 @@ function CountMissingSymbols {
# Ensure input file exist
if (!(Test-Path $PackagePath)) {
throw "Input file does not exist: $PackagePath"
Write-PipelineTaskError "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
# Extensions for which we'll look for symbols
$RelevantExtensions = @(".dll", ".exe", ".so", ".dylib")
$RelevantExtensions = @('.dll', '.exe', '.so', '.dylib')
# How many files are missing symbol information
$MissingSymbols = 0
@@ -76,41 +78,38 @@ function CountMissingSymbols {
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$PackageGuid = New-Guid
$ExtractPath = Join-Path -Path $ExtractPath -ChildPath $PackageGuid
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath "Symbols"
$SymbolsPath = Join-Path -Path $ExtractPath -ChildPath 'Symbols'
[System.IO.Compression.ZipFile]::ExtractToDirectory($PackagePath, $ExtractPath)
# Makes easier to reference `symbol tool`
Push-Location $SymbolToolPath
Get-ChildItem -Recurse $ExtractPath |
Where-Object {$RelevantExtensions -contains $_.Extension} |
ForEach-Object {
if ($_.FullName -Match "\\ref\\") {
Write-Host "`t Ignoring reference assembly file" $_.FullName
if ($_.FullName -Match '\\ref\\') {
Write-Host "`t Ignoring reference assembly file " $_.FullName
return
}
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--microsoft-symbol-server" $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName "--internal-server" $SymbolsPath
$SymbolsOnMSDL = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--microsoft-symbol-server' $SymbolsPath
$SymbolsOnSymWeb = FirstMatchingSymbolDescriptionOrDefault $_.FullName '--internal-server' $SymbolsPath
Write-Host -NoNewLine "`t Checking file" $_.FullName "... "
Write-Host -NoNewLine "`t Checking file " $_.FullName "... "
if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
Write-Host "Symbols found on MSDL (" $SymbolsOnMSDL ") and SymWeb (" $SymbolsOnSymWeb ")"
Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
}
else {
$MissingSymbols++
if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
Write-Host "No symbols found on MSDL or SymWeb!"
Write-Host 'No symbols found on MSDL or SymWeb!'
}
else {
if ($SymbolsOnMSDL -eq $null) {
Write-Host "No symbols found on MSDL!"
Write-Host 'No symbols found on MSDL!'
}
else {
Write-Host "No symbols found on SymWeb!"
Write-Host 'No symbols found on SymWeb!'
}
}
}
@@ -129,30 +128,61 @@ function CheckSymbolsAvailable {
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$FileName = $_.Name
# These packages from Arcade-Services include some native libraries that
# our current symbol uploader can't handle. Below is a workaround until
# we get issue: https://github.com/dotnet/arcade/issues/2457 sorted.
if ($FileName -Match "Microsoft\.DotNet\.Darc\.") {
if ($FileName -Match 'Microsoft\.DotNet\.Darc\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
elseif ($FileName -Match "Microsoft\.DotNet\.Maestro\.Tasks\.") {
elseif ($FileName -Match 'Microsoft\.DotNet\.Maestro\.Tasks\.') {
Write-Host "Ignoring Arcade-services file: $FileName"
Write-Host
return
}
Write-Host "Validating $FileName "
$Status = CountMissingSymbols "$InputPath\$FileName"
if ($Status -ne 0) {
Write-Error "Missing symbols for $Status modules in the package $FileName"
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message "Missing symbols for $Status modules in the package $FileName"
ExitWithExitCode $exitCode
}
Write-Host
}
}
CheckSymbolsAvailable
function InstallDotnetSymbol {
$dotnetSymbolPackageName = 'dotnet-symbol'
$dotnetRoot = InitializeDotNetCli -install:$true
$dotnet = "$dotnetRoot\dotnet.exe"
$toolList = & "$dotnet" tool list --global
if (($toolList -like "*$dotnetSymbolPackageName*") -and ($toolList -like "*$dotnetSymbolVersion*")) {
Write-Host "dotnet-symbol version $dotnetSymbolVersion is already installed."
}
else {
Write-Host "Installing dotnet-symbol version $dotnetSymbolVersion..."
Write-Host 'You may need to restart your command window if this is the first dotnet tool you have installed.'
& "$dotnet" tool install $dotnetSymbolPackageName --version $dotnetSymbolVersion --verbosity "minimal" --global
}
}
try {
. $PSScriptRoot\post-build-utils.ps1
Add-Type -AssemblyName System.IO.Compression.FileSystem
InstallDotnetSymbol
CheckSymbolsAvailable
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'CheckSymbols' -Message $_
ExitWithExitCode 1
}
@@ -0,0 +1,64 @@
param(
[Parameter(Mandatory=$true)][string] $SourceRepo,
[Parameter(Mandatory=$true)][int] $ChannelId,
[Parameter(Mandatory=$true)][string] $MaestroApiAccessToken,
[Parameter(Mandatory=$false)][string] $MaestroApiEndPoint = 'https://maestro-prod.westus2.cloudapp.azure.com',
[Parameter(Mandatory=$false)][string] $MaestroApiVersion = '2019-01-16'
)
try {
. $PSScriptRoot\post-build-utils.ps1
# Get all the $SourceRepo subscriptions
$normalizedSourceRepo = $SourceRepo.Replace('dnceng@', '')
$subscriptions = Get-MaestroSubscriptions -SourceRepository $normalizedSourceRepo -ChannelId $ChannelId
if (!$subscriptions) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message "No subscriptions found for source repo '$normalizedSourceRepo' in channel '$ChannelId'"
ExitWithExitCode 0
}
$subscriptionsToTrigger = New-Object System.Collections.Generic.List[string]
$failedTriggeredSubscription = $false
# Get all enabled subscriptions that need dependency flow on 'everyBuild'
foreach ($subscription in $subscriptions) {
if ($subscription.enabled -and $subscription.policy.updateFrequency -like 'everyBuild' -and $subscription.channel.id -eq $ChannelId) {
Write-Host "Should trigger this subscription: ${$subscription.id}"
[void]$subscriptionsToTrigger.Add($subscription.id)
}
}
foreach ($subscriptionToTrigger in $subscriptionsToTrigger) {
try {
Write-Host "Triggering subscription '$subscriptionToTrigger'."
Trigger-Subscription -SubscriptionId $subscriptionToTrigger
Write-Host 'done.'
}
catch
{
Write-Host "There was an error while triggering subscription '$subscriptionToTrigger'"
Write-Host $_
Write-Host $_.ScriptStackTrace
$failedTriggeredSubscription = $true
}
}
if ($subscriptionsToTrigger.Count -eq 0) {
Write-Host "No subscription matched source repo '$normalizedSourceRepo' and channel ID '$ChannelId'."
}
elseif ($failedTriggeredSubscription) {
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message 'At least one subscription failed to be triggered...'
ExitWithExitCode 1
}
else {
Write-Host 'All subscriptions were triggered successfully!'
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'TriggerSubscriptions' -Message $_
ExitWithExitCode 1
}
+9 -10
View File
@@ -1,8 +1,8 @@
[CmdletBinding(PositionalBinding=$false)]
Param(
[string] $configuration = "Debug",
[string] $configuration = 'Debug',
[string] $task,
[string] $verbosity = "minimal",
[string] $verbosity = 'minimal',
[string] $msbuildEngine = $null,
[switch] $restore,
[switch] $prepareMachine,
@@ -32,7 +32,7 @@ function Print-Usage() {
}
function Build([string]$target) {
$logSuffix = if ($target -eq "Execute") { "" } else { ".$target" }
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
@@ -46,33 +46,32 @@ function Build([string]$target) {
}
try {
if ($help -or (($null -ne $properties) -and ($properties.Contains("/help") -or $properties.Contains("/?")))) {
if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Print-Usage
exit 0
}
if ($task -eq "") {
Write-Host "Missing required parameter '-task <value>'" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Missing required parameter '-task <value>'" -ForegroundColor Red
Print-Usage
ExitWithExitCode 1
}
$taskProject = GetSdkTaskProject $task
if (!(Test-Path $taskProject)) {
Write-Host "Unknown task: $task" -ForegroundColor Red
Write-PipelineTelemetryError -Category 'Build' -Message "Unknown task: $task" -ForegroundColor Red
ExitWithExitCode 1
}
if ($restore) {
Build "Restore"
Build 'Restore'
}
Build "Execute"
Build 'Execute'
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Build' -Message $_
ExitWithExitCode 1
}
+13
View File
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
<packageSources>
<clear />
<add key="guardian" value="https://securitytools.pkgs.visualstudio.com/_packaging/Guardian/nuget/v3/index.json" />
</packageSources>
<disabledPackageSources>
<clear />
</disabledPackageSources>
</configuration>
+110
View File
@@ -0,0 +1,110 @@
Param(
[string] $GuardianPackageName, # Required: the name of guardian CLI package (not needed if GuardianCliLocation is specified)
[string] $NugetPackageDirectory, # Required: directory where NuGet packages are installed (not needed if GuardianCliLocation is specified)
[string] $GuardianCliLocation, # Optional: Direct location of Guardian CLI executable if GuardianPackageName & NugetPackageDirectory are not specified
[string] $Repository=$env:BUILD_REPOSITORY_NAME, # Required: the name of the repository (e.g. dotnet/arcade)
[string] $BranchName=$env:BUILD_SOURCEBRANCH, # Optional: name of branch or version of gdn settings; defaults to master
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY, # Required: the directory where source files are located
[string] $ArtifactsDirectory = (Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY ('artifacts')), # Required: the directory where build artifacts are located
[string] $AzureDevOpsAccessToken, # Required: access token for dnceng; should be provided via KeyVault
[string[]] $SourceToolsList, # Optional: list of SDL tools to run on source code
[string[]] $ArtifactToolsList, # Optional: list of SDL tools to run on built artifacts
[bool] $TsaPublish=$False, # Optional: true will publish results to TSA; only set to true after onboarding to TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaBranchName=$env:BUILD_SOURCEBRANCH, # Optional: required for TSA publish; defaults to $(Build.SourceBranchName); TSA is the automated framework used to upload test results as bugs.
[string] $TsaRepositoryName=$env:BUILD_REPOSITORY_NAME, # Optional: TSA repository name; will be generated automatically if not submitted; TSA is the automated framework used to upload test results as bugs.
[string] $BuildNumber=$env:BUILD_BUILDNUMBER, # Optional: required for TSA publish; defaults to $(Build.BuildNumber)
[bool] $UpdateBaseline=$False, # Optional: if true, will update the baseline in the repository; should only be run after fixing any issues which need to be fixed
[bool] $TsaOnboard=$False, # Optional: if true, will onboard the repository to TSA; should only be run once; TSA is the automated framework used to upload test results as bugs.
[string] $TsaInstanceUrl, # Optional: only needed if TsaOnboard or TsaPublish is true; the instance-url registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the codebase registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaProjectName, # Optional: only needed if TsaOnboard or TsaPublish is true; the name of the project registered with TSA; TSA is the automated framework used to upload test results as bugs.
[string] $TsaNotificationEmail, # Optional: only needed if TsaOnboard is true; the email(s) which will receive notifications of TSA bug filings (e.g. [email protected]); TSA is the automated framework used to upload test results as bugs.
[string] $TsaCodebaseAdmin, # Optional: only needed if TsaOnboard is true; the aliases which are admins of the TSA codebase (e.g. DOMAIN\alias); TSA is the automated framework used to upload test results as bugs.
[string] $TsaBugAreaPath, # Optional: only needed if TsaOnboard is true; the area path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $TsaIterationPath, # Optional: only needed if TsaOnboard is true; the iteration path where TSA will file bugs in AzDO; TSA is the automated framework used to upload test results as bugs.
[string] $GuardianLoggerLevel='Standard', # Optional: the logger level for the Guardian CLI; options are Trace, Verbose, Standard, Warning, and Error
[string[]] $CrScanAdditionalRunConfigParams, # Optional: Additional Params to custom build a CredScan run config in the format @("xyz:abc","sdf:1")
[string[]] $PoliCheckAdditionalRunConfigParams # Optional: Additional Params to custom build a Policheck run config in the format @("xyz:abc","sdf:1")
)
try {
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
. $PSScriptRoot\..\tools.ps1
#Replace repo names to the format of org/repo
if (!($Repository.contains('/'))) {
$RepoName = $Repository -replace '(.*?)-(.*)', '$1/$2';
}
else{
$RepoName = $Repository;
}
if ($GuardianPackageName) {
$guardianCliLocation = Join-Path $NugetPackageDirectory (Join-Path $GuardianPackageName (Join-Path 'tools' 'guardian.cmd'))
} else {
$guardianCliLocation = $GuardianCliLocation
}
$workingDirectory = (Split-Path $SourceDirectory -Parent)
$ValidPath = Test-Path $guardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Invalid Guardian CLI Location.'
ExitWithExitCode 1
}
& $(Join-Path $PSScriptRoot 'init-sdl.ps1') -GuardianCliLocation $guardianCliLocation -Repository $RepoName -BranchName $BranchName -WorkingDirectory $workingDirectory -AzureDevOpsAccessToken $AzureDevOpsAccessToken -GuardianLoggerLevel $GuardianLoggerLevel
$gdnFolder = Join-Path $workingDirectory '.gdn'
if ($TsaOnboard) {
if ($TsaCodebaseName -and $TsaNotificationEmail -and $TsaCodebaseAdmin -and $TsaBugAreaPath) {
Write-Host "$guardianCliLocation tsa-onboard --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-onboard --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-onboard failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not onboard to TSA -- not all required values ($TsaCodebaseName, $TsaNotificationEmail, $TsaCodebaseAdmin, $TsaBugAreaPath) were specified.'
ExitWithExitCode 1
}
}
if ($ArtifactToolsList -and $ArtifactToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $ArtifactsDirectory -GdnFolder $gdnFolder -ToolsList $ArtifactToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($SourceToolsList -and $SourceToolsList.Count -gt 0) {
& $(Join-Path $PSScriptRoot 'run-sdl.ps1') -GuardianCliLocation $guardianCliLocation -WorkingDirectory $workingDirectory -TargetDirectory $SourceDirectory -GdnFolder $gdnFolder -ToolsList $SourceToolsList -AzureDevOpsAccessToken $AzureDevOpsAccessToken -UpdateBaseline $UpdateBaseline -GuardianLoggerLevel $GuardianLoggerLevel -CrScanAdditionalRunConfigParams $CrScanAdditionalRunConfigParams -PoliCheckAdditionalRunConfigParams $PoliCheckAdditionalRunConfigParams
}
if ($UpdateBaseline) {
& (Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $RepoName -BranchName $BranchName -GdnFolder $GdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Update baseline'
}
if ($TsaPublish) {
if ($TsaBranchName -and $BuildNumber) {
if (-not $TsaRepositoryName) {
$TsaRepositoryName = "$($Repository)-$($BranchName)"
}
Write-Host "$guardianCliLocation tsa-publish --all-tools --repository-name `"$TsaRepositoryName`" --branch-name `"$TsaBranchName`" --build-number `"$BuildNumber`" --codebase-name `"$TsaCodebaseName`" --notification-alias `"$TsaNotificationEmail`" --codebase-admin `"$TsaCodebaseAdmin`" --instance-url `"$TsaInstanceUrl`" --project-name `"$TsaProjectName`" --area-path `"$TsaBugAreaPath`" --iteration-path `"$TsaIterationPath`" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel"
& $guardianCliLocation tsa-publish --all-tools --repository-name "$TsaRepositoryName" --branch-name "$TsaBranchName" --build-number "$BuildNumber" --onboard $True --codebase-name "$TsaCodebaseName" --notification-alias "$TsaNotificationEmail" --codebase-admin "$TsaCodebaseAdmin" --instance-url "$TsaInstanceUrl" --project-name "$TsaProjectName" --area-path "$TsaBugAreaPath" --iteration-path "$TsaIterationPath" --working-directory $workingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian tsa-publish failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
} else {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message 'Could not publish to TSA -- not all required values ($TsaBranchName, $BuildNumber) were specified.'
ExitWithExitCode 1
}
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
exit 1
}
@@ -0,0 +1,80 @@
param(
[Parameter(Mandatory=$true)][string] $InputPath, # Full path to directory where artifact packages are stored
[Parameter(Mandatory=$true)][string] $ExtractPath # Full path to directory where the packages will be extracted
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
# `tools.ps1` checks $ci to perform some actions. Since the post-build
# scripts don't necessarily execute in the same agent that run the
# build.ps1/sh script this variable isn't automatically set.
$ci = $true
$disableConfigureToolsetImport = $true
function ExtractArtifacts {
if (!(Test-Path $InputPath)) {
Write-Host "Input Path does not exist: $InputPath"
ExitWithExitCode 0
}
$Jobs = @()
Get-ChildItem "$InputPath\*.nupkg" |
ForEach-Object {
$Jobs += Start-Job -ScriptBlock $ExtractPackage -ArgumentList $_.FullName
}
foreach ($Job in $Jobs) {
Wait-Job -Id $Job.Id | Receive-Job
}
}
try {
. $PSScriptRoot\..\tools.ps1
$ExtractPackage = {
param(
[string] $PackagePath # Full path to a NuGet package
)
if (!(Test-Path $PackagePath)) {
Write-PipelineTelemetryError -Category 'Build' -Message "Input file does not exist: $PackagePath"
ExitWithExitCode 1
}
$RelevantExtensions = @('.dll', '.exe', '.pdb')
Write-Host -NoNewLine 'Extracting ' ([System.IO.Path]::GetFileName($PackagePath)) '...'
$PackageId = [System.IO.Path]::GetFileNameWithoutExtension($PackagePath)
$ExtractPath = Join-Path -Path $using:ExtractPath -ChildPath $PackageId
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Directory]::CreateDirectory($ExtractPath);
try {
$zip = [System.IO.Compression.ZipFile]::OpenRead($PackagePath)
$zip.Entries |
Where-Object {$RelevantExtensions -contains [System.IO.Path]::GetExtension($_.Name)} |
ForEach-Object {
$TargetFile = Join-Path -Path $ExtractPath -ChildPath $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, $TargetFile, $true)
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
finally {
$zip.Dispose()
}
}
Measure-Command { ExtractArtifacts }
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+63
View File
@@ -0,0 +1,63 @@
Param(
[string] $GuardianCliLocation,
[string] $Repository,
[string] $BranchName='master',
[string] $WorkingDirectory,
[string] $AzureDevOpsAccessToken,
[string] $GuardianLoggerLevel='Standard'
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
. $PSScriptRoot\..\tools.ps1
# Don't display the console progress UI - it's a huge perf hit
$ProgressPreference = 'SilentlyContinue'
# Construct basic auth from AzDO access token; construct URI to the repository's gdn folder stored in that repository; construct location of zip file
$encodedPat = [Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$AzureDevOpsAccessToken"))
$escapedRepository = [Uri]::EscapeDataString("/$Repository/$BranchName/.gdn")
$uri = "https://dev.azure.com/dnceng/internal/_apis/git/repositories/sdl-tool-cfg/Items?path=$escapedRepository&versionDescriptor[versionOptions]=0&`$format=zip&api-version=5.0-preview.1"
$zipFile = "$WorkingDirectory/gdn.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
$gdnFolder = (Join-Path $WorkingDirectory '.gdn')
try {
# We try to download the zip; if the request fails (e.g. the file doesn't exist), we catch it and init guardian instead
Write-Host 'Downloading gdn folder from internal config repostiory...'
Invoke-WebRequest -Headers @{ "Accept"="application/zip"; "Authorization"="Basic $encodedPat" } -Uri $uri -OutFile $zipFile
if (Test-Path $gdnFolder) {
# Remove the gdn folder if it exists (it shouldn't unless there's too much caching; this is just in case)
Remove-Item -Force -Recurse $gdnFolder
}
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipFile, $WorkingDirectory)
Write-Host $gdnFolder
ExitWithExitCode 0
} catch [System.Net.WebException] { } # Catch and ignore webexception
try {
# if the folder does not exist, we'll do a guardian init and push it to the remote repository
Write-Host 'Initializing Guardian...'
Write-Host "$GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel"
& $GuardianCliLocation init --working-directory $WorkingDirectory --logger-level $GuardianLoggerLevel
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian init failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We create the mainbaseline so it can be edited later
Write-Host "$GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline"
& $GuardianCliLocation baseline --working-directory $WorkingDirectory --name mainbaseline
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Build' -Message "Guardian baseline failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
& $(Join-Path $PSScriptRoot 'push-gdn.ps1') -Repository $Repository -BranchName $BranchName -GdnFolder $gdnFolder -AzureDevOpsAccessToken $AzureDevOpsAccessToken -PushReason 'Initialize gdn folder'
ExitWithExitCode 0
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+4
View File
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Guardian.Cli" version="0.7.2"/>
</packages>
+65
View File
@@ -0,0 +1,65 @@
Param(
[string] $Repository,
[string] $BranchName='master',
[string] $GdnFolder,
[string] $AzureDevOpsAccessToken,
[string] $PushReason
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
try {
. $PSScriptRoot\..\tools.ps1
# We create the temp directory where we'll store the sdl-config repository
$sdlDir = Join-Path $env:TEMP 'sdl'
if (Test-Path $sdlDir) {
Remove-Item -Force -Recurse $sdlDir
}
Write-Host "git clone https://dnceng:`$[email protected]/dnceng/internal/_git/sdl-tool-cfg $sdlDir"
git clone https://dnceng:$AzureDevOpsAccessToken@dev.azure.com/dnceng/internal/_git/sdl-tool-cfg $sdlDir
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git clone failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# We copy the .gdn folder from our local run into the git repository so it can be committed
$sdlRepositoryFolder = Join-Path (Join-Path (Join-Path $sdlDir $Repository) $BranchName) '.gdn'
if (Get-Command Robocopy) {
Robocopy /S $GdnFolder $sdlRepositoryFolder
} else {
rsync -r $GdnFolder $sdlRepositoryFolder
}
# cd to the sdl-config directory so we can run git there
Push-Location $sdlDir
# git add . --> git commit --> git push
Write-Host 'git add .'
git add .
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git add failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host "git -c user.email=`"[email protected]`" -c user.name=`"Dotnet Bot`" commit -m `"$PushReason for $Repository/$BranchName`""
git -c user.email="[email protected]" -c user.name="Dotnet Bot" commit -m "$PushReason for $Repository/$BranchName"
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git commit failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
Write-Host 'git push'
git push
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Git push failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
# Return to the original directory
Pop-Location
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+69
View File
@@ -0,0 +1,69 @@
Param(
[string] $GuardianCliLocation,
[string] $WorkingDirectory,
[string] $TargetDirectory,
[string] $GdnFolder,
[string[]] $ToolsList,
[string] $UpdateBaseline,
[string] $GuardianLoggerLevel='Standard',
[string[]] $CrScanAdditionalRunConfigParams,
[string[]] $PoliCheckAdditionalRunConfigParams
)
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
try {
. $PSScriptRoot\..\tools.ps1
# We store config files in the r directory of .gdn
Write-Host $ToolsList
$gdnConfigPath = Join-Path $GdnFolder 'r'
$ValidPath = Test-Path $GuardianCliLocation
if ($ValidPath -eq $False)
{
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Invalid Guardian CLI Location."
ExitWithExitCode 1
}
$configParam = @('--config')
foreach ($tool in $ToolsList) {
$gdnConfigFile = Join-Path $gdnConfigPath "$tool-configure.gdnconfig"
Write-Host $tool
# We have to manually configure tools that run on source to look at the source directory only
if ($tool -eq 'credscan') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" TargetDirectory < $TargetDirectory `" `" OutputType < pre `" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " TargetDirectory < $TargetDirectory " "OutputType < pre" $(If ($CrScanAdditionalRunConfigParams) {$CrScanAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
if ($tool -eq 'policheck') {
Write-Host "$GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args `" Target < $TargetDirectory `" $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})"
& $GuardianCliLocation configure --working-directory $WorkingDirectory --tool $tool --output-path $gdnConfigFile --logger-level $GuardianLoggerLevel --noninteractive --force --args " Target < $TargetDirectory " $(If ($PoliCheckAdditionalRunConfigParams) {$PoliCheckAdditionalRunConfigParams})
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian configure for $tool failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
$configParam+=$gdnConfigFile
}
Write-Host "$GuardianCliLocation run --working-directory $WorkingDirectory --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam"
& $GuardianCliLocation run --working-directory $WorkingDirectory --tool $tool --baseline mainbaseline --update-baseline $UpdateBaseline --logger-level $GuardianLoggerLevel $configParam
if ($LASTEXITCODE -ne 0) {
Write-PipelineTelemetryError -Force -Category 'Sdl' -Message "Guardian run for $ToolsList using $configParam failed with exit code $LASTEXITCODE."
ExitWithExitCode $LASTEXITCODE
}
}
catch {
Write-Host $_.ScriptStackTrace
Write-PipelineTelemetryError -Category 'Sdl' -Message $_
ExitWithExitCode 1
}
+89
View File
@@ -0,0 +1,89 @@
parameters:
enable: 'false' # Whether the SDL validation job should execute or not
overrideParameters: '' # Optional: to override values for parameters.
additionalParameters: '' # Optional: parameters that need user specific values eg: '-SourceToolsList @("abc","def") -ArtifactToolsList @("ghi","jkl")'
# There is some sort of bug (has been reported) in Azure DevOps where if this parameter is named
# 'continueOnError', the parameter value is not correctly picked up.
# This can also be remedied by the caller (post-build.yml) if it does not use a nested parameter
sdlContinueOnError: false # optional: determines whether to continue the build if the step errors;
dependsOn: '' # Optional: dependencies of the job
artifactNames: '' # Optional: patterns supplied to DownloadBuildArtifacts
# Usage:
# artifactNames:
# - 'BlobArtifacts'
# - 'Artifacts_Windows_NT_Release'
jobs:
- job: Run_SDL
dependsOn: ${{ parameters.dependsOn }}
displayName: Run SDL tool
condition: eq( ${{ parameters.enable }}, 'true')
variables:
- group: DotNet-VSTS-Bot
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
name: Hosted VS2017
steps:
- checkout: self
clean: true
- ${{ if ne(parameters.artifactNames, '') }}:
- ${{ each artifactName in parameters.artifactNames }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: ${{ artifactName }}
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- ${{ if eq(parameters.artifactNames, '') }}:
- task: DownloadBuildArtifacts@0
displayName: Download Build Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: specific files
itemPattern: "**"
downloadPath: $(Build.ArtifactStagingDirectory)\artifacts
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\BlobArtifacts
displayName: Extract Blob Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- powershell: eng/common/sdl/extract-artifact-packages.ps1
-InputPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
-ExtractPath $(Build.ArtifactStagingDirectory)\artifacts\PackageArtifacts
displayName: Extract Package Artifacts
continueOnError: ${{ parameters.sdlContinueOnError }}
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
- task: NuGetCommand@2
displayName: 'Install Guardian'
inputs:
restoreSolution: $(Build.SourcesDirectory)\eng\common\sdl\packages.config
feedsToUse: config
nugetConfigPath: $(Build.SourcesDirectory)\eng\common\sdl\NuGet.config
externalFeedCredentials: GuardianConnect
restoreDirectory: $(Build.SourcesDirectory)\.packages
- ${{ if ne(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1 ${{ parameters.overrideParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
- ${{ if eq(parameters.overrideParameters, '') }}:
- powershell: eng/common/sdl/execute-all-sdl-tools.ps1
-GuardianPackageName Microsoft.Guardian.Cli.0.7.2
-NugetPackageDirectory $(Build.SourcesDirectory)\.packages
-AzureDevOpsAccessToken $(dn-bot-dotnet-build-rw-code-rw)
${{ parameters.additionalParameters }}
displayName: Execute SDL
continueOnError: ${{ parameters.sdlContinueOnError }}
+75 -64
View File
@@ -1,64 +1,33 @@
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
parameters:
# Job schema parameters - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
cancelTimeoutInMinutes: ''
condition: ''
continueOnError: false
container: ''
continueOnError: false
dependsOn: ''
displayName: ''
steps: []
pool: ''
steps: []
strategy: ''
timeoutInMinutes: ''
variables: []
workspace: ''
# Job base template specific parameters
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
# See schema documentation - https://github.com/dotnet/arcade/blob/master/Documentation/AzureDevOps/TemplateSchema.md
artifacts: ''
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
enableTelemetry: false
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: define the helix type for telemetry (example: 'build/product/')
helixType: ''
# Required: name of the job
enablePublishUsingPipelines: false
name: ''
# Optional: should run as a public build even in the internal project
# if 'true', the build won't run any of the internal only steps, even if it is running in non-public projects.
preSteps: []
runAsPublic: false
# Internal resources (telemetry, microbuild) can only be accessed from non-public projects,
# and some (Microbuild) should only be applied to non-PR cases for internal builds.
jobs:
- job: ${{ parameters.name }}
@@ -90,7 +59,7 @@ jobs:
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
- ${{ if ne(parameters.enableTelemetry, 'false') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- ${{ each variable in parameters.variables }}:
@@ -122,21 +91,12 @@ jobs:
workspace: ${{ parameters.workspace }}
steps:
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendStartTelemetry@0
displayName: 'Send Helix Start Telemetry'
inputs:
helixRepo: ${{ parameters.helixRepo }}
${{ if ne(parameters.helixType, '') }}:
helixType: ${{ parameters.helixType }}
buildConfig: $(_BuildConfig)
runAsPublic: ${{ parameters.runAsPublic }}
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if ne(parameters.preSteps, '') }}:
- ${{ each preStep in parameters.preSteps }}:
- ${{ preStep }}
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- task: MicroBuildSigningPlugin@2
displayName: Install MicroBuild plugin
inputs:
@@ -148,6 +108,16 @@ jobs:
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- task: NuGetAuthenticate@0
- ${{ if or(eq(parameters.artifacts.download, 'true'), ne(parameters.artifacts.download, '')) }}:
- task: DownloadPipelineArtifact@2
inputs:
buildType: current
artifactName: ${{ coalesce(parameters.artifacts.download.name, 'Artifacts_$(Agent.OS)_$(_BuildConfig)') }}
targetPath: ${{ coalesce(parameters.artifacts.download.path, 'artifacts') }}
itemPattern: ${{ coalesce(parameters.artifacts.download.pattern, '**') }}
- ${{ each step in parameters.steps }}:
- ${{ step }}
@@ -160,20 +130,60 @@ jobs:
env:
TeamName: $(_TeamName)
- ${{ if eq(parameters.enableTelemetry, 'true') }}:
# Telemetry tasks are built from https://github.com/dotnet/arcade-extensions
- task: sendEndTelemetry@0
displayName: 'Send Helix End Telemetry'
continueOnError: ${{ parameters.continueOnError }}
condition: always()
- ${{ if ne(parameters.artifacts.publish, '') }}:
- ${{ if or(eq(parameters.artifacts.publish.artifacts, 'true'), ne(parameters.artifacts.publish.artifacts, '')) }}:
- task: CopyFiles@2
displayName: Gather binaries for publish to artifacts
inputs:
SourceFolder: 'artifacts/bin'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/bin'
- task: CopyFiles@2
displayName: Gather packages for publish to artifacts
inputs:
SourceFolder: 'artifacts/packages'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/packages'
- task: PublishBuildArtifacts@1
displayName: Publish pipeline artifacts
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts'
PublishLocation: Container
ArtifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.logs, 'true'), ne(parameters.artifacts.publish.logs, '')) }}:
- publish: artifacts/log
artifact: ${{ coalesce(parameters.artifacts.publish.logs.name, 'Logs_Build_$(Agent.Os)_$(_BuildConfig)') }}
displayName: Publish logs
continueOnError: true
condition: always()
- ${{ if or(eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- ${{ if and(ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
SourceFolder: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)/AssetManifest'
TargetFolder: '$(Build.ArtifactStagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/AssetManifests'
PublishLocation: Container
ArtifactName: AssetManifests
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- ${{ if ne(parameters.enablePublishBuildArtifacts, 'false') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_$(Agent.JobName)
ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
continueOnError: true
condition: always()
@@ -187,7 +197,7 @@ jobs:
continueOnError: true
condition: always()
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(variables['_PublishUsingPipelines'], 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), ne(parameters.enablePublishUsingPipelines, 'true'), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: CopyFiles@2
displayName: Gather Asset Manifests
inputs:
@@ -195,6 +205,7 @@ jobs:
TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), eq(variables['_DotNetPublishToBlobFeed'], 'true'))
- task: PublishBuildArtifacts@1
displayName: Push Asset Manifests
inputs:
+95
View File
@@ -0,0 +1,95 @@
parameters:
steps: [] # optional -- any additional steps that need to happen before pulling down the performance repo and sending the performance benchmarks to helix (ie building your repo)
variables: [] # optional -- list of additional variables to send to the template
jobName: '' # required -- job name
displayName: '' # optional -- display name for the job. Will use jobName if not passed
pool: '' # required -- name of the Build pool
container: '' # required -- name of the container
osGroup: '' # required -- operating system for the job
extraSetupParameters: '' # optional -- extra arguments to pass to the setup script
frameworks: ['netcoreapp3.0'] # optional -- list of frameworks to run against
continueOnError: 'false' # optional -- determines whether to continue the build if the step errors
dependsOn: '' # optional -- dependencies of the job
timeoutInMinutes: 320 # optional -- timeout for the job
enableTelemetry: false # optional -- enable for telemetry
jobs:
- template: ../jobs/jobs.yml
parameters:
dependsOn: ${{ parameters.dependsOn }}
enableTelemetry: ${{ parameters.enableTelemetry }}
enablePublishBuildArtifacts: true
continueOnError: ${{ parameters.continueOnError }}
jobs:
- job: '${{ parameters.jobName }}'
${{ if ne(parameters.displayName, '') }}:
displayName: '${{ parameters.displayName }}'
${{ if eq(parameters.displayName, '') }}:
displayName: '${{ parameters.jobName }}'
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
variables:
- ${{ each variable in parameters.variables }}:
- ${{ if ne(variable.name, '') }}:
- name: ${{ variable.name }}
value: ${{ variable.value }}
- ${{ if ne(variable.group, '') }}:
- group: ${{ variable.group }}
- IsInternal: ''
- HelixApiAccessToken: ''
- HelixPreCommand: ''
- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq( parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'set "PERFLAB_UPLOAD_TOKEN=$(PerfCommandUploadToken)"'
- IsInternal: -Internal
- ${{ if ne(parameters.osGroup, 'Windows_NT') }}:
- HelixPreCommand: 'export PERFLAB_UPLOAD_TOKEN="$(PerfCommandUploadTokenLinux)"'
- IsInternal: --internal
- group: DotNet-HelixApi-Access
- group: dotnet-benchview
workspace:
clean: all
pool:
${{ parameters.pool }}
container: ${{ parameters.container }}
strategy:
matrix:
${{ each framework in parameters.frameworks }}:
${{ framework }}:
_Framework: ${{ framework }}
steps:
- checkout: self
clean: true
# Run all of the steps to setup repo
- ${{ each step in parameters.steps }}:
- ${{ step }}
- powershell: $(Build.SourcesDirectory)\eng\common\performance\performance-setup.ps1 $(IsInternal) -Framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Windows)
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Build.SourcesDirectory)/eng/common/performance/performance-setup.sh $(IsInternal) --framework $(_Framework) ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Unix)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments)
displayName: Run ci setup script
# Run perf testing in helix
- template: /eng/common/templates/steps/perf-send-to-helix.yml
parameters:
HelixSource: '$(HelixSourcePrefix)/$(Build.Repository.Name)/$(Build.SourceBranch)' # sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'test/performance/$(Kind)/$(_Framework)/$(Architecture)'
HelixAccessToken: $(HelixApiAccessToken)
HelixTargetQueues: $(Queue)
HelixPreCommands: $(HelixPreCommand)
Creator: $(Creator)
WorkItemTimeout: 4:00 # 4 hours
WorkItemDirectory: '$(WorkItemDirectory)' # WorkItemDirectory can not be empty, so we send it some docs to keep it happy
CorrelationPayloadDirectory: '$(PayloadDirectory)' # it gets checked out to a folder with shorter path than WorkItemDirectory so we can avoid file name too long exceptions
@@ -37,6 +37,12 @@ jobs:
- name: _BuildConfig
value: ${{ parameters.configuration }}
- group: Publish-Build-Assets
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
steps:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
@@ -47,6 +53,10 @@ jobs:
downloadPath: '$(Build.StagingDirectory)/Download'
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: NuGetAuthenticate@0
- task: PowerShell@2
displayName: Publish Build Assets
inputs:
@@ -59,6 +69,23 @@ jobs:
/p:Configuration=$(_BuildConfig)
condition: ${{ parameters.condition }}
continueOnError: ${{ parameters.continueOnError }}
- task: powershell@2
displayName: Create ReleaseConfigs Artifact
inputs:
targetType: inline
script: |
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(BARBuildId)
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value "$(DefaultChannels)"
Add-Content -Path "$(Build.StagingDirectory)/ReleaseConfigs.txt" -Value $(IsStableBuild)
- task: PublishBuildArtifacts@1
displayName: Publish ReleaseConfigs Artifact
inputs:
PathtoPublish: '$(Build.StagingDirectory)/ReleaseConfigs.txt'
PublishLocation: Container
ArtifactName: ReleaseConfigs
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
+28 -46
View File
@@ -1,19 +1,10 @@
parameters:
# Optional: 'true' if failures in job.yml job should not fail the job
# See schema documentation in /Documentation/AzureDevOps/TemplateSchema.md
continueOnError: false
# Optional: Enable installing Microbuild plugin
# if 'true', these "variables" must be specified in the variables object or as part of the queue matrix
# _TeamName - the name of your team
# _SignType - 'test' or 'real'
enableMicrobuild: false
# Optional: Include PublishBuildArtifacts task
enablePublishBuildArtifacts: false
# Optional: Enable publishing to the build asset registry
enablePublishBuildAssets: false
# Optional: Enable publishing using release pipelines
enablePublishUsingPipelines: false
@@ -23,19 +14,9 @@ parameters:
# Optional: Include toolset dependencies in the generated graph files
includeToolset: false
# Optional: Include PublishTestResults task
enablePublishTestResults: false
# Optional: enable sending telemetry
# if enabled then the 'helixRepo' parameter should also be specified
enableTelemetry: false
# Required: A collection of jobs to run - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#job
jobs: []
# Optional: define the helix repo for telemetry (example: 'dotnet/arcade')
helixRepo: ''
# Optional: Override automatically derived dependsOn value for "publish build assets" job
publishBuildAssetsDependsOn: ''
@@ -62,29 +43,30 @@ jobs:
name: ${{ job.job }}
- ${{ if and(eq(parameters.enablePublishBuildAssets, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if and(eq(parameters.graphFileGeneration.enabled, true), eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
- template: ../job/publish-build-assets.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
dependsOn:
- ${{ if ne(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.publishBuildAssetsDependsOn }}:
- ${{ job.job }}
- ${{ if eq(parameters.publishBuildAssetsDependsOn, '') }}:
- ${{ each job in parameters.jobs }}:
- ${{ job.job }}
pool:
vmImage: vs2017-win2016
runAsPublic: ${{ parameters.runAsPublic }}
publishUsingPipelines: ${{ parameters.enablePublishUsingPipelines }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
- ${{ if eq(parameters.graphFileGeneration.enabled, true) }}:
- template: ../job/generate-graph-files.yml
parameters:
continueOnError: ${{ parameters.continueOnError }}
includeToolset: ${{ parameters.graphFileGeneration.includeToolset }}
dependsOn:
- Asset_Registry_Publish
pool:
vmImage: vs2017-win2016
@@ -0,0 +1,172 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
/p:PublishToMSDL=false
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=true
/p:ChecksumsTargetStaticFeed=$(InternalChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(InternalChecksumsBlobFeedKey)
/p:InstallersTargetStaticFeed=$(InternalInstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(InternalInstallersBlobFeedKey)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:PublishToMSDL=false
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -0,0 +1,178 @@
parameters:
artifactsPublishingAdditionalParameters: ''
dependsOn:
- Validate
publishInstallersAndChecksums: false
symbolPublishingAdditionalParameters: ''
stageName: ''
channelName: ''
channelId: ''
transportFeed: ''
shippingFeed: ''
symbolsFeed: ''
# If the channel name is empty, no links will be generated
akaMSChannelName: ''
stages:
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- template: ../common-variables.yml
displayName: ${{ parameters.channelName }} Publishing
jobs:
- template: ../setup-maestro-vars.yml
- job: publish_symbols
displayName: Symbol Publishing
dependsOn: setupMaestroVars
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
variables:
- group: DotNet-Symbol-Server-Pats
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PdbArtifacts/**
BlobArtifacts/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishToSymbolServers -restore -msbuildEngine dotnet
/p:DotNetSymbolServerTokenMsdl=$(microsoft-symbol-server-pat)
/p:DotNetSymbolServerTokenSymWeb=$(symweb-symbol-server-pat)
/p:PDBArtifactsDirectory='$(Build.ArtifactStagingDirectory)/PDBArtifacts/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:SymbolPublishingExclusionsFile='$(Build.SourcesDirectory)/eng/SymbolPublishingExclusionsFile.txt'
/p:Configuration=Release
${{ parameters.symbolPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'SymbolPublishing'
- job: publish_assets
displayName: Publish Assets
dependsOn: setupMaestroVars
timeoutInMinutes: 120
variables:
- name: BARBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.BARBuildId'] ]
- name: IsStableBuild
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.IsStableBuild'] ]
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
- name: ArtifactsCategory
value: ${{ coalesce(variables._DotNetArtifactsCategory, '.NETCore') }}
condition: or(contains(dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'], format('[{0}]', ${{ parameters.channelId }} )), eq(dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'], ${{ parameters.channelId }}))
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Build Assets
continueOnError: true
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
downloadType: 'specific'
itemPattern: |
PackageArtifacts/**
BlobArtifacts/**
AssetManifests/**
downloadPath: '$(Build.ArtifactStagingDirectory)'
- task: NuGetToolInstaller@1
displayName: 'Install NuGet.exe'
# This is necessary whenever we want to publish/restore to an AzDO private feed
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Publish Assets
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task PublishArtifactsInManifest -restore -msbuildEngine dotnet
/p:ArtifactsCategory=$(ArtifactsCategory)
/p:IsStableBuild=$(IsStableBuild)
/p:IsInternalBuild=$(IsInternalBuild)
/p:RepositoryName=$(Build.Repository.Name)
/p:CommitSha=$(Build.SourceVersion)
/p:NugetPath=$(NuGetExeToolPath)
/p:AzdoTargetFeedPAT='$(dn-bot-dnceng-universal-packages-rw)'
/p:AzureStorageTargetFeedPAT='$(dotnetfeed-storage-access-key-1)'
/p:BARBuildId=$(BARBuildId)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(MaestroApiAccessToken)'
/p:ManifestsBasePath='$(Build.ArtifactStagingDirectory)/AssetManifests/'
/p:BlobBasePath='$(Build.ArtifactStagingDirectory)/BlobArtifacts/'
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts/'
/p:Configuration=Release
/p:PublishInstallersAndChecksums=${{ parameters.publishInstallersAndChecksums }}
/p:InstallersTargetStaticFeed=$(InstallersBlobFeedUrl)
/p:InstallersAzureAccountKey=$(dotnetcli-storage-key)
/p:ChecksumsTargetStaticFeed=$(ChecksumsBlobFeedUrl)
/p:ChecksumsAzureAccountKey=$(dotnetclichecksums-storage-key)
/p:AzureDevOpsStaticShippingFeed='${{ parameters.shippingFeed }}'
/p:AzureDevOpsStaticShippingFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticTransportFeed='${{ parameters.transportFeed }}'
/p:AzureDevOpsStaticTransportFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:AzureDevOpsStaticSymbolsFeed='${{ parameters.symbolsFeed }}'
/p:AzureDevOpsStaticSymbolsFeedKey='$(dn-bot-dnceng-artifact-feeds-rw)'
/p:LatestLinkShortUrlPrefix=dotnet/'${{ parameters.akaMSChannelName }}'
/p:AkaMSClientId=$(akams-client-id)
/p:AkaMSClientSecret=$(akams-client-secret)
${{ parameters.artifactsPublishingAdditionalParameters }}
- template: ../../steps/publish-logs.yml
parameters:
StageLabel: '${{ parameters.stageName }}'
JobLabel: 'AssetsPublishing'
- template: ../../steps/add-build-to-channel.yml
parameters:
ChannelId: ${{ parameters.channelId }}
@@ -0,0 +1,99 @@
variables:
- group: AzureDevOps-Artifact-Feeds-Pats
- group: DotNet-Blob-Feed
- group: DotNet-DotNetCli-Storage
- group: DotNet-MSRC-Storage
- group: Publish-Build-Assets
# .NET Core 3.1 Dev
- name: PublicDevRelease_31_Channel_Id
value: 128
# .NET Core 5 Dev
- name: NetCore_5_Dev_Channel_Id
value: 131
# .NET Eng - Validation
- name: Net_Eng_Validation_Channel_Id
value: 9
# .NET Eng - Latest
- name: Net_Eng_Latest_Channel_Id
value: 2
# .NET 3 Eng - Validation
- name: NET_3_Eng_Validation_Channel_Id
value: 390
# .NET 3 Eng
- name: NetCore_3_Tools_Channel_Id
value: 344
# .NET Core 3.0 Internal Servicing
- name: InternalServicing_30_Channel_Id
value: 184
# .NET Core 3.0 Release
- name: PublicRelease_30_Channel_Id
value: 19
# .NET Core 3.1 Release
- name: PublicRelease_31_Channel_Id
value: 129
# General Testing
- name: GeneralTesting_Channel_Id
value: 529
# .NET Core 3.1 Blazor Features
- name: NetCore_31_Blazor_Features_Channel_Id
value: 531
# .NET Core Experimental
- name: NetCore_Experimental_Channel_Id
value: 562
# Whether the build is internal or not
- name: IsInternalBuild
value: ${{ and(ne(variables['System.TeamProject'], 'public'), contains(variables['Build.SourceBranch'], 'internal')) }}
# Default Maestro++ API Endpoint and API Version
- name: MaestroApiEndPoint
value: "https://maestro-prod.westus2.cloudapp.azure.com"
- name: MaestroApiAccessToken
value: $(MaestroAccessToken)
- name: MaestroApiVersion
value: "2019-01-16"
- name: SourceLinkCLIVersion
value: 3.0.0
- name: SymbolToolVersion
value: 1.0.1
# Feed Configurations
# These should include the suffix "/index.json"
# Default locations for Installers and checksums
# Public Locations
- name: ChecksumsBlobFeedUrl
value: https://dotnetclichecksums.blob.core.windows.net/dotnet/index.json
- name: InstallersBlobFeedUrl
value: https://dotnetcli.blob.core.windows.net/dotnet/index.json
# Private Locations
- name: InternalChecksumsBlobFeedUrl
value: https://dotnetclichecksumsmsrc.blob.core.windows.net/dotnet/index.json
- name: InternalChecksumsBlobFeedKey
value: $(dotnetclichecksumsmsrc-storage-key)
- name: InternalInstallersBlobFeedUrl
value: https://dotnetclimsrc.blob.core.windows.net/dotnet/index.json
- name: InternalInstallersBlobFeedKey
value: $(dotnetclimsrc-access-key)
# Skip component governance and codesign validation for SDL. These jobs
# create no content.
- name: skipComponentGovernanceDetection
value: true
- name: runCodesignValidationInjection
value: false
@@ -0,0 +1,354 @@
parameters:
enableSourceLinkValidation: false
enableSigningValidation: true
enableSymbolValidation: false
enableNugetValidation: true
publishInstallersAndChecksums: false
SDLValidationParameters:
enable: false
continueOnError: false
params: ''
artifactNames: ''
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
symbolPublishingAdditionalParameters: ''
artifactsPublishingAdditionalParameters: ''
signingValidationAdditionalParameters: ''
# Which stages should finish execution before post-build stages start
validateDependsOn:
- build
publishDependsOn:
- Validate
# Channel ID's instantiated in this file.
# When adding a new channel implementation the call to `check-channel-consistency.ps1`
# needs to be updated with the new channel ID
NetEngLatestChannelId: 2
NetEngValidationChannelId: 9
NetCoreDev5ChannelId: 131
GeneralTestingChannelId: 529
NETCoreToolingDevChannelId: 548
NETCoreToolingReleaseChannelId: 549
NETInternalToolingChannelId: 551
NETCoreExperimentalChannelId: 562
NetEngServicesIntChannelId: 678
NetEngServicesProdChannelId: 679
Net5Preview1ChannelId: 737
Net5Preview2ChannelId: 738
stages:
- stage: Validate
dependsOn: ${{ parameters.validateDependsOn }}
displayName: Validate
variables:
- template: common-variables.yml
jobs:
- template: setup-maestro-vars.yml
- job:
displayName: Post-build Checks
dependsOn: setupMaestroVars
variables:
- name: InitialChannels
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.InitialChannels'] ]
- name: PromoteToMaestroChannelId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.PromoteToMaestroChannelId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell@2
displayName: Maestro Channels Consistency
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/check-channel-consistency.ps1
arguments: -PromoteToChannels "$(InitialChannels)[$(PromoteToMaestroChannelId)]"
-AvailableChannelIds ${{parameters.NetEngLatestChannelId}},${{parameters.NetEngValidationChannelId}},${{parameters.NetCoreDev5ChannelId}},${{parameters.GeneralTestingChannelId}},${{parameters.NETCoreToolingDevChannelId}},${{parameters.NETCoreToolingReleaseChannelId}},${{parameters.NETInternalToolingChannelId}},${{parameters.NETCoreExperimentalChannelId}},${{parameters.NetEngServicesIntChannelId}},${{parameters.NetEngServicesProdChannelId}},${{parameters.Net5Preview1ChannelId}},${{parameters.Net5Preview2ChannelId}}
- job:
displayName: NuGet Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableNugetValidation }}, 'true')
pool:
vmImage: 'windows-2019'
variables:
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/nuget-validation.ps1
arguments: -PackagesPath $(Build.ArtifactStagingDirectory)/PackageArtifacts/
-ToolDestinationPath $(Agent.BuildDirectory)/Extract/
- job:
displayName: Signing Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSigningValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Package Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
# otherwise it'll complain about accessing a private feed.
- task: NuGetAuthenticate@0
displayName: 'Authenticate to AzDO Feeds'
- task: PowerShell@2
displayName: Enable cross-org publishing
inputs:
filePath: eng\common\enable-cross-org-publishing.ps1
arguments: -token $(dn-bot-dnceng-artifact-feeds-rw)
- task: PowerShell@2
displayName: Validate
inputs:
filePath: eng\common\sdk-task.ps1
arguments: -task SigningValidation -restore -msbuildEngine dotnet
/p:PackageBasePath='$(Build.ArtifactStagingDirectory)/PackageArtifacts'
/p:SignCheckExclusionsFile='$(Build.SourcesDirectory)/eng/SignCheckExclusionsFile.txt'
${{ parameters.signingValidationAdditionalParameters }}
- template: ../steps/publish-logs.yml
parameters:
StageLabel: 'Validation'
JobLabel: 'Signing'
- job:
displayName: SourceLink Validation
dependsOn: setupMaestroVars
condition: eq( ${{ parameters.enableSourceLinkValidation }}, 'true')
variables:
- template: common-variables.yml
- name: AzDOProjectName
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOProjectName'] ]
- name: AzDOPipelineId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOPipelineId'] ]
- name: AzDOBuildId
value: $[ dependencies.setupMaestroVars.outputs['setReleaseVars.AzDOBuildId'] ]
pool:
vmImage: 'windows-2019'
steps:
- task: DownloadBuildArtifacts@0
displayName: Download Blob Artifacts
inputs:
buildType: specific
buildVersionToDownload: specific
project: $(AzDOProjectName)
pipeline: $(AzDOPipelineId)
buildId: $(AzDOBuildId)
artifactName: BlobArtifacts
- task: PowerShell@2
displayName: Validate
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/sourcelink-validation.ps1
arguments: -InputPath $(Build.ArtifactStagingDirectory)/BlobArtifacts/
-ExtractPath $(Agent.BuildDirectory)/Extract/
-GHRepoName $(Build.Repository.Name)
-GHCommit $(Build.SourceVersion)
-SourcelinkCliVersion $(SourceLinkCLIVersion)
continueOnError: true
- template: /eng/common/templates/job/execute-sdl.yml
parameters:
enable: ${{ parameters.SDLValidationParameters.enable }}
dependsOn: setupMaestroVars
additionalParameters: ${{ parameters.SDLValidationParameters.params }}
continueOnError: ${{ parameters.SDLValidationParameters.continueOnError }}
artifactNames: ${{ parameters.SDLValidationParameters.artifactNames }}
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NetCore_Dev5_Publish'
channelName: '.NET Core 5 Dev'
akaMSChannelName: 'net5/dev'
channelId: ${{ parameters.NetCoreDev5ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview1_Publish'
channelName: '.NET 5 Preview 1'
akaMSChannelName: 'net5/preview1'
channelId: ${{ parameters.Net5Preview1ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net5_Preview2_Publish'
channelName: '.NET 5 Preview 2'
akaMSChannelName: 'net5/preview2'
channelId: ${{ parameters.Net5Preview2ChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Latest_Publish'
channelName: '.NET Eng - Latest'
akaMSChannelName: 'eng/daily'
channelId: ${{ parameters.NetEngLatestChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Validation_Publish'
channelName: '.NET Eng - Validation'
akaMSChannelName: 'eng/validation'
channelId: ${{ parameters.NetEngValidationChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'General_Testing_Publish'
channelName: 'General Testing'
akaMSChannelName: 'generaltesting'
channelId: ${{ parameters.GeneralTestingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Dev_Publishing'
channelName: '.NET Core Tooling Dev'
channelId: ${{ parameters.NETCoreToolingDevChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Tooling_Release_Publishing'
channelName: '.NET Core Tooling Release'
channelId: ${{ parameters.NETCoreToolingReleaseChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-internal-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NET_Internal_Tooling_Publishing'
channelName: '.NET Internal Tooling'
channelId: ${{ parameters.NETInternalToolingChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet-tools-internal-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'NETCore_Experimental_Publishing'
channelName: '.NET Core Experimental'
channelId: ${{ parameters.NETCoreExperimentalChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Int_Publish'
channelName: '.NET Eng Services - Int'
channelId: ${{ parameters.NetEngServicesIntChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
- template: \eng\common\templates\post-build\channels\generic-public-channel.yml
parameters:
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
dependsOn: ${{ parameters.publishDependsOn }}
publishInstallersAndChecksums: ${{ parameters.publishInstallersAndChecksums }}
symbolPublishingAdditionalParameters: ${{ parameters.symbolPublishingAdditionalParameters }}
stageName: 'Net_Eng_Services_Prod_Publish'
channelName: '.NET Eng Services - Prod'
channelId: ${{ parameters.NetEngServicesProdChannelId }}
transportFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
shippingFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
symbolsFeed: 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng-symbols/nuget/v3/index.json'
@@ -0,0 +1,81 @@
jobs:
- job: setupMaestroVars
displayName: Setup Maestro Vars
variables:
- template: common-variables.yml
- name: BuildId
value: $[ coalesce(variables.BARBuildId, 0) ]
- name: PromoteToChannelId
value: $[ coalesce(variables.PromoteToMaestroChannelId, 0) ]
pool:
vmImage: 'windows-2019'
steps:
- checkout: none
- task: DownloadBuildArtifacts@0
displayName: Download Release Configs
condition: eq(variables.PromoteToChannelId, 0)
inputs:
buildType: current
artifactName: ReleaseConfigs
- task: PowerShell@2
name: setReleaseVars
displayName: Set Release Configs Vars
inputs:
targetType: inline
script: |
try {
if ($Env:PromoteToChannelId -eq 0) {
$Content = Get-Content $(Build.StagingDirectory)/ReleaseConfigs/ReleaseConfigs.txt
$BarId = $Content | Select -Index 0
$Channels = ""
$Content | Select -Index 1 | ForEach-Object { $Channels += "$_ ," }
$IsStableBuild = $Content | Select -Index 2
$AzureDevOpsProject = $Env:System_TeamProject
$AzureDevOpsBuildDefinitionId = $Env:System_DefinitionId
$AzureDevOpsBuildId = $Env:Build_BuildId
$PromoteToMaestroChannelId = 0
}
else {
$buildApiEndpoint = "${Env:MaestroApiEndPoint}/api/builds/${Env:BARBuildId}?api-version=${Env:MaestroApiVersion}"
$apiHeaders = New-Object 'System.Collections.Generic.Dictionary[[String],[String]]'
$apiHeaders.Add('Accept', 'application/json')
$apiHeaders.Add('Authorization',"Bearer ${Env:MAESTRO_API_TOKEN}")
$buildInfo = try { Invoke-WebRequest -Method Get -Uri $buildApiEndpoint -Headers $apiHeaders | ConvertFrom-Json } catch { Write-Host "Error: $_" }
$BarId = $Env:BARBuildId
$Channels = 'None'
#TODO: Fix this once this issue is done: https://github.com/dotnet/arcade/issues/3834
$IsStableBuild = 'False'
$AzureDevOpsProject = $buildInfo.azureDevOpsProject
$AzureDevOpsBuildDefinitionId = $buildInfo.azureDevOpsBuildDefinitionId
$AzureDevOpsBuildId = $buildInfo.azureDevOpsBuildId
$PromoteToMaestroChannelId = $Env:PromoteToMaestroChannelId
}
Write-Host "##vso[task.setvariable variable=BARBuildId;isOutput=true]$BarId"
Write-Host "##vso[task.setvariable variable=InitialChannels;isOutput=true]$Channels"
Write-Host "##vso[task.setvariable variable=IsStableBuild;isOutput=true]$IsStableBuild"
Write-Host "##vso[task.setvariable variable=AzDOProjectName;isOutput=true]$AzureDevOpsProject"
Write-Host "##vso[task.setvariable variable=AzDOPipelineId;isOutput=true]$AzureDevOpsBuildDefinitionId"
Write-Host "##vso[task.setvariable variable=AzDOBuildId;isOutput=true]$AzureDevOpsBuildId"
Write-Host "##vso[task.setvariable variable=PromoteToMaestroChannelId;isOutput=true]$PromoteToMaestroChannelId"
}
catch {
Write-Host $_
Write-Host $_.Exception
Write-Host $_.ScriptStackTrace
exit 1
}
env:
MAESTRO_API_TOKEN: $(MaestroApiAccessToken)
@@ -0,0 +1,13 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Triggering subscriptions
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/trigger-subscriptions.ps1
arguments: -SourceRepo $(Build.Repository.Uri)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -0,0 +1,13 @@
parameters:
ChannelId: 0
steps:
- task: PowerShell@2
displayName: Add Build to Channel
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/post-build/add-build-to-channel.ps1
arguments: -BuildId $(BARBuildId)
-ChannelId ${{ parameters.ChannelId }}
-MaestroApiAccessToken $(MaestroApiAccessToken)
-MaestroApiEndPoint $(MaestroApiEndPoint)
-MaestroApiVersion $(MaestroApiVersion)
@@ -0,0 +1,66 @@
# Please remember to update the documentation if you make changes to these parameters!
parameters:
HelixSource: 'pr/default' # required -- sources must start with pr/, official/, prodcon/, or agent/
HelixType: 'tests/default/' # required -- Helix telemetry which identifies what type of data this is; should include "test" for clarity and must end in '/'
HelixBuild: $(Build.BuildNumber) # required -- the build number Helix will use to identify this -- automatically set to the AzDO build number
HelixTargetQueues: '' # required -- semicolon delimited list of Helix queues to test on; see https://helix.dot.net/ for a list of queues
HelixAccessToken: '' # required -- access token to make Helix API requests; should be provided by the appropriate variable group
HelixPreCommands: '' # optional -- commands to run before Helix work item execution
HelixPostCommands: '' # optional -- commands to run after Helix work item execution
WorkItemDirectory: '' # optional -- a payload directory to zip up and send to Helix; requires WorkItemCommand; incompatible with XUnitProjects
CorrelationPayloadDirectory: '' # optional -- a directory to zip up and send to Helix as a correlation payload
IncludeDotNetCli: false # optional -- true will download a version of the .NET CLI onto the Helix machine as a correlation payload; requires DotNetCliPackageType and DotNetCliVersion
DotNetCliPackageType: '' # optional -- either 'sdk' or 'runtime'; determines whether the sdk or runtime will be sent to Helix; see https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
DotNetCliVersion: '' # optional -- version of the CLI to send to Helix; based on this: https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases.json
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Send job to Helix' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
continueOnError: false # optional -- determines whether to continue the build if the step errors; defaults to false
steps:
- powershell: $(Build.SourcesDirectory)\eng\common\msbuild.ps1 $(Build.SourcesDirectory)\eng\common\performance\perfhelixpublish.proj /restore /t:Test /bl:$(Build.SourcesDirectory)\artifacts\log\$env:BuildConfig\SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Windows)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $BUILD_SOURCESDIRECTORY/eng/common/msbuild.sh $BUILD_SOURCESDIRECTORY/eng/common/performance/perfhelixpublish.proj /restore /t:Test /bl:$BUILD_SOURCESDIRECTORY/artifacts/log/$BuildConfig/SendToHelix.binlog
displayName: ${{ parameters.DisplayNamePrefix }} (Unix)
env:
BuildConfig: $(_BuildConfig)
HelixSource: ${{ parameters.HelixSource }}
HelixType: ${{ parameters.HelixType }}
HelixBuild: ${{ parameters.HelixBuild }}
HelixTargetQueues: ${{ parameters.HelixTargetQueues }}
HelixAccessToken: ${{ parameters.HelixAccessToken }}
HelixPreCommands: ${{ parameters.HelixPreCommands }}
HelixPostCommands: ${{ parameters.HelixPostCommands }}
WorkItemDirectory: ${{ parameters.WorkItemDirectory }}
CorrelationPayloadDirectory: ${{ parameters.CorrelationPayloadDirectory }}
IncludeDotNetCli: ${{ parameters.IncludeDotNetCli }}
DotNetCliPackageType: ${{ parameters.DotNetCliPackageType }}
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
@@ -0,0 +1,23 @@
parameters:
StageLabel: ''
JobLabel: ''
steps:
- task: Powershell@2
displayName: Prepare Binlogs to Upload
inputs:
targetType: inline
script: |
New-Item -ItemType Directory $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
Move-Item -Path $(Build.SourcesDirectory)/artifacts/log/Debug/* $(Build.SourcesDirectory)/PostBuildLogs/${{parameters.StageLabel}}/${{parameters.JobLabel}}/
continueOnError: true
condition: always()
- task: PublishBuildArtifacts@1
displayName: Publish Logs
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/PostBuildLogs'
PublishLocation: Container
ArtifactName: PostBuildLogs
continueOnError: true
condition: always()
@@ -23,6 +23,7 @@ parameters:
EnableXUnitReporter: false # optional -- true enables XUnit result reporting to Mission Control
WaitForWorkItemCompletion: true # optional -- true will make the task wait until work items have been completed and fail the build if work items fail. False is "fire and forget."
IsExternal: false # [DEPRECATED] -- doesn't do anything, jobs are external if HelixAccessToken is empty and Creator is set
HelixBaseUri: 'https://helix.dot.net/' # optional -- sets the Helix API base URI (allows targeting int)
Creator: '' # optional -- if the build is external, use this to specify who is sending the job
DisplayNamePrefix: 'Run Tests' # optional -- rename the beginning of the displayName of the steps in AzDO
condition: succeeded() # optional -- condition for step to execute; defaults to succeeded()
@@ -55,6 +56,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, eq(variables['Agent.Os'], 'Windows_NT'))
@@ -85,6 +87,7 @@ steps:
DotNetCliVersion: ${{ parameters.DotNetCliVersion }}
EnableXUnitReporter: ${{ parameters.EnableXUnitReporter }}
WaitForWorkItemCompletion: ${{ parameters.WaitForWorkItemCompletion }}
HelixBaseUri: ${{ parameters.HelixBaseUri }}
Creator: ${{ parameters.Creator }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
condition: and(${{ parameters.condition }}, ne(variables['Agent.Os'], 'Windows_NT'))
+222 -158
View File
@@ -5,7 +5,7 @@
[bool]$ci = if (Test-Path variable:ci) { $ci } else { $false }
# Build configuration. Common values include 'Debug' and 'Release', but the repository may use other names.
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { "Debug" }
[string]$configuration = if (Test-Path variable:configuration) { $configuration } else { 'Debug' }
# Set to true to output binary log from msbuild. Note that emitting binary log slows down the build.
# Binary log must be enabled on CI.
@@ -24,7 +24,7 @@
[bool]$restore = if (Test-Path variable:restore) { $restore } else { $true }
# Adjusts msbuild verbosity level.
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { "minimal" }
[string]$verbosity = if (Test-Path variable:verbosity) { $verbosity } else { 'minimal' }
# Set to true to reuse msbuild nodes. Recommended to not reuse on CI.
[bool]$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { !$ci }
@@ -35,23 +35,29 @@
# Specifies which msbuild engine to use for build: 'vs', 'dotnet' or unspecified (determined based on presence of tools.vs in global.json).
[string]$msbuildEngine = if (Test-Path variable:msbuildEngine) { $msbuildEngine } else { $null }
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.ps1
[string]$dotnetInstallScriptVersion = if (Test-Path variable:dotnetInstallScriptVersion) { $dotnetInstallScriptVersion } else { 'v1' }
# True to use global NuGet cache instead of restoring packages to repository-local directory.
[bool]$useGlobalNuGetCache = if (Test-Path variable:useGlobalNuGetCache) { $useGlobalNuGetCache } else { !$ci }
# An array of names of processes to stop on script exit if prepareMachine is true.
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @("msbuild", "dotnet", "vbcscompiler") }
$processesToStopOnExit = if (Test-Path variable:processesToStopOnExit) { $processesToStopOnExit } else { @('msbuild', 'dotnet', 'vbcscompiler') }
$disableConfigureToolsetImport = if (Test-Path variable:disableConfigureToolsetImport) { $disableConfigureToolsetImport } else { $null }
set-strictmode -version 2.0
$ErrorActionPreference = "Stop"
$ErrorActionPreference = 'Stop'
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
function Create-Directory([string[]] $path) {
if (!(Test-Path $path)) {
New-Item -path $path -force -itemType "Directory" | Out-Null
New-Item -path $path -force -itemType 'Directory' | Out-Null
}
}
@@ -76,7 +82,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
$finished = $false
try {
while (-not $process.WaitForExit(100)) {
while (-not $process.WaitForExit(100)) {
# Non-blocking loop done to allow ctr-c interrupts
}
@@ -84,7 +90,7 @@ function Exec-Process([string]$command, [string]$commandArgs) {
return $global:LASTEXITCODE = $process.ExitCode
}
finally {
# If we didn't finish then an error occured or the user hit ctrl-c. Either
# If we didn't finish then an error occurred or the user hit ctrl-c. Either
# way kill the process
if (-not $finished) {
$process.Kill()
@@ -92,69 +98,10 @@ function Exec-Process([string]$command, [string]$commandArgs) {
}
}
function Write-PipelineTaskError {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Message,
[Parameter(Mandatory = $false)]
[string]$Type = 'error',
[string]$ErrCode,
[string]$SourcePath,
[string]$LineNumber,
[string]$ColumnNumber,
[switch]$AsOutput)
if(!$ci) {
if($Type -eq 'error') {
Write-Error $Message
return
}
elseif ($Type -eq 'warning') {
Write-Warning $Message
return
}
}
if(($Type -ne 'error') -and ($Type -ne 'warning')) {
Write-Host $Message
return
}
if(-not $PSBoundParameters.ContainsKey('Type')) {
$PSBoundParameters.Add('Type', 'error')
}
Write-LogIssue @PSBoundParameters
}
function Write-PipelineSetVariable {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Name,
[string]$Value,
[switch]$Secret,
[switch]$AsOutput)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'setvariable' -Data $Value -Properties @{
'variable' = $Name
'issecret' = $Secret
} -AsOutput:$AsOutput
}
}
function Write-PipelinePrependPath {
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[switch]$AsOutput)
if($ci) {
Write-LoggingCommand -Area 'task' -Event 'prependpath' -Data $Path -AsOutput:$AsOutput
}
}
function InitializeDotNetCli([bool]$install) {
# createSdkLocationFile parameter enables a file being generated under the toolset directory
# which writes the sdk's location into. This is only necessary for cmd --> powershell invocations
# as dot sourcing isn't possible.
function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
if (Test-Path variable:global:_DotNetInstallDir) {
return $global:_DotNetInstallDir
}
@@ -177,7 +124,7 @@ function InitializeDotNetCli([bool]$install) {
# Find the first path on %PATH% that contains the dotnet.exe
if ($useInstalledDotNetCli -and (-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -eq $null)) {
$dotnetCmd = Get-Command "dotnet.exe" -ErrorAction SilentlyContinue
$dotnetCmd = Get-Command 'dotnet.exe' -ErrorAction SilentlyContinue
if ($dotnetCmd -ne $null) {
$env:DOTNET_INSTALL_DIR = Split-Path $dotnetCmd.Path -Parent
}
@@ -190,13 +137,13 @@ function InitializeDotNetCli([bool]$install) {
if ((-not $globalJsonHasRuntimes) -and ($env:DOTNET_INSTALL_DIR -ne $null) -and (Test-Path(Join-Path $env:DOTNET_INSTALL_DIR "sdk\$dotnetSdkVersion"))) {
$dotnetRoot = $env:DOTNET_INSTALL_DIR
} else {
$dotnetRoot = Join-Path $RepoRoot ".dotnet"
$dotnetRoot = Join-Path $RepoRoot '.dotnet'
if (-not (Test-Path(Join-Path $dotnetRoot "sdk\$dotnetSdkVersion"))) {
if ($install) {
InstallDotNetSdk $dotnetRoot $dotnetSdkVersion
} else {
Write-PipelineTaskError "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to find dotnet with SDK version '$dotnetSdkVersion'"
ExitWithExitCode 1
}
}
@@ -204,13 +151,32 @@ function InitializeDotNetCli([bool]$install) {
$env:DOTNET_INSTALL_DIR = $dotnetRoot
}
# Creates a temporary file under the toolset dir.
# The following code block is protecting against concurrent access so that this function can
# be called in parallel.
if ($createSdkLocationFile) {
do {
$sdkCacheFileTemp = Join-Path $ToolsetDir $([System.IO.Path]::GetRandomFileName())
}
until (!(Test-Path $sdkCacheFileTemp))
Set-Content -Path $sdkCacheFileTemp -Value $dotnetRoot
try {
Rename-Item -Force -Path $sdkCacheFileTemp 'sdk.txt'
} catch {
# Somebody beat us
Remove-Item -Path $sdkCacheFileTemp
}
}
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
# It also ensures that VS msbuild will use the downloaded sdk targets.
$env:PATH = "$dotnetRoot;$env:PATH"
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
# Make Sure that our bootstrapped dotnet cli is available in future steps of the Azure Pipelines build
Write-PipelinePrependPath -Path $dotnetRoot
Write-PipelineSetVariable -Name 'DOTNET_MULTILEVEL_LOOKUP' -Value '0'
Write-PipelineSetVariable -Name 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE' -Value '1'
@@ -218,20 +184,54 @@ function InitializeDotNetCli([bool]$install) {
}
function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot "dotnet-install.ps1"
$installScript = Join-Path $dotnetRoot 'dotnet-install.ps1'
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
Invoke-WebRequest "https://dot.net/v1/dotnet-install.ps1" -OutFile $installScript
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
$maxRetries = 5
$retries = 1
$uri = "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1"
while($true) {
try {
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
break
}
catch {
Write-Host "Failed to download '$uri'"
Write-Error $_.Exception.Message -ErrorAction Continue
}
if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
throw "Unable to download file in $maxRetries attempts."
}
}
}
return $installScript
}
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = "") {
function InstallDotNetSdk([string] $dotnetRoot, [string] $version, [string] $architecture = '') {
InstallDotNet $dotnetRoot $version $architecture
}
function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $architecture = "", [string] $runtime = "", [bool] $skipNonVersionedFiles = $false) { $installScript = GetDotNetInstallScript $dotnetRoot
function InstallDotNet([string] $dotnetRoot,
[string] $version,
[string] $architecture = '',
[string] $runtime = '',
[bool] $skipNonVersionedFiles = $false,
[string] $runtimeSourceFeed = '',
[string] $runtimeSourceFeedKey = '') {
$installScript = GetDotNetInstallScript $dotnetRoot
$installParameters = @{
Version = $version
@@ -242,15 +242,37 @@ function InstallDotNet([string] $dotnetRoot, [string] $version, [string] $archit
if ($runtime) { $installParameters.Runtime = $runtime }
if ($skipNonVersionedFiles) { $installParameters.SkipNonVersionedFiles = $skipNonVersionedFiles }
& $installScript @installParameters
if ($lastExitCode -ne 0) {
Write-PipelineTaskError -Message "Failed to install dotnet cli (exit code '$lastExitCode')."
ExitWithExitCode $lastExitCode
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from public location."
# Only the runtime can be installed from a custom [private] location.
if ($runtime -and ($runtimeSourceFeed -or $runtimeSourceFeedKey)) {
if ($runtimeSourceFeed) { $installParameters.AzureFeed = $runtimeSourceFeed }
if ($runtimeSourceFeedKey) {
$decodedBytes = [System.Convert]::FromBase64String($runtimeSourceFeedKey)
$decodedString = [System.Text.Encoding]::UTF8.GetString($decodedBytes)
$installParameters.FeedCredential = $decodedString
}
try {
& $installScript @installParameters
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Failed to install dotnet runtime '$runtime' from custom location '$runtimeSourceFeed'."
ExitWithExitCode 1
}
} else {
ExitWithExitCode 1
}
}
}
#
# Locates Visual Studio MSBuild installation.
# Locates Visual Studio MSBuild installation.
# The preference order for MSBuild to use is as follows:
#
# 1. MSBuild from an active VS command prompt
@@ -266,16 +288,16 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { "15.9" }
$vsMinVersion = [Version]::new($vsMinVersionStr)
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { '15.9' }
$vsMinVersion = [Version]::new($vsMinVersionStr)
# Try msbuild command available in the environment.
if ($env:VSINSTALLDIR -ne $null) {
$msbuildCmd = Get-Command "msbuild.exe" -ErrorAction SilentlyContinue
$msbuildCmd = Get-Command 'msbuild.exe' -ErrorAction SilentlyContinue
if ($msbuildCmd -ne $null) {
# Workaround for https://github.com/dotnet/roslyn/issues/35793
# Due to this issue $msbuildCmd.Version returns 0.0.0.0 for msbuild.exe 16.2+
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split(@('-', '+'))[0])
$msbuildVersion = [Version]::new((Get-Item $msbuildCmd.Path).VersionInfo.ProductVersion.Split([char[]]@('-', '+'))[0])
if ($msbuildVersion -ge $vsMinVersion) {
return $global:_MSBuildExe = $msbuildCmd.Path
@@ -295,7 +317,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
InitializeVisualStudioEnvironmentVariables $vsInstallDir $vsMajorVersion
} else {
if (Get-Member -InputObject $GlobalJson.tools -Name "xcopy-msbuild") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'xcopy-msbuild') {
$xcopyMSBuildVersion = $GlobalJson.tools.'xcopy-msbuild'
$vsMajorVersion = $xcopyMSBuildVersion.Split('.')[0]
} else {
@@ -303,9 +325,12 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
$xcopyMSBuildVersion = "$vsMajorVersion.$($vsMinVersion.Minor).0-alpha"
}
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
$vsInstallDir = $null
if ($xcopyMSBuildVersion.Trim() -ine "none") {
$vsInstallDir = InitializeXCopyMSBuild $xcopyMSBuildVersion $install
}
if ($vsInstallDir -eq $null) {
throw "Unable to find Visual Studio that has required version and components installed"
throw 'Unable to find Visual Studio that has required version and components installed'
}
}
@@ -316,7 +341,7 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =
function InitializeVisualStudioEnvironmentVariables([string] $vsInstallDir, [string] $vsMajorVersion) {
$env:VSINSTALLDIR = $vsInstallDir
Set-Item "env:VS$($vsMajorVersion)0COMNTOOLS" (Join-Path $vsInstallDir "Common7\Tools\")
$vsSdkInstallDir = Join-Path $vsInstallDir "VSSDK\"
if (Test-Path $vsSdkInstallDir) {
Set-Item "env:VSSDK$($vsMajorVersion)0Install" $vsSdkInstallDir
@@ -329,7 +354,7 @@ function InstallXCopyMSBuild([string]$packageVersion) {
}
function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
$packageName = "RoslynTools.MSBuild"
$packageName = 'RoslynTools.MSBuild'
$packageDir = Join-Path $ToolsDir "msbuild\$packageVersion"
$packagePath = Join-Path $packageDir "$packageName.$packageVersion.nupkg"
@@ -340,55 +365,61 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
Create-Directory $packageDir
Write-Host "Downloading $packageName $packageVersion"
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://dotnet.myget.org/F/roslyn-tools/api/v2/package/$packageName/$packageVersion/" -OutFile $packagePath
Unzip $packagePath $packageDir
}
return Join-Path $packageDir "tools"
return Join-Path $packageDir 'tools'
}
#
# Locates Visual Studio instance that meets the minimal requirements specified by tools.vs object in global.json.
#
# The following properties of tools.vs are recognized:
# "version": "{major}.{minor}"
# "version": "{major}.{minor}"
# Two part minimal VS version, e.g. "15.9", "16.0", etc.
# "components": ["componentId1", "componentId2", ...]
# "components": ["componentId1", "componentId2", ...]
# Array of ids of workload components that must be available in the VS instance.
# See e.g. https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-enterprise?view=vs-2017
#
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# Returns JSON describing the located VS instance (same format as returned by vswhere),
# or $null if no instance meeting the requirements is found on the machine.
#
function LocateVisualStudio([object]$vsRequirements = $null){
if (Get-Member -InputObject $GlobalJson.tools -Name "vswhere") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'vswhere') {
$vswhereVersion = $GlobalJson.tools.vswhere
} else {
$vswhereVersion = "2.5.2"
$vswhereVersion = '2.5.2'
}
$vsWhereDir = Join-Path $ToolsDir "vswhere\$vswhereVersion"
$vsWhereExe = Join-Path $vsWhereDir "vswhere.exe"
$vsWhereExe = Join-Path $vsWhereDir 'vswhere.exe'
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host "Downloading vswhere"
Invoke-WebRequest "https://github.com/Microsoft/vswhere/releases/download/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
Write-Host 'Downloading vswhere'
try {
Invoke-WebRequest "https://netcorenativeassets.blob.core.windows.net/resource-packages/external/windows/vswhere/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
}
}
if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$args = @("-latest", "-prerelease", "-format", "json", "-requires", "Microsoft.Component.MSBuild")
if (Get-Member -InputObject $vsRequirements -Name "version") {
$args += "-version"
$args = @('-latest', '-prerelease', '-format', 'json', '-requires', 'Microsoft.Component.MSBuild', '-products', '*')
if (Get-Member -InputObject $vsRequirements -Name 'version') {
$args += '-version'
$args += $vsRequirements.version
}
if (Get-Member -InputObject $vsRequirements -Name "components") {
if (Get-Member -InputObject $vsRequirements -Name 'components') {
foreach ($component in $vsRequirements.components) {
$args += "-requires"
$args += '-requires'
$args += $component
}
}
}
$vsInfo =& $vsWhereExe $args | ConvertFrom-Json
@@ -412,28 +443,27 @@ function InitializeBuildTool() {
# Initialize dotnet cli if listed in 'tools'
$dotnetRoot = $null
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
$dotnetRoot = InitializeDotNetCli -install:$restore
}
if ($msbuildEngine -eq "dotnet") {
if ($msbuildEngine -eq 'dotnet') {
if (!$dotnetRoot) {
Write-PipelineTaskError "/global.json must specify 'tools.dotnet'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "/global.json must specify 'tools.dotnet'."
ExitWithExitCode 1
}
$buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }
$buildTool = @{ Path = Join-Path $dotnetRoot 'dotnet.exe'; Command = 'msbuild'; Tool = 'dotnet'; Framework = 'netcoreapp2.1' }
} elseif ($msbuildEngine -eq "vs") {
try {
$msbuildPath = InitializeVisualStudioMSBuild -install:$restore
} catch {
Write-PipelineTaskError $_
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
ExitWithExitCode 1
}
$buildTool = @{ Path = $msbuildPath; Command = ""; Tool = "vs"; Framework = "net472" }
} else {
Write-PipelineTaskError "Unexpected value of -msbuildEngine: '$msbuildEngine'."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unexpected value of -msbuildEngine: '$msbuildEngine'."
ExitWithExitCode 1
}
@@ -442,15 +472,15 @@ function InitializeBuildTool() {
function GetDefaultMSBuildEngine() {
# Presence of tools.vs indicates the repo needs to build using VS msbuild on Windows.
if (Get-Member -InputObject $GlobalJson.tools -Name "vs") {
return "vs"
}
if (Get-Member -InputObject $GlobalJson.tools -Name "dotnet") {
return "dotnet"
if (Get-Member -InputObject $GlobalJson.tools -Name 'vs') {
return 'vs'
}
Write-PipelineTaskError "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
if (Get-Member -InputObject $GlobalJson.tools -Name 'dotnet') {
return 'dotnet'
}
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "-msbuildEngine must be specified, or /global.json must specify 'tools.dotnet' or 'tools.vs'."
ExitWithExitCode 1
}
@@ -459,9 +489,9 @@ function GetNuGetPackageCachePath() {
# Use local cache on CI to ensure deterministic build,
# use global cache in dev builds to avoid cost of downloading packages.
if ($useGlobalNuGetCache) {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile ".nuget\packages"
$env:NUGET_PACKAGES = Join-Path $env:UserProfile '.nuget\packages'
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot ".packages"
$env:NUGET_PACKAGES = Join-Path $RepoRoot '.packages'
}
}
@@ -474,12 +504,14 @@ function GetSdkTaskProject([string]$taskName) {
}
function InitializeNativeTools() {
if (Get-Member -InputObject $GlobalJson -Name "native-tools") {
$nativeArgs=""
if (-Not (Test-Path variable:DisableNativeToolsetInstalls) -And (Get-Member -InputObject $GlobalJson -Name "native-tools")) {
$nativeArgs= @{}
if ($ci) {
$nativeArgs = "-InstallDirectory $ToolsDir"
$nativeArgs = @{
InstallDirectory = "$ToolsDir"
}
}
Invoke-Expression "& `"$PSScriptRoot/init-tools-native.ps1`" $nativeArgs"
& "$PSScriptRoot/init-tools-native.ps1" @nativeArgs
}
}
@@ -501,14 +533,14 @@ function InitializeToolset() {
}
if (-not $restore) {
Write-PipelineTaskError "Toolset version $toolsetVersion has not been restored."
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Toolset version $toolsetVersion has not been restored."
ExitWithExitCode 1
}
$buildTool = InitializeBuildTool
$proj = Join-Path $ToolsetDir "restore.proj"
$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "ToolsetRestore.binlog") } else { "" }
$proj = Join-Path $ToolsetDir 'restore.proj'
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'ToolsetRestore.binlog') } else { '' }
'<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' | Set-Content $proj
@@ -530,7 +562,7 @@ function ExitWithExitCode([int] $exitCode) {
}
function Stop-Processes() {
Write-Host "Killing running build processes..."
Write-Host 'Killing running build processes...'
foreach ($processName in $processesToStopOnExit) {
Get-Process -Name $processName -ErrorAction SilentlyContinue | Stop-Process
}
@@ -544,9 +576,21 @@ function Stop-Processes() {
function MSBuild() {
if ($pipelinesLog) {
$buildTool = InitializeBuildTool
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if ($ci -and $buildTool.Tool -eq 'dotnet') {
dotnet nuget locals http-cache -c
$env:NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS = 20
$env:NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS = 20
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS' -Value '20'
Write-PipelineSetVariable -Name 'NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS' -Value '20'
}
$toolsetBuildProject = InitializeToolset
$path = Split-Path -parent $toolsetBuildProject
$path = Join-Path $path (Join-Path $buildTool.Framework "Microsoft.DotNet.Arcade.Sdk.dll")
$path = Join-Path $path (Join-Path $buildTool.Framework 'Microsoft.DotNet.Arcade.Sdk.dll')
$args += "/logger:$path"
}
@@ -561,11 +605,13 @@ function MSBuild() {
function MSBuild-Core() {
if ($ci) {
if (!$binaryLog) {
throw "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Binary log must be enabled in CI build.'
ExitWithExitCode 1
}
if ($nodeReuse) {
throw "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -Category 'Build' -Message 'Node reuse must be disabled in CI build.'
ExitWithExitCode 1
}
}
@@ -573,8 +619,11 @@ function MSBuild-Core() {
$cmdArgs = "$($buildTool.Command) /m /nologo /clp:Summary /v:$verbosity /nr:$nodeReuse /p:ContinuousIntegrationBuild=$ci"
if ($warnAsError) {
$cmdArgs += " /warnaserror /p:TreatWarningsAsErrors=true"
if ($warnAsError) {
$cmdArgs += ' /warnaserror /p:TreatWarningsAsErrors=true'
}
else {
$cmdArgs += ' /p:TreatWarningsAsErrors=false'
}
foreach ($arg in $args) {
@@ -582,31 +631,31 @@ function MSBuild-Core() {
$cmdArgs += " `"$arg`""
}
}
$exitCode = Exec-Process $buildTool.Path $cmdArgs
if ($exitCode -ne 0) {
Write-PipelineTaskError "Build failed."
Write-PipelineTelemetryError -Category 'Build' -Message 'Build failed.'
$buildLog = GetMSBuildBinaryLogCommandLineArgument $args
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
if ($buildLog -ne $null) {
Write-Host "See log: $buildLog" -ForegroundColor DarkGray
}
ExitWithExitCode $exitCode
}
}
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
function GetMSBuildBinaryLogCommandLineArgument($arguments) {
foreach ($argument in $arguments) {
if ($argument -ne $null) {
$arg = $argument.Trim()
if ($arg.StartsWith("/bl:", "OrdinalIgnoreCase")) {
return $arg.Substring("/bl:".Length)
}
if ($arg.StartsWith("/binaryLogger:", "OrdinalIgnoreCase")) {
return $arg.Substring("/binaryLogger:".Length)
if ($arg.StartsWith('/bl:', "OrdinalIgnoreCase")) {
return $arg.Substring('/bl:'.Length)
}
if ($arg.StartsWith('/binaryLogger:', 'OrdinalIgnoreCase')) {
return $arg.Substring('/binaryLogger:'.Length)
}
}
}
@@ -614,16 +663,16 @@ function GetMSBuildBinaryLogCommandLineArgument($arguments) {
return $null
}
. $PSScriptRoot\LoggingCommandFunctions.ps1
. $PSScriptRoot\pipeline-logging-functions.ps1
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$ToolsetDir = Join-Path $ArtifactsDir "toolset"
$ToolsDir = Join-Path $RepoRoot ".tools"
$LogDir = Join-Path (Join-Path $ArtifactsDir "log") $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir "tmp") $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot "global.json") | ConvertFrom-Json
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$ArtifactsDir = Join-Path $RepoRoot 'artifacts'
$ToolsetDir = Join-Path $ArtifactsDir 'toolset'
$ToolsDir = Join-Path $RepoRoot '.tools'
$LogDir = Join-Path (Join-Path $ArtifactsDir 'log') $configuration
$TempDir = Join-Path (Join-Path $ArtifactsDir 'tmp') $configuration
$GlobalJson = Get-Content -Raw -Path (Join-Path $RepoRoot 'global.json') | ConvertFrom-Json
# true if global.json contains a "runtimes" section
$globalJsonHasRuntimes = if ($GlobalJson.tools.PSObject.Properties.Name -Match 'runtimes') { $true } else { $false }
@@ -636,3 +685,18 @@ Write-PipelineSetVariable -Name 'Artifacts.Toolset' -Value $ToolsetDir
Write-PipelineSetVariable -Name 'Artifacts.Log' -Value $LogDir
Write-PipelineSetVariable -Name 'TEMP' -Value $TempDir
Write-PipelineSetVariable -Name 'TMP' -Value $TempDir
# Import custom tools configuration, if present in the repo.
# Note: Import in global scope so that the script set top-level variables without qualification.
if (!$disableConfigureToolsetImport) {
$configureToolsetScript = Join-Path $EngRoot 'configure-toolset.ps1'
if (Test-Path $configureToolsetScript) {
. $configureToolsetScript
if ((Test-Path variable:failOnConfigureToolsetError) -And $failOnConfigureToolsetError) {
if ((Test-Path variable:LastExitCode) -And ($LastExitCode -ne 0)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'configure-toolset.ps1 returned a non-zero exit code'
ExitWithExitCode $LastExitCode
}
}
}
}
Regular → Executable
+114 -102
View File
@@ -41,10 +41,14 @@ fi
# Configures warning treatment in msbuild.
warn_as_error=${warn_as_error:-true}
# True to attempt using .NET Core already that meets requirements specified in global.json
# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
# Enable repos to use a particular version of the on-line dotnet-install scripts.
# default URL: https://dot.net/v1/dotnet-install.sh
dotnetInstallScriptVersion=${dotnetInstallScriptVersion:-'v1'}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
if [[ "$ci" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
@@ -52,78 +56,6 @@ else
use_global_nuget_cache=${use_global_nuget_cache:-true}
fi
function EmitError {
if [[ "$ci" != true ]]; then
echo "$@" >&2
return
fi
message_type="error"
sourcepath=''
linenumber=''
columnnumber=''
error_code=''
while [[ $# -gt 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-type|-t)
message_type=$2
shift
;;
-sourcepath|-s)
sourcepath=$2
shift
;;
-linenumber|-l)
linenumber=$2
shift
;;
-columnnumber|-col)
columnnumber=$2
shift
;;
-code|-c)
error_code=$2
shift
;;
*)
break
;;
esac
shift
done
message='##vso[task.logissue'
message="$message type=$message_type"
if [ -n "$sourcepath" ]; then
message="$message;sourcepath=$sourcepath"
else
message="$message;sourcepath=${BASH_SOURCE[1]}"
fi
if [ -n "$linenumber" ]; then
message="$message;linenumber=$linenumber"
else
message="$message;linenumber=${BASH_LINENO[0]}"
fi
if [ -n "$columnnumber" ]; then
message="$message;columnnumber=$columnnumber"
fi
if [ -n "$error_code" ]; then
message="$message;code=$error_code"
fi
message="$message]$*"
echo "$message"
}
# Resolve any symlinks in the given path.
function ResolvePath {
local path=$1
@@ -149,7 +81,7 @@ function ReadGlobalVersion {
local pattern="\"$key\" *: *\"(.*)\""
if [[ ! $line =~ $pattern ]]; then
EmitError "Error: Cannot find \"$key\" in $global_json_file"
Write-PipelineTelemetryError -category 'Build' "Error: Cannot find \"$key\" in $global_json_file"
ExitWithExitCode 1
fi
@@ -210,7 +142,7 @@ function InitializeDotNetCli {
if [[ "$install" == true ]]; then
InstallDotNetSdk "$dotnet_root" "$dotnet_sdk_version"
else
EmitError "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
Write-PipelineTelemetryError -category 'InitializeToolset' "Unable to find dotnet with SDK version '$dotnet_sdk_version'"
ExitWithExitCode 1
fi
fi
@@ -218,14 +150,10 @@ function InitializeDotNetCli {
# Add dotnet to PATH. This prevents any bare invocation of dotnet in custom
# build steps from using anything other than what we've downloaded.
export PATH="$dotnet_root:$PATH"
Write-PipelinePrependPath -path "$dotnet_root"
if [[ $ci == true ]]; then
# Make Sure that our bootstrapped dotnet cli is avaliable in future steps of the Azure Pipelines build
echo "##vso[task.prependpath]$dotnet_root"
echo "##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP]0"
echo "##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE]1"
fi
Write-PipelineSetVariable -name "DOTNET_MULTILEVEL_LOOKUP" -value "0"
Write-PipelineSetVariable -name "DOTNET_SKIP_FIRST_TIME_EXPERIENCE" -value "1"
# return value
_InitializeDotNetCli="$dotnet_root"
@@ -244,7 +172,7 @@ function InstallDotNetSdk {
function InstallDotNet {
local root=$1
local version=$2
GetDotNetInstallScript "$root"
local install_script=$_GetDotNetInstallScript
@@ -263,15 +191,59 @@ function InstallDotNet {
fi
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg || {
local exit_code=$?
EmitError "Failed to install dotnet SDK (exit code '$exit_code')."
ExitWithExitCode $exit_code
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from public location (exit code '$exit_code')."
if [[ -n "$runtimeArg" ]]; then
local runtimeSourceFeed=''
if [[ -n "${6:-}" ]]; then
runtimeSourceFeed="--azure-feed $6"
fi
local runtimeSourceFeedKey=''
if [[ -n "${7:-}" ]]; then
decodedFeedKey=`echo $7 | base64 --decode`
runtimeSourceFeedKey="--feed-credential $decodedFeedKey"
fi
if [[ -n "$runtimeSourceFeed" || -n "$runtimeSourceFeedKey" ]]; then
bash "$install_script" --version $version --install-dir "$root" $archArg $runtimeArg $skipNonVersionedFilesArg $runtimeSourceFeed $runtimeSourceFeedKey || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to install dotnet SDK from custom location '$runtimeSourceFeed' (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
ExitWithExitCode $exit_code
fi
fi
}
}
function with_retries {
local maxRetries=5
local retries=1
echo "Trying to run '$@' for maximum of $maxRetries attempts."
while [[ $((retries++)) -le $maxRetries ]]; do
"$@"
if [[ $? == 0 ]]; then
echo "Ran '$@' successfully."
return 0
fi
timeout=$((2**$retries-1))
echo "Failed to execute '$@'. Waiting $timeout seconds before next attempt ($retries out of $maxRetries)." 1>&2
sleep $timeout
done
echo "Failed to execute '$@' for $maxRetries times." 1>&2
return 1
}
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
local install_script_url="https://dot.net/v1/dotnet-install.sh"
local install_script_url="https://dot.net/$dotnetInstallScriptVersion/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
@@ -280,12 +252,19 @@ function GetDotNetInstallScript {
# Use curl if available, otherwise use wget
if command -v curl > /dev/null; then
curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script"
with_retries curl "$install_script_url" -sSL --retry 10 --create-dirs -o "$install_script" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
else
wget -q -O "$install_script" "$install_script_url"
with_retries wget -v -O "$install_script" "$install_script_url" || {
local exit_code=$?
Write-PipelineTelemetryError -category 'InitializeToolset' "Failed to acquire dotnet install script (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
fi
fi
# return value
_GetDotNetInstallScript="$install_script"
}
@@ -294,11 +273,11 @@ function InitializeBuildTool {
if [[ -n "${_InitializeBuildTool:-}" ]]; then
return
fi
InitializeDotNetCli $restore
# return values
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildTool="$_InitializeDotNetCli/dotnet"
_InitializeBuildToolCommand="msbuild"
_InitializeBuildToolFramework="netcoreapp2.1"
}
@@ -317,11 +296,14 @@ function GetNuGetPackageCachePath {
}
function InitializeNativeTools() {
if [[ -n "${DisableNativeToolsetInstalls:-}" ]]; then
return
fi
if grep -Fq "native-tools" $global_json_file
then
local nativeArgs=""
if [[ "$ci" == true ]]; then
nativeArgs="-InstallDirectory $tools_dir"
nativeArgs="--installDirectory $tools_dir"
fi
"$_script_dir/init-tools-native.sh" $nativeArgs
fi
@@ -349,7 +331,7 @@ function InitializeToolset {
fi
if [[ "$restore" != true ]]; then
EmitError "Toolset version $toolsetVersion has not been restored."
Write-PipelineTelemetryError -category 'InitializeToolset' "Toolset version $toolset_version has not been restored."
ExitWithExitCode 2
fi
@@ -359,14 +341,14 @@ function InitializeToolset {
if [[ "$binary_log" == true ]]; then
bl="/bl:$log_dir/ToolsetRestore.binlog"
fi
echo '<Project Sdk="Microsoft.DotNet.Arcade.Sdk"/>' > "$proj"
MSBuild-Core "$proj" $bl /t:__WriteToolsetLocation /clp:ErrorsOnly\;NoSummary /p:__ToolsetLocationOutputFile="$toolset_location_file"
local toolset_build_proj=`cat "$toolset_location_file"`
if [[ ! -a "$toolset_build_proj" ]]; then
EmitError "Invalid toolset path: $toolset_build_proj"
Write-PipelineTelemetryError -category 'Build' "Invalid toolset path: $toolset_build_proj"
ExitWithExitCode 3
fi
@@ -393,6 +375,18 @@ function MSBuild {
if [[ "$pipelines_log" == true ]]; then
InitializeBuildTool
InitializeToolset
# Work around issues with Azure Artifacts credential provider
# https://github.com/dotnet/arcade/issues/3932
if [[ "$ci" == true ]]; then
"$_InitializeBuildTool" nuget locals http-cache -c
export NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS=20
export NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS=20
Write-PipelineSetVariable -name "NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS" -value "20"
Write-PipelineSetVariable -name "NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS" -value "20"
fi
local toolset_dir="${_InitializeToolset%/*}"
local logger_path="$toolset_dir/$_InitializeBuildToolFramework/Microsoft.DotNet.Arcade.Sdk.dll"
args=( "${args[@]}" "-logger:$logger_path" )
@@ -404,12 +398,12 @@ function MSBuild {
function MSBuild-Core {
if [[ "$ci" == true ]]; then
if [[ "$binary_log" != true ]]; then
EmitError "Binary log must be enabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Binary log must be enabled in CI build."
ExitWithExitCode 1
fi
if [[ "$node_reuse" == true ]]; then
EmitError "Node reuse must be disabled in CI build."
Write-PipelineTelemetryError -category 'Build' "Node reuse must be disabled in CI build."
ExitWithExitCode 1
fi
fi
@@ -423,7 +417,7 @@ function MSBuild-Core {
"$_InitializeBuildTool" "$_InitializeBuildToolCommand" /m /nologo /clp:Summary /v:$verbosity /nr:$node_reuse $warnaserror_switch /p:TreatWarningsAsErrors=$warn_as_error /p:ContinuousIntegrationBuild=$ci "$@" || {
local exit_code=$?
EmitError "Build failed (exit code '$exit_code')."
Write-PipelineTelemetryError -category 'Build' "Build failed (exit code '$exit_code')."
ExitWithExitCode $exit_code
}
}
@@ -431,6 +425,8 @@ function MSBuild-Core {
ResolvePath "${BASH_SOURCE[0]}"
_script_dir=`dirname "$_ResolvePath"`
. "$_script_dir/pipeline-logging-functions.sh"
eng_root=`cd -P "$_script_dir/.." && pwd`
repo_root=`cd -P "$_script_dir/../.." && pwd`
artifacts_dir="$repo_root/artifacts"
@@ -457,7 +453,23 @@ mkdir -p "$toolset_dir"
mkdir -p "$temp_dir"
mkdir -p "$log_dir"
if [[ $ci == true ]]; then
export TEMP="$temp_dir"
export TMP="$temp_dir"
fi
Write-PipelineSetVariable -name "Artifacts" -value "$artifacts_dir"
Write-PipelineSetVariable -name "Artifacts.Toolset" -value "$toolset_dir"
Write-PipelineSetVariable -name "Artifacts.Log" -value "$log_dir"
Write-PipelineSetVariable -name "Temp" -value "$temp_dir"
Write-PipelineSetVariable -name "TMP" -value "$temp_dir"
# Import custom tools configuration, if present in the repo.
if [ -z "${disable_configure_toolset_import:-}" ]; then
configure_toolset_script="$eng_root/configure-toolset.sh"
if [[ -a "$configure_toolset_script" ]]; then
. "$configure_toolset_script"
fi
fi
# TODO: https://github.com/dotnet/arcade/issues/1468
# Temporary workaround to avoid breaking change.
# Remove once repos are updated.
if [[ -n "${useInstalledDotNetCli:-}" ]]; then
use_installed_dotnet_cli="$useInstalledDotNetCli"
fi
View File
File renamed without changes.
-51
View File
@@ -1,51 +0,0 @@
<Project DefaultTargets="PublishPackages">
<!-- TODO: move properties imported from here into a common props file -->
<!-- <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dir.props))\dir.props" /> -->
<!-- Use an explicit SDK import so that arcade uses the build
configuration computed in dir.common.props. -->
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
<Import Project="$(MSBuildThisFileDirectory)\..\.packages\microsoft.dotnet.build.tasks.feed\$(MicrosoftDotNetBuildTasksFeedVersion)\build\Microsoft.DotNet.Build.Tasks.Feed.targets" />
<PropertyGroup>
<!-- Set the TargetFramework just to make the SDK happy -->
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<Target Name="PublishPackages">
<ItemGroup>
<PackagesToPublish Include="$(MSBuildThisFileDirectory)\..\artifacts\*.nupkg" />
</ItemGroup>
<ItemGroup>
<ItemsToPushToBlobFeed Include="@(PackagesToPublish)" />
</ItemGroup>
<Error Condition=" '$(AzureFeedUrl)' == '' " Text="AzureFeedUrl must be set" />
<Error Condition=" '$(AzureAccountKey)' == '' " Text="AzureAccountKey must be set" />
<Error Condition=" '$(BUILD_REPOSITORY_URI)' == '' " Text="BUILD_REPOSITORY_URI must be set" />
<Error Condition=" '$(BUILD_SOURCEBRANCH)' == '' " Text="BUILD_SOURCEBRANCH must be set" />
<Error Condition=" '$(BUILD_BUILDNUMBER)' == '' " Text="BUILD_BUILDNUMBER must be set" />
<Error Condition=" '$(BUILD_SOURCEVERSION)' == '' " Text="BUILD_SOURCEVERSION must be set" />
<PropertyGroup>
<AssetManifestFilePath>$(ArtifactsLogDir)AssetManifest\$(AssetManifest)</AssetManifestFilePath>
</PropertyGroup>
<Message Importance="Low" Text="BUILD_REPOSITORY_URI: $(BUILD_REPOSITORY_URI)" />
<PushToBlobFeed ExpectedFeedUrl="$(AzureFeedUrl)"
AccountKey="$(AzureAccountKey)"
ItemsToPush="@(ItemsToPushToBlobFeed)"
ManifestBuildData="Location=$(AzureFeedUrl)"
ManifestRepoUri="$(BUILD_REPOSITORY_URI)"
ManifestBranch="$(BUILD_SOURCEBRANCH)"
ManifestBuildId="$(BUILD_BUILDNUMBER)"
ManifestCommit="$(BUILD_SOURCEVERSION)"
AssetManifestPath="$(AssetManifestFilePath)" />
</Target>
</Project>
Vendored Submodule
+1
Submodule external/bdwgc added at 6e8db07c8d.
+1 -1
+1 -1
+1 -1
Loaded 100 of 3524 files, more files were not shown because too many files have changed in this diff. Show more