Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge from dotnet/runtime #1322

Merged
merged 134 commits into from
Jul 15, 2021
Merged

Conversation

jkotas
Copy link
Member

@jkotas jkotas commented Jul 14, 2021

No description provided.

adamsitnik and others added 30 commits July 10, 2021 20:38
…55456)

* add test project with file locking disabled via config file

* use O_TRUNC when file locking is disabled, avoid ftruncate syscall

* Apply suggestions from code review

Co-authored-by: Stephen Toub <[email protected]>
* Turn off test FlushAsync_ThrowsIfWriterReaderWithException
* Turn off System.Net.NameResolution.Tests.GetHost* tests for SLES
…ocation only (#55448)

* Prototype allocation profiler

* Add callback for pinned objects

* Fix the build issue caused by corprof.idl change

* Improve the test

* Misc changes for the tests

Co-authored-by: Andrew Au <[email protected]>
Co-authored-by: Yauk Jia <[email protected]>
* Fixing MetricEventSource tests

I've seen two recent non-deterministic failures:
1. After starting the EventListener there is a delay of one collection
interval (previously 0.3 seconds) where we expect calls to counter.Add
and Histogram.Record() to update stats. On a fast machine this code
would run in under a millisecond but in some test runs it still wasn't
happening fast enough.
2. We were seeing error events from a previous test get observed in a
later test because session id was being ignored for error events.

Fixes:
1. Added an initial collection interval where no counter APIs will be
invoked and the test will delay up to 60 seconds waiting for this.
Hopefully this delay makes it more likely that the Add/Record APIs
are ready to execute promptly once the 2nd interval begins. I
also increased the intervals to 1 second. If that still isn't sufficient
we can either further increase the collection intervals or disable the
tests. I also moved these tests to outerloop because
they are slow and noisy on the console, but it may have a side
benefit lessening the impact if there are future timing related failures.
2. Tightened up the session id matching so only error events with
empty id or the expected id are allowed.
* W^X support

This change is the last part of enabling the W^X support. It adds the
actual executable allocator that handles all double mapped memory
allocations and creating the writeable mappings.

The platform specific functionality is placed in a new minipal that is
going to be a basis for future removal of Windows APIs usage from the
native runtime.

The last state of the change was tested on all the platforms that we
support using coreclr pri 1 tests with both the W^X enabled and disabled
using the COMPlus_EnableWriteXorExecute variable.

The debugger changes were tested using the managed debugger testing
suite on Windows x64, x86 and on Apple Silicon so far. Further testing
on other platforms is in progress.

* Replace LeafLock in UMEntryThunkFreeList by a new lock

* Also allocate LoaderHeapFreeBlock from regular heap.

* Set the W^X default to disabled
* Fixing some issues in the generic math implementation

* Adding generic math tests for integer types
* Move PosixSignalRegistration to corelib

* Fix trimming issue on mobile targets
…5450)

Generate the same code as in the non-aot case, i.e. compare
the vtable with a constant.
Previously exceptions that escaped user provided callbacks for observable instruments would have terminated all further metric collection. Now those exceptions are caught and reported, only potentially interfering with other observable instruments in the same collection interval.
* use updated fedora image with msquic

* fix ping
Updating Docker image tag references that are obsolete and replaced by references to mcr.microsoft.com/dotnet-buildtools/prereqs. See dotnet/dotnet-docker#2848.
* reduce timeout in SetListenerTimeoutWorksWithSmallTimeout

* replace AssertArrayEqual with AssertExtensions.SequenceEqual

Co-authored-by: Geoffrey Kizer <[email protected]>
Fixes #47583. Resolves #52287, #54807 and similar issues, without changing customer application code.

Introduces an AppContext switch `System.Net.DisableIPv6` and environment variable `DOTNET_SYSTEM_NET_DISABLEIPV6` to emulate the lack of OS-level IPv6 support. This is useful to workaround dual-stack socket issues when the OS reports support of IPv6, but some other underlying infrastructure element (typically VPN) doesn't function well with IPv6 request.

For consistency, this switch also impacts NameResolution and Ping, not only Sockets.
Together with #55483 this updates all logging to use msquic handles instead of hash code - for listeners, connections and streams. The format of handle is the same as msquic one.
_innerWebSocket is already disposed by after abort.
Add crash report to createdump for Linux Watson

For Linux Watson the crash report will contains the .NET Core version, the faulting process name (the module/assembly containing Main) and the managed exception info (including the exception HRESULT) and thread stack trace of the thread the caused the crash.

Add the CLRDATA_MODULE_IS_MAIN_MODULE flag to the DAC's IXCLRDataModule::GetFlags API.

Add code to get the managed method name and write it out as "method_name" (even on MacOS).

Add native frame symbolization (unmanaged_name) using dladdr.

Only get the image info once and save in a global

Demangle the stack frame symbols

Only write PH_HDR_CANARY section if neccessary
…es (#55445)

This fixes dotnet/runtime#55097 - which allows us to support C# nullability analysis once again in hot reload deltas.

Specifically we allow EnC deltas to include modifications of existing rows in the CustomAttribute table as long as the Parent and Type columns stay the same (that is: a custom attribute modification that still applies to the same element - and uses the same custom attribute constructor, but may have a different value).

To support this, we add a new BaselineInfo:any_modified_rows array that keeps track for each table whether any rows have been modified (as opposed to added) by any EnC delta. When the runtime calls mono_metadata_decode_row, if there have been any deltas that modified a row in the requested table, we call hot_reload_effective_table_slow which find the latest delta that modified that row. If there have only been additions, we stop at the first delta that added the row we're looking for, if there are modifications, we look through all the deltas and return the latest one.

* [hot_reload] Add test for updating custom attribute ctor values

   Just changing the arguments of a custom attribute constructor should generate an update to the CustomAttributes table with the same parent and .ctor.  That kind of change should be allowed by Mono and CoreCLR

* [hot_reload] Allow custom attribute row modifications if Parent and Type unchanged.

   Allows updates to the constructor arguments (or property values)

* [hot_reload] Implement table lookup of modified rows

   Add a bool array on the base image to keep track of whether each table had any row modifications (as opposed to row additions) in any generation.

   If there was a modification, take the slow path in mono_image_effective_table even if the index we're looking at is in the base image.

   Update hot_reload_effective_table_slow so that if there was a modified row, we look at all the deltas to see if there's an even later update to that row.  (When there are only additions, keep same behavior as before - only look
as far as the generation that added the row we wanted to find).

   Also refine the assertion in hot_reload_relative_delta_index to properly account for EnCMap entries that correspond to modifications - in that case we might stop searching a metadata delta before we hit the end of the table if the
EnCmap entries start pointing to rows that are past the one we wanted to look up.

* Update the CustomAttributeUpdates test to check attribute value

   Check that we get the updated custom attribute string property value.

* Re-enable nullability for hot reload tests

   Mono can now deal with the custom attributes modifications that Roslyn emits
* Used VectorXYZ.Create for constants in Base64

* Used VectorXYZ.Create for constants in BitArray

* Remove conditional compilation

It's only built for NetCoreAppCurrent, so no need to special case older runtimes.
* snatpshot

* snapshot

* add csproj'

* update

* more ckeanup

* more cleanup

* feedback from review

* fix mail tests

* feedabck from review

* feedback from review

* make constructor private
tannergooding and others added 26 commits July 14, 2021 00:34
This change moves the RSAOpenSsl class (and the RSA.Create() internal class on Linux) to use EVP_PKEY
for the import and export operations, making the interaction with the underlying library based on
PKCS#8 PrivateKeyInfo and X.509 SubjectPublicKeyInfo key blobs.

A large portion of the code is just from splitting the key helpers files to not need all of the encrypted PKCS#8 support
in the System.Security.Cryptography.OpenSsl library, as the encrypted PKCS#8 <-> unencrypted PKCS#8 work provided
by the base classes is sufficient.

Once PKCS#8 and SPKI became the primary modes of interaction the spanified import and export routines for those got
overridden in RSAOpenSsl to avoid SPKI->RSAParameters->SPKI-style conversions.

This change completely eliminates SafeRsaHandle, and the only time that an `RSA*` is used at all is in the legacy RSAOpenSsl(IntPtr) constructor.
For some reason we have 2 of them, and only need 1.
* Compression.ZipFile support for Unix Permissions

When running on Unix, capture the file's permissions on ZipFile Create and write the captured file permissions on ZipFile Extract.

Fix dotnet#1548
* Renamed AllowDraftHttp3 to AllowHttp3AndQuic app context switch

* Usage of AllowHttp3AndQuic switch in S.N.Quic and enabling it in tests

* Renamed AppContext switch to Http3Support
Fixes #41300

Previously DiagnosticSourceEventSource would only iterate and recognize properties that were declared on the most derived type.
Now it can capture properties that were inherited from a base class too.
* Fix deug after hotreload.

* do not change mono.proj

* fix formatting.

* Remove extra space.
* Implement method calls and method calls with simple parameters.

* removing useless change.

* Support const char.
Support object.

* Adding more tests as suggested by @pavelsavara.

* Adding a test calling a method that returns void and change an attribute value.

* Changing what @pavelsavara suggested.

* Fix merge.
…630)

It causes the output to be a bitcode file as well, which is
passed to the emscripten link step, which will compile it
to wasm sequentially, slowing down linking.

Fixes dotnet/runtime#54935.
…used to copy (#55598)

valuetypes containing references. This is needed to make sure llvm keeps the valuetypes
on the C stack for GC tracking.

Fixes random crashes when running the System.Collections.NonGeneric testsuite.
Follow-up for NITs and cosmetic changes from #55505
…ncurrentRequestsSuccessfullyHandled

Reverts #55572. The test is still failing.
* Turn on loc PRs

* oops

Co-authored-by: Jeremy Koritzinsky <[email protected]>

* Keep OneLocBuild running only in main

Co-authored-by: Jeremy Koritzinsky <[email protected]>
We have a lot of code that passes raw pointers to managed objects around when they should really be using roots instead (i.e. root.value) so that if the GC relocates the object, they won't be using the old address. This PR migrates a bunch of that code so that it uses root objects.

A lot of code remains that does use raw objects, but I read over all of it and the cases appear to be safe because they immediately return it (or return it without calling any other functions).
We previously set an arbitrary cut-off of 2MB max array size. That was before we would trim arrays in response to memory pressure.  This now raises the limit as high as we can while maintaining the current power-of-two-based scheme.
@jkotas jkotas merged commit b4c3760 into dotnet:feature/NativeAOT Jul 15, 2021
@jkotas jkotas deleted the merge-runtime branch July 15, 2021 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.