DynamicDataSource caching causes problems when running tests in parallel.

[Haplois]

After 2.2.4, as long as the values in data source are serializable, every row for the specific test case is considered a separate test case. If our assumptions break parallel test runs that's a regression. Is it possible for you to share a quick repro with us?

@Haplois I will share a complete project when I have more time, but in general, consider a test case like this:

[
  DataTestMethod,
  DynamicData(dynamicDataSourceName: nameof(GetData), DynamicDataSourceType.Method)
]
public async Task Create_Tasks_Requests_From_Invalid_Tasks_Definitions_Should_Return_BadRequest_Errors(Events.APIGatewayProxyRequest request, Requests.CreateTaskCommand.Arguments arguments)
{
    // test code
}

with GetData similar to this:

public static IEnumerable<object[]> GetData() =>
 new [] {
    new [] object {
         // initialize pair of objects
    },
    new [] object {
         // initialize pair of objects
    },
 }

If I run such test cases in parallel (method level parallelism) then I see all kinds of exception suggesting that the the test framework attempts to pull the enumerable on different threads. I have plenty of data sources structured like this:

public static IEnumerable<object[]> GetData()
{
    yield new InputData {
       // ...
    };

    yield new InputData {
       // ...
    };

    // etc.
}

These obviously can't be pulled in parallel either.

Originally posted by @paul-michalik in #871 (comment)

AB#1408168

[Haplois]

This class needs to be able work concurrently.

testfx/src/Adapter/MSTest.CoreAdapter/Helpers/DataSerializationHelper.cs

Lines 14 to 135 in 86845cf

	internal static class DataSerializationHelper
	{
	private static readonly Dictionary<Type, DataContractJsonSerializer> SerializerCache = new Dictionary<Type, DataContractJsonSerializer>();
	private static readonly DataContractJsonSerializerSettings SerializerSettings = new DataContractJsonSerializerSettings()
	{
	UseSimpleDictionaryFormat = true,
	EmitTypeInformation = System.Runtime.Serialization.EmitTypeInformation.Always
	};
	/// <summary>
	/// Serializes the date in such a way that won't throw exceptions during deserialization in Test Platform.
	/// The result can be deserialized using <see cref="Deserialize(string[])"/> method.
	/// </summary>
	/// <param name="data">Data array to serialize.</param>
	/// <returns>Serialzed array.</returns>
	public static string[] Serialize(object[] data)
	{
	if (data == null)
	{
	return null;
	}
	var serializedData = new string[data.Length * 2];
	for (int i = 0; i < data.Length; i++)
	{
	var typeIndex = i * 2;
	var dataIndex = typeIndex + 1;
	if (data[i] == null)
	{
	serializedData[typeIndex] = null;
	serializedData[dataIndex] = null;
	continue;
	}
	var type = data[i].GetType();
	var typeName = type.AssemblyQualifiedName;
	serializedData[typeIndex] = typeName;
	var serializer = GetSerializer(type);
	using (var memoryStream = new MemoryStream())
	{
	serializer.WriteObject(memoryStream, data[i]);
	var serializerData = memoryStream.ToArray();
	serializedData[dataIndex] = Encoding.UTF8.GetString(serializerData, 0, serializerData.Length);
	}
	}
	return serializedData;
	}
	/// <summary>
	/// Deserialzes the data serialzed by <see cref="Serialize(object[])" /> method.
	/// </summary>
	/// <param name="serializedData">Serialized data array to deserialize.</param>
	/// <returns>Deserialized array.</returns>
	public static object[] Deserialize(string[] serializedData)
	{
	if (serializedData == null || serializedData.Length % 2 != 0)
	{
	return null;
	}
	var length = serializedData.Length / 2;
	var data = new object[length];
	for (int i = 0; i < length; i++)
	{
	var typeIndex = i * 2;
	var typeName = serializedData[typeIndex];
	var serializedValue = serializedData[typeIndex + 1];
	if (serializedValue == null || typeName == null)
	{
	data[i] = null;
	continue;
	}
	var serializer = GetSerializer(typeName);
	var serialzedDataBytes = Encoding.UTF8.GetBytes(serializedValue);
	using (var memoryStream = new MemoryStream(serialzedDataBytes))
	{
	data[i] = serializer.ReadObject(memoryStream);
	}
	}
	return data;
	}
	private static DataContractJsonSerializer GetSerializer(string typeName)
	{
	var serializer = SerializerCache.SingleOrDefault(i => i.Key.FullName == typeName);
	if (serializer.Value != null)
	{
	return serializer.Value;
	}
	var type = Type.GetType(typeName);
	if (type != null)
	{
	return GetSerializer(type);
	}
	return GetSerializer(typeof(object));
	}
	private static DataContractJsonSerializer GetSerializer(Type type)
	{
	if (SerializerCache.ContainsKey(type))
	{
	return SerializerCache[type];
	}
	return SerializerCache[type] = new DataContractJsonSerializer(type, SerializerSettings);
	}
	}

[jr01]

@Haplois - just to add to this. I intermittently get this exception with 2.2.5:

  Message: 
    Object reference not set to an instance of an object.

  Stack Trace: 
    <>c__DisplayClass4_0.<GetSerializer>b__0(KeyValuePair`2 i)
    Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
    DataSerializationHelper.GetSerializer(String typeName)
    DataSerializationHelper.Deserialize(String[] serializedData)
    TestMethodRunner.RunTestMethod()
    TestMethodRunner.Execute()

Changing the SerializerCache to a ConcurrentDictionary should fix this.

[Haplois]

Fixed by #998