Skip to content
This repository has been archived by the owner on Oct 11, 2022. It is now read-only.

Commit

Permalink
Fix Framework Extensibilty for Trait Attributes (#54)
Browse files Browse the repository at this point in the history 
* Fixing the RFC for Framework extensibility for attributes. Removing the unnecessary references to Description, WorkItem attribute.

* Removing redundancies.

* Changing the name as per comment
  • Loading branch information
@singhsarab
singhsarab authored Sep 18, 2018
1 parent 23b3bb3 commit 2652c9a
Showing 1 changed file with 90 additions and 96 deletions.
186 changes: 90 additions & 96 deletions ...k-Extensibility-Description-Attributes.md → ...amework-Extensibility-Trait-Attributes.md
View file
Original file line number Diff line number Diff line change
@@ -1,96 +1,90 @@
# RFC 001- Framework Extensibility for Description Attributes

## Summary
This details the MSTest V2 framework extensibility for attributes that describe a test.

## Motivation
It is a requirement for teams to have description attributes which are strongly typed over a test method as opposed to just a KeyvaluePair<string,string>. This allows teams to have a standard across the team which is less error prone and more natural to specify. Users can also filter tests based on the values of these attributes.

## Detailed Design

### Requirements
1. One should be able to have a strongly typed desciption attribute on a test method.
2. One should be able to filter in VS IDE based on this description attribute.
3. One should be able to see this information in Test Reporting.
4. This extensibility should also guarantee that attributes in MSTest V1 which are brought into MSTest V2 with this extensibility model should be source compatible.

### Proposed solution
The test framework currently has a TestProperty attribute which can be used to define custom traits as a KeyValuePair<string,string>. The definition of this attribute is as below:
```csharp
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public sealed class TestPropertyAttribute : Attribute
{
/// <summary>
/// Initializes a new instance of the <see cref="TestPropertyAttribute"/> class.
/// </summary>
/// <param name="name">
/// The name.
/// </param>
/// <param name="value">
/// The value.
/// </param>
public TestPropertyAttribute(string name, string value)
{
this.Name = name;
this.Value = value;
}

/// <summary>
/// Gets the name.
/// </summary>
public string Name { get; }

/// <summary>
/// Gets the value.
/// </summary>
public string Value { get; }
}
```
Here is how the test methods are decorated with this attribute:
```csharp
[TestMethod]
[TestProperty("WorkItem","234")]
public void TestMethod()
{
}
```
Users can then filter tests in VS IDE Test Explorer based on this Test Property. The query string that would filter the test above would look like
```
Trait:"WorkItem" Trait:"234"
```
This TestProperty is also filled into the TestPlatform's TestCase object which makes it available for reporting in the various loggers that can be plugged into the TestPlatform.

To provide extension writers with the ability to have strongly typed attributes to achieve what TestProperty above achieves, the proposal is to make TestProperty a non-sealed class allowing classes to extend it like below:
```csharp
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public class WorkItemAttribute : TestPropertyAttribute
{
/// <summary>
/// Initializes a new instance of the <see cref="WorkItemAttribute"/> class.
/// </summary>
/// <param name="workItemId">
/// The Id of the workitem.
/// </param>
public WorkItemAttribute(int workItemId)
: base ("WorkItem", workItemId.ToString())
{
}
}
```
And test methods would be decorated in a much more convenient form as below:
```csharp
[TestMethod]
[WorkItem(234)]
public void TestMethod()
{
}
```
This would provide extension writers with the same level of functionality that TestProperty already has with filtering and reporting plus the added ease of having a strongly typed description.

### Requirements from Test Platform
1. The attributes that were already part of MSTest V1(Description, WorkItem, CssIteration and CssProjectStructure) should be part of the trx logger output. Currently only owner and priority are part of the trx log.
2. Custom attributes should also show up in the trx logger. This is a bigger change since it might require changes in the trx schema.

## Open Questions
1. Should this attribute be opened up to be at a class and assembly level like TestCategory Attribute?
2. Is filtering needed at the console level for these extension attributes? Currently the TestCaseFilter switch at the console level only supports - TestCategory, Priority, FullyQualifiedName, Name, ClassName. This is a finite list. In order to support filtering for extended attributes this list needs to be made dynamic.
# RFC 001- Framework Extensibility for Trait Attributes

## Summary
This details the MSTest V2 framework extensibility for attributes that are traits for a test.

## Motivation
It is a requirement for teams to have trait attributes which are strongly typed over a test method as opposed to just a KeyvaluePair<string,string>. This allows teams to have a standard across the team which is less error prone and more natural to specify. Users can also filter tests based on the values of these attributes.

## Detailed Design

### Requirements
1. One should be able to have a strongly typed trait attribute on a test method.
2. One should be able to filter in VS IDE based on this trait attribute.
3. One should be able to see this information in Test Reporting.
4. This extensibility should also guarantee that attributes in MSTest V1 which are brought into MSTest V2 with this extensibility model should be source compatible.

### Proposed solution
The test framework currently has a TestProperty attribute which can be used to define custom traits as a KeyValuePair<string,string>. The definition of this attribute is as below:
```csharp
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public sealed class TestPropertyAttribute : Attribute
{
/// <summary>
/// Initializes a new instance of the <see cref="TestPropertyAttribute"/> class.
/// </summary>
/// <param name="name">
/// The name.
/// </param>
/// <param name="value">
/// The value.
/// </param>
public TestPropertyAttribute(string name, string value)
{
this.Name = name;
this.Value = value;
}

/// <summary>
/// Gets the name.
/// </summary>
public string Name { get; }

/// <summary>
/// Gets the value.
/// </summary>
public string Value { get; }
}
```

This TestProperty is also filled into the TestPlatform's TestCase object which makes it available for reporting in the various loggers that can be plugged into the TestPlatform.

To provide extension writers with the ability to have strongly typed attributes to achieve what TestProperty above achieves, the proposal is to make TestProperty a non-sealed class allowing classes to extend it like below:
```csharp
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public class CustomTraitPropertyAttribute : TestPropertyAttribute
{
/// <summary>
/// Initializes a new instance of the <see cref="CustomTraitPropertyAttribute"/> class.
/// </summary>
/// <param name="traitData">
/// Data associated with trait property
/// </param>
public CustomTraitPropertyAttribute(int traitData)
: base ("CustomTraitProperty", traitData.ToString())
{
}
}
```
And test methods would be decorated in a much more convenient form as below:
```csharp
[TestMethod]
[CustomTraitProperty(234)]
public void TestMethod()
{
}
```

Users can then filter tests in VS IDE Test Explorer based on this Test Property. The query string that would filter the test above would look like
```
Trait:"CustomTraitProperty" Trait:"234"
```

This would provide extension writers with the same level of functionality that TestProperty already has with filtering and reporting plus the added ease of having a strongly typed trait.

### Requirements from Test Platform
1. Custom attributes should also show up in the trx logger. This is a bigger change since it might require changes in the trx schema.

## Open Questions
1. Should this attribute be opened up to be at a class and assembly level like TestCategory Attribute?
2. Is filtering needed at the console level for these extension attributes? Currently the TestCaseFilter switch at the console level only supports - TestCategory, Priority, FullyQualifiedName, Name, ClassName. This is a finite list. In order to support filtering for extended attributes this list needs to be made dynamic.

0 comments on commit 2652c9a

Please sign in to comment.