Report on Types and Versions (#340)

To improve the visibility of which types are in which version, generate a `versions_matrix.md` file in the root folder for each service, detailing which types exist in which version. 

For `microsoft.powerbi` the matrix is simple:

|                                    | v20160129 |
|------------------------------------|-----------|
| AzureSku                           | v20160129 |
| AzureSkuName                       | v20160129 |
| AzureSkuStatusName                 | v20160129 |
| AzureSkuStatusTier                 | v20160129 |
| AzureSkuTier                       | v20160129 |
| AzureSku_Status                    | v20160129 |
| WorkspaceCollection                | v20160129 |
| WorkspaceCollection_Status         | v20160129 |
| WorkspaceCollectionsSpecApiVersion | v20160129 |
| WorkspaceCollectionsSpecType       | v20160129 |
| WorkspaceCollections_Spec          | v20160129 |

For `microsoft.authorization` it's quite informative:

|                                                 | v20150101 | v20160901 | v20170901 | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
|-------------------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
| Identity                                        |           |           |           |           | v20180501 | v20190101 | v20190601 | v20190901 |
| IdentityStatusType                              |           |           |           |           | v20180501 | v20190101 | v20190601 | v20190901 |
| IdentityType                                    |           |           |           |           | v20180501 | v20190101 | v20190601 | v20190901 |
| Identity_Status                                 |           |           |           |           | v20180501 | v20190101 | v20190601 | v20190901 |
| Lock                                            | v20150101 | v20160901 |           |           |           |           |           |           |
| LocksSpecApiVersion                             | v20150101 | v20160901 |           |           |           |           |           |           |
| LocksSpecPropertiesLevel                        | v20150101 |           |           |           |           |           |           |           |
| LocksSpecType                                   | v20150101 | v20160901 |           |           |           |           |           |           |
| Locks_Spec                                      | v20150101 | v20160901 |           |           |           |           |           |           |
| Locks_Spec_Properties                           | v20150101 |           |           |           |           |           |           |           |
| ManagementLockObject_Status                     | v20150101 | v20160901 |           |           |           |           |           |           |
| ManagementLockOwner_Status                      |           | v20160901 |           |           |           |           |           |           |
| ManagementLockProperties                        |           | v20160901 |           |           |           |           |           |           |
| ManagementLockPropertiesLevel                   |           | v20160901 |           |           |           |           |           |           |
| ManagementLockPropertiesStatusLevel             | v20150101 | v20160901 |           |           |           |           |           |           |
| ManagementLockProperties_Status                 | v20150101 | v20160901 |           |           |           |           |           |           |
| ParameterValuesValue_Status                     |           |           |           |           |           |           |           | v20190901 |
| Permission                                      |           |           | v20170901 |           |           |           |           |           |
| PolicyAssignment                                |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignmentProperties                      |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignmentPropertiesEnforcementMode       |           |           |           |           |           |           | v20190601 | v20190901 |
| PolicyAssignmentPropertiesStatusEnforcementMode |           |           |           |           |           |           | v20190601 | v20190901 |
| PolicyAssignmentProperties_Status               |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignment_Status                         |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignmentsSpecApiVersion                 |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignmentsSpecType                       |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicyAssignments_Spec                          |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicySku                                       |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| PolicySku_Status                                |           |           |           | v20180301 | v20180501 | v20190101 | v20190601 | v20190901 |
| RoleAssignment                                  |           |           | v20170901 |           |           |           |           |           |
| RoleAssignmentProperties                        |           |           | v20170901 |           |           |           |           |           |
| RoleAssignmentsSpecApiVersion                   |           |           | v20170901 |           |           |           |           |           |
| RoleAssignmentsSpecType                         |           |           | v20170901 |           |           |           |           |           |
| RoleAssignments_Spec                            |           |           | v20170901 |           |           |           |           |           |
| RoleDefinition                                  |           |           | v20170901 |           |           |           |           |           |
| RoleDefinitionProperties                        |           |           | v20170901 |           |           |           |           |           |
| RoleDefinitionsSpecApiVersion                   |           |           | v20170901 |           |           |           |           |           |
| RoleDefinitionsSpecType                         |           |           | v20170901 |           |           |           |           |           |
| RoleDefinitions_Spec                            |           |           | v20170901 |           |           |           |           |           |


For `microsoft.network`, less so: 😁 
![image](https://user-images.githubusercontent.com/1272094/100169550-a21dee80-2f28-11eb-94fb-81a499424dae.png)

hack/generator/pkg/codegen/code_generator.go

@@ -80,6 +80,7 @@ func corePipelineStages(idFactory astmodel.IdentifierFactory, configuration *con
 		applyExportFilters(configuration),
 		stripUnreferencedTypeDefinitions(),
 		replaceAnyTypeWithJSON(),
+		reportOnTypesAndVersions(configuration),
 
 		createArmTypesAndCleanKubernetesTypes(idFactory),
 		applyKubernetesResourceInterface(idFactory),

hack/generator/pkg/codegen/pipeline_report_type_versions.go

@@ -0,0 +1,104 @@
+/*
+ * Copyright (c) Microsoft Corporation.
+ * Licensed under the MIT license.
+ */
+
+package codegen
+
+import (
+	"context"
+	"github.com/Azure/k8s-infra/hack/generator/pkg/astmodel"
+	"github.com/Azure/k8s-infra/hack/generator/pkg/config"
+	"github.com/Azure/k8s-infra/hack/generator/pkg/reporting"
+	"github.com/pkg/errors"
+	"io/ioutil"
+	kerrors "k8s.io/apimachinery/pkg/util/errors"
+	"os"
+	"path"
+	"strings"
+)
+
+// reportOnTypesAndVersions creates a pipeline stage that removes any wrapper types prior to actual code generation
+func reportOnTypesAndVersions(configuration *config.Configuration) PipelineStage {
+
+	return MakePipelineStage(
+		"reportTypesAndVersions",
+		"Generate reports on types and versions in each package",
+		func(ctx context.Context, types astmodel.Types) (astmodel.Types, error) {
+			report := NewPackagesMatrixReport()
+			report.Summarize(types)
+			err := report.WriteTo(configuration.OutputPath)
+			return types, err
+		})
+}
+
+type PackagesMatrixReport struct {
+	// A separate table for each package
+	tables map[string]*reporting.Table
+}
+
+func NewPackagesMatrixReport() *PackagesMatrixReport {
+	return &PackagesMatrixReport{
+		tables: make(map[string]*reporting.Table),
+	}
+}
+
+func (report *PackagesMatrixReport) Summarize(types astmodel.Types) {
+	for _, t := range types {
+		typeName := t.Name().Name()
+		packageName := report.ServiceName(t.Name().PackageReference)
+		packageVersion := t.Name().PackageReference.PackageName()
+		table, ok := report.tables[packageName]
+		if !ok {
+			table = reporting.NewTable()
+			report.tables[packageName] = table
+		}
+
+		table.SetCell(typeName, packageVersion, packageVersion)
+	}
+}
+
+func (report *PackagesMatrixReport) WriteTo(outputPath string) error {
+	var errs []error
+	for pkg, table := range report.tables {
+		err := report.WriteTableTo(table, pkg, outputPath)
+		if err != nil {
+			errs = append(errs, err)
+		}
+	}
+
+	return kerrors.NewAggregate(errs)
+}
+
+func (report *PackagesMatrixReport) ServiceName(ref astmodel.PackageReference) string {
+	pathBits := strings.Split(ref.PackagePath(), "/")
+	index := len(pathBits) - 1
+	if index > 0 {
+		index--
+	}
+
+	return pathBits[index]
+}
+
+func (report *PackagesMatrixReport) WriteTableTo(table *reporting.Table, pkg string, outputPath string) error {
+	table.SortColumns(func(left string, right string) bool {
+		return left < right
+	})
+	table.SortRows(func(top string, bottom string) bool {
+		return top < bottom
+	})
+
+	var buffer strings.Builder
+	table.WriteTo(&buffer)
+
+	outputFolder := path.Join(outputPath, pkg)
+	if _, err := os.Stat(outputFolder); os.IsNotExist(err) {
+		err = os.MkdirAll(outputFolder, 0700)
+		if err != nil {
+			return errors.Wrapf(err, "Unable to create directory %q", outputFolder)
+		}
+	}
+
+	destination := path.Join(outputFolder, "versions_matrix.md")
+	return ioutil.WriteFile(destination, []byte(buffer.String()), 0600)
+}

hack/generator/pkg/reporting/table.go

@@ -0,0 +1,182 @@
+/*
+ * Copyright (c) Microsoft Corporation.
+ * Licensed under the MIT license.
+ */
+
+package reporting
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+)
+
+type Table struct {
+	// Captions for each row
+	rows []string
+	// Length of longest row caption
+	rowWidth int
+	// Captions for each column
+	cols []string
+	// Width required for each column
+	colWidths []int
+	// Cell content, arranged per row, then per column
+	cells map[string]map[string]string
+}
+
+func NewTable() *Table {
+	return &Table{
+		cells: make(map[string]map[string]string),
+	}
+}
+
+// Rows returns a slice containing the captions of all the rows of the table
+// A new slice is returned to avoid violations of encapsulation
+func (t *Table) Rows() []string {
+	var result []string
+	result = append(result, t.rows...)
+	return result
+}
+
+// AddRow adds the specified row to the table if it doesn't already exist
+func (t *Table) AddRow(row string) {
+	if t.indexOfRow(row) == -1 {
+		t.rows = append(t.rows, row)
+		if len(row) > t.rowWidth {
+			t.rowWidth = len(row)
+		}
+	}
+}
+
+// SortRows allows rows to be sorted by caption
+func (t *Table) SortRows(less func(top string, bottom string) bool) {
+	sort.Slice(t.rows, func(i, j int) bool {
+		return less(t.rows[i], t.rows[j])
+	})
+}
+
+// Columns returns a slice containing the captions of all the columns of the table
+// A new slice is returned to avoid violations of encapsulation
+func (t *Table) Columns() []string {
+	var result []string
+	result = append(result, t.cols...)
+	return result
+}
+
+// AddColumn adds the specified column to the table if it doesn't already exist
+func (t *Table) AddColumn(col string) {
+	index := t.indexOfColumn(col)
+	if index == -1 {
+		t.cols = append(t.cols, col)
+		t.colWidths = append(t.colWidths, len(col))
+	}
+}
+
+// SortColumns allows columns to be sorted by caption
+func (t *Table) SortColumns(less func(left string, right string) bool) {
+	sort.Slice(t.cols, func(i, j int) bool {
+		return less(t.cols[i], t.cols[j])
+	})
+}
+
+// SetCell sets the content of a given cell of the table
+func (t *Table) SetCell(row string, col string, cell string) {
+	t.AddColumn(col)
+	t.AddRow(row)
+	rowCells := t.getRowCells(row)
+	rowCells[col] = cell
+
+	index := t.indexOfColumn(col)
+	if len(cell) > t.colWidths[index] {
+		t.colWidths[index] = len(cell)
+	}
+}
+
+func (t *Table) WriteTo(buffer *strings.Builder) {
+	buffer.WriteString(t.renderHeader())
+	buffer.WriteString(t.renderDivider())
+	for _, r := range t.rows {
+		buffer.WriteString(t.renderRow(r))
+	}
+}
+
+func (t *Table) getRowCells(row string) map[string]string {
+	if m, ok := t.cells[row]; ok {
+		return m
+	}
+
+	result := make(map[string]string)
+	t.cells[row] = result
+	return result
+}
+
+func (t *Table) renderHeader() string {
+	var result strings.Builder
+
+	result.WriteString(fmt.Sprintf("| %*s |", t.rowWidth, ""))
+
+	for _, c := range t.cols {
+		result.WriteString(" ")
+		result.WriteString(c)
+		result.WriteString(" |")
+	}
+
+	result.WriteString("\n")
+	return result.String()
+}
+
+func (t *Table) renderDivider() string {
+	var result strings.Builder
+
+	result.WriteString("|")
+	for i := -2; i < t.rowWidth; i++ {
+		result.WriteRune('-')
+	}
+	result.WriteRune('|')
+
+	for w := range t.cols {
+		width := t.colWidths[w]
+		for i := -2; i < width; i++ {
+			result.WriteRune('-')
+		}
+
+		result.WriteRune('|')
+	}
+
+	result.WriteString("\n")
+	return result.String()
+}
+
+func (t *Table) renderRow(row string) string {
+	var result strings.Builder
+	cells := t.getRowCells(row)
+
+	result.WriteString(fmt.Sprintf("| %*s |", -t.rowWidth, row))
+	for i, c := range t.cols {
+		content := cells[c]
+		result.WriteString(fmt.Sprintf(" %*s |", -t.colWidths[i], content))
+	}
+
+	result.WriteString("\n")
+	return result.String()
+}
+
+func (t *Table) indexOfRow(row string) int {
+	for i, r := range t.rows {
+		if r == row {
+			return i
+		}
+	}
+
+	return -1
+}
+
+func (t *Table) indexOfColumn(col string) int {
+	for i, c := range t.cols {
+		if c == col {
+			return i
+		}
+	}
+
+	return -1
+}

hack/generator/pkg/reporting/table_test.go

@@ -0,0 +1,45 @@
+/*
+ * Copyright (c) Microsoft Corporation.
+ * Licensed under the MIT license.
+ */
+
+package reporting
+
+import (
+	"fmt"
+	"github.com/sebdah/goldie/v2"
+	"strings"
+	"testing"
+)
+
+func TestTable_StepByStep_GivesExpectedResults(t *testing.T) {
+	steps := []struct {
+		row  string
+		col  string
+		cell string
+	}{
+		{"1", "prime", "(yes)"},
+		{"1", "square", "yes"},
+		{"2", "prime", "yes"},
+		{"3", "prime", "yes"},
+		{"3", "triangle", "yes"},
+		{"4", "square", "yes"},
+		{"5", "prime", "yes"},
+		{"6", "triangle", "yes"},
+		{"7", "prime", "yes"},
+		{"9", "square", "yes"},
+		{"10", "triangle", "yes"},
+	}
+
+	table := NewTable()
+	g := goldie.New(t)
+	for i, s := range steps {
+		table.SetCell(s.row, s.col, s.cell)
+
+		var buff strings.Builder
+		table.WriteTo(&buff)
+
+		testName := fmt.Sprintf("%s_step_%d", t.Name(), i)
+		g.Assert(t, testName, []byte(buff.String()))
+	}
+}

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_0.golden

@@ -0,0 +1,3 @@
+|   | prime |
+|---|-------|
+| 1 | (yes) |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_1.golden

@@ -0,0 +1,3 @@
+|   | prime | square |
+|---|-------|--------|
+| 1 | (yes) | yes    |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_10.golden

@@ -0,0 +1,11 @@
+|    | prime | square | triangle |
+|----|-------|--------|----------|
+| 1  | (yes) | yes    |          |
+| 2  | yes   |        |          |
+| 3  | yes   |        | yes      |
+| 4  |       | yes    |          |
+| 5  | yes   |        |          |
+| 6  |       |        | yes      |
+| 7  | yes   |        |          |
+| 9  |       | yes    |          |
+| 10 |       |        | yes      |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_2.golden

@@ -0,0 +1,4 @@
+|   | prime | square |
+|---|-------|--------|
+| 1 | (yes) | yes    |
+| 2 | yes   |        |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_3.golden

@@ -0,0 +1,5 @@
+|   | prime | square |
+|---|-------|--------|
+| 1 | (yes) | yes    |
+| 2 | yes   |        |
+| 3 | yes   |        |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_4.golden

@@ -0,0 +1,5 @@
+|   | prime | square | triangle |
+|---|-------|--------|----------|
+| 1 | (yes) | yes    |          |
+| 2 | yes   |        |          |
+| 3 | yes   |        | yes      |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_5.golden

@@ -0,0 +1,6 @@
+|   | prime | square | triangle |
+|---|-------|--------|----------|
+| 1 | (yes) | yes    |          |
+| 2 | yes   |        |          |
+| 3 | yes   |        | yes      |
+| 4 |       | yes    |          |

hack/generator/pkg/reporting/testdata/TestTable_StepByStep_GivesExpectedResults_step_6.golden

@@ -0,0 +1,7 @@
+|   | prime | square | triangle |
+|---|-------|--------|----------|
+| 1 | (yes) | yes    |          |
+| 2 | yes   |        |          |
+| 3 | yes   |        | yes      |
+| 4 |       | yes    |          |
+| 5 | yes   |        |          |