feat: use go1.19 documentation parsing to parse documentation blocks (#…

…87)

This change makes the initial switch over to using the go 1.19+ pattern
of documentation parsing which supports new constructs like lists and
links and tweaks the parsing behavior of some other existing constructs.
Links are not yet functional but lists and other constructs should be
parsed correctly as of this change.

BREAKING CHANGE: Renderer functions no longer end with newlines and
golang 1.19's documentation parser behaves differently for some inputs
than the previous parser

README.md

@@ -5,6 +5,8 @@
 [![Go Reference](https://pkg.go.dev/badge/github.com/princjef/gomarkdoc.svg)](https://pkg.go.dev/github.com/princjef/gomarkdoc)
 [![codecov](https://codecov.io/gh/princjef/gomarkdoc/branch/master/graph/badge.svg?token=171XNH5XLT)](https://codecov.io/gh/princjef/gomarkdoc)
 
+
+
 # gomarkdoc
 
 ```go
@@ -13,7 +15,7 @@ import "github.com/princjef/gomarkdoc"
 
 Package gomarkdoc formats documentation for one or more packages as markdown for usage outside of the main https://pkg.go.dev site. It supports custom templates for tweaking representation of documentation at fine\-grained levels, exporting both exported and unexported symbols, and custom formatters for different backends.
 
-\# Command Line Usage
+### Command Line Usage
 
 If you want to use this package as a command\-line tool, you can install the command by running the following on go 1.16\+:
 
@@ -64,11 +66,11 @@ The gomarkdoc command processes each of the provided packages, generating docume
 gomarkdoc --output doc.md .
 ```
 
-\# Package Specifiers
+### Package Specifiers
 
 The gomarkdoc tool supports generating documentation for both local packages and remote ones. To specify a local package, start the name of the package with a period \(.\) or specify an absolute path on the filesystem. All other package signifiers are assumed to be remote packages. You may specify both local and remote packages in the same command invocation as separate arguments.
 
-\# Output Redirection
+### Output Redirection
 
 By default, the documentation generated by the gomarkdoc command is sent to standard output, where it can be redirected to a file. This can be useful if you want to perform additional modifications to the documentation or send it somewhere other than a file. However, keep in mind that there are some inconsistencies in how various shells/platforms handle redirected command output \(for example, Powershell encodes in UTF\-16, not UTF\-8\). As a result, the \-\-output option described below is recommended for most use cases.
 
@@ -84,51 +86,35 @@ gomarkdoc --output '{{.Dir}}/README.md' ./...
 
 You can see all of the data available to the output template in the PackageSpec struct in the github.com/princjef/gomarkdoc/cmd/gomarkdoc package.
 
-\# Template Overrides
+### Template Overrides
 
 The documentation information that is output is formatted using a series of text templates for the various components of the overall documentation which get generated. Higher level templates contain lower level templates, but any template may be replaced with an override template using the \-\-template/\-t option. The full list of templates that may be overridden are:
 
-```
-- file:    generates documentation for a file containing one or more
-  packages, depending on how the tool is configured. This is the
-  root template for documentation generation.
+- file: generates documentation for a file containing one or more packages, depending on how the tool is configured. This is the root template for documentation generation.
 
 - package: generates documentation for an entire package.
 
-- type:    generates documentation for a single type declaration, as well
-  as any related functions/methods.
+- type: generates documentation for a single type declaration, as well as any related functions/methods.
 
-- func:    generates documentation for a single function or method. It may
-  be referenced from within a type, or directly in the package,
-  depending on nesting.
+- func: generates documentation for a single function or method. It may be referenced from within a type, or directly in the package, depending on nesting.
 
-- value:   generates documentation for a single variable or constant
-  declaration block within a package.
+- value: generates documentation for a single variable or constant declaration block within a package.
 
-- index:   generates an index of symbols within a package, similar to what
-  is seen for godoc.org. The index links to types, funcs,
-  variables, and constants generated by other templates, so it may
-  need to be overridden as well if any of those templates are
-  changed in a material way.
+- index: generates an index of symbols within a package, similar to what is seen for godoc.org. The index links to types, funcs, variables, and constants generated by other templates, so it may need to be overridden as well if any of those templates are changed in a material way.
 
-- example: generates documentation for a single example for a package or
-  one of its symbols. The example is generated alongside whichever
-  symbol it represents, based on the standard naming conventions
-  outlined in https://blog.golang.org/examples#TOC_4.
+- example: generates documentation for a single example for a package or one of its symbols. The example is generated alongside whichever symbol it represents, based on the standard naming conventions outlined in https://blog.golang.org/examples#TOC_4 .
 
-- doc:     generates the freeform documentation block for any of the above
-  structures that can contain a documentation section.
+- doc: generates the freeform documentation block for any of the above structures that can contain a documentation section.
 
-- import:  generates the import code used to pull in a package.
-```
+- import: generates the import code used to pull in a package.
 
 Overriding with the \-t option uses a key\-vaule pair mapping a template name to the file containing the contents of the override template to use. Specified template files must exist:
 
 ```
 gomarkdoc -t package=custom-package.gotxt -t doc=custom-doc.gotxt .
 ```
 
-\# Additional Options
+### Additional Options
 
 As with the godoc tool itself, only exported symbols will be shown in documentation. This can be expanded to include all symbols in a package by adding the \-\-include\-unexported/\-u flag.
 
@@ -182,13 +168,13 @@ Some features of gomarkdoc rely on being able to detect information from the git
 gomarkdoc --repository.url "https://github.com/princjef/gomarkdoc" --repository.default-branch master --repository.path / -o README.md .
 ```
 
-\# Configuring via File
+### Configuring via File
 
 If you want to reuse configuration options across multiple invocations, you can specify a file in the folder where you invoke gomarkdoc containing configuration information that you would otherwise provide on the command line. This file may be a JSON, TOML, YAML, HCL, env, or Java properties file, but the name is expected to start with .gomarkdoc \(e.g. .gomarkdoc.yml\).
 
 All configuration options are available with the camel\-cased form of their long name \(e.g. \-\-include\-unexported becomes includeUnexported\). Template overrides are specified as a map, rather than a set of key\-value pairs separated by =. Options provided on the command line override those provided in the configuration file if an option is present in both.
 
-\# Programmatic Usage
+### Programmatic Usage
 
 While most users will find the command line utility sufficient for their needs, this package may also be used programmatically by installing it directly, rather than its command subpackage. The programmatic usage provides more flexibility when selecting what packages to work with and what components to generate documentation for.
 
@@ -237,7 +223,7 @@ func main() {
 }
 ```
 
-\# Examples
+### Examples
 
 This project uses itself to generate the README files in github.com/princjef/gomarkdoc and its subdirectories. To see the commands that are run to generate documentation for this repository, take a look at the Doc\(\) and DocVerify\(\) functions in magefile.go and the .gomarkdoc.yml file in the root of this repository. To run these commands in your own project, simply replace \`go run ./cmd/gomarkdoc\` with \`gomarkdoc\`.
 
@@ -257,7 +243,7 @@ Know of another project that is using gomarkdoc? Open an issue with a descriptio
   - [func WithTemplateOverride(name, tmpl string) RendererOption](<#func-withtemplateoverride>)
 
 
-## type [Renderer](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L15-L19>)
+## type [Renderer](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L16-L20>)
 
 Renderer provides capabilities for rendering various types of documentation with the configured format and templates.
 
@@ -267,78 +253,76 @@ type Renderer struct {
 }
 ```
 
-### func [NewRenderer](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L30>)
+### func [NewRenderer](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L31>)
 
 ```go
 func NewRenderer(opts ...RendererOption) (*Renderer, error)
 ```
 
 NewRenderer initializes a Renderer configured using the provided options. If nothing special is provided, the created renderer will use the default set of templates and the GitHubFlavoredMarkdown.
 
-### func \(\*Renderer\) [Example](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L140>)
+### func \(\*Renderer\) [Example](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L181>)
 
 ```go
 func (out *Renderer) Example(ex *lang.Example) (string, error)
 ```
 
 Example renders an example's documentation to a string. You can change the rendering of the example by overriding the "example" template or one of the templates it references.
 
-### func \(\*Renderer\) [File](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L112>)
+### func \(\*Renderer\) [File](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L153>)
 
 ```go
 func (out *Renderer) File(file *lang.File) (string, error)
 ```
 
 File renders a file containing one or more packages to document to a string. You can change the rendering of the file by overriding the "file" template or one of the templates it references.
 
-### func \(\*Renderer\) [Func](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L126>)
+### func \(\*Renderer\) [Func](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L167>)
 
 ```go
 func (out *Renderer) Func(fn *lang.Func) (string, error)
 ```
 
 Func renders a function's documentation to a string. You can change the rendering of the package by overriding the "func" template or one of the templates it references.
 
-### func \(\*Renderer\) [Package](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L119>)
+### func \(\*Renderer\) [Package](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L160>)
 
 ```go
 func (out *Renderer) Package(pkg *lang.Package) (string, error)
 ```
 
 Package renders a package's documentation to a string. You can change the rendering of the package by overriding the "package" template or one of the templates it references.
 
-### func \(\*Renderer\) [Type](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L133>)
+### func \(\*Renderer\) [Type](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L174>)
 
 ```go
 func (out *Renderer) Type(typ *lang.Type) (string, error)
 ```
 
 Type renders a type's documentation to a string. You can change the rendering of the type by overriding the "type" template or one of the templates it references.
 
-## type [RendererOption](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L22>)
+## type [RendererOption](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L23>)
 
 RendererOption configures the renderer's behavior.
 
 ```go
 type RendererOption func(renderer *Renderer) error
 ```
 
-### func [WithFormat](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L102>)
+### func [WithFormat](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L143>)
 
 ```go
 func WithFormat(format format.Format) RendererOption
 ```
 
 WithFormat changes the renderer to use the format provided instead of the default format.
 
-### func [WithTemplateOverride](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L88>)
+### func [WithTemplateOverride](<https://github.com/princjef/gomarkdoc/blob/master/renderer.go#L129>)
 
 ```go
 func WithTemplateOverride(name, tmpl string) RendererOption
 ```
 
 WithTemplateOverride adds a template that overrides the template with the provided name using the value provided in the tmpl parameter.
 
-
-
 Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

cmd/gomarkdoc/README.md

@@ -34,6 +34,4 @@ type PackageSpec struct {
 }
 ```
 
-
-
 Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

format/README.md

@@ -476,6 +476,4 @@ func (f *PlainMarkdown) RawHeader(level int, text string) (string, error)
 
 RawHeader converts the provided text into a header of the provided level without escaping the header text. The level is expected to be at least 1.
 
-
-
 Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

format/devops_test.go

@@ -25,7 +25,7 @@ func TestCodeBlock(t *testing.T) {
 	var f format.AzureDevOpsMarkdown
 	res, err := f.CodeBlock("go", "Line 1\nLine 2")
 	is.NoErr(err)
-	is.Equal(res, "```go\nLine 1\nLine 2\n```\n\n")
+	is.Equal(res, "```go\nLine 1\nLine 2\n```")
 }
 
 func TestCodeBlock_noLanguage(t *testing.T) {
@@ -34,7 +34,7 @@ func TestCodeBlock_noLanguage(t *testing.T) {
 	var f format.AzureDevOpsMarkdown
 	res, err := f.CodeBlock("", "Line 1\nLine 2")
 	is.NoErr(err)
-	is.Equal(res, "```\nLine 1\nLine 2\n```\n\n")
+	is.Equal(res, "```\nLine 1\nLine 2\n```")
 }
 
 func TestHeader(t *testing.T) {
@@ -43,14 +43,14 @@ func TestHeader(t *testing.T) {
 		level  int
 		result string
 	}{
-		{"header text", 1, "# header text\n\n"},
-		{"level 2", 2, "## level 2\n\n"},
-		{"level 3", 3, "### level 3\n\n"},
-		{"level 4", 4, "#### level 4\n\n"},
-		{"level 5", 5, "##### level 5\n\n"},
-		{"level 6", 6, "###### level 6\n\n"},
-		{"other level", 12, "###### other level\n\n"},
-		{"with * escape", 2, "## with \\* escape\n\n"},
+		{"header text", 1, "# header text"},
+		{"level 2", 2, "## level 2"},
+		{"level 3", 3, "### level 3"},
+		{"level 4", 4, "#### level 4"},
+		{"level 5", 5, "##### level 5"},
+		{"level 6", 6, "###### level 6"},
+		{"other level", 12, "###### other level"},
+		{"with * escape", 2, "## with \\* escape"},
 	}
 
 	for _, test := range tests {
@@ -79,8 +79,8 @@ func TestRawHeader(t *testing.T) {
 		level  int
 		result string
 	}{
-		{"header text", 1, "# header text\n\n"},
-		{"with * escape", 2, "## with * escape\n\n"},
+		{"header text", 1, "# header text"},
+		{"with * escape", 2, "## with * escape"},
 	}
 
 	for _, test := range tests {
@@ -173,7 +173,7 @@ func TestListEntry(t *testing.T) {
 	var f format.AzureDevOpsMarkdown
 	res, err := f.ListEntry(0, "list entry text")
 	is.NoErr(err)
-	is.Equal(res, "- list entry text\n")
+	is.Equal(res, "- list entry text")
 }
 
 func TestListEntry_nested(t *testing.T) {
@@ -182,7 +182,7 @@ func TestListEntry_nested(t *testing.T) {
 	var f format.AzureDevOpsMarkdown
 	res, err := f.ListEntry(2, "nested text")
 	is.NoErr(err)
-	is.Equal(res, "    - nested text\n")
+	is.Equal(res, "    - nested text")
 }
 
 func TestListEntry_empty(t *testing.T) {

format/formatcore/README.md

@@ -40,23 +40,23 @@ func CodeBlock(code string) string
 
 CodeBlock wraps the provided code as a code block. Language syntax highlighting is not supported.
 
-## func [Escape](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L138>)
+## func [Escape](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L136>)
 
 ```go
 func Escape(text string) string
 ```
 
 Escape escapes the special characters in the provided text, but leaves URLs found intact. Note that the URLs included must begin with a scheme to skip the escaping.
 
-## func [GFMAccordion](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L101>)
+## func [GFMAccordion](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L99>)
 
 ```go
 func GFMAccordion(title, body string) string
 ```
 
 GFMAccordion generates a collapsible content. The accordion's visible title while collapsed is the provided title and the expanded content is the body.
 
-## func [GFMAccordionHeader](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L114>)
+## func [GFMAccordionHeader](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L112>)
 
 ```go
 func GFMAccordionHeader(title string) string
@@ -70,62 +70,60 @@ The GFMAccordionHeader is expected to be used in conjunction with GFMAccordionTe
 accordion := GFMAccordionHeader("Accordion Title") + "Accordion Body" + GFMAccordionTerminator()
 ```
 
-## func [GFMAccordionTerminator](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L121>)
+## func [GFMAccordionTerminator](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L119>)
 
 ```go
 func GFMAccordionTerminator() string
 ```
 
 GFMAccordionTerminator generates the code necessary to terminate an accordion after the body. It is expected to be used in conjunction with GFMAccordionHeader\(\). See GFMAccordionHeader for a full description.
 
-## func [GFMCodeBlock](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L45>)
+## func [GFMCodeBlock](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L43>)
 
 ```go
 func GFMCodeBlock(language, code string) string
 ```
 
 GFMCodeBlock wraps the provided code as a code block and tags it with the provided language \(or no language if the empty string is provided\), using the triple backtick format from GitHub Flavored Markdown.
 
-## func [Header](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L51>)
+## func [Header](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L49>)
 
 ```go
 func Header(level int, text string) (string, error)
 ```
 
 Header converts the provided text into a header of the provided level. The level is expected to be at least 1.
 
-## func [Link](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L74>)
+## func [Link](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L72>)
 
 ```go
 func Link(text, href string) string
 ```
 
 Link generates a link with the given text and href values.
 
-## func [ListEntry](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L89>)
+## func [ListEntry](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L87>)
 
 ```go
 func ListEntry(depth int, text string) string
 ```
 
 ListEntry generates an unordered list entry with the provided text at the provided zero\-indexed depth. A depth of 0 is considered the topmost level of list.
 
-## func [Paragraph](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L126>)
+## func [Paragraph](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L124>)
 
 ```go
 func Paragraph(text string) string
 ```
 
-Paragraph formats a paragraph with the provided text as the contents.
+Paragraph formats a paragraph with the provided text as the contents
 
-## func [PlainText](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L175>)
+## func [PlainText](<https://github.com/princjef/gomarkdoc/blob/master/format/formatcore/base.go#L173>)
 
 ```go
 func PlainText(text string) string
 ```
 
 PlainText converts a markdown string to the plain text that appears in the rendered output.
 
-
-
 Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)