Allow custom sources for Javadoc and Scaladoc #1385

Iltotore · 2021-06-23T13:43:39Z

This pull request allows the user to define custom sources for Javadoc/Scaladoc though processedDocSources:

JavaModule#docJar now uses processedDocSources instead of allSources
ScalaModule#docJar now uses processedDocSources instead of os.walk(compile().classes.path).map(PathRef(_))

Fixes #1384 .

Signed-off-by: Iltotore <[email protected]>

lefou

I'm not that happy with the name processedDocSources, as I immediately asked myself what "processed" could mean. Why not just docSources? Some scaladoc should indicate the purpose of the new target.

Also, one motivation to customize the docSources could be to adapt the set of sources files, so it might be probably worth a though, if we could feed files instead of dirs.

It looks like some unrelated formatting changes went into this PR.

scalalib/src/ScalaModule.scala

Iltotore · 2021-06-24T06:00:50Z

It looks like some unrelated formatting changes went into this PR.

Yes, I just noticed my IDE reformatted the code. If needed, I can undo this reformat.

I'm not that happy with the name processedDocSources, as I immediately asked myself what "processed" could mean. Why not just docSources?

I also thook about docSources but this method already exists and it isn't the same feature. According to scaladocs, docSources is for static (not processed by scaladoc) files.

What if we have, for some obscure reason, a tasty file and we just want to have it in our doc without processing it ?

lefou · 2021-06-24T09:13:26Z

It looks like some unrelated formatting changes went into this PR.

Yes, I just noticed my IDE reformatted the code. If needed, I can undo this reformat.

That would be nice. If you enable scalafmt in your IDE, it should automatically use the .scalafmt.conf from the repo.

I'm not that happy with the name processedDocSources, as I immediately asked myself what "processed" could mean. Why not just docSources?

I also thook about docSources but this method already exists and it isn't the same feature. According to scaladocs, docSources is for static (not processed by scaladoc) files.

What if we have, for some obscure reason, a tasty file and we just want to have it in our doc without processing it ?

I must have overlooked the existing but quite new docSources. Unfortunately the existing docSources has also misleading name, as it does not contain sources (to be process by the doc tool) but static resources to be bundled as-is. I guess we should rename to docResources and deprecate the old name for the long-term benefit. The whole docSources seems to be unsupported outside of Scala 3, so we might as well clear it up without deprecation cycles, which also allows us to re-use it's name now.

Besides, the "processed" prefix is misleading, as these sources are going to be processed by the doc tool.

Iltotore · 2021-06-24T09:25:35Z

That would be nice. If you enable scalafmt in your IDE, it should automatically use the .scalafmt.conf from the repo.

That's what I did. I will push the code formatted using scalafmt

I guess we should rename to docResources

Do you want me to commit this change then, use docSources instead of processed ?

lefou · 2021-06-25T13:21:50Z

I guess we should rename to docResources

Do you want me to commit this change then, use docSources instead of processed ?

Yes, please!

To avoid conflicts with users who used that target to provide static resources, we should probable output a warning in case the outcome of docSources isn't empty but does not contain any scala or java source files. Something like: "Since mill after 0.9.8 docSources should only be used to provide source files. To provide static resources, please use docResources."

… now docResources) Signed-off-by: Iltotore <[email protected]>

Iltotore

Earlier we talked about reformatting modified files using ScalaFMT but it indicated that they're already formatted correctly.

Iltotore · 2021-06-25T14:15:36Z

Once this change is approved, I will add the suggested warning.

lefou · 2021-06-25T15:59:47Z

Earlier we talked about reformatting modified files using ScalaFMT but it indicated that they're already formatted correctly.

Scalafmt isn't going to remove new lines which you introduced earlier.

scalalib/src/JavaModule.scala

scalalib/src/ScalaModule.scala

Signed-off-by: Iltotore <[email protected]>

scalalib/src/JavaModule.scala

scalalib/src/ScalaModule.scala

Co-authored-by: Tobias Roeser <[email protected]>

Signed-off-by: Iltotore <[email protected]>

scalalib/src/ScalaModule.scala

Signed-off-by: Iltotore <[email protected]>

Iltotore · 2021-07-01T16:23:39Z

This last change works for Scala 3 and Scala 3 Milestone. Haven't tested with Scala 2 yet but it should as I simply applied the same changes (except for tasty files filter) and it also uses packageWithZinc. I'll test it when available.

EDIT: Seems to work in Scala 2.x.

lefou · 2021-07-08T09:53:18Z

There are still failing tests related to scaladoc.

Iltotore · 2021-07-08T20:27:08Z

Actually, I don't really understand where this error comes from:

-- Error: C:\Users\rapha\IdeaProjects\mill\target\workspace\mill\scalalib\DottyDocTests\eval\multiple\multidocs\compile\dest\classes\pkg\SomeClass.tasty:2:407 
2 |??u?6??p?V3☻?@???o?t☻?3???p?V2☻?‼↨ ?☺??☺???p?V3???u?3?3?q?p?3?u?Pu?Pu?3?v?\r☻?3????)3????)u?3???p?Vu?3?R3♥?♠↨???)3???p?V3?↨?????3?↨\♦♦↨????3?↨?????u?3?t♥?P3☻??p?V3?↨‼↨??3☻???p?V3☻?@??☺?????
??☺????~???????????????????????????????????????????????????????☺?????☺?????☻?}???????????????????????????????????????/** A doc comment. */▬?☻?/** A doc comment. */▬?☻?/** A doc comment. */▬?
  |

                          ^
  |                                                illegal character '\u0000'

Seems like an encoding error but if I'm not wrong, I didn't change something about file encoding.

Failing tests are related to pre-release Dotty. I'll check the code tomorrow.

EDIT: After some researches, the compiled classes already have illegal characters once compiled. ~~I'm not sure this issue comes from this PR~~ the master branch doesn't have this issue so it comes from my fork. I don't know if it is an old bug patched in master or if it is caused by my changes. I'll update my fork with the upstream.

lefou · 2021-07-09T12:22:11Z

scalalib/src/ScalaModule.scala

@@ -185,6 +201,8 @@ trait ScalaModule extends JavaModule { outer =>
      )
  }

+  override def docSources: Sources = T.sources(compile().classes.path)


This seem not correct for early dotty / scala 3 versions (isDotty(scalaVersion()) || isScala3Milestone(scalaVersion())) which use dottydoc and consumes scala source files.
Only Scala 3 releases (isScala3(scalaVersion())) use scaladoc 3 tool, which consumes tasty files.

Iltotore · 2021-07-09T14:43:13Z

The last commit should fix this issue. Tests passed in local.

lefou

Your PR looks good now. There are still some newly introduced but unwanted newlines. Can you please remove these, and we are good to go.

scalalib/src/ScalaModule.scala

lefou

Looks good to me. Thank you!

feat: Add processedDocSources target

72abdf2

Signed-off-by: Iltotore <[email protected]>

lefou reviewed Jun 23, 2021

View reviewed changes

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

refactor: Rename processedDocSources to docSources (old docSources is…

eb9abe8

… now docResources) Signed-off-by: Iltotore <[email protected]>

Iltotore commented Jun 25, 2021

View reviewed changes

lefou requested changes Jun 25, 2021

View reviewed changes

scalalib/src/JavaModule.scala Outdated Show resolved Hide resolved

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

scalalib/src/ScalaModule.scala Show resolved Hide resolved

Iltotore added 4 commits June 26, 2021 11:29

feat: Use Sources for docSources

e793429

Signed-off-by: Iltotore <[email protected]>

docs: Document new docSources

ec90f48

Signed-off-by: Iltotore <[email protected]>

fix: omitted docSources for Dottydoc

93e9bdd

Signed-off-by: Iltotore <[email protected]>

docs: Reference docResources in docSources and vice-versa

521a415

Signed-off-by: Iltotore <[email protected]>

Iltotore requested a review from lefou June 26, 2021 10:01

lefou reviewed Jun 27, 2021

View reviewed changes

scalalib/src/JavaModule.scala Outdated Show resolved Hide resolved

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

Iltotore and others added 2 commits June 27, 2021 13:32

docs: Fix documentation of docJar

534bcad

Co-authored-by: Tobias Roeser <[email protected]>

fix: Unintended filter of tasty files for Scaladoc 2.x

c214aa7

Signed-off-by: Iltotore <[email protected]>

lefou requested changes Jun 28, 2021

View reviewed changes

scalalib/src/ScalaModule.scala Show resolved Hide resolved

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

fix: Unexpected usage of source directories instead of source files

a8c5a0a

Signed-off-by: Iltotore <[email protected]>

Merge branch 'com-lihaoyi:main' into main

c32aad3

lefou requested changes Jul 9, 2021

View reviewed changes

fix: Wrong input for Dottydoc

efc3632

lefou reviewed Jul 13, 2021

View reviewed changes

style: Reverse wrong code format in ScalaModule

06d39c3

Iltotore added 2 commits July 14, 2021 09:38

style: Reverse wrong code format in ScalaModule

7f979e4

Merge remote-tracking branch 'origin/main' into main

6c6e6c5

lefou requested changes Jul 14, 2021

View reviewed changes

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

Only feed compile output when using Scala 3, else use all source files.

c5ce54a

lefou reviewed Jul 14, 2021

View reviewed changes

scalalib/src/ScalaModule.scala Outdated Show resolved Hide resolved

Fixing compile issue

72ac21e

lefou approved these changes Jul 14, 2021

View reviewed changes

lefou merged commit a6eef45 into com-lihaoyi:main Jul 14, 2021

lefou modified the milestones: after 0.9.8 (0.10), 0.9.9, after 0.9.9 (0.10) Jul 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow custom sources for Javadoc and Scaladoc #1385

Allow custom sources for Javadoc and Scaladoc #1385

Iltotore commented Jun 23, 2021

lefou left a comment

Iltotore commented Jun 24, 2021 •

edited

Loading

lefou commented Jun 24, 2021

Iltotore commented Jun 24, 2021 •

edited

Loading

lefou commented Jun 25, 2021

Iltotore left a comment

Iltotore commented Jun 25, 2021 •

edited

Loading

lefou commented Jun 25, 2021

Iltotore commented Jul 1, 2021 •

edited

Loading

lefou commented Jul 8, 2021

Iltotore commented Jul 8, 2021 •

edited

Loading

lefou Jul 9, 2021

Iltotore commented Jul 9, 2021 •

edited

Loading

lefou left a comment

lefou left a comment

Allow custom sources for Javadoc and Scaladoc #1385

Allow custom sources for Javadoc and Scaladoc #1385

Conversation

Iltotore commented Jun 23, 2021

lefou left a comment

Choose a reason for hiding this comment

Iltotore commented Jun 24, 2021 • edited Loading

lefou commented Jun 24, 2021

Iltotore commented Jun 24, 2021 • edited Loading

lefou commented Jun 25, 2021

Iltotore left a comment

Choose a reason for hiding this comment

Iltotore commented Jun 25, 2021 • edited Loading

lefou commented Jun 25, 2021

Iltotore commented Jul 1, 2021 • edited Loading

lefou commented Jul 8, 2021

Iltotore commented Jul 8, 2021 • edited Loading

lefou Jul 9, 2021

Choose a reason for hiding this comment

Iltotore commented Jul 9, 2021 • edited Loading

lefou left a comment

Choose a reason for hiding this comment

lefou left a comment

Choose a reason for hiding this comment

Iltotore commented Jun 24, 2021 •

edited

Loading

Iltotore commented Jun 24, 2021 •

edited

Loading

Iltotore commented Jun 25, 2021 •

edited

Loading

Iltotore commented Jul 1, 2021 •

edited

Loading

Iltotore commented Jul 8, 2021 •

edited

Loading

Iltotore commented Jul 9, 2021 •

edited

Loading