add :common subpackage without libdparse
#54
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WebFreak001
commented
May 20, 2021
Comment on lines
+39
to
+97
| static Comment parse(string text, string[string] macros, bool removeUnknown, | ||
| string delegate(string) highlightFn) | ||
| out(retVal) | ||
| { | ||
| assert(retVal.sections.length >= 2); | ||
| } | ||
| do | ||
| { | ||
| import std.algorithm : find; | ||
| import ddoc.macros : expand; | ||
|
|
||
| auto sections = splitSections(text); | ||
| string[string] sMacros = macros; | ||
| auto m = sections.find!(p => p.name == "Macros"); | ||
| const e = sections.find!(p => p.name == "Escapes"); | ||
| auto p = sections.find!(p => p.name == "Params"); | ||
| if (m.length) | ||
| { | ||
| if (!doMapping(m[0])) | ||
| throw new DdocParseException("Unable to parse Key/Value pairs", m[0].content); | ||
| foreach (kv; m[0].mapping) | ||
| sMacros[kv[0]] = kv[1]; | ||
| } | ||
| if (e.length) | ||
| { | ||
| assert(0, "Escapes not handled yet"); | ||
| } | ||
| if (p.length) | ||
| { | ||
| if (!doMapping(p[0])) | ||
| throw new DdocParseException("Unable to parse Key/Value pairs", p[0].content); | ||
| foreach (ref kv; p[0].mapping) | ||
| kv[1] = expand(Lexer(highlightFn(kv[1])), sMacros, removeUnknown); | ||
| } | ||
|
|
||
| foreach (ref Section sec; sections) | ||
| { | ||
| if (sec.name != "Macros" && sec.name != "Escapes" && sec.name != "Params") | ||
| sec.content = expand(Lexer(highlightFn(sec.content)), sMacros, removeUnknown); | ||
| } | ||
| return Comment(sections); | ||
| } | ||
| } | ||
|
|
||
| private: | ||
| bool doMapping(ref Section s) | ||
| { | ||
| import ddoc.macros : KeyValuePair, parseKeyValuePair; | ||
|
|
||
| auto lex = Lexer(s.content); | ||
| KeyValuePair[] pairs; | ||
| if (parseKeyValuePair(lex, pairs)) | ||
| { | ||
| foreach (idx, kv; pairs) | ||
| s.mapping ~= kv; | ||
| return true; | ||
| } | ||
| return false; | ||
| } |
There was a problem hiding this comment.
this code is pure copy-paste from ddoc.comments, but changing the legacy parameter and making it more flexible with a callback.
Hackerpilot
reviewed
May 20, 2021
Comment on lines
+6
to
+7
| * Copyright: © 2014 Economic Modeling Specialists, Intl. | ||
| * Authors: Brian Schott, Mathias 'Geod24' Lang, Jan Jurzitza |
There was a problem hiding this comment.
This is probably not accurate for a new file.
There was a problem hiding this comment.
most of the content is copy pasted, so I kept the authors
Hackerpilot
reviewed
May 20, 2021
Hackerpilot
reviewed
May 20, 2021
| ], | ||
| "dependencies": { | ||
| "libdparse": ">=0.13.0 <0.18.0" | ||
| } | ||
| "libdparse": ">=0.13.0 <1.0.0", |
Hackerpilot
approved these changes
May 20, 2021
adds ddoc.unhighlight and ddoc.types modules for more useful usage. Unhighlight can be used to emit embedded D code as-is, instead of parsing it and (potentially) causing issues as well as needing a dependency on libdparse. The ddoc.types module now contains the Comment struct that was in ddoc.comments and has a new parseUnexpanded call, which puts the split sections directly into the comment, without expanding, which is commonly used for example in D-Scanner to just check if a comment is `ditto` or contains some section with a given name.
|
Thank you for doing this! |
|
@WebFreak001 This seems to have broken something in libbdoc. I tried using the newest version of dscanner (that contains the version of libddoc with this PR), but ended up with an error [1]: I think that the breakage comes from here: Any ideas for a quick fix? |
|
hm no logic has changed though, all the old functions should still have the same execution. Do you know which file this happens on? |
|
yes, it is |
|
can't reproduce locally (issue with GC deallocating when memory is limited?) |
|
Have you tried running dscanner (commit: 2400d9c9e7763c264412048021d0bd8f977d7e0c) on phobos? (via make -f posix.mak dscanner). This fails on both my machine and the phobos testing pipeline |
|
oh I'll try that later, I tried |
|
I investigated this issue and this PR isn't the culprit. The issue might be with dscanner after all. |
|
It turns out dscanner starts failing phobos with the UTFException since: dlang-community/D-Scanner@a40492b . It looks like the libdparse upgrade introduced the regression. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
adds
ddoc.unhighlightandddoc.typesmodules for more useful usage. Unhighlight can be used to emit embedded D code as-is, instead of parsing it and (potentially) causing issues as well as needing adependency on libdparse.
The
ddoc.typesmodule now contains the Comment struct that was in ddoc.comments and has a newparseUnexpandedandparsecall, which does what parseComment has done before, but with unhighlight, without expanding, which is commonly used for example in D-Scanner to just check if a comment isdittoor contains some section with a given name.With only minor modifications to D-Scanner just the
libddoc:commonsubpackage can be used there, avoiding keeping libddoc libdparse up-to-date and synced.Makes the root libdparse version any version up to
<1.0.0, because only tokenization is used and updates should not be needed that often. Having the separate common package makes it less likely that things will break for packages not needing libdparse, other packages will manually specify versions that will make ddoc use older versions if necessary.