Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-71139] Fail fast when serializing invalid XML 1.1 data #7875

Merged
merged 2 commits into from
Apr 26, 2023

Conversation

jglick
Copy link
Member

@jglick jglick commented Apr 24, 2023

See JENKINS-71139. Whether the usage that was broken was valid to begin with is open to interpretation; given jenkinsci/junit-plugin#521 this just fails during the write rather than the read.

It is possible there are other plugins saving arbitrary user-generated text content via XStream which might be affected, though it seems unlikely any would be as widely used as junit. For a general defense, PrettyPrintWriter.writeText (and .writeAttributeValue) could quietly replace NULs with some placeholder such as ^@ and not pretend to round-trip them.

Testing done

The suppressed tests fail with

javax.xml.stream.XMLStreamException: ParseError at [row,col]:[3,13]
Message: The reference to entity "y" must end with the ';' delimiter.

If the main source change from #7778 is commented out, they pass (but then of course testEmojiEscaped fails).

This behavior seems to be intentional in the StAX driver: https://github.com/x-stream/xstream/blob/289ae780001c31d7d5d75e0d58608c13f44549a2/xstream/src/test/com/thoughtworks/xstream/io/xml/StaxReaderTest.java#L64-L67

Proposed changelog entries

Do not write NUL values to XML files. A technically illegal � could be written to Jenkins XML files but could no longer be read. Now the write will fail as well (regression in 2.398).

Proposed upgrade guidelines

Jenkins XML files can no longer save text content with the ASCII NUL character (U+0000). In particular, if you are using the junit plugin to publish test results, be sure to update it to at least 1198.ve38db_d1b_c975 to avoid problems with new builds. (Test results published with older versions of the plugin will remain unreadable.)

Maintainer checklist

Before the changes are marked as ready-for-merge:

  • There are at least two (2) approvals for the pull request and no outstanding requests for change.
  • Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
  • Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
  • Proper changelog labels are set so that the changelog can be generated automatically.
  • If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
  • If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

@basil basil changed the title [JENKINS-71139] Reproducing XStream2 problem with NUL [JENKINS-71139] Fail fast when serializing invalid XML 1.1 data Apr 24, 2023
Copy link
Member

@basil basil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres does not accept the null character in strings, either, so I think it is reasonable to reject serialization and deserialization of strings with the null character and adapt the rest of the Jenkins ecosystem to comply with the XML 1.1 standard if it does not already. This JUnit issue is the only case I am aware of where we were not in compliance, as no other issues have been reported. I was about to look into what it would take to get PrettyPrintWriter to fail fast when provided with input that does not comply with the XML 1.1 standard, but looks like you beat me to it in commit a659507. If other cases are discovered where we do not comply with the XML 1.1 standard, I can fix them up and/or guide users to clean up the invalid data. As you wrote in Jira,

Seems unlikely to affect many users

@MarkEWaite MarkEWaite added bug For changelog: Minor bug. Will be listed after features regression-fix Pull request that fixes a regression in one of the previous Jenkins releases and removed bug For changelog: Minor bug. Will be listed after features labels Apr 25, 2023
@MarkEWaite
Copy link
Contributor

This PR is now ready for merge. We will merge it after approximately 24 hours if there is no negative feedback.

@MarkEWaite MarkEWaite merged commit 458c686 into jenkinsci:master Apr 26, 2023
krisstern pushed a commit to krisstern/jenkins that referenced this pull request Apr 26, 2023
…insci#7875)

* [JENKINS-71139] Reproducing `XStream2` problem with NUL

* Arguably better to switch from “quirks” to XML 1.1 mode, failing during write not just read

(cherry picked from commit 458c686)
@jglick jglick deleted the xml-JENKINS-71139 branch April 26, 2023 18:17
return new PrettyPrintWriter(out, getNameCoder());
return new PrettyPrintWriter(out, PrettyPrintWriter.XML_1_1, getNameCoder());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Causes JENKINS-71182

NotMyFault pushed a commit to NotMyFault/jenkins that referenced this pull request May 15, 2023
…insci#7875)

* [JENKINS-71139] Reproducing `XStream2` problem with NUL

* Arguably better to switch from “quirks” to XML 1.1 mode, failing during write not just read

(cherry picked from commit 458c686)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression-fix Pull request that fixes a regression in one of the previous Jenkins releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants