Skip to content

[EMAIL-209] Return re-readable attachment data sources from MimeMessageParser - #403

Open
sharanggupta wants to merge 1 commit into
apache:masterfrom
sharanggupta:EMAIL-209-repeatable-attachment-stream
Open

sharanggupta wants to merge 1 commit into
apache:masterfrom
sharanggupta:EMAIL-209-repeatable-attachment-stream

Conversation

@sharanggupta

Copy link
Copy Markdown

Fixes EMAIL-209.

What

Since 1.6.0, the attachment data sources of MimeMessageParser can be read only once. createDataSource() opens the part's input stream when the message is parsed and wraps that one stream in an InputStreamDataSource, whose getInputStream() returns it again on every call. A second read gets the stream where the first one stopped, so code that reads an attachment twice, as in the report, gets empty or truncated content. The DataSource contract says that "a new InputStream object must be returned each time this method is called, and the stream must be positioned at the beginning of the data." In 1.5, the attachment was a ByteArrayDataSource, which could be read any number of times, but it held a copy of every attachment in memory, which EMAIL-207 removed.

How

createDataSource() now returns a small private AttachmentDataSource that keeps the part's own DataSource, with the base content type and the name as before. Its getInputStream() asks the part's data source for a new stream on each call; for a parsed message that is a MimePartDataSource, which decodes the content again each time. So every read starts at the beginning of the data, and the content is still not copied into memory, which keeps the fix for EMAIL-207. As with InputStreamDataSource, getOutputStream() throws UnsupportedOperationException.

createDataSource() still opens the part's stream once while parsing, and closes it right away, so that an exception from opening the part's content is still thrown by parse() as before, and not only when the attachment is read.

The change is made in both the Jakarta and the javax modules. InputStreamDataSource itself is unchanged, since it is public API, but the parser no longer uses it.

Two consequences worth knowing about:

  • An attachment's data source now keeps a reference to its part, and through it to the message, instead of to an open stream of the part. Code that keeps an attachment after dropping the parser and the message keeps the message reachable. That is the price of reading the content again on demand without a copy.
  • Each read gets the current content from the part's data source. For a parsed message this is always the same content. For a message built in memory with a FileDataSource or a URLDataSource attachment and parsed without serialising it, the file or URL is read again on each call, like these data sources do themselves.

Tests

MimeMessageParserTest in both modules:

  • testAttachmentCanBeReadTwice reads the 7bit text attachment of multipart-text-attachment.eml twice, and checks that getOutputStream() throws UnsupportedOperationException.
  • testBase64AttachmentCanBeReadTwice reads the base64 PDF attachment of attachment-only.eml twice and compares the bytes.

On master the second reads are empty: expected: <test> but was: <>, and array lengths differ, expected: <7555> but was: <0>. With this change both tests pass. testParseNoHeaderSeperatorWithOutOfMemory from EMAIL-207 still passes.

The default Maven goal (clean verify apache-rat:check checkstyle:check pmd:check pmd:cpd-check spotbugs:check javadoc:javadoc) passes locally on Java 17.

Before you push a pull request, review this list:

  • Read the contribution guidelines for this project.
  • Read the ASF Generative Tooling Guidance if you use Artificial Intelligence (AI).
  • I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute? Claude (Anthropic) helped analyse the regression and draft the change and tests; I reviewed them against the code and ran the build.
  • Run a successful build using the default Maven goal with mvn; that's mvn on the command line by itself.
  • Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible, but it is a best practice.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Each commit in the pull request should have a meaningful subject line and body. Note that a maintainer may squash commits during the merge process.

…geParser

Since EMAIL-207, MimeMessageParser wrapped the one input stream of each
attachment in an InputStreamDataSource, so a second getInputStream()
call returned the consumed stream, contrary to the DataSource contract.
Return a data source that opens a new stream of the part's content on
each call instead, which still keeps no copy of the content. The part's
stream is still opened once while parsing so that errors are reported
by parse() as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant