Add support for BIO_FP_TEXT in file BIOs

[WillChilds-Klein]

Issues

Addresses CryptoAlg-2036

Notes

OpenSSL defines a "close flag" BIO_FP_TEXT which is used by callers to indicate that file-backed BIOs should be opened in "text" mode as opposed to the default of raw "binary" mode. Of platforms currently supported by AWS-LC, this is only meaningful on windows. From linux's fopen man page:

        The mode string can also include the letter 'b' either as a last
        character or as a character between the characters in any of the
        two-character strings described above.  This is strictly for
        compatibility with ISO C and has no effect; the 'b' is ignored on
        all POSIX conforming systems, including Linux.  (Other systems
        may treat text files and binary files differently, and adding the
        'b' may be a good idea if you do I/O to a binary file and expect
        that your program may be ported to non-UNIX environments.)

And from Windows:

t     Open in text (translated) mode. Carriage return-line feed (CR-LF)
      combinations are translated into single line feeds (LF) on input and LF
      characters are translated to CR-LF combinations on output. Also, CTRL+Z is
      interpreted as an end-of-file character on input.
b     Open in binary (untranslated) mode; translations involving
      carriage-return and line feed characters are suppressed.

The terminology here is a bit confusing. In text mode, "Input" here refers to reading files and "output" refers to writing files to disk. So, if an application is using conventional unixy LF line endings (\n) and then writes a line of text to a file, the Windows CRT will instead write a CRLF line ending (\r\n). When reading from a file, the CRT will translate CRLF back to LF transparently to the application. In binary mode, what you see is what you get and no translation occurs.

OpenSSL supports setting text mode for a variety of BIO operations (including when opening files with fopen), but only sets this flag internally when calling BIO_new_fp. Note that BIO_new_fp operates on open FILE *s, and does not itself open the file. Instead, it uses the Windows CRT-specific function _setmode to set binary/text mode on the already opened file.

To implement this functionality in AWS-LC, we only set translation mode in functions using the BIO_C_SET_FILE_PTR control directive: BIO_set_fp and BIO_new_fp. AWS-LC's BIO functions that call fopen along the way do not provide the caller a parameter for specifying flags, so we're limited here by the interface.

Finally, it's a little odd to refer to BIO_FP_TEXT as a "close flag", as the behavior it determines has nothing to do with how or if the file is closed. However, this is how OpenSSL's code refers to it, so it's the terminology we use here.

Links

Documentation links for the Windows-specific functions used in this change can be found here:

• _setmode: used to set translation mode on an open file
• _fileno: used to convert a FILE * to a file descriptor

Testing

• CI runs with new platform-aware test

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

[justsmth]

Not part of this PR, but I noticed that BIO_get_fp also doesn't have much test coverage.