Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace strtold with non locale dependent version #337

Closed
wants to merge 14 commits into from
Closed
171 changes: 128 additions & 43 deletions src/json.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8800,64 +8800,149 @@ class basic_json
return result;
}

/*!
@brief parse floating point number
// non locale aware isdigit
// Microsoft in 1252 codepage and others may classify additional single-byte characters as digits using std::isdigit
constexpr bool nl_isdigit(const char c) const
{
return c >= '0' and c <= '9';
}

This function (and its overloads) serves to select the most approprate
standard floating point number parsing function based on the type
supplied via the first parameter. Set this to @a
static_cast<number_float_t*>(nullptr).
/*!
@brief parse string to floating point number

@param[in] type the @ref number_float_t in use
This function is a partial reimplementation of the strtold in order to meet needs of JSON number

@param[in,out] endptr recieves a pointer to the first character after
the number
@param[in] str the string we will parse

@return the floating point number
*/
long double str_to_float_t(long double* /* type */, char** endptr) const
long double strtojnum(const char *str) const
{
return std::strtold(reinterpret_cast<typename string_t::const_pointer>(m_start), endptr);
}
long double result = 0;
char cp = *str;
int exp = 0; // exponent
{
const bool negative_sign = cp == '-';

/*!
@brief parse floating point number
if (cp == '-' or cp == '+')
{
++str;
}

This function (and its overloads) serves to select the most approprate
standard floating point number parsing function based on the type
supplied via the first parameter. Set this to @a
static_cast<number_float_t*>(nullptr).
// read in fractional part of number, until an 'e' is reached.
// count digits after decimal point.
while (nl_isdigit(cp = *str))
{
result = result * 10 + (cp - '0');
++str;
}

@param[in] type the @ref number_float_t in use
if (cp == '.')
{
while (nl_isdigit(cp = *++str))
{
result = result * 10 + (cp - '0');
--exp;
}
}

@param[in,out] endptr recieves a pointer to the first character after
the number
// if negative number, reverse sign
if (negative_sign)
{
result *= -1;
}
}

@return the floating point number
*/
double str_to_float_t(double* /* type */, char** endptr) const
{
return std::strtod(reinterpret_cast<typename string_t::const_pointer>(m_start), endptr);
}
// read in explicit exponent and calculate real exponent.
if (*str == 'e' or *str == 'E')
{
cp = *++str;

/*!
@brief parse floating point number
const bool negative_exp = cp == '-'; // read in exponent sign (+/-)
const bool plus_or_minus = (cp == '-' or cp == '+');
if (plus_or_minus)
{
cp = *++str;
}

This function (and its overloads) serves to select the most approprate
standard floating point number parsing function based on the type
supplied via the first parameter. Set this to @a
static_cast<number_float_t*>(nullptr).
int count = 0; // exponent calculation
if (not nl_isdigit(cp))
{
if (plus_or_minus)
{
*--str;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there a '*' here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look! @nlohmann want to incorp these into your feature branch?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes no sense. Please have a look at https://github.com/nlohmann/json/blob/feature/strtold/src/json.hpp.re2c#L8139 where I started an overworked version of this PR.

}

@param[in] type the @ref number_float_t in use
*--str;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

goto skip_loop;
}

@param[in,out] endptr recieves a pointer to the first character after
the number
while (nl_isdigit(cp))
{
constexpr int imax = std::numeric_limits<int>::max();

@return the floating point number
*/
float str_to_float_t(float* /* type */, char** endptr) const
{
return std::strtof(reinterpret_cast<typename string_t::const_pointer>(m_start), endptr);
if ((imax - std::abs(exp) - (cp - '0')) / 10 > count)
{
count *= 10;
count += cp - '0';
}
else
{
count = imax - exp;
break;
}

cp = *++str;
}
skip_loop:
exp += negative_exp ? -count : count;
}

// adjust number by powers of ten specified by format and exponent.
constexpr std::array<long double, 9> powerof10 = {
{1.e1L, 1.e2L, 1.e4L, 1.e8L, 1.e16L, 1.e32L, 1.e64L, 1.e128L, 1.e256L}
};

// round to INF if our exponent is larger than representable number
if (exp > std::numeric_limits<long double>::max_exponent10)
{
constexpr long double inf = std::numeric_limits<long double>::infinity();
result = (result < 0) ? -inf : inf;
}
// round to zero if our exponent is smaller than representable number
else if (exp < std::numeric_limits<long double>::min_exponent10)
{
result = 0.0L;
}
// iteratively divide result for negative exp
else if (exp < 0)
{
// make exp positive for loop below
exp *= -1;

// check enabled exp bits on lookup powerof10 lookup table
for (std::size_t count = 0; exp; ++count, exp >>= 1)
{
if (exp & 1)
{
result /= powerof10[count];
}
}
}
// iteratively multiply result for positive exp
else
{
// check enabled exp bits on lookup powerof10 lookup table
for (std::size_t count = 0; exp; ++count, exp >>= 1)
{
if (exp & 1)
{
result *= powerof10[count];
}
}
}

return result;
}

/*!
Expand Down Expand Up @@ -8958,8 +9043,8 @@ class basic_json
}
else
{
// parse with strtod
result.m_value.number_float = str_to_float_t(static_cast<number_float_t*>(nullptr), NULL);
// convert string by json number format to floating point
result.m_value.number_float = strtojnum(reinterpret_cast<typename string_t::const_pointer>(m_start));
}

// save the type
Expand Down
Loading