Skip to content

Commit

Permalink
Merge branch 'master' into sync-fork
Browse files Browse the repository at this point in the history
  • Loading branch information
catherio authored Jan 22, 2025
2 parents 268b660 + 493108e commit 6c8998f
Show file tree
Hide file tree
Showing 28 changed files with 804 additions and 183 deletions.
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Changelog

## 3.10.14-post2

### Fixed

- Updated tests to correctly expect NaN/Infinity support and overflow handling
- Fix behavior on 0-dimensional arrays/tensors
- Fix behavior where Infinity/NaN were incorrectly being written as strings

## 3.10.14-post1

### Added

- Support serializing NaN and Infinity
- Support serializing PyTorch tensors when numpy serialization is enabled.


## 3.10.15

Expand Down
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ library for JSON and is more correct than the standard json library or other
third-party libraries. It serializes
[dataclass](https://github.com/ijl/orjson?tab=readme-ov-file#dataclass),
[datetime](https://github.com/ijl/orjson?tab=readme-ov-file#datetime),
[numpy](https://github.com/ijl/orjson?tab=readme-ov-file#numpy), and
[numpy](https://github.com/ijl/orjson?tab=readme-ov-file#numpy), [PyTorch](https://github.com/ijl/orjson?tab=readme-ov-file#pytorch), and
[UUID](https://github.com/ijl/orjson?tab=readme-ov-file#uuid) instances natively.

[orjson.dumps()](https://github.com/ijl/orjson?tab=readme-ov-file#serialize) is
Expand Down Expand Up @@ -798,6 +798,19 @@ orjson natively serializes `numpy.ndarray` and individual
`numpy.uintp`, `numpy.intp`, `numpy.datetime64`, and `numpy.bool`
instances.

### pytorch

orjson natively serializes PyTorch tensors (`torch.Tensor`) by converting them to numpy arrays. This requires both numpy and PyTorch to be installed, and the `OPT_SERIALIZE_NUMPY` option to be enabled:

```python
>>> import orjson, torch
>>> tensor = torch.tensor([[1, 2], [3, 4]])
>>> orjson.dumps(tensor, option=orjson.OPT_SERIALIZE_NUMPY)
b'[[1,2],[3,4]]'
```

The tensor must be on CPU and have a dtype that can be converted to a numpy array. GPU tensors are automatically moved to CPU before serialization.

orjson is compatible with both numpy v1 and v2.

orjson is faster than all compared libraries at serializing
Expand Down
30 changes: 12 additions & 18 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ fn main() {
println!("cargo:rustc-check-cfg=cfg(Py_3_8)");
println!("cargo:rustc-check-cfg=cfg(Py_3_9)");
println!("cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)");
println!("cargo:rustc-check-cfg=cfg(yyjson_allow_inf_and_nan)");

let python_config = pyo3_build_config::get();
for cfg in python_config.build_script_outputs() {
Expand Down Expand Up @@ -61,25 +62,18 @@ fn main() {
panic!("ORJSON_DISABLE_YYJSON and --features=yyjson both enabled.")
}
} else {
match cc::Build::new()
// Compile yyjson
cc::Build::new()
.file("include/yyjson/yyjson.c")
.include("include/yyjson")
.define("YYJSON_DISABLE_NON_STANDARD", "1")
.define("YYJSON_DISABLE_UTF8_VALIDATION", "1")
.define("YYJSON_DISABLE_UTILS", "1")
.define("YYJSON_DISABLE_WRITER", "1")
.try_compile("yyjson")
{
Ok(_) => {
println!("cargo:rustc-cfg=feature=\"yyjson\"");
}
Err(_) => {
if env::var("CARGO_FEATURE_YYJSON").is_ok() {
panic!(
"yyjson was enabled but the build failed. To build with a different backend do not specify the feature."
)
}
}
.compile("yyjson");

// Link against Python
let python_config = pyo3_build_config::get();
for cfg in python_config.build_script_outputs() {
println!("{cfg}");
}

println!("cargo:rustc-cfg=feature=\"yyjson\"");
println!("cargo:rustc-cfg=yyjson_allow_inf_and_nan");
}
}
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip28.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[NaN]
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip29.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[Infinity]
1 change: 1 addition & 0 deletions data/roundtrip/roundtrip30.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[NaN,Infinity,-Infinity]
Loading

0 comments on commit 6c8998f

Please sign in to comment.