Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upSpeed up boolean encoding/decoding #68
Open
Comments
|
It's weird that this slower than what orjson does. That's the implementation of unsafe {
PyObject::from_borrowed_ptr(
py,
if *self {
ffi::Py_True()
} else {
ffi::Py_False()
},
)
}And that's the implementation of debug_assert!(
!ptr.is_null() && ffi::Py_REFCNT(ptr) > 0,
format!("REFCNT: {:?} - {:?}", ptr, ffi::Py_REFCNT(ptr))
);
ffi::Py_INCREF(ptr);
PyObject(NonNull::new_unchecked(ptr))So in theory this should compile down to the same as orjson. You could try using |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From our benchmarks we can see that we are consistently slower than everyone else when serializing/deserializing boolean values. We should fix that.
orjson is using an
unsafeblock to create a reference to a boolean:https://github.com/ijl/orjson/blob/03d55e99a953ce93cedc05f03e4b63b0bcbbcc7a/src/decode.rs#L81-L96
This avoids additional allocations.
For comparison, this is our code at the moment:
hyperjson/src/lib.rs
Lines 475 to 480 in ded13b4
I wonder if we could achieve comparable performance without using
unsafe.@konstin, any idea? Maybe there was a recent development in pyo3 that we could leverage here?