Replacing long int type with int64_t #3739
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The code changes involve updating the type `long` or `long int`. The type `long int` is replaced with `int64_t` in several places to ensure compatibility and improve code clarity.
WalkthroughWalkthroughThe recent updates across various source files primarily focus on enhancing data type consistency for array and index size handling. Changes include shifting from Changes
Recent Review DetailsConfiguration used: CodeRabbit UI Files selected for processing (1)
Files skipped from review as they are similar to previous changes (1)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
for more information, see https://pre-commit.ci
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## devel #3739 +/- ##
==========================================
- Coverage 82.19% 82.18% -0.01%
==========================================
Files 513 513
Lines 47642 47642
Branches 2980 2980
==========================================
- Hits 39159 39156 -3
- Misses 7572 7575 +3
Partials 911 911 ☔ View full report in Codecov by Sentry. |
njzjz
approved these changes
May 4, 2024
should it be included? |
|
After giving it some thought, because this observed behavior with |
mtaillefumier
pushed a commit
to mtaillefumier/deepmd-kit
that referenced
this pull request
Sep 18, 2024
Following the discussion in deepmodeling#3657 this pull request addresses the usage of `long` or `long int` by replacing them with `int64_t` in multiple instances. This change aims to enhance code compatibility across different platforms and improve code clarity. The `int64_t` type is a feature introduced in C++11, defined in the [`<cstdint>`](https://en.cppreference.com/w/cpp/header/cstdint) header. Due to historical reasons, the compilation behavior of `int64_t` is platform- and system-specific. On Linux, `int64_t` is compiled to `long`, whereas on macOS, it's compiled to `long long`. In relevant codebases such as PyTorch and TensorFlow, `int64_t` is preferred over explicit declarations of `long` or `long long`. Consequently, for precompiled libraries, on Linux, symbols are defined exclusively to `long`, while on macOS, symbols are defined exclusively based on `long long`. For these reasons, `data_ptr<long int>()` is unable to compile on macOS. ## References * https://stackoverflow.com/questions/67584843/pytorch-tensordata-ptrlong-long-not-working-on-linux * https://github.com/llvm/llvm-project/blob/c7910ee1f0af64501bf068cdfec154ea359ff832/clang/test/Preprocessor/init.c ## Examples ### Example 1 For the code used here, `torch::from_blob`, is defined using ```cpp inline at::Tensor from_blob( void* data, at::IntArrayRef sizes, const at::TensorOptions& options = at::TensorOptions()) { ``` where `IntArrayRef` is defined as ```cpp using IntArrayRef = c10::ArrayRef<int64_t>; ``` ### Example 2 Dumping the symbols in `libtorch_cpu.dylib` on macOS ``` -> % nm -gU libtorch_cpu.dylib | llvm-cxxfilt | grep TensorBase | grep ::data_ptr 0000000002153398 T c10::Float8_e5m2* at::TensorBase::data_ptr<c10::Float8_e5m2>() const 0000000002153524 T c10::Float8_e4m3fn* at::TensorBase::data_ptr<c10::Float8_e4m3fn>() const 0000000002152738 T c10::Half* at::TensorBase::data_ptr<c10::Half>() const 00000000021536b0 T c10::qint8* at::TensorBase::data_ptr<c10::qint8>() const 00000000021539c8 T c10::qint32* at::TensorBase::data_ptr<c10::qint32>() const 000000000215383c T c10::quint8* at::TensorBase::data_ptr<c10::quint8>() const 0000000002152bdc T c10::complex<c10::Half>* at::TensorBase::data_ptr<c10::complex<c10::Half>>() const 0000000002152ef4 T c10::complex<double>* at::TensorBase::data_ptr<c10::complex<double>>() const 0000000002152d68 T c10::complex<float>* at::TensorBase::data_ptr<c10::complex<float>>() const 000000000215320c T c10::BFloat16* at::TensorBase::data_ptr<c10::BFloat16>() const 0000000002153ce0 T c10::quint2x4* at::TensorBase::data_ptr<c10::quint2x4>() const 0000000002153b54 T c10::quint4x2* at::TensorBase::data_ptr<c10::quint4x2>() const 0000000002152108 T signed char* at::TensorBase::data_ptr<signed char>() const 0000000002153080 T bool* at::TensorBase::data_ptr<bool>() const 0000000002152a50 T double* at::TensorBase::data_ptr<double>() const 00000000021528c4 T float* at::TensorBase::data_ptr<float>() const 0000000002151f7c T unsigned char* at::TensorBase::data_ptr<unsigned char>() const 0000000002152420 T int* at::TensorBase::data_ptr<int>() const 0000000002152294 T short* at::TensorBase::data_ptr<short>() const 00000000021525ac T long long* at::TensorBase::data_ptr<long long>() const ``` dumping symbols in `libtorch_cpu.dylib` on Linux ``` -> % nm -gU libtorch_cpu.so | c++filt | grep TensorBase | grep ::data_ptr 00000000031ec0d0 T c10::Float8_e5m2* at::TensorBase::data_ptr<c10::Float8_e5m2>() const 00000000031ec2f0 T c10::Float8_e4m3fn* at::TensorBase::data_ptr<c10::Float8_e4m3fn>() const 00000000031ec730 T c10::Float8_e4m3fnuz* at::TensorBase::data_ptr<c10::Float8_e4m3fnuz>() const 00000000031ec510 T c10::Float8_e5m2fnuz* at::TensorBase::data_ptr<c10::Float8_e5m2fnuz>() const 00000000031eb030 T c10::Half* at::TensorBase::data_ptr<c10::Half>() const 00000000031ec950 T c10::qint8* at::TensorBase::data_ptr<c10::qint8>() const 00000000031ecd80 T c10::qint32* at::TensorBase::data_ptr<c10::qint32>() const 00000000031ecb70 T c10::quint8* at::TensorBase::data_ptr<c10::quint8>() const 00000000031eb660 T c10::complex<c10::Half>* at::TensorBase::data_ptr<c10::complex<c10::Half> >() const 00000000031eba80 T c10::complex<double>* at::TensorBase::data_ptr<c10::complex<double> >() const 00000000031eb870 T c10::complex<float>* at::TensorBase::data_ptr<c10::complex<float> >() const 00000000031ebeb0 T c10::BFloat16* at::TensorBase::data_ptr<c10::BFloat16>() const 00000000031ed1c0 T c10::quint2x4* at::TensorBase::data_ptr<c10::quint2x4>() const 00000000031ecfa0 T c10::quint4x2* at::TensorBase::data_ptr<c10::quint4x2>() const 00000000031ea7f0 T signed char* at::TensorBase::data_ptr<signed char>() const 00000000031ebca0 T bool* at::TensorBase::data_ptr<bool>() const 00000000031eb450 T double* at::TensorBase::data_ptr<double>() const 00000000031eb240 T float* at::TensorBase::data_ptr<float>() const 00000000031ea5d0 T unsigned char* at::TensorBase::data_ptr<unsigned char>() const 00000000031eac10 T int* at::TensorBase::data_ptr<int>() const 00000000031ed5e0 T unsigned int* at::TensorBase::data_ptr<unsigned int>() const 00000000031eae20 T long* at::TensorBase::data_ptr<long>() const 00000000031ed7f0 T unsigned long* at::TensorBase::data_ptr<unsigned long>() const 00000000031eaa00 T short* at::TensorBase::data_ptr<short>() const 00000000031ed3d0 T unsigned short* at::TensorBase::data_ptr<unsigned short>() const ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Refactor** - Improved data type consistency across various components for handling larger data sizes more reliably. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Following the discussion in #3657 this pull request addresses the usage of
longorlong intby replacing them withint64_tin multiple instances. This change aims to enhance code compatibility across different platforms and improve code clarity.The
int64_ttype is a feature introduced in C++11, defined in the<cstdint>header. Due to historical reasons, the compilation behavior ofint64_tis platform- and system-specific. On Linux,int64_tis compiled tolong, whereas on macOS, it's compiled tolong long.In relevant codebases such as PyTorch and TensorFlow,
int64_tis preferred over explicit declarations oflongorlong long. Consequently, for precompiled libraries, on Linux, symbols are defined exclusively tolong, while on macOS, symbols are defined exclusively based onlong long.For these reasons,
data_ptr<long int>()is unable to compile on macOS.References
Examples
Example 1
For the code used here,
torch::from_blob, is defined usingwhere
IntArrayRefis defined asExample 2
Dumping the symbols in
libtorch_cpu.dylibon macOSdumping symbols in
libtorch_cpu.dylibon LinuxSummary by CodeRabbit