crashes when calling c++ in arm android #15006

yyccR · 2023-01-09T11:30:56Z

System information (version)

OpenVINO => 2022.3.0
Operating System / Platform => Android(arm64)
Compiler => Visual Studio 2017

Detailed description

I was going to use openvino in flutter by calling c++ instead of using the Java api, but it crashed when calling c++ to initialize the model, here is the error msg:
Abort message: 'terminating with uncaught exception of type ov::Exception: Can't get absolute file path for [/data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk!/lib/arm64-v8a/libopenvino.so], err = No such file or directory'

here is the c++ code:

#include <iostream>
#include "memory"
#include "openvino/runtime/runtime.hpp"

int initOpenvinoModel(std::string &plugin_xml, std::string &xml_model_path, std::string &bin_model_path) {

    std::shared_ptr<ov::Core> core = std::make_shared<ov::Core>(plugin_xml);
    // std::shared_ptr<ov::Model> model = core.read_model(xml_model_path, bin_model_path);
    // core.compile_model(model, "CPU");
    return 0;
}

extern "C" __attribute__((visibility("default"))) __attribute__((used))
int initOpenvinoModel_c(char* plugin_xml_path, char* xml_model_path, char* bin_model_path) {
    std::string plugin_xml_path_s(plugin_xml_path);
    std::string xml_model_path_s(xml_model_path);
    std::string bin_model_path_s(bin_model_path);
    return initOpenvinoModel(plugin_xml_path_s, xml_model_path_s, bin_model_path_s);
}

and I have build the openvino with arm cpu plugin, below is the shared library and plugins.xml:

libopenvino.so
libopenvino_arm_cpu_plugin.so
libopenvino_auto_batch_plugin.so
libopenvino_auto_plugin.so
libopenvino_hetero_plugin.so

<ie>
    <plugins>
        <plugin name="AUTO" location="libopenvino_auto_plugin.so">
        </plugin>
        <plugin name="BATCH" location="libopenvino_auto_batch_plugin.so">
        </plugin>
        <plugin name="CPU" location="libopenvino_arm_cpu_plugin.so">
        </plugin>
        <plugin name="HETERO" location="libopenvino_hetero_plugin.so">
        </plugin>
        <plugin name="MULTI" location="libopenvino_auto_plugin.so">
        </plugin>
    </plugins>
</ie>

I have linked the libopenvino.so in the project and include all the header files, this is my CMakeLists.txt

cmake_minimum_required(VERSION 3.17.0)
project(test)
set(CMAKE_CXX_STANDARD 17)

include_directories(${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/include)

set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/libs/opencv-mobile-4.5.4-android/sdk/native/jni)
find_package(OpenCV REQUIRED core imgproc highgui)

add_library(openvino SHARED IMPORTED)
set_target_properties(openvino PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino.so)

aux_source_directory(${CMAKE_SOURCE_DIR}/openvino_cpp openvino_src_lists)
add_library(test SHARED ${openvino_src_lists})
target_link_libraries(test ${OpenCV_LIBS})
target_link_libraries(test openvino)

And I've put all the plugin shared libraries in the same directory as libopevino.so。

I also put plugins.xml, openvino_model.xml, openvino_model.bin in the assert directory and copy these three files to the android system when I start the application so that c++ programs can read them.

The error below

Abort message: 'terminating with uncaught exception of type ov::Exception: Can't get absolute file path for [/data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk!/lib/arm64-v8a/libopenvino.so], err = No such file or directory'

seems that openvino c++ wants to find the libopenvino library directory and get the plugin shared library, but cannot access it when android is not root

So I've put all the plugin.so in the asset directory, and change the plugins.xml into :

<ie>
    <plugins>
        <plugin name="AUTO" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_plugin.so">
        </plugin>
        <plugin name="BATCH" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_batch_plugin.so">
        </plugin>
        <plugin name="CPU" location="/data/user/0/com.test.test/app_flutter/libopenvino_arm_cpu_plugin.so">
        </plugin>
        <plugin name="HETERO" location="/data/user/0/com.test.test/app_flutter/libopenvino_hetero_plugin.so">
        </plugin>
        <plugin name="MULTI" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_plugin.so">
        </plugin>
    </plugins>
</ie>

The location in plugins.xml is the path to the android system file, which can be read by c++ programs, and is the target copy path to the asset file。

After trying, I still get the same error message.

please let me know if you have a solution.

Thank you so much.

The text was updated successfully, but these errors were encountered:

ilya-lavrenov · 2023-01-09T20:14:53Z

@dkurt do you have suggestions?

dkurt · 2023-01-09T20:47:49Z

And I've put all the plugin shared libraries in the same directory as libopevino.so

Which directory? /data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk! must be a temporal directory which is available only in Java runtime. Also, running C++ requires rooted device, is your device rooted? If so, can you please put the binary and libraries in some regular location.

yyccR · 2023-01-10T01:58:26Z

@dkurt Thanks，At first my project directory tree:

android
├──app
├────├──src
├────────├──mian

lib
├──my_flutter_calling_cpp_code.dart

assert
├── plugins.xml
├── model.xml
├── mdoel.bin

cpp
├── openvino-2022.3.0
│	├── arm64-v8a
│	│   ├── libopenvino.so
│	│   ├── libopenvino_arm_cpu_plugin.so
│	│   ├── libopenvino_auto_batch_plugin.so
│	│   ├── libopenvino_auto_plugin.so
│	│   ├── libopenvino_hetero_plugin.so
│	├── armeabi-v7a
│	│   ├── libopenvino.so
│	│   ├── libopenvino_arm_cpu_plugin.so
│	│   ├── libopenvino_auto_batch_plugin.so
│	│   ├── libopenvino_auto_plugin.so
│	│   ├── libopenvino_hetero_plugin.so
│	│── include
│	│   ├── ie
│	│   ├── ngraph
│	│   ├── openvino
├── openvino_cpp
│	├── test.cpp
├── CMakeLists.txt

But when got this error msg:

Abort message: 'terminating with uncaught exception of type ov::Exception: Can't get absolute file path for [/data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk!/lib/arm64-v8a/libopenvino.so], err = No such file or directory'

I change my project tree into:


android
├──app
├────├──src
├────────├──mian

lib
├──my_flutter_calling_cpp_code.dart

assert
├── plugins.xml
├── model.xml
├── mdoel.bin
├── libopenvino_arm_cpu_plugin.so
├── libopenvino_auto_batch_plugin.so
├── libopenvino_auto_plugin.so
├── libopenvino_hetero_plugin.so

cpp
├── openvino-2022.3.0
│	├── arm64-v8a
│	│   ├── libopenvino.so

│	├── armeabi-v7a
│	│   ├── libopenvino.so
│	│── include
│	│   ├── ie
│	│   ├── ngraph
│	│   ├── openvino
├── openvino_cpp
│	├── test.cpp
├── CMakeLists.txt

When the application starts, I copy all the libraries in the assert directory to the android readable directory，and then change the location of all the plugin.so in the xml file:

<ie>
    <plugins>
        <plugin name="AUTO" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_plugin.so">
        </plugin>
        <plugin name="BATCH" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_batch_plugin.so">
        </plugin>
        <plugin name="CPU" location="/data/user/0/com.test.test/app_flutter/libopenvino_arm_cpu_plugin.so">
        </plugin>
        <plugin name="HETERO" location="/data/user/0/com.test.test/app_flutter/libopenvino_hetero_plugin.so">
        </plugin>
        <plugin name="MULTI" location="/data/user/0/com.test.test/app_flutter/libopenvino_auto_plugin.so">
        </plugin>
    </plugins>
</ie>

This directory /data/user/0/com.test.test/app_flutter/ is readable for c++ program. But still got the same error:

Abort message: 'terminating with uncaught exception of type ov::Exception: Can't get absolute file path for [/data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk!/lib/arm64-v8a/libopenvino.so], err = No such file or directory'

The error seems that openvino c++ want to access directory /data/app/com.test.test-YcefHGmNSq6Rurhx-jV-xQ==/base.apk! which was denied, because my device not rooted,

I also tried to manually load all the plugin.so myself, here is CMakeLists.txt:

cmake_minimum_required(VERSION 3.17.0)
project(test)
set(CMAKE_CXX_STANDARD 17)

include_directories(${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/include)

set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/libs/opencv-mobile-4.5.4-android/sdk/native/jni)
find_package(OpenCV REQUIRED core imgproc highgui)

add_library(openvino SHARED IMPORTED)
set_target_properties(openvino PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino.so)

add_library(openvino_auto_plugin SHARED IMPORTED)
set_target_properties(openvino_auto_plugin PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino_auto_plugin.so)
add_library(openvino_auto_batch_plugin SHARED IMPORTED)
set_target_properties(openvino_auto_batch_plugin PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino_auto_batch_plugin.so)
add_library(openvino_hetero_plugin SHARED IMPORTED)
set_target_properties(openvino_hetero_plugin PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino_hetero_plugin.so)
add_library(openvino_arm_cpu_plugin SHARED IMPORTED)
set_target_properties(openvino_arm_cpu_plugin PROPERTIES IMPORTED_LOCATION
        ${CMAKE_CURRENT_LIST_DIR}/libs/openvino-2022.3.0/${ANDROID_ABI}/libopenvino_arm_cpu_plugin.so)

aux_source_directory(${CMAKE_SOURCE_DIR}/openvino_cpp openvino_src_lists)
add_library(test SHARED ${openvino_src_lists})
target_link_libraries(test ${OpenCV_LIBS})
target_link_libraries(test openvino)
target_link_libraries(sai openvino_auto_plugin)
target_link_libraries(sai openvino_auto_batch_plugin)
target_link_libraries(sai openvino_hetero_plugin)
target_link_libraries(sai openvino_arm_cpu_plugin)

But another error occur:

Failed to load dynamic library 'libtest.so': dlopen failed: library "/Users/yang/AndroidStudioProjects/test/cpp/openvino-2022.3.0/arm64-v8a/libopenvino_auto_plugin.so" not found

As you mentioned, running C++ requires rooted device, I think this is a little unfriendly when you want to deploy openvino on arm Android.

dkurt · 2023-01-10T11:48:21Z

There is a similar report dkurt/openvino_java#18, however I cannot reproduce the problem. Can you verify that Java API works?

yyccR · 2023-01-10T14:49:27Z

Thanks, I think I know what to after looking at the java api source code: https://github.com/openvinotoolkit/openvino_contrib/blob/master/modules/java_api/org/intel/openvino/compatibility/IECore.java#L45

I should load the libopenvino.so library manually in dart code instead of in the camkeLists.txt.

dkurt · 2023-01-12T13:07:46Z

@yyccR, can you please check your Android API Level? In example, using CPU-Z app (System tab)

If it's below 31, probably you tried the binaries with higher API requirement. For example, in the case with similar error message dkurt/openvino_java#18 binaries from ANDROID_PLATFORM=32 were used on device with API level 29 while device with API level 31 was fine.

Try rebuild OpenVINO with lower ANDROID_PLATFORM

yyccR · 2023-01-13T01:49:23Z

Thanks, @dkurt
this is my cmake build flags:

cmake -DCMAKE_BUILD_TYPE=Release \
      -DTHREADING=SEQ \
      -DIE_EXTRA_MODULES="/Users/yang/opt/openvino_android/openvino_contrib/modules" \
      -DCMAKE_TOOLCHAIN_FILE="/Users/yang/Library/Android/sdk/ndk/21.4.7075529/build/cmake/android.toolchain.cmake" \
      -DANDROID_ABI=arm64-v8a \
      -DANDROID_PLATFORM=30 \
      -DENABLE_SAMPLES=OFF \
      -DENABLE_OPENCV=OFF \
      -DENABLE_CLDNN=OFF \
      -DENABLE_VPU=OFF \
      -DENABLE_GNA=OFF \
      -DENABLE_MYRIAD=OFF \
      -DENABLE_TESTS=OFF \
      -DENABLE_GAPI_TESTS=OFF \
      -DENABLE_INTEL_MYRIAD=OFF \
      -DENABLE_INTEL_MYRIAD_COMMON=OFF \
      -DBUILD_java_api=ON \
      -DARM_COMPUTE_SCONS_JOBS=$(nproc) \
      -DENABLE_BEH_TESTS=OFF ..

it was built with ANDROID_PLATFORM=30 and my android build.gradle is:

android {
    compileSdkVersion 33
    ndkVersion 	"21.4.7075529"
     defaultConfig {
        minSdkVersion 24
        targetSdkVersion 31
        externalNativeBuild {
            cmake {
               abiFilters 'armeabi-v7a', 'arm64-v8a'
            }
        }
    }
}

This error Can't get absolute file path libopenvino.so is mainly about Android permissions. If we are not root, we cannot directly access files in path /data/app, so if we want to call openvino from c++ inside androind, we need to package all c++ source files together with libopenvino.so, And use the java code System. LoadLibrary ("/readable/path/to/libopenvino. So "); or the dart code DynamicLibrary. Open ("/readable/path/to/libopenvino. So "); to load the dynamic library manually.

This loading method is very different from other inference frameworks such as onnxruntime, tflite, ncnn, paddle-lite.

ilya-lavrenov · 2023-01-15T07:00:25Z

This error Can't get absolute file path libopenvino.so is mainly about Android permissions

Do you know whether is happens during reading model through the frontend library or compiling for inference?

Will it be convenient for Android scenarios to build OpenVINO with plugins / frontends as a single libopenvino.so library?
Currently, you can build OpenVINO statically and all the code will be built in into a final binary file. In this case, Can't get absolute file path libopenvino.so will not happen, since the code path leading to it - just not called.

Thanks,
Ilya.

yyccR · 2023-01-15T14:30:24Z

@ilya-lavrenov

build OpenVINO statically and all the code will be built in into a final binary file

Thanks, I will try it and report the result。

Do you know whether is happens during reading model through the frontend library or compiling for inference?

I haven't tried to do that.

yyccR · 2023-01-16T02:32:23Z

@ilya-lavrenov
Static library seems not work for ARM CPU plugin, below my cmake build flags:

cmake -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_SHARED_LIBS=OFF \
      -DTHREADING=SEQ \
      -DIE_EXTRA_MODULES="/Users/yang/opt/openvino_android/openvino_contrib/modules" \
      -DCMAKE_TOOLCHAIN_FILE="/Users/yang/Library/Android/sdk/ndk/21.4.7075529/build/cmake/android.toolchain.cmake" \
      -DANDROID_ABI=arm64-v8a \
      -DANDROID_PLATFORM=30 \
      -DENABLE_SAMPLES=OFF \
      -DENABLE_OPENCV=OFF \
      -DENABLE_CLDNN=OFF \
      -DENABLE_VPU=OFF \
      -DENABLE_GNA=OFF \
      -DENABLE_MYRIAD=OFF \
      -DENABLE_TESTS=OFF \
      -DENABLE_GAPI_TESTS=OFF \
      -DENABLE_INTEL_MYRIAD=OFF \
      -DENABLE_INTEL_MYRIAD_COMMON=OFF \
      -DBUILD_java_api=ON \
      -DARM_COMPUTE_SCONS_JOBS=$(nproc) \
      -DENABLE_BEH_TESTS=OFF ..

got error:

CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_transformations" that is not in any export set.
CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_opset" that is not in any export set.

dkurt · 2023-01-16T07:17:05Z

@yyccR,t Sorry if missed, but have you managed to workaround the problem finally?

In dkurt/openvino_java#18 where Android device with API level 29 is used we rebuild OpenVINO with ANDROID_PLATFORM=26 and use special actions order:

Put all OpenVINO resources to assets
Copy in runtime all the assets to temp/cache folder
Load libtbb,so, libtbbmalloc.so, libopenvino.so by System.load. Other .so should not be loaded - OpenVINO will load them from C++.

ilya-lavrenov · 2023-01-16T09:31:11Z

CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_transformations" that is not in any export set.
CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_opset" that is not in any export set.

This is fixed on master branch and I ported to 2022.3 as well.
You can try once again

yyccR · 2023-01-17T03:00:51Z

@ilya-lavrenov

thanks, I have pull the latest master branch of openvino and the branch-2022.3 of openvino_contrib, still got the same error.

yyccR · 2023-01-17T03:06:53Z

@dkurt
Thanks bro, I decide to use the java api finally, and it works.

I found that when I built openvino_contrib with the ndk 21.4.7075529, no java api jar were produced, but they were when I upgrade the ndk version.

dkurt · 2023-01-17T05:23:07Z

@ilya-lavrenov

thanks, I have pull the latest master branch of openvino and the branch-2022.3 of openvino_contrib, still got the same error.

The same error about openvino_arm_cpu_plugin or initial libopenvino.so?

yyccR · 2023-01-17T05:40:08Z

The same error about openvino_arm_cpu_plugin or initial libopenvino.so?

Same error when cmake configure static library:

CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_transformations" that is not in any export set.
CMake Error: install(EXPORT "OpenVINOTargets" ...) includes target "openvino_arm_cpu_plugin" which requires target "openvino_arm_cpu_opset" that is not in any export set.

and the cmake flags:

cmake -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_SHARED_LIBS=OFF \
      -DTHREADING=SEQ \
      -DIE_EXTRA_MODULES="/Users/yang/opt/openvino_android/openvino_contrib/modules" \
      -DCMAKE_TOOLCHAIN_FILE="/Users/yang/Library/Android/sdk/ndk/21.4.7075529/build/cmake/android.toolchain.cmake" \
      -DANDROID_ABI=arm64-v8a \
      -DANDROID_PLATFORM=30 \
      -DENABLE_SAMPLES=OFF \
      -DENABLE_OPENCV=OFF \
      -DENABLE_CLDNN=OFF \
      -DENABLE_VPU=OFF \
      -DENABLE_GNA=OFF \
      -DENABLE_MYRIAD=OFF \
      -DENABLE_TESTS=OFF \
      -DENABLE_GAPI_TESTS=OFF \
      -DENABLE_INTEL_MYRIAD=OFF \
      -DENABLE_INTEL_MYRIAD_COMMON=OFF \
      -DBUILD_java_api=ON \
      -DARM_COMPUTE_SCONS_JOBS=$(nproc) \
      -DENABLE_BEH_TESTS=OFF ..

avitial · 2023-01-27T17:53:42Z

@yyccR seems you got onto the right path with Java API, can this issue be closed?

yyccR added bug Something isn't working support_request labels Jan 9, 2023

avitial added platform: android OpenVINO on Android category: CPP API OpenVINO CPP API bindings and removed bug Something isn't working labels Jan 9, 2023

avitial assigned dkurt Jan 12, 2023

ilya-lavrenov assigned vurusovs Jan 15, 2023

yyccR closed this as completed Jan 28, 2023

ly0303521 mentioned this issue May 21, 2024

[Bug]: crashes when calling c++ in arm android #24601

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crashes when calling c++ in arm android #15006

crashes when calling c++ in arm android #15006

yyccR commented Jan 9, 2023 •

edited

Loading

ilya-lavrenov commented Jan 9, 2023

dkurt commented Jan 9, 2023 •

edited

Loading

yyccR commented Jan 10, 2023 •

edited

Loading

dkurt commented Jan 10, 2023

yyccR commented Jan 10, 2023

dkurt commented Jan 12, 2023 •

edited

Loading

yyccR commented Jan 13, 2023 •

edited

Loading

ilya-lavrenov commented Jan 15, 2023 •

edited

Loading

yyccR commented Jan 15, 2023 •

edited

Loading

yyccR commented Jan 16, 2023

dkurt commented Jan 16, 2023 •

edited

Loading

ilya-lavrenov commented Jan 16, 2023

yyccR commented Jan 17, 2023

yyccR commented Jan 17, 2023

dkurt commented Jan 17, 2023 •

edited

Loading

yyccR commented Jan 17, 2023

avitial commented Jan 27, 2023

crashes when calling c++ in arm android #15006

crashes when calling c++ in arm android #15006

Comments

yyccR commented Jan 9, 2023 • edited Loading

System information (version)

Detailed description

ilya-lavrenov commented Jan 9, 2023

dkurt commented Jan 9, 2023 • edited Loading

yyccR commented Jan 10, 2023 • edited Loading

dkurt commented Jan 10, 2023

yyccR commented Jan 10, 2023

dkurt commented Jan 12, 2023 • edited Loading

yyccR commented Jan 13, 2023 • edited Loading

ilya-lavrenov commented Jan 15, 2023 • edited Loading

yyccR commented Jan 15, 2023 • edited Loading

yyccR commented Jan 16, 2023

dkurt commented Jan 16, 2023 • edited Loading

ilya-lavrenov commented Jan 16, 2023

yyccR commented Jan 17, 2023

yyccR commented Jan 17, 2023

dkurt commented Jan 17, 2023 • edited Loading

yyccR commented Jan 17, 2023

avitial commented Jan 27, 2023

yyccR commented Jan 9, 2023 •

edited

Loading

dkurt commented Jan 9, 2023 •

edited

Loading

yyccR commented Jan 10, 2023 •

edited

Loading

dkurt commented Jan 12, 2023 •

edited

Loading

yyccR commented Jan 13, 2023 •

edited

Loading

ilya-lavrenov commented Jan 15, 2023 •

edited

Loading

yyccR commented Jan 15, 2023 •

edited

Loading

dkurt commented Jan 16, 2023 •

edited

Loading

dkurt commented Jan 17, 2023 •

edited

Loading