Flexiv AIDK (AI Development Kit), is a development kit to enable flexible AI service calling and control with client.
| Supported OS | Supported processor | Supported language | Required compiler kit |
|---|---|---|---|
| Linux (Ubuntu 18/20/22 tested) | x86_64 | C++, Python | build-essential |
Before using c++ and python AIDK api, please make sure NoemaEdge is running.
Tips: for experienced users, if robot can run plan to send request to AI correctly, then everything is ready.
Here takes project GRASPNET as an example. To setup NoemaEdge App, take the following steps:
- Launch NoemaEdge
- Image loading and management
- License generation
- License import
- Software package download and import
- Software package configuration
- Program switch and logs
Finally, start environment and start program InferApp. Now the NoemaEdge App is ready and its status is shown below.
Note: If NoemaEdge is running remotely, please make sure the AIDK machine have network connection with it. For more details, please consult Flexiv members for help.
The C++ interface of Flexiv AIDK is packed into a unified modern CMake library named flexiv_aidk, which can be configured via CMake.
-
In a new Terminal, install C++ compiler, Git, and CMake (with GUI) using the package manager:
sudo apt install build-essential git cmake cmake-qt-gui -y -
Choose a directory for installing
flexiv_aidklibrary and all its dependencies. For example, a new folder namedaidk_installunder the home directory. -
In a new Terminal, use CMake to configure flexiv_aidk:
cd release mkdir build && cd build cmake .. -DCMAKE_INSTALL_PREFIX=~/aidk_install -
Compile and install
flexiv_aidklibrary:cd release/build cmake --build . --target install --config Release -
The user project can now find and link to the installed
flexiv_aidklibrary:cd release/example mkdir build && cd build cmake .. -DCMAKE_INSTALL_PREFIX=~/aidk_install cmake --build . --config Release -j 4Note:
CMAKE_INSTALL_PREFIXis set to the same directory whereflexiv_aidkwas installed to. -
To run an compiled example program:
./test_aidk_compute [address] [config_path] [total_num] [detect_version] ./test_aidk_compute_image [address] [config_path] [total_num] ./test_aidk_others [address] [config_path] [version] ./test_aidk_camera [address] [camera_id] [total_num]e.g. to communicate with NoemaEdge App (version v3.1.0) running in remote machine with ip 10.24.14.101:
./test_aidk_compute 10.24.14.101 ../../config/GRASPNET.json 1 v2 ./test_aidk_compute_image 10.24.14.101 ../../config/GRASPNET_IMAGE.json 1 ./test_aidk_others 10.24.14.101 ../../config/GRASPNET.json v3.1.0 ./test_aidk_camera 10.24.14.101 CAMID_0 3Note: Port
18203is used for detect service and port18201is used for camera service, andsudois not required unless prompted by the program.
A brief instruction is provided below and applies to Linux.
To run an example Python program:
cd release/example_py
python3 test_aidk_compute.py --ip [address] --config [config_path] --num [num] --detect-version [version]
python3 test_aidk_compute_image.py --ip [address] --config [config_path] --num [num]
For example, to communicate with local running NoemaEdge:
python3 test_aidk_compute.py --ip 127.0.0.1 --config ../config/GRASPNET.json --num 1 --detect-version v2
The log is shown below:
INFO:root:Namespace(config='../config/GRASPNET.json', detect_version=v2, ip='127.0.0.1', num=1)
INFO:root:Test config:
{'project': 'GRASPNET', 'command': {'command': 'POSE6D', 'obj_name': 'GRASPNET_graspnet_0', 'coordinate_id': 0, 'camera_id': 'CAMID_0', 'instruction_id': -1, 'custom': 'test_0', 'camera_pose': [0.55, -0.034, 0.66, 0.0, -0.7071068, 0.7071068, 0.0]}, 'keys': ['obj_pose', 'double_value']}
INFO:root:detect 0: 370.9 ms, 2.7 Hz, instruction -1
INFO:root:state: True
INFO:root:current detected object names: ['GRASPNET_graspnet_0'], current detected object nums: [1]
INFO:root:key: obj_pose
INFO:root:[0.5027717547217662, -0.003570836059272227, 0.25158332987576737, 0.08253609149323858, -0.3200245424326347, -0.943391842633212, -0.027998519512926395]
INFO:root:key: double_value
INFO:root:0.07259449357263434
Here is an example to get grasp pose in world frame using AIDK and RDK. Remember to put flexivrdk.so under relesae/example_py in advance, then run
python3 test_grasping_with_rdk.py --ai-ip 127.0.0.1 --config ../config/GRASPNET.json --robot-ip 192.168.2.100 --local-ip 192.168.2.107
and details can be found in test_grasping_with_rdk.py:
robot = flexivrdk.Robot(robot_ip, local_ip)
while not robot.isConnected():
time.sleep(0.001)
robot_states = flexivrdk.RobotStates()
robot.getRobotStates(robot_states)
camera_pose = robot_states.camPose
# use aidk to get grasp pose
client = aidk.AIDKClient(ai_ip)
tic = time.time()
config['camera_pose'] = camera_pose
state = client.detect(
**config,
)
key = "obj_pose"
parse_state, result_list = client.parse_result(config["obj_name"], key, -1)
grasp_pose = result_list[0].vect[0]
The example script test_aidk_others.py supports version>=v2.11.1 andtest_aidk_compute.py supports version>=2.10.0.
For NoemaEdge version < v3.2.0, supported custom are:
"POS3D", "POSE6D", "GRASP_POSE", "KEYPOINT", "BBOX", "CLASSIFY", "GET_INT_VALUE", "GET_DOUBLE_VALUE", "KEYPOINT3D", "CUSTOM".
For NoemaEdge version >= v3.2.0, supported custom can be arbitrary command defined as workflow name in project editor.
Supported keys are: "bbox", "keypoints", "positions", "obj_pose", "valid","double_value", "int_value", "name".
Note: For Noema version >= v3.2.0, make sure "command" in config json is always "CUSTOM", and "custom" is the real workflow command.
| API | function | detail | Noema version |
|---|---|---|---|
| detect | computing | send a detect request | >= v2.10.0 |
| detect_with_image | computing | send a detect request with image | >= v2.10.0 |
| get_detected_obj_names | computing | function to get all detected object names | >= v2.10.0 |
| get_detected_obj_nums | computing | function to get all detected object nums | >= v2.10.0 |
| get_detected_obj_num | computing | function to get detected object number based of object name | >= v2.10.0 |
| get_detected_time | computing | function to get timestamp of detect request | >= v3.2.0 |
| parse_result | computing | function to parse detection result | >= v2.10.0 |
| get_camera_intrinsic | computing | function to get camera intrinsic | >= v2.10.0 |
| reload_configs | configure | reload project config | >= v2.11.1 |
| save_configs | configure | save current project config | >= v2.11.1 |
| get_images | others | get color and depth images from camera | >= v3.2.0 |
| get_camera_intrinsics | others | get intrinsics from camera | >= v3.2.0 |
| is_ready | others | check whether AI Noema Edge is ready | >= v2.10.0 |
| get_current_state | others | get current status of AI Noema Edge | >= v2.10.0 |
| warmup | others | warmup current project, if configured on NoemaEdge | >= v2.11.1 |
| all_direct_command | others | all direct commands | >= v2.11.1 |
| get_runtime_info | others | function to get AIApp runtime info | >= v3.1.0 |
| list_remote_files | others | list file and dir list under remote directory | >= v3.1.0 |
| get_file_info | others | function to get remote file info | >= v3.1.0 |
| send_file | others | function to send local file to remote file path | >= v3.1.0 |
| receive_file | others | function to receive remote file to local file path | >= v3.1.0 |
| send_folder | others | function to copy local folder to remote folder | >= v3.1.0 |
| receive_folder | others | function to copy remote folder to local folder | >= v3.1.0 |
| remove_path | others | function to remove remote files or folders | >= v3.1.0 |
| make_remote_directory | others | function to make directory in remote | >= v3.1.0 |
| get_direct_setting_variables | others | function to get variables | >= v3.1.0 |
| set_direct_setting_variables | others | function to set variables | >= v3.1.0 |