diff --git a/.gitignore b/.gitignore index 2eb6e5e1a2..7270945558 100644 --- a/.gitignore +++ b/.gitignore @@ -17,7 +17,6 @@ python/build_cpu.sh python/fastdeploy/ThirdPartyNotices* *.so* fpython/astdeploy/libs/third_libs -fastdeploy/core/config.h fastdeploy/pybind/main.cc python/fastdeploy/libs/lib* python/fastdeploy/libs/third_libs @@ -29,7 +28,6 @@ python/scripts/process_libraries.py .DS_Store miniprogram_npm node_modules -.DS_Store dist etc lib @@ -40,6 +38,7 @@ yalc.* .yalc examples/vision/collect_quantize_cc.sh examples/vision/tests_quantize +ppocr_keys_v1.txt fastdeploy/LICENSE fastdeploy/ThirdPartyNotices.txt FastDeployCSharp.cmake diff --git a/examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md b/examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md index 7da6ce6fd9..f25b3ee6b5 100755 --- a/examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md +++ b/examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md @@ -65,7 +65,7 @@ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg # x.y.z为镜像版本号，需参照serving文档替换为数字 docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 -docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash +docker run --gpus all -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.5-21.10 bash docker exec -it -u root fastdeploy bash ``` @@ -81,7 +81,7 @@ fastdeployserver --model-repository=/ocr_serving/models ``` 参数: - - `model-repository`(required): 整套模型streaming_pp_tts存放的路径. + - `model-repository`(required): 整套模型PP-OCRv3存放的路径. - `http-port`(optional): HTTP服务的端口号. 默认: `8000`. 本示例中未使用该端口. - `grpc-port`(optional): GRPC服务的端口号. 默认: `8001`. - `metrics-port`(optional): 服务端指标的端口号. 默认: `8002`. 本示例中未使用该端口. diff --git a/serving/README.md b/serving/README.md index 0e7d2e13fd..270d58832e 100644 --- a/serving/README.md +++ b/serving/README.md @@ -6,7 +6,7 @@ FastDeploy builds an end-to-end serving deployment based on [Triton Inference Server](https://github.com/triton-inference-server/server). The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance. -> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/serving/simple_serving) for its usage. +> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README.md) for its usage. ## Prepare the environment diff --git a/serving/README_CN.md b/serving/README_CN.md index ab0245b1db..f7e165003b 100644 --- a/serving/README_CN.md +++ b/serving/README_CN.md @@ -4,48 +4,59 @@ ## 简介 -FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server)搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块，并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。 +FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server) +搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块，并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。 + +> +FastDeploy同时还提供了基于Python搭建的服务化部署能力，只需要通过Python即可启动服务，可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README_CN.md) +了解其用法。 -> FastDeploy同时还提供了基于Python搭建的服务化部署能力，只需要通过Python即可启动服务，可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/serving/simple_serving)了解其用法。 ## 准备环境 ### 环境要求 + - Linux -- 如果使用GPU镜像， 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU，如T4使用的NVIDIA Driver可以是418.40+、440.33+、450.51+、460.27+) +- 如果使用GPU镜像， 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU，如T4使用的NVIDIA + Driver可以是418.40+、440.33+、450.51+、460.27+) ### 获取镜像 #### CPU镜像 + CPU镜像仅支持Paddle/ONNX模型在CPU上进行服务化部署，支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime + ``` shell docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-cpu-only-21.10 ``` #### GPU镜像 + GPU镜像支持Paddle/ONNX模型在GPU/CPU上进行服务化部署，支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime + ``` docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-gpu-cuda11.4-trt8.5-21.10 ``` 用户也可根据自身需求，参考如下文档自行编译镜像 + - [FastDeploy服务化部署镜像编译说明](docs/zh_CN/compile.md) ## 其它文档 + - [模型仓库目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型仓库目录) - [模型配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项) - [服务化部署示例](docs/zh_CN/demo.md) (服务化部署示例) - [客户端访问说明](docs/zh_CN/client.md) (客户端访问说明) - [Serving可视化部署](docs/zh_CN/vdl_management.md) (Serving可视化部署) - ### 服务化部署示例 -| 任务场景 | 模型 | -|---|-----------------------------------------------------------------------------------| -| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) | -| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) | -| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) | -| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) | -| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) | -| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) | -| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) | +| 任务场景 | 模型 | +|----------------|-------------------------------------------------------------------------------------------| +| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) | +| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) | +| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) | +| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) | +| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) | +| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) | +| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) |