Skip to content

Commit

Permalink
[Doc] add readme for js packages (#421)
Browse files Browse the repository at this point in the history
* add contributor

* add package readme

* refine ocr readme

* refine ocr readme
  • Loading branch information
LDOUBLEV authored Oct 24, 2022
1 parent 828a81d commit a823d4e
Show file tree
Hide file tree
Showing 14 changed files with 715 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[中文版](./README_cn.md)

# detect

detect model is used to detect the position of label frame in the image.

<img src="https://img.shields.io/npm/v/@paddle-js-models/detect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/detect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/detect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/detect" alt="downloads">

# Usage

```js
import * as det from '@paddle-js-models/detect';

// Load model
await det.load();

// Get label index, confidence and coordinates
const res = await det.detect(img);

res.forEach(item => {
// Get label index
console.log(item[0]);
// Get label confidence
console.log(item[1]);
// Get label left coordinates
console.log(item[2]);
// Get label top coordinates
console.log(item[3]);
// Get label right coordinates
console.log(item[4]);
// Get label bottom coordinates
console.log(item[5]);
});
```

# effect
![img.png](https://user-images.githubusercontent.com/43414102/153805288-80f289bf-ca92-4788-b1dd-44854681a03f.png)

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[English](./README.md)

# detect

detect模型用于检测图像中label框选位置。

<img src="https://img.shields.io/npm/v/@paddle-js-models/detect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/detect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/detect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/detect" alt="downloads">

# 使用

```js
import * as det from '@paddle-js-models/detect';

// 模型加载
await det.load();

// 获取label对应索引、置信度、检测框选坐标
const res = await det.detect(img);

res.forEach(item => {
// 获取label对应索引
console.log(item[0]);
// 获取label置信度
console.log(item[1]);
// 获取检测框选left顶点
console.log(item[2]);
// 获取检测框选top顶点
console.log(item[3]);
// 获取检测框选right顶点
console.log(item[4]);
// 获取检测框选bottom顶点
console.log(item[5]);
});
```

# 效果
![img.png](https://user-images.githubusercontent.com/43414102/153805288-80f289bf-ca92-4788-b1dd-44854681a03f.png)

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[中文版](./README_cn.md)

# Facedetect

Facedetect is used for face detection in image. It provides a simple interface. At the same time, you can use your own model.

<img src="https://img.shields.io/npm/v/@paddle-js-models/facedetect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/facedetect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/facedetect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/facedetect" alt="downloads">

# Usage

```js
import { FaceDetector } from '@paddle-js-models/facedetect';

const faceDetector = new FaceDetector();
await faceDetector.init();
// Required parameter:imgEle(HTMLImageElement)
// Optional parameter: shrink, threshold
// Result is face area information. It includes left, top, width, height, confidence
const res = await faceDetector.detect(
imgEle,
{ shrink: 0.4, threshold: 0.6 }
);
```

# Performance
+ **multi small-sized face**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/small.png"/>

+ **single big-sized face**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/big.png"/>

# Postprocess
This model has a better recognition effect for small-sized faces, and the image will be shrink before prediction, so it is necessary to transform the prediction output data.
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/example.png"/>
The red line indicates the predicted output result, and the green line indicates the converted result. dx dy fw fh are known parameters.

# Reference
[original model link](https://github.com/PaddlePaddle/PaddleHub/tree/release/v2.2/modules/image/face_detection/pyramidbox_lite_mobile)
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
[English](./README.md)

# Facedetect

Facedetect 实现图像中的人脸检测,提供的接口简单,支持用户传入模型。

<img src="https://img.shields.io/npm/v/@paddle-js-models/facedetect?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/facedetect" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/facedetect?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/facedetect" alt="downloads">

# 使用

```js
import { FaceDetector } from '@paddle-js-models/facedetect';

const faceDetector = new FaceDetector();
await faceDetector.init();
// 使用时必传图像元素(HTMLImageElement),支持指定图片缩小比例(shrink)、置信阈值(threshold)
// 结果为人脸区域信息,包括:左侧 left,上部 top,区域宽 width,区域高 height,置信度 confidence
const res = await faceDetector.detect(
imgEle,
{ shrink: 0.4, threshold: 0.6 }
);
```

# 效果
+ **多个小尺寸人脸**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/small.png"/>

+ **单个大尺寸人脸**
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/big.png"/>

# 数据后处理
此人脸检测模型对小尺寸人脸具有更好的识别效果,图像在预测前会进行缩小,因此需要对预测输出数据进行变换,及为**数据后处理过程**。示意如下:
<img width="500" src="https://mms-voice-fe.cdn.bcebos.com/pdmodel/face/detection/pic/example.png"/>
红线标识的是预测输出结果,绿线标识的是经过转换后的结果,二者变换过程所涉及到的 dx dy fw fh均为已知量。

# 参考
[源模型链接](https://github.com/PaddlePaddle/PaddleHub/tree/release/v2.2/modules/image/face_detection/pyramidbox_lite_mobile)
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
[中文版](./README_cn.md)

# humanseg

A real-time human-segmentation model. You can use it to change background. The output of the model is gray value. Model supplies simple api for users.

Api drawHumanSeg can draw human segmentation with a specified background.
Api blurBackground can draw human segmentation with a blurred origin background.
Api drawMask can draw the background without human.


<img src="https://img.shields.io/npm/v/@paddle-js-models/humanseg?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/humanseg" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/humanseg?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/humanseg" alt="downloads">

# Usage

```js

import * as humanseg from '@paddle-js-models/humanseg';

// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();

// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);

// get the gray value [2, 398, 224] or [2, 288, 160];
const { data } = await humanseg.getGrayValue(img);

// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas1, back_canvas) ;

// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
humanseg.blurBackground(data, canvas2) ;

// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
humanseg.drawMask(data, canvas3, back_canvas);

```

## gpu pipeline

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg/lib/index_gpu';

// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();

// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);


// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;

// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.blurBackground(input, canvas2) ;

// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);

```

# Online experience

image human segmentation:https://paddlejs.baidu.com/humanseg

video-streaming human segmentation:https://paddlejs.baidu.com/humanStream

# Performance

<img width="800" src="https://user-images.githubusercontent.com/10822846/126873788-1e2d4984-274f-45be-8716-2a87ddda8c75.png"/>
<img width="800" src="https://user-images.githubusercontent.com/10822846/126873838-e5b68c9b-279f-4cb4-ae90-6aaaecd06aa4.png"/>


# Used in Video Meeting
<p>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872499-c3fd680e-a01b-4daa-b0cb-acd3290862bd.gif"/>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872930-4f4c5c5d-5c51-44fe-b2d6-3f83c4e124bc.png"/>
</p>
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
[English](./README.md)

# 人像分割

实时的人像分割模型。使用者可以用于背景替换。需要使用接口 getGrayValue 获取灰度值。
然后使用接口 drawHumanSeg 绘制分割出来的人像,实现背景替换;使用接口 blurBackground 实现背景虚化;也可以使用 drawMask 接口绘制背景,可以配置参数来获取全黑背景或者原图背景。

<img src="https://img.shields.io/npm/v/@paddle-js-models/humanseg?color=success" alt="version"> <img src="https://img.shields.io/bundlephobia/min/@paddle-js-models/humanseg" alt="size"> <img src="https://img.shields.io/npm/dm/@paddle-js-models/humanseg?color=orange" alt="downloads"> <img src="https://img.shields.io/npm/dt/@paddle-js-models/humanseg" alt="downloads">

# 使用

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg';

// 默认下载 398x224 shape 的模型,默认执行预热
await humanseg.load();

// 指定下载更轻量模型, 该模型 shape 288x160,预测过程会更快,但会有少许精度损失
// await humanseg.load(true, true);


// 获取分割后的像素 alpha 值,大小为 [2, 398, 224] 或者 [2, 288, 160]
const { data } = await humanseg.getGrayValue(img);

// 获取 background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// 背景替换, 使用 back_canvas 作为新背景实现背景替换
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas1, back_canvas) ;

// 背景虚化
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
humanseg.blurBackground(data, canvas2) ;

// 绘制人型遮罩,在新背景上隐藏人像
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
humanseg.drawMask(data, canvas3, back_canvas);

```

## gpu pipeline

```js

// 引入 humanseg sdk
import * as humanseg from '@paddle-js-models/humanseg/lib/index_gpu';

// 默认下载 398x224 shape 的模型,默认执行预热
await humanseg.load();

// 指定下载更轻量模型, 该模型 shape 288x160,预测过程会更快,但会有少许精度损失
// await humanseg.load(true, true);


// 获取 background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;

// 背景替换, 使用 back_canvas 作为新背景实现背景替换
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;

// 背景虚化
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.blurBackground(input, canvas2) ;

// 绘制人型遮罩,在新背景上隐藏人像
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);

```

# 在线体验

图片人像分割:https://paddlejs.baidu.com/humanseg

基于视频流人像分割:https://paddlejs.baidu.com/humanStream

# 效果

从左到右:原图、背景虚化、背景替换、人型遮罩

<img width="800" src="https://user-images.githubusercontent.com/10822846/126873788-1e2d4984-274f-45be-8716-2a87ddda8c75.png"/>
<img width="800" src="https://user-images.githubusercontent.com/10822846/126873838-e5b68c9b-279f-4cb4-ae90-6aaaecd06aa4.png"/>


# 视频会议
<p>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872499-c3fd680e-a01b-4daa-b0cb-acd3290862bd.gif"/>
<img width="400" src="https://user-images.githubusercontent.com/10822846/126872930-4f4c5c5d-5c51-44fe-b2d6-3f83c4e124bc.png"/>
</p>

Loading

0 comments on commit a823d4e

Please sign in to comment.