Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(virtual-background-guide): updated virtual background guide to support v2 changes #390

Merged
merged 2 commits into from
Sep 13, 2024
Merged

fix(virtual-background-guide): updated virtual background guide to support v2 changes #390

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
33cca11
fix(virtual-background-guide): updated virtual background guide to su…
ravindra-dyte Sep 13, 2024
e0eb7ed
fix(typo): fixed typo in mechanism
ravindra-dyte Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions docs/guides/capabilities/video/add-virtual-background.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ If you are using the web-core CDN script bundle, you can add the package by incl
If you prefer using a specific version of the package, you can modify the source URL by appending the desired version number:

```js
<script src="https://cdn.jsdelivr.net/npm/@dytesdk/video-background-transformer@1.1.3/dist/index.iife.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@dytesdk/video-background-transformer@2.0.3/dist/index.iife.js"></script>
```

Whenever a new release of the package is available, you can simply update the version number in the source URL to upgrade to the latest version.
Whenever a new minor release of the package is available, you can simply update the version number in the source URL to upgrade to the latest version.

## Initialize

Expand All @@ -40,13 +40,37 @@ import DyteVideoBackgroundTransformer from '@dytesdk/video-background-transforme

If you installed the package using a script tag, the `DyteVideoBackgroundTransformer` will be automatically available for use once the installation process is complete.

To begin using the Dyte Background Transformer, you need to initialize it by calling the `init` method. Follow the steps below to initialize the transformer:
2.x versions of DyteVideoBackgroundTransformer use their own rendering mechanism thus require you to disable the default per frame canvas rendering.

```js
const videoBackgroundTransformer = await DyteVideoBackgroundTransformer.init();
await meeting.self.setVideoMiddlewareGlobalConfig({ disablePerFrameCanvasRendering: true });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this now do we

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.0.1 & 2.0.2 still require this line to be added by the devs. 2.0.3 onwards, we are internally doing this.

I have kept it here for reference. Also calling this method multiple times is safe and doesn't cause any side effects.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a note saying not required post 2.0.3?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is added exclusively to highlight it as a breaking change for any custom middlewares that our clients might have.

If any middleware is broken, migration could be done by referring to company branding sample.

Default middleware design: https://github.com/dyte-in/streams-middleware-samples/blob/main/video/company-branding.js
Design if per frame rendering is disabled: https://github.com/dyte-in/streams-middleware-samples/blob/main/video/middleware-with-per-frame-rendering-disabled/company-branding.js

```

Make sure to use the `await` keyword before the `init` method call to ensure the initialization process is completed before proceeding further. Once the initialization is successful, the `videoBackgroundTransformer` object will be ready for use in your application.
For reference, A middleware such as [company branding](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/company-branding.js) with the default per frame rendering can be rewritten without per frame rendering as [this](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/middleware-with-per-frame-rendering-disabled/company-branding.js).

:::info Note
If you don't want to disable the default per frame rendering mechanism due to a custom middleware, please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with respective web-core 1.x versions.
:::

Now that we have disabled the per frame rendering, We can initialise the DyteVideoBackgroundTransformer.

```js
const videoBackgroundTransformer = await DyteVideoBackgroundTransformer.init({
meeting: meeting,
segmentationConfig: {
pipeline: 'canvas2dCpu', // 'webgl2' | 'canvas2dCpu'
},
});

```

To customise DyteVideoBackgroundTransformer configs, please refer to the [video background transfomer](https://www.npmjs.com/package/@dytesdk/video-background-transformer?activeTab=readme) NPM package.

:::info Note
DyteVideoBackgroundTransformer's 2.x version requires you to use web-core 2.x versions. Please refer to our web-core [upgrade guide](https://docs.dyte.io/web-core/upgrade).

If, In case you don't want to upgrade SDKs yet, Please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with respective web-core 1.x versions.
:::

## Usage

Expand Down
8 changes: 8 additions & 0 deletions docs/react-ui-kit/build-pre-call-ui/build-your-own/add-middleware.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,16 @@ export default function CustomMeetingPreview() {
if (!meeting) {
return;
}
/**
* @dytesdk/ui-kit-addons/video-background uses @dytesdk/video-background-transformer.
* To customise DyteVideoBackgroundTransformer configs, please refer to https://www.npmjs.com/package/@dytesdk/video-background-transformer?activeTab=readme.
*
*/
const videoBackground = new VideoBackground({
modes: ['blur', 'virtual'],
segmentationConfig: {
pipeline: 'canvas2dCpu', // 'webgl2' | 'canvas2dCpu'
},
images: [
'https://assets.dyte.io/backgrounds/bg_0.jpg',
'https://assets.dyte.io/backgrounds/bg_1.jpg',
Expand Down
29 changes: 26 additions & 3 deletions docs/react-web-core/pre-call/3-virtual-bg.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,37 @@ import DyteVideoBackgroundTransformer from '@dytesdk/video-background-transforme

If you installed the package using a script tag, the `DyteVideoBackgroundTransformer` will be automatically available for use once the installation process is complete.

2.x versions of DyteVideoBackgroundTransformer use their own rendering mechanism thus require you to disable the default per frame canvas rendering.

```js
await meeting.self.setVideoMiddlewareGlobalConfig({ disablePerFrameCanvasRendering: true });
```

For reference, A middleware such as [company branding](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/company-branding.js) with the default per frame rendering can be rewritten without per frame rendering as [this](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/middleware-with-per-frame-rendering-disabled/company-branding.js).

:::info Note
If you don't want to disable the default per frame rendering mechanism due to a custom middleware, please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with react-web-core 1.x versions.
:::

Now that we have disabled the per frame rendering, We can initialise the DyteVideoBackgroundTransformer.

```js
const videoBackgroundTransformer = await DyteVideoBackgroundTransformer.init({
meeting: meeting
});
meeting: meeting,
segmentationConfig: {
pipeline: 'canvas2dCpu', // 'webgl2' | 'canvas2dCpu'
},
});

await meeting.self.setVideoMiddlewareGlobalConfig({ disablePerFrameCanvasRendering: true });
```

To customise DyteVideoBackgroundTransformer configs, please refer to the [video background transfomer](https://www.npmjs.com/package/@dytesdk/video-background-transformer?activeTab=readme) NPM package.

:::info Note
DyteVideoBackgroundTransformer's 2.x version requires you to use web-core 2.x versions. Please refer to our web-core [upgrade guide](https://docs.dyte.io/web-core/upgrade).

If, In case you don't want to upgrade SDKs yet, Please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with react-web-core 1.x versions.
:::

### Add an image as a background

Expand Down
30 changes: 27 additions & 3 deletions docs/web-core/pre-call/3-virtual-bg.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,38 @@ import DyteVideoBackgroundTransformer from '@dytesdk/video-background-transforme

If you installed the package using a script tag, the `DyteVideoBackgroundTransformer` will be automatically available for use once the installation process is complete.

2.x versions of DyteVideoBackgroundTransformer use their own rendering mechanism thus require you to disable the default per frame canvas rendering.

```js
await meeting.self.setVideoMiddlewareGlobalConfig({ disablePerFrameCanvasRendering: true });
```

For reference, A middleware such as [company branding](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/company-branding.js) with the default per frame rendering can be rewritten without per frame rendering as [this](https://github.com/dyte-in/streams-middleware-samples/blob/main/video/middleware-with-per-frame-rendering-disabled/company-branding.js).

:::info Note
If you don't want to disable the default per frame rendering mechanism due to a custom middleware, please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with web-core 1.x versions.
:::

Now that we have disabled the per frame rendering, We can initialise the DyteVideoBackgroundTransformer.

```js
const videoBackgroundTransformer = await DyteVideoBackgroundTransformer.init({
meeting: meeting
});
meeting: meeting,
segmentationConfig: {
pipeline: 'canvas2dCpu', // 'webgl2' | 'canvas2dCpu'
},
});

await meeting.self.setVideoMiddlewareGlobalConfig({ disablePerFrameCanvasRendering: true });
```

To customise DyteVideoBackgroundTransformer configs, please refer to the [video background transfomer](https://www.npmjs.com/package/@dytesdk/video-background-transformer?activeTab=readme) NPM package.

:::info Note
DyteVideoBackgroundTransformer's 2.x version requires you to use web-core 2.x versions. Please refer to our web-core [upgrade guide](https://docs.dyte.io/web-core/upgrade).

If, In case you don't want to upgrade SDKs yet, Please use the [legacy version](https://www.npmjs.com/package/@dytesdk/video-background-transformer/v/1.1.8) (README included) of DyteVideoBackgroundTransformer that is compatible with web-core 1.x versions.
:::

### Add an image as a background

To incorporate an image as a background, create a static background video middleware using the `createStaticBackgroundVideoMiddleware` method provided by the `videoBackgroundTransformer` object.
Expand Down
Loading