NIFI-3785: Added feature to move controller services between process … #8965

Freedom9339 · 2024-06-13T20:19:02Z

…groups with the new UI

Summary

NIFI-3785: Added an option on the controller services grid that moves a controller service to a specified process group. The controller service can be moved to the parent process group or a child process group. However, it can only be moved one level at a time. If there are any scope conflicts with referencing components, the move will fail displaying the reason.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Apache NiFi Jira issue created

Pull Request Tracking

Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

Pull Request based on current revision of the main branch
Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

Build completed using mvn clean install -P contrib-check
- JDK 21

Licensing

New dependencies are compatible with the Apache License 2.0 according to the License Policy
New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

Documentation formatting appears as expected in rendered files

Freedom9339 · 2024-06-13T20:21:14Z

@markap14 @exceptionfactory I've resubmitted this PR with the changes done to the new UI. The backend is the same as it was in the closed PR(#7734), but the UI changes were done against the new UI.

Freedom9339 · 2024-06-14T19:02:35Z

@markobean

exceptionfactory · 2024-06-14T21:44:58Z

Thanks for putting in the work to refactor this for the new UI @Freedom9339.

Tagging @mcgilman and @rfellows for potential review.

mcgilman · 2024-06-14T21:54:45Z

Thanks for the mention @exceptionfactory! I'll have a look...

mcgilman

Thanks for the PR @Freedom9339! I've left a number of comments below that mostly appear to be from using the Enable/Disable Service dialog has a starting point. Since that dialog needs to be used for both Services in the Flow and in the Controller Settings there area a number of considerations there that are not needed here.

In addition to the items below, please ensure you run prettier and that this PR isn't introducing any new lint issues (I think there are currently 31 left that we're slowly trying to work through). From nifi/nifi-frontend/src/main/frontend please run:

$ npx nx prettier:format
$ npx nx lint

I will defer to @exceptionfactory or @markap14 for a more thorough review of the back end changes.

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.reducer.ts

...i/common/controller-service/controller-service-table/controller-service-table.component.html

.../ui/common/controller-service/move-controller-service/move-controller-service.component.html

...pp/ui/common/controller-service/move-controller-service/move-controller-service.component.ts

nifi-frontend/src/main/frontend/apps/nifi/src/app/state/shared/index.ts

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

.../common/controller-service/move-controller-service/move-controller-service.component.spec.ts

Freedom9339 · 2024-07-23T13:21:48Z

@mcgilman Just checking in. Any updates on this?

mcgilman · 2024-07-23T15:08:56Z

@mcgilman Just checking in. Any updates on this?

Thanks for the ping. Sorry for the delay, I'll try to get some more eyes on this PR this week.

Freedom9339 · 2024-08-21T17:36:01Z

@mcgilman Sorry the pinging again. Just want to make sure this doesn't fall through the cracks.

mcgilman · 2024-08-23T21:43:56Z

@mcgilman Sorry the pinging again. Just want to make sure this doesn't fall through the cracks.

I just pulled down the latest. After rebasing against current main there were build failures due to changes that have landed after your PR was created. So I tried to run the PR as-is without rebasing and I'm still seeing issues. When running through the dev server it fails to serve with

Application bundle generation failed. [13.461 seconds]

✘ [ERROR] Undefined function.
╷
109 │ $surface-light-palette: mat.define-palette($surface-palette, 600, 100, 900);
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
╵
libs/shared/src/assets/themes/supplemental.scss 109:25 @import
apps/nifi/src/styles.scss 62:9 root stylesheet [plugin angular-sass]
angular:styles/global:styles:1:8:
  1 │ @import 'apps/nifi/src/styles.scss';

I then tried running with the built application but I'm seeing this error in dev tools when attempting to open the MoveControllerService Component.

TypeError: g is not iterable
    at chunk-VIZEOFFB.js:70:66450
    at Array.forEach (<anonymous>)
    at o.loadChildOptions (chunk-VIZEOFFB.js:70:66333)
    at new o (chunk-VIZEOFFB.js:70:65423)
    at o.ɵfac [as factory] (chunk-VIZEOFFB.js:70:67428)
    at hn (chunk-ZKVQE3GD.js:7:23746)
    at TC (chunk-ZKVQE3GD.js:7:63202)
    at mn.create (chunk-ZKVQE3GD.js:7:61795)
    at Vg.createComponent (chunk-ZKVQE3GD.js:7:64954)
    at t.attachComponentPortal (chunk-6XYXU2WR.js:1:110675)

Once the PR is updated I should be able to get some more eyes on it quickly.

Freedom9339 · 2024-08-26T13:58:21Z

@mcgilman I fixed the issue caused by rebasing and pushed. Thank You!

mcgilman

Thanks for the rebase! I've left some additional feedback below.

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

...d/src/main/frontend/apps/nifi/src/app/pages/flow-designer/state/controller-services/index.ts

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

Freedom9339 · 2024-09-03T15:39:33Z

@mcgilman I've completed the change/fixes requested. Thank You!

Freedom9339 · 2024-09-05T18:47:20Z

@mcgilman Sorry, I just realized my latest commit didn't go through. It's there now.

Freedom9339 · 2024-10-08T16:58:37Z

@mcgilman Any update on this? I just did a fresh rebase.

mcgilman

Thanks for the updates @Freedom9339! I've left a few more comments below.

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

...rc/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.html

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

...rc/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.html

Freedom9339 · 2024-10-16T19:05:23Z

@mcgilman Thank you for the review. I've made the requested changes. Thank You.

mcgilman

@Freedom9339 Thanks again for the updates! It's looking good. I've left a couple comments and I've noted a number of nit-picky coding style things to better follow some of the conventions currently practiced.

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

...rc/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.html

mcgilman · 2024-10-22T23:04:19Z

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

+
+        // build the form
+        this.moveControllerServiceForm = this.formBuilder.group({
+            processGroups: new FormControl('Process Group', Validators.required)


Process Group is not a valid value for this field. We could initialize the value to null but we're setting the value below. We could instead determine the value before creating the form and then just use the correct value when creating the FormControl.

Not sure what you mean by this. "Process Group" is what is shown whenever there is no process group selected. The only time this happens is if there are no valid process groups available.

The first argument to the FormControl is the value. The string Process Group is not a valid value. What you see in the control when no value is select comes from the mat-label in the mat-form-field. If you want something else than the mat-label value you can specify a placeholder on the mat-select.

I see. I've removed the initial value.

.../src/app/pages/flow-designer/ui/move-controller-service/move-controller-service.component.ts

nifi-frontend/src/main/frontend/apps/nifi/src/app/pages/flow-designer/service/flow.service.ts

...esigner/ui/controller-service/move-controller-service/move-controller-service.component.html

mcgilman

Thanks for the updates @Freedom9339. I've left a few comments below. I didn't call out a specific line but the new move-options endpoint should not be providing options and exposing details for parent or child Process Groups that the end user does not have permissions to. Please ensure your trying out all these scenarios when your PR changes.

Thanks!

mcgilman · 2024-11-18T20:52:22Z

...rontend/src/main/frontend/apps/nifi/src/app/pages/flow-designer/state/flow/flow.selectors.ts

+export const selectProcessGroupFlow = createSelector(
+    selectFlowState,
+    (state: FlowState) => state.flow.processGroupFlow
+);
+


This is no longer used and can be removed.

mcgilman · 2024-11-18T20:53:02Z

...-designer/ui/controller-service/move-controller-service/move-controller-service.component.ts

+import {
+    ControllerServiceEntity,
+    ControllerServiceReferencingComponent,
+    ControllerServiceReferencingComponentEntity


This is unused and is causing a lint warning.

mcgilman · 2024-11-18T20:53:18Z

...-designer/ui/controller-service/move-controller-service/move-controller-service.component.ts

+import { NifiSpinnerDirective } from 'apps/nifi/src/app/ui/common/spinner/nifi-spinner.directive';
+import { MoveControllerServiceDialogRequestSuccess } from '../../../state/controller-services';
+import { moveControllerService } from '../../../state/controller-services/controller-services.actions';
+import { BreadcrumbEntity } from '../../../state/shared';


This is unused and is causing a lint warning.

mcgilman · 2024-11-18T21:16:16Z

...k/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ControllerServiceResource.java

+        controllerServiceDTO.getReferencingComponents().forEach(e -> {
+            if (processGroup.getProcessGroup().findProcessor(e.getId()) == null
+                    && processGroup.getProcessGroup().findControllerService(e.getId(), true, false) == null) {
+                conflictingComponents.add("[" + e.getComponent().getName() + "]");


If I take away permissions from a component that references the Service I'm trying to remove this line generates a NullPointerException

2024-11-18 16:04:51,256 ERROR [NiFi Web Server-76] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException: Cannot invoke "org.apache.nifi.web.api.dto.ControllerServiceReferencingComponentDTO.getName()" because the return value of "org.apache.nifi.web.api.entity.ControllerServiceReferencingComponentEntity.getComponent()" is null. Returning Internal Server Error response. java.lang.NullPointerException: Cannot invoke "org.apache.nifi.web.api.dto.ControllerServiceReferencingComponentDTO.getName()" because the return value of "org.apache.nifi.web.api.entity.ControllerServiceReferencingComponentEntity.getComponent()" is null at org.apache.nifi.web.api.ControllerServiceResource.lambda$generateProcessGroupOption$18(ControllerServiceResource.java:751)

mcgilman · 2024-11-18T21:18:01Z

...-designer/ui/controller-service/move-controller-service/move-controller-service.component.ts

+    selector: 'move-controller-service',
+    standalone: true,
+    templateUrl: './move-controller-service.component.html',
+    imports: [


A number of the imports below are no longer used. Once they are removed from here, please ensure the the corresponding import above is removed.

Removed unused imports.

Freedom9339 · 2024-12-19T20:30:34Z

@mcgilman Sorry for the delay. I addressed the issues you mentioned and added more user validation to not expose any unauthorized components. I've tested every scenario I could think of. Please let me know if there is one I missed.

…groups with the new UI

… component to flow-designer. Added disabling options that have a move conflict

…upflow in the state and just load the info needed.

…I call. Moved move dialog to ui/controller-service.

Freedom9339 · 2025-02-12T17:31:13Z

@mcgilman Any update on this? I've just rebased to fix merge conflicts.

mcgilman

Thanks for the updates and the mention @Freedom9339! The front end changes are starting to look good. I've noted a couple minor things below. I've also noted a few things on the back end code but we'll want to get a closer look from @markap14 or @exceptionfactory before we sign off.

mcgilman · 2025-02-19T00:11:12Z

...esigner/ui/controller-service/move-controller-service/move-controller-service.component.html

+                                        <div
+                                            *ngIf="option.description != undefined; else valid"
+                                            class="pointer fa fa-warning has-errors caution-color"
+                                            style="padding: 3px"></div>


I'm not sure we should be using the warning icon here. When we use this, the user can hover to reveal the issue. However, here the icon cannot be interacted with and instead the warning is revealed as a tooltip on the menu item.

mcgilman · 2025-02-19T00:18:26Z

...k/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ControllerServiceResource.java

+                if (authorizableProcessGroupCurrent.getProcessGroup().getParent() != null) {
+                    final ProcessGroupAuthorizable authorizableProcessGroupParent = lookup.getProcessGroup(authorizableProcessGroupCurrent.getProcessGroup().getParent().getIdentifier());
+                    if (authorizableProcessGroupParent.getAuthorizable().isAuthorized(authorizer, RequestAction.READ, user)
+                            && authorizableProcessGroupParent.getAuthorizable().isAuthorized(authorizer, RequestAction.WRITE, user)) {
+                        options.add(generateProcessGroupOption(controllerServiceDTO, authorizableProcessGroupParent, lookup, user));
+                    }
+                }
+
+                authorizableProcessGroupCurrent.getProcessGroup().getProcessGroups().forEach(processGroup -> {
+                    final ProcessGroupAuthorizable authorizableProcessGroup = lookup.getProcessGroup(processGroup.getIdentifier());
+                    if (authorizableProcessGroup.getAuthorizable().isAuthorized(authorizer, RequestAction.READ, user)
+                            && authorizableProcessGroup.getAuthorizable().isAuthorized(authorizer, RequestAction.WRITE, user)) {
+                        options.add(generateProcessGroupOption(controllerServiceDTO, authorizableProcessGroup, lookup, user));
+                    }
+                });


Both parent and child options are added to the same collection. Do you think it would be helpful to differentiate between the parent Process Group from the child Process Groups in the listing shown to the user. I'm not suggesting we have to do this, just looking for feedback if you think it may be helpful to the user especially when dealing with a large flow.

The options collection is already ordered and should position the parent Process Group first but it'll only be present when the user has permissions.

mcgilman · 2025-02-19T00:22:13Z

...k/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ControllerServiceResource.java

+        return generateOkResponse(options).build();
+    }
+
+    private ProcessGroupOptionEntity generateProcessGroupOption(ControllerServiceDTO controllerServiceDTO, ProcessGroupAuthorizable processGroup, AuthorizableLookup lookup, NiFiUser user) {


This sort of logic is typically within the service facade. The Resource tier handles request validation, authorization, and request replication. It would defer to the service facade to handle the actual logic of the request.

mcgilman · 2025-02-19T00:25:36Z

...k/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ControllerServiceResource.java

+                    @SecurityRequirement(name = "Read - /flow")
+            }
+    )
+    public Response getProcessGroupOptions(


This endpoint needs to be replicated across the cluster. NiFi supports an extensible authorization model. That considered, for endpoints like we the call would be replicated to each node in the cluster. When in the response merger, we need to essentially return the intersection of options that are approved and in common for all nodes in the cluster.

mcgilman · 2025-02-19T00:31:11Z

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

+                        })
+                    ),
+                    catchError((errorResponse: HttpErrorResponse) =>
+                        of(ErrorActions.snackBarError({ error: this.errorHelper.getErrorString(errorResponse) }))


Since this action happens with the dialog open and the dialog remains open, we should be using a banner error here. This will show the error in a banner within the dialog.

mcgilman · 2025-02-19T00:35:37Z

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.effects.ts

+                            return of(
+                                ErrorActions.snackBarError({ error: this.errorHelper.getErrorString(errorResponse) })
+                            );


The effects for this action does not dispatch so this return has no effect. You'll want to replace this with this.store.dispatch(...)

mcgilman · 2025-02-19T00:37:27Z

...ps/nifi/src/app/pages/flow-designer/state/controller-services/controller-services.reducer.ts

@@ -99,6 +100,17 @@ export const controllerServicesReducer = createReducer(
            draftState.saving = false;
        });
    }),
+    on(moveControllerServiceSuccess, (state, { response }) => {


moveControllerService should be added to the reducer above to ensure that the saving flag is set to true.

ChrisSamo632 changed the title ~~NIFI-3785.1: Added feature to move controller services between process …~~ NIFI-3785: Added feature to move controller services between process … Jun 13, 2024

mcgilman self-requested a review June 14, 2024 21:54

mcgilman requested changes Jun 17, 2024

View reviewed changes

mcgilman added the new ui Pull requests for work relating to the new user interface being developed. label Jun 20, 2024

Freedom9339 force-pushed the nifi-3785.1 branch from 2de68fc to 6c4fab1 Compare June 25, 2024 18:01

Freedom9339 requested a review from mcgilman June 27, 2024 12:36

Freedom9339 force-pushed the nifi-3785.1 branch from fc7e2c1 to 9392904 Compare July 3, 2024 17:12

Freedom9339 force-pushed the nifi-3785.1 branch from 9392904 to 9009d70 Compare August 26, 2024 13:56

mcgilman reviewed Aug 30, 2024

View reviewed changes

Freedom9339 force-pushed the nifi-3785.1 branch from 9009d70 to 5a86dbe Compare September 3, 2024 15:34

Freedom9339 force-pushed the nifi-3785.1 branch from c368f43 to 712413e Compare October 8, 2024 16:57

mcgilman reviewed Oct 9, 2024

View reviewed changes

Freedom9339 force-pushed the nifi-3785.1 branch from 712413e to 275c18d Compare October 16, 2024 19:02

mcgilman reviewed Oct 22, 2024

View reviewed changes

Freedom9339 force-pushed the nifi-3785.1 branch 2 times, most recently from 59ef48f to 47df567 Compare October 28, 2024 17:15

mcgilman reviewed Oct 30, 2024

View reviewed changes

...esigner/ui/controller-service/move-controller-service/move-controller-service.component.html Outdated Show resolved Hide resolved

Freedom9339 force-pushed the nifi-3785.1 branch from 47df567 to 94b6dd4 Compare November 12, 2024 16:28

mcgilman reviewed Nov 18, 2024

View reviewed changes

Freedom9339 force-pushed the nifi-3785.1 branch from 94b6dd4 to bedb313 Compare December 19, 2024 20:25

Freedom9339 added 12 commits February 12, 2025 16:19

NIFI-3785: Added feature to move controller services between process …

8db98eb

…groups with the new UI

NIFI-3785: Fixed issue caused by rebasing

631ac19

NIFI-3785: Various feedback requests completed. Moved move controller…

be400f9

… component to flow-designer. Added disabling options that have a move conflict

NIFI-3785: removed need for second api call

45ef64a

NIFI-3785: Fixed dropdown width. Stopped storing the whole processgro…

c2cffcb

…upflow in the state and just load the info needed.

NIFI-3785: Rebased.

baee3a1

NIFI-3785: Fixed recursion issue. Removed need to clone.

57fd4d7

NIFI-3785: Added protection against null root. Applied suggestions.

ac5c842

NIFI-3785: Moved the logic for getting process group options to an AP…

3ee6c4e

…I call. Moved move dialog to ui/controller-service.

NIFI-3785: Removed invalid initial process group value.

713fccc

NIFI-3785: Fixed user authorization errors. Removed unused imports.

f197809

NIFI-3785: Fixed Rebasing issues

06a1e13

Freedom9339 force-pushed the nifi-3785.1 branch from bedb313 to 06a1e13 Compare February 12, 2025 17:29

mcgilman requested changes Feb 19, 2025

View reviewed changes

NIFI-3785: Added feature to move controller services between process … #8965

Are you sure you want to change the base?

NIFI-3785: Added feature to move controller services between process … #8965

Conversation

Freedom9339 commented Jun 13, 2024 • edited Loading

Summary

Tracking

Issue Tracking

Pull Request Tracking

Pull Request Formatting

Verification

Build

Licensing

Documentation

Freedom9339 commented Jun 13, 2024

Freedom9339 commented Jun 14, 2024

exceptionfactory commented Jun 14, 2024

mcgilman commented Jun 14, 2024

mcgilman left a comment

Choose a reason for hiding this comment

Freedom9339 commented Jul 23, 2024

mcgilman commented Jul 23, 2024

Freedom9339 commented Aug 21, 2024

mcgilman commented Aug 23, 2024

Freedom9339 commented Aug 26, 2024

mcgilman left a comment

Choose a reason for hiding this comment

Freedom9339 commented Sep 3, 2024

Freedom9339 commented Sep 5, 2024

Freedom9339 commented Oct 8, 2024

mcgilman left a comment

Choose a reason for hiding this comment

Freedom9339 commented Oct 16, 2024

mcgilman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcgilman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Freedom9339 commented Dec 19, 2024

Freedom9339 commented Feb 12, 2025

mcgilman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Freedom9339 commented Jun 13, 2024 •

edited

Loading