docs(refactor): refactor of the overview guide (#10460)

Signed-off-by: prmellor <[email protected]>

documentation/assemblies/configuring/assembly-config.adoc

@@ -39,11 +39,13 @@ Use `KafkaNodePool` resources to configure distinct groups of nodes within a Kaf
 .Common configuration
 Common configuration is configured independently for each component, such as the following:
 
+* Bootstrap servers for host/port connection to a Kafka cluster
 * Metrics configuration
 * Healthchecks and liveness probes
 * Resource limits and requests (CPU/Memory)
 * Logging frequency
 * JVM options for maximum and minimum memory allocation
+* Adding additional volumes and volume mounts
 
 .Config maps to centralize configuration
 For specific areas of configuration, namely metrics, logging, and external configuration for Kafka Connect connectors, you can also use `ConfigMap` resources.
@@ -121,9 +123,12 @@ include::../../modules/operators/proc-disabling-fips-mode-cluster-operator.adoc[
 //`KafkaConnect` resource config
 include::../../modules/configuring/con-config-kafka-connect.adoc[leveloffset=+1]
 //running multiple Kafka Connect instances
-include::../../modules/configuring/con-config-kafka-connect-multiple-instances.adoc[leveloffset=+1]
+include::../../modules/configuring/con-config-kafka-connect-multiple-instances.adoc[leveloffset=+2]
 //if authorization is enabled, configure the Kafka Connect user for read/write access rights
 include::../../modules/configuring/proc-config-kafka-connect-user-authorization.adoc[leveloffset=+2]
+
+//`KafkaConnector` resource config
+include::../../modules/configuring/con-config-kafka-connector.adoc[leveloffset=+1]
 //Procedure to manually pause or stop a Kafka connector
 include::../../modules/configuring/proc-manual-stop-pause-connector.adoc[leveloffset=+2]
 //Procedure to manually restart a Kafka connector

documentation/assemblies/deploying/assembly-deploy-intro.adoc

@@ -7,7 +7,7 @@
 
 //Standard introduction text for the product
 [role="_abstract"]
-include::../../shared/snip-intro-text.adoc[leveloffset=+1]
+Strimzi simplifies the process of running link:https://kafka.apache.org/[Apache Kafka^] within a Kubernetes cluster.
 
 This guide provides instructions for deploying and managing Strimzi.
 Deployment options and steps are covered using the example installation files included with Strimzi. 

documentation/modules/con-common-configuration-properties.adoc

@@ -667,3 +667,61 @@ jvmOptions:
   gcLoggingEnabled: true
 # ...
 ----
+
+[id='con-common-configuration-volumes-{context}']
+== Additional volumes
+
+Strimzi supports specifying additional volumes and volume mounts in the following components:
+
+* Kafka
+* Kafka Connect
+* Kafka Bridge
+* Kafka MirrorMaker2
+* Entity Operator
+* Cruise Control
+* Kafka Exporter
+* Zookeeper
+* User Operator
+* Topic Operator
+
+All additional mounted paths are located inside `/mnt` to ensure compatibility with future Kafka and Strimzi updates.
+
+Supported Volume Types
+
+* Secret
+* ConfigMap
+* EmptyDir
+* PersistentVolumeClaims
+
+.Example configuration for additional volumes
+[source,yaml,subs=attributes+]
+----
+kind: Kafka
+spec:
+  kafka:
+    # ...
+    template:
+      pod:
+        volumes:
+          - name: example-secret
+            secret:
+              secretName: secret-name
+          - name: example-configmap
+            configMap:
+              name: config-map-name
+          - name: temp
+            emptyDir: {}
+          - name: example-pvc-volume
+            persistentVolumeClaim:
+              claimName: myclaim
+      kafkaContainer:
+        volumeMounts:
+          - name: example-secret
+            mountPath: /mnt/secret-volume
+          - name: example-configmap
+            mountPath: /mnt/cm-volume
+          - name: temp
+            mountPath: /mnt/temp
+          - name: example-pvc-volume
+            mountPath: /mnt/data
+----

documentation/modules/configuring/con-config-kafka-bridge.adoc

@@ -12,7 +12,7 @@ In order to prevent issues arising when client consumer requests are processed b
 Additionally, each independent Kafka Bridge instance must have a replica.
 A Kafka Bridge instance has its own state which is not shared with another instances.
 
-For a deeper understanding of the Kafka Bridge cluster configuration options, refer to the link:{BookURLConfiguring}[Strimzi Custom Resource API Reference^].
+For a deeper understanding of the Kafka Bridge and its cluster configuration options, refer to the link:{BookURLBridge}[Using the Kafka Bridge^] and the {BookURLConfiguring}[Strimzi Custom Resource API Reference^].
 
 .Example `KafkaBridge` custom resource configuration
 [source,yaml,subs="+quotes,attributes"]
@@ -111,7 +111,7 @@ spec:
     type: opentelemetry # <16>
 ----
 <1> The number of replica nodes.
-<2> Bootstrap server for connection to the target Kafka cluster. Use the name of the Kafka cluster as the <cluster_name>.
+<2> Bootstrap address for connection to the target Kafka cluster. The address takes the format `<cluster_name>-kafka-bootstrap:<port_number>`. The Kafka cluster doesn't need to be managed by Strimzi or deployed to a Kubernetes cluster.
 <3> TLS configuration for encrypted connections to the Kafka cluster, with trusted certificates stored in X.509 format within the specified secrets.
 <4> Authentication for the Kafka Bridge cluster, specified as mTLS, token-based OAuth, SASL-based SCRAM-SHA-256/SCRAM-SHA-512, or PLAIN.
 By default, the Kafka Bridge connects to Kafka brokers without authentication.
@@ -126,9 +126,4 @@ By default, the Kafka Bridge connects to Kafka brokers without authentication.
 <13> Optional: Container image configuration, which is recommended only in special situations.
 <14> Template customization. Here a pod is scheduled with anti-affinity, so the pod is not scheduled on nodes with the same hostname.
 <15> Environment variables are set for distributed tracing.
-<16> Distributed tracing is enabled by using OpenTelemetry.
-
-[role="_additional-resources"]
-.Additional resources
-
-* link:{BookURLBridge}[Using the Kafka Bridge^]
+<16> Distributed tracing is enabled by using OpenTelemetry.

documentation/modules/configuring/con-config-kafka-connect-multiple-instances.adoc

@@ -3,7 +3,7 @@
 // assembly-config.adoc
 
 [id='con-config-kafka-connect-multiple-instances-{context}']
-== Configuring Kafka Connect for multiple instances
+= Configuring Kafka Connect for multiple instances
 
 [role="_abstract"]
 By default, Strimzi configures the group ID and names of the internal topics used by Kafka Connect.

documentation/modules/configuring/con-config-kafka-connect.adoc

@@ -11,24 +11,15 @@ Update the `spec` properties of the `KafkaConnect` custom resource to configure
 Use Kafka Connect to set up external data connections to your Kafka cluster.
 Use the properties of the `KafkaConnect` resource to configure your Kafka Connect deployment.
 
-For a deeper understanding of the Kafka Connect cluster configuration options, refer to the link:{BookURLConfiguring}[Strimzi Custom Resource API Reference^].
-
-.KafkaConnector configuration
-`KafkaConnector` resources allow you to create and manage connector instances for Kafka Connect in a Kubernetes-native way.
+You can also use the `KafkaConnect` resource to specify the following:
 
-In your Kafka Connect configuration, you enable KafkaConnectors for a Kafka Connect cluster by adding the `strimzi.io/use-connector-resources` annotation.
-You can also add a `build` configuration so that Strimzi automatically builds a container image with the connector plugins you require for your data connections.
-External configuration for Kafka Connect connectors is specified through the `externalConfiguration` property.
+* Connector plugin configuration to build a container image that includes the plugins to make connections
+* Configuration for the Kafka Connect worker pods that run connectors
+* An annotation to enable use of the `KafkaConnector` resource to manage connectors
 
-To manage connectors, you can use use `KafkaConnector` custom resources or the Kafka Connect REST API.
-`KafkaConnector` resources must be deployed to the same namespace as the Kafka Connect cluster they link to.
-For more information on using these methods to create, reconfigure, or delete connectors, see xref:using-kafka-connect-with-plug-ins-{context}[Adding connectors].
+The Cluster Operator manages Kafka Connect clusters deployed using the `KafkaConnect` resource and connectors created using the `KafkaConnector` resource.
 
-Connector configuration is passed to Kafka Connect as part of an HTTP request and stored within Kafka itself.
-ConfigMaps and Secrets are standard Kubernetes resources used for storing configurations and confidential data.
-You can use ConfigMaps and Secrets to configure certain elements of a connector.
-You can then reference the configuration values in HTTP REST commands, which keeps the configuration separate and more secure, if needed.
-This method applies especially to confidential data, such as usernames, passwords, or certificates.
+For a deeper understanding of the Kafka Connect cluster configuration options, refer to the link:{BookURLConfiguring}[Strimzi Custom Resource API Reference^].
 
 .Handling high volumes of messages
 You can tune the configuration to handle high volumes of messages.
@@ -169,14 +160,17 @@ spec:
     type: opentelemetry # <21>
 ----
 <1> Use `KafkaConnect`.
-<2> Enables KafkaConnectors for the Kafka Connect cluster.
+<2> Enables the use of `KafkaConnector` resources to start, stop, and manage connector instances.
 <3> The number of replica nodes for the workers that run tasks.
 <4> Authentication for the Kafka Connect cluster, specified as mTLS, token-based OAuth, SASL-based SCRAM-SHA-256/SCRAM-SHA-512, or PLAIN.
 By default, Kafka Connect connects to Kafka brokers using a plain text connection.
-<5> Bootstrap server for connection to the Kafka cluster.
+<5> Bootstrap address for connection to the Kafka cluster. The address takes the format `<cluster_name>-kafka-bootstrap:<port_number>`. The Kafka cluster doesn't need to be managed by Strimzi or deployed to a Kubernetes cluster.
 <6> TLS configuration for encrypted connections to the Kafka cluster, with trusted certificates stored in X.509 format within the specified secrets.
-<7> Kafka Connect configuration of workers (not connectors).
+<7> Kafka Connect configuration of workers (not connectors) that run connectors and their tasks.
 Standard Apache Kafka configuration may be provided, restricted to those properties not managed directly by Strimzi.
+In this example, JSON convertors are specified. 
+A replication factor of 3 is set for the internal topics used by Kafka Connect (minimum requirement for production environment). 
+Changing the replication factor after the topics have been created has no effect.
 <8> Build configuration properties for building a container image with connector plugins automatically.
 <9> (Required) Configuration of the container registry where new images are pushed.
 <10> (Required) List of connector plugins and their artifacts to add to the new container image. Each plugin must be configured with at least one `artifact`.

documentation/modules/configuring/con-config-kafka-connector.adoc

@@ -0,0 +1,45 @@
+// Module included in the following assemblies:
+//
+// assembly-config.adoc
+
+[id='con-kafka-connector-config-{context}']
+= Configuring Kafka Connect connectors
+
+[role="_abstract"]
+The `KafkaConnector` resource provides a Kubernetes-native approach to management of connectors by the Cluster Operator.
+To create, delete, or reconfigure connectors with `KafkaConnector` resources, you must set the `use-connector-resources` annotation to `true`  in your `KafkaConnect` custom resource.
+
+.Annotation to enable KafkaConnectors
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaConnectApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect-cluster
+  annotations:
+    strimzi.io/use-connector-resources: "true"
+  # ...
+----
+
+When the `use-connector-resources` annotation is enabled in your `KafkaConnect` configuration, you must define and manage connectors using `KafkaConnector` resources. 
+
+NOTE: Alternatively, you can manage connectors using the Kafka Connect REST API instead of `KafkaConnector` resources.
+To use the API, you must remove the `strimzi.io/use-connector-resources` annotation to use `KafkaConnector` resources in the `KafkaConnect` the resource.
+
+`KafkaConnector` resources provide the configuration needed to create connectors within a Kafka Connect cluster, which interacts with a Kafka cluster as specified in the `KafkaConnect` configuration. 
+The Kafka cluster does not need to be managed by Strimzi or deployed to a Kubernetes cluster.
+
+.Kafka components contained in the same Kubernetes cluster
+image:overview/kafka-concepts-kafka-connector.png[Kafka and Kafka Connect clusters]
+
+The configuration also specifies how the connector instances interact with external data systems, including any required authentication methods. Additionally, you must define the data to watch. 
+For example, in a source connector that reads data from a database, the configuration might include the database name. 
+You can also define where this data should be placed in Kafka by specifying the target topic name.
+
+Use the `tasksMax` property to specify the maximum number of tasks.
+For instance, a source connector with `tasksMax: 2` might split the import of source data into two tasks.
+
+include::../../shared/snip-example-source-connector-config.adoc[]
+
+To include external connector configurations, such as user access credentials stored in a secret, use the `externalConfiguration` property of the `KafkaConnect` resource. 
+You can also load values using xref:assembly-loading-config-with-providers-str[configuration providers]. 

documentation/modules/configuring/con-config-mirrormaker2.adoc

@@ -233,11 +233,11 @@ spec:
 <4> Specification for the Kafka clusters being synchronized.
 <5> Cluster alias for the source Kafka cluster.
 <6> Authentication for the source cluster, specified as mTLS, token-based OAuth, SASL-based SCRAM-SHA-256/SCRAM-SHA-512, or PLAIN.
-<7> Bootstrap server for connection to the source Kafka cluster.
+<7> Bootstrap address for connection to the source Kafka cluster. The address takes the format `<cluster_name>-kafka-bootstrap:<port_number>`. The Kafka cluster doesn't need to be managed by Strimzi or deployed to a Kubernetes cluster.
 <8> TLS configuration for encrypted connections to the Kafka cluster, with trusted certificates stored in X.509 format within the specified secrets.
 <9> Cluster alias for the target Kafka cluster.
 <10> Authentication for the target Kafka cluster is configured in the same way as for the source Kafka cluster.
-<11> Bootstrap server for connection to the target Kafka cluster.
+<11> Bootstrap address for connection to the target Kafka cluster. The address takes the format `<cluster_name>-kafka-bootstrap:<port_number>`. The Kafka cluster doesn't need to be managed by Strimzi or deployed to a Kubernetes cluster.
 <12> Kafka Connect configuration.
 Standard Apache Kafka configuration may be provided, restricted to those properties not managed directly by Strimzi.
 <13> TLS encryption for the target Kafka cluster is configured in the same way as for the source Kafka cluster.

documentation/modules/deploying/con-switching-api-to-kafka-connector.adoc

@@ -3,7 +3,7 @@
 // assembly-deploy-kafka-connect-with-plugins.adoc
 
 [id='con-switching-api-to-kafka-connector-{context}']
-= Switching from using the Kafka Connect API to using KafkaConnector custom resources
+= Switching to using `KafkaConnector` custom resources
 
 [role="_abstract"]
 You can switch from using the Kafka Connect API to using `KafkaConnector` custom resources to manage your connectors.

documentation/modules/deploying/proc-deploy-kafka-connect-new-image-from-base.adoc

@@ -3,7 +3,7 @@
 // assembly-deploy-kafka-connect-with-plugins.adoc
 
 [id='creating-new-image-from-base-{context}']
-= Building a new container image with connector plugins from the Kafka Connect base image
+= Building new container images with connector plugins from the Kafka Connect base image
 
 [role="_abstract"]
 Create a custom Docker image with connector plugins from the Kafka Connect base image.

documentation/modules/deploying/proc-deploy-kafka-connect-using-kafka-connect-build.adoc

@@ -3,12 +3,13 @@
 // assembly-deploy-kafka-connect-with-plugins.adoc
 
 [id='creating-new-image-using-kafka-connect-build-{context}']
-= Building a new container image with connector plugins automatically
+= Building new container images with connector plugins automatically
 
 [role="_abstract"]
 Configure Kafka Connect so that Strimzi automatically builds a new container image with additional connectors.
 You define the connector plugins using the `.spec.build.plugins` property of the `KafkaConnect` custom resource.
-Strimzi will automatically download and add the connector plugins into a new container image.
+
+Strimzi automatically downloads and adds the connector plugins into a new container image.
 The container is pushed into the container repository specified in `.spec.build.output` and automatically used in the Kafka Connect deployment.
 
 .Prerequisites