Skip to content

Commit b2d23a8

Browse files
chore(external docs): update sink tutorials with Data Volume tag changes (#18148)
* Update tutorials Signed-off-by: Stephen Wakely <[email protected]> * Made clearer grouping by source and service Signed-off-by: Stephen Wakely <[email protected]> --------- Signed-off-by: Stephen Wakely <[email protected]>
1 parent a7c95dd commit b2d23a8

File tree

2 files changed

+37
-14
lines changed

2 files changed

+37
-14
lines changed

docs/tutorials/sinks/1_basic_sink.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ is deserialized to the fields in this struct so the user can customise the
3333
sink's behaviour.
3434

3535
```rust
36-
#[configurable_component(sink("basic", "Basic sink."))]
36+
#[configurable_component(sink("basic"))]
3737
#[derive(Clone, Debug)]
3838
/// A basic sink that dumps its output to stdout.
3939
pub struct BasicConfig {

docs/tutorials/sinks/2_http_sink.md

+36-13
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ use crate::{
1616
http::HttpClient,
1717
internal_events::SinkRequestBuildError,
1818
};
19+
use vector_core::config::telemetry;
1920
use bytes::Bytes;
2021
```
2122

@@ -81,12 +82,12 @@ struct BasicEncoder;
8182
The Encoder must implement the [`Encoder`][encoder] trait:
8283

8384
```rust
84-
impl Encoder<Event> for BasicEncoder {
85+
impl encoding::Encoder<Event> for BasicEncoder {
8586
fn encode_input(
8687
&self,
8788
input: Event,
8889
writer: &mut dyn std::io::Write,
89-
) -> std::io::Result<usize> {
90+
) -> std::io::Result<(usize, GroupedCountByteSize)> {
9091
}
9192
}
9293
```
@@ -98,16 +99,25 @@ sending batches of events, or they may send a completely different type if each
9899
event is processed in some way prior to encoding.
99100

100101
[`encode_input`][encoder_encode_input] serializes the event to a String and
101-
writes these bytes:
102+
writes these bytes. The function also creates a [`GroupedCountByteSize`]
103+
[grouped_count_byte_size] object. This object tracks the size of the event
104+
that is sent by the sink, optionally grouped by the source and service that
105+
originated the event if Vector has been configured to do so. It is necessary to
106+
calculate the sizes in this function since the encode function sometimes drops
107+
fields from the event prior to encoding. We need the size to be calculated after
108+
these fields have been dropped.
102109

103110
```rust
104111
fn encode_input(
105112
&self,
106113
input: Event,
107114
writer: &mut dyn std::io::Write,
108-
) -> std::io::Result<usize> {
115+
) -> std::io::Result<(usize, GroupedCountByteSize)> {
116+
let mut byte_size = telemetry().create_request_count_byte_size();
117+
byte_size.add_event(&input, input.estimated_json_encoded_size_of());
118+
109119
let event = serde_json::to_string(&input).unwrap();
110-
write_all(writer, 1, event.as_bytes()).map(|()| event.len())
120+
write_all(writer, 1, event.as_bytes()).map(|()| (event.len(), byte_size))
111121
}
112122
```
113123

@@ -152,8 +162,12 @@ We need to implement a number of traits for the request to access these fields:
152162

153163
```rust
154164
impl MetaDescriptive for BasicRequest {
155-
fn get_metadata(&self) -> RequestMetadata {
156-
self.metadata
165+
fn get_metadata(&self) -> &RequestMetadata {
166+
&self.metadata
167+
}
168+
169+
fn metadata_mut(&mut self) -> &mut RequestMetadata {
170+
&mut self.metadata
157171
}
158172
}
159173

@@ -249,7 +263,7 @@ when sending the event to an `amqp` server.
249263
mut input: Event,
250264
) -> (Self::Metadata, RequestMetadataBuilder, Self::Events) {
251265
let finalizers = input.take_finalizers();
252-
let metadata_builder = RequestMetadataBuilder::from_events(&input);
266+
let metadata_builder = RequestMetadataBuilder::from_event(&input);
253267
(finalizers, metadata_builder, input)
254268
}
255269
```
@@ -338,7 +352,12 @@ that will be invoked to send the actual data.
338352
match client.call(req).await {
339353
Ok(response) => {
340354
if response.status().is_success() {
341-
Ok(BasicResponse { byte_size })
355+
Ok(BasicResponse {
356+
byte_size,
357+
json_size: request
358+
.metadata
359+
.into_events_estimated_json_encoded_byte_size(),
360+
})
342361
} else {
343362
Err("received error response")
344363
}
@@ -359,18 +378,21 @@ The return from our service must be an object that implements the
359378
```rust
360379
struct BasicResponse {
361380
byte_size: usize,
381+
json_size: GroupedCountByteSize,
362382
}
363383

364384
impl DriverResponse for BasicResponse {
365385
fn event_status(&self) -> EventStatus {
366386
EventStatus::Delivered
367387
}
368388

369-
fn events_sent(&self) -> RequestCountByteSize {
370-
// (events count, byte size)
371-
CountByteSize(1, self.byte_size).into()
389+
fn events_sent(&self) -> &GroupedCountByteSize {
390+
&self.json_size
372391
}
373-
}
392+
393+
fn bytes_sent(&self) -> Option<usize> {
394+
Some(self.byte_size)
395+
}}
374396
```
375397

376398
Vector calls the methods in this trait to determine if the event was delivered successfully.
@@ -492,3 +514,4 @@ BODY:
492514
[sinkbuilder_ext_into_driver]: https://rust-doc.vector.dev/vector/sinks/util/builder/trait.sinkbuilderext#method.into_driver
493515
[stream_filter_map]: https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.filter_map
494516
[driver]: https://rust-doc.vector.dev/vector_core/stream/struct.driver
517+
[grouped_count_byte_size]: https://rust-doc.vector.dev/vector_common/request_metadata/enum.groupedcountbytesize

0 commit comments

Comments
 (0)