Skip to content

Commit

Permalink
docs(examples): add grafana cloud tempo recipe (#3879)
Browse files Browse the repository at this point in the history
* docs(examples): add datastore resource to sls example

* docs(recipes): grafana cloud tempo

* docs(recipes): grafana cloud edit infra

* docs(recipes): edit code link

* docs(recipe): add poke grafana cloud

* docs(recipes): update grafana cloud readme

* docs(recipes): grafana
  • Loading branch information
Adnan Rahić authored May 30, 2024
1 parent b8d2667 commit 334a85d
Show file tree
Hide file tree
Showing 26 changed files with 3,865 additions and 1,938 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
---
id: running-tracetest-with-grafana-cloud-tempo-pokeshop
title: Pokeshop and Grafana Cloud Tempo
description: Quick start on how to configure the Pokeshop Demo app with OpenTelemetry traces, Grafana Tempo as a trace data store, and Tracetest for enhancing your E2E and integration tests with trace-based testing.
hide_table_of_contents: true
keywords:
- tracetest
- trace-based testing
- observability
- distributed tracing
- testing
- grafana cloud
- grafana tempo
- grafana traces
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::note
[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/tracetest-grafana-cloud-tempo-pokeshop)
:::

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.

[Grafana Tempo](https://grafana.com/oss/tempo/) is an open-source, high-scale distributed tracing data store. It utilizes object storage to minimize costs, allowing you to store more information at a lower cost. It is deeply integrated with Grafana, Prometheus, and Loki. Grafana Tempo is also available as a paid hosted service at Grafana Cloud.

[Grafana Cloud](https://grafana.com/products/cloud/) is a managed service offering from Grafana Labs that provides scalable and secure monitoring, visualization, and alerting for metrics, logs, and traces. It integrates seamlessly with various data sources, enabling users to create dynamic, interactive dashboards and gain insights into their system performance and reliability. With features like team collaboration, alerting, and plugins, Grafana Cloud is designed to simplify observability and help teams efficiently monitor their applications and infrastructure.

[Pokeshop API](https://docs.tracetest.io/live-examples/pokeshop/overview) is a testing ground, the team at Tracetest has implemented a sample instrumented API around the [PokeAPI](https://pokeapi.co/).

## Pokeshop API with Grafana Cloud Tempo, OpenTelemetry and Tracetest

This is a simple quick start guide on how to configure a fully instrumented API to be used with Tracetest for enhancing your E2E and integration tests with trace-based testing. The infrastructure will use Grafana Cloud Tempo as the trace data store and the Pokeshop API to generate the telemetry data.

## Prerequisites

**Tracetest Account**:

- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/installation) docs.
- Create an [environment](/concepts/environments).
- Create an [environment token](/concepts/environment-tokens).
- Have access to the environment's [agent API key](/configuration/agent).

**Grafana Cloud Account**

- Sign up to [Grafana Cloud](https://grafana.com/auth/sign-in/).
- [Create tokens for both reading and writing traces by following this guide](https://grafana.com/blog/2021/04/13/how-to-send-traces-to-grafana-clouds-tempo-service-with-opentelemetry-collector/).

**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.

## Run This Quckstart Example

The example below is provided as part of the Tracetest project. You can download and run the example by following these steps:

Clone the Tracetest project and go to the Grafana Cloud Tempo Pokeshop Quickstart:

```bash
git clone https://github.com/kubeshop/tracetest
cd tracetest/examples/tracetest-grafana-cloud-tempo-pokeshop
```

Follow these instructions to run the quick start:

1. Copy the `.env.template` file to `.env`.
2. Log into the [Tracetest app](https://app.tracetest.io/).
3. Fill out the [token](https://docs.tracetest.io/concepts/environment-tokens) and [API key](https://docs.tracetest.io/concepts/agent) details by editing your `.env` file. You can find these values in the Settings area for your environment.
4. Run `docker compose up -d`.
5. This example is configured to use the Grafana Tempo Tracing Backend. Ensure the environment you're using to run this example is configured to use the Grafana Tempo Tracing Backend by clicking on Settings, Tracing Backend, Grafana Tempo, Save. Or, use the CLI as explained below.
6. Run tests from the Tracetest Web UI by accessing the app with the URL `http://demo-api:8081/pokemon/import`.

Follow along with the sections below for an in detail breakdown of what the example you just ran did and how it works.

## Project Structure

The project contains the [Tracetest Agent](/getting-started/installation#install-the-tracetest-agent), and the [Pokeshop Demo app](/live-examples/pokeshop/overview).

The `docker-compose.yaml` file in the root directory of the quick start runs the Pokeshop Demo app, [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector-contrib), and the [Tracetest Agent](/concepts/agent) setup.

## Configuring the Pokeshop Demo App

The Pokeshop API is a fully instrumented REST API that makes use of different services to mimic a real life scenario.

It is instrumented using the [OpenTelemetry standards for Node.js](https://opentelemetry.io/docs/instrumentation/js/getting-started/nodejs/), sending the data to the ADOT collector that will be pushing the telemetry information to both the AWS X-Ray service.

This is a [fragment from the main tracing file](https://github.com/kubeshop/pokeshop/blob/master/api/src/telemetry/tracing.ts) from the [Pokeshop API repo](https://github.com/kubeshop/pokeshop).

Configure the `.env` like shown below.

```bash
TRACETEST_API_KEY="<YOUR_TRACETEST_API_KEY>"
TRACETEST_API_TOKEN="<YOUR_TRACETEST_TOKEN>"
```

## Configuring OpenTelemetry Collector

Configure OpenTelemetry Collector to send traces to Grafana Cloud Tempo.

```yaml title="collector.config.yaml"
receivers:
otlp:
protocols:
grpc:
http:

exporters:
logging:
verbosity: detailed
otlp:
endpoint: tempo-us-central1.grafana.net:443
headers:
authorization: Basic <base64 encoded username:password>

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp]
```
Fill the authorization header in the `collector.config.yaml` file from your Grafana Tempo Setup. It should be encoded `base64` with the format of `username:token`. Follow [this guide to learn how](https://grafana.com/blog/2021/04/13/how-to-send-traces-to-grafana-clouds-tempo-service-with-opentelemetry-collector/).

## Configuring Grafana Tempo

Configure Tempo as a Tracing Backend:

```yaml title="tracetest-tracing-backend.yaml"
---
type: DataStore
spec:
id: tempo-cloud
name: Tempo
type: tempo
tempo:
type: http
http:
url: https://tempo-us-central1.grafana.net/tempo
headers:
authorization: Basic <base64 encoded username:password>
tls: {}
```

Fill in the details of your Grafana Cloud Tempo instance by using the HTTP integration `authorization` header. It should be encoded `base64` with the format of `username:token`. Follow [this guide to learn how](https://grafana.com/blog/2021/04/13/how-to-send-traces-to-grafana-clouds-tempo-service-with-opentelemetry-collector/).

```bash
tracetest config -t <YOUR_API_TOKEN>
tracetest apply datastore -f ./tracetest-tracing-backend.yaml
```

## Run the Pokeshop Demo App, OpenTelemetry Collector, and Tracetest Agent with Docker Compose

The [`docker-compose.yaml` file](https://github.com/kubeshop/tracetest/blob/main/examples/tracetest-grafana-cloud-tempo-pokeshop/docker-compose.yaml) in the root directory contains the Pokeshop Demo app services.

The [`docker-compose.yaml` file](https://github.com/kubeshop/tracetest/blob/main/examples/tracetest-grafana-cloud-tempo-pokeshop/docker-compose.yaml) also contains the Tracetest Agent and OpenTelemetry Collector.

To start it, run this command:

```bash
docker compose up -d
```

This will start the Pokeshop Demo app, OpenTelemetry Collector, and Tracetest Agent.

## Run Tracetest Tests

1. Open [Tracetest](https://app.tracetest.io/)
2. [Configure Grafana Tempo as a tracing backend](/configuration/connecting-to-data-stores/tempo) if you have not already as explained above.
3. Start creating tests! Make sure to use the `http://demo-api:8081/pokemon/import` URL in your test creation.
4. To trigger tests in the CLI, first [install the CLI](/cli/cli-installation-reference), [configure it](/cli/configuring-your-cli), and [run a test](/cli/running-tests). From the root of the quick start directory, run:

```bash
tracetest configure -t <YOUR_API_TOKEN>
tracetest run test -f ./tests/test.yaml
```

## Learn More

Feel free to check out our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!
Loading

0 comments on commit 334a85d

Please sign in to comment.