chore: [k217] fix: sanatize structured metadata at query time

[grafanabot]

Backport 3bf7fa9 from #13983

---

There's a bug in structured metadata where Loki can accept characters that are invalid in prometheus label names. A subsequent PR will reject inputs but the data that's already been ingested is not queryable.

As a workaround, this PR sanatizes structured metadata label names at query time. In the case where no bad inputs exist, there isn't a meaningful performance difference. Otherwise, there is 1 alloc per bad input. We could do this with byte slices to avoid incurring allocs, but doing it this way is the most straightforward way to ensure we aren't missing multi-byte characters that may exist in label names.

benchmark before:

goos: darwin
goarch: arm64
pkg: github.com/grafana/loki/v3/pkg/logql/log
Benchmark_Pipeline/pipeline_bytes-12              270484              4416 ns/op            1795 B/op         34 allocs/op
Benchmark_Pipeline/pipeline_string-12             266888              4432 ns/op            1859 B/op         35 allocs/op
Benchmark_Pipeline/line_extractor_bytes-12                                        241034              4940 ns/op            1489 B/op         33 allocs/op
Benchmark_Pipeline/line_extractor_string-12                                       243536              4933 ns/op            1489 B/op         33 allocs/op
Benchmark_Pipeline/label_extractor_bytes-12                                       236311              5224 ns/op            1489 B/op         33 allocs/op
Benchmark_Pipeline/label_extractor_string-12                                      239662              5015 ns/op            1489 B/op         33 allocs/op

benchmark after

goos: darwin
goarch: arm64
pkg: github.com/grafana/loki/v3/pkg/logql/log
Benchmark_Pipeline/pipeline_bytes-12              244771              5156 ns/op            1795 B/op         34 allocs/op
Benchmark_Pipeline/pipeline_string-12             244720              4869 ns/op            1859 B/op         35 allocs/op
Benchmark_Pipeline/pipeline_bytes_no_invalid_structured_metadata-12               238662              5326 ns/op            1555 B/op         35 allocs/op
Benchmark_Pipeline/pipeline_string_with_invalid_structured_metadata-12            228170              5219 ns/op            1664 B/op         37 allocs/op
Benchmark_Pipeline/line_extractor_bytes-12                                        206654              5378 ns/op            1490 B/op         33 allocs/op
Benchmark_Pipeline/line_extractor_string-12                                       222890              5429 ns/op            1490 B/op         33 allocs/op
Benchmark_Pipeline/label_extractor_bytes-12                                       218815              5601 ns/op            1490 B/op         33 allocs/op
Benchmark_Pipeline/label_extractor_string-12                                      208882              5470 ns/op            1490 B/op         33 allocs/op