Problems with Cloudflare analytics engine Clickhouse after upgrading to 3.0.0 #558

spelexander · 2024-05-12T16:48:35Z

I use Cloudflare Analytics Engine and Grafana cloud. In the official documentation for Cloudflare Analytics Engine, using this plugin is the recommended option for integrating with Grafana.

I have been using Altinity plugin with Grafana for some time, but due to the Angular deprecation warnings I decided to upgrade today.

Since upgrading my plugin to 3.0.0 I have been unable to:

Edit query settings (e.g. to select a database, table or DateTime column)

- Edit the plugin configuration in Grafana settings

Steps to reproduce:

Follow the setup guide here: https://developers.cloudflare.com/analytics/analytics-engine/grafana/
Create a new datasource
You will be unable to Save and Test the datasource without a crash in Grafana

Error message:

An unexpected error happened
Details
TypeError: Cannot assign to read only property 'dataSourceUrl' of object '#<Object>'

    at https://renewview.grafana.net/public/plugins/vertamedia-clickhouse-datasource/module.js?_cache=3.0.0:2:3699851
    at div
    at vt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:48:2348)
    at o (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:1:172)
    at form
    at Pt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:56:1914)
    at At (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:56:1464)
    at f (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:239:18087)
    at div
    at div
    at div
    at div
    at te (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:902:4503)
    at d (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:63:30037)
    at c
    at div
    at K (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:239:22026)
    at wt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:56:4946)
    at Bt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:56:5358)
    at j (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:29171)
    at j (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:31348)
    at Gt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/Connections.a1ad302de305309a8646.js:64:2099)
    at Suspense
    at l (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:63:125457)
    at er (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:184:11347)
    at j (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:29171)
    at co (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1035:4448)
    at lr (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1035:11997)
    at Nt (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:17387)
    at j (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:31348)
    at main
    at div
    at div
    at div
    at kn (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:184:2441)
    at div
    at kr (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:184:6891)
    at co (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1035:4448)
    at lr (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1035:11997)
    at hs (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1035:11333)
    at fr (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:17686)
    at j (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1044:25387)
    at v (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:111:8970)
    at _ (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1377:112624)
    at c (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:479:21361)
    at l (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:63:125457)
    at g (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:63:125966)
    at et (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/330.3e128f9f2fef2b8f895b.js:1016:530)
    at Ll (https://grafana-assets.grafana.net/grafana/11.1.0-70648/public/build/2540.a40dde18ed1e1774799b.js:195:127)

The text was updated successfully, but these errors were encountered:

Slach · 2024-05-13T06:33:59Z

duplicated #516
fixed in 3.1.0 which was submit on Friday
let's wait when review from Grafana Labs finish

caglarsayin · 2024-05-16T10:22:40Z

I cannot believe how long it takes

Slach · 2024-05-16T11:44:52Z

@caglarsayin 3.1.0 was released recently, please check it

dangeredwolf · 2024-10-19T16:24:16Z

Same issue on 3.2.3

Slach · 2024-10-19T20:47:15Z

@dangeredwolf which grafana-server version do you use?

dangeredwolf · 2024-10-19T20:49:28Z

@Slach I'm using Grafana Cloud, version 11.4.0-77383

Slach · 2024-10-19T20:52:00Z

When you said "same"
do you mean you see in console
TypeError: Cannot assign to read only property 'dataSourceUrl' of object '#<Object>' ?

or just "save & test" doesn't work?

dangeredwolf · 2024-10-19T20:56:31Z

I had the issue where the settings was broken with TypeError: Cannot assign to read only property 'dataSourceUrl' of object '#<Object>' (I found this GitHub issue by googling the error), and I also have the issue where I cannot set the timestamp column. Though now it's letting me access the settings without crashing this time.

dangeredwolf · 2024-10-19T20:59:29Z

I am also using Cloudflare Analytics Engine and I am unable to set the timestamp column as the timestamp. In fact, it does not list any databases, tables, or columns.

I can get the data to properly display in Grafana, but I cannot filter it by an time range because I have been unable to set the timestamp column. Per spec, it is available under the timestamp column.

Slach · 2024-10-20T16:02:19Z

Ok. let's try to figure out

could you share

curl -vvv -X POST "https://api.cloudflare.com/client/v4/accounts/<your-account-id>/analytics_engine/sql" -H "Authorization: Bearer <your-api-token>" --data "SELECT version()"

without sensitive credentials

dangeredwolf · 2024-10-20T16:14:58Z

I ran it and

< HTTP/2 422 
...
* Connection #0 to host api.cloudflare.com left intact
Input was invalid: unknown function call: version

Huh... So there's that.

Slach · 2024-10-20T16:21:49Z

sorry, try Version() instead version()

Slach · 2024-10-20T16:24:26Z

ok.
try

curl -vvv -X POST "https://api.cloudflare.com/client/v4/accounts/<your-account-id>/analytics_engine/sql" -H "Authorization: Bearer <your-api-token>" --data "SHOW CREATE TABLE system.columns"

dangeredwolf · 2024-10-20T16:27:31Z

unknown function call: Version

unsupported SQL statement type: SHOW CREATE TABLE system.columns

🫠

Slach · 2024-10-20T16:56:52Z

last check

same parameters but different --data

--data "SELECT name,database,table,type FROM system.columns WHERE type LIKE '%Date%' OR type LIKE '%DateTime%' OR type = 'UInt32' ORDER BY type,name FORMAT JSON"

dangeredwolf · 2024-10-20T18:01:21Z

unsupported binary operator: LIKE

Sorry if these outputs aren't more helpful, seems Cloudflare's SQL API is a bit limited :/

Even though ironically Cloudflare still recommends this specific plugin for use with Analytics Engine

Slach · 2024-10-20T19:36:54Z

check
--data "SELECT name,database,table,type FROM system.columns WHERE substring(type,1,4) = 'Date' OR type = 'UInt32' ORDER BY type,name FORMAT JSON"

looks like they tested on 2.x

…ers analytics engine, look details inhttps://github.com//issues/558

dangeredwolf · 2024-10-21T16:55:04Z

cannot specify a database when selecting a dataset: system

Slach · 2024-10-22T08:00:53Z

hm, looks like cloudflare workers analytics engine have too many restrictions

Slach · 2024-10-22T08:07:01Z

did you try just setup database and table and time related column, manually in UI, instead of try to select it from drop down?

dangeredwolf · 2024-10-22T15:28:45Z

did you try just setup database and table and time related column, manually in UI, instead of try to select it from drop down?

By manually in the UI do you mean typing it in and hitting enter? Cause if so, I did try that but Grafana won’t let me. If I’m missing something else though or if there’s a configuration file that I can manually edit I’d love to know about it

Slach · 2024-10-24T09:44:35Z

@dangeredwolf are you sure do you use latest version of plugin?
i just tried manual typing and just press enter in grafana cloud and everything works as expected.

dangeredwolf · 2024-10-24T11:47:27Z

@Slach 3.2.3 is the latest version, right?

Slach · 2024-10-24T11:49:36Z

yes, i created data source
after that i succesfully add panel

could you save screencast to reproduce whats failed on your side?

dangeredwolf · 2024-10-24T12:13:07Z

I am able to add it but am unable to set Database or Columns in the UI even though I can write SQL queries to do so
https://github.com/user-attachments/assets/8ba5332b-d056-42ab-8aef-61588a9ff55a

If I could somehow manually force timestamp as the timestamp column in the configuration somewhere it might be possible to work around this, but I'm not sure how to do that.

Slach · 2024-10-25T11:40:46Z

on shared screencast you didn't try to use keyboard

try just press left click to field and instead of trying select from drop down
type database and table name in edit field and press ENTER it shall added and $timeFilter and $timeSeries shall works after that

spelexander mentioned this issue May 12, 2024

Latest version of Altinity or Clickhouse plugins for Grafana does not work with Cloudflare Analytics Engine endpoint cloudflare/cloudflare-docs#14532

Closed

Slach closed this as completed May 13, 2024

Slach reopened this Oct 20, 2024

Slach added this to the 3.2.4 milestone Oct 20, 2024

Slach added a commit that referenced this issue Oct 20, 2024

try apply system.columns related workaround to allows CloudFlare work…

10db683

…ers analytics engine, look details inhttps://github.com//issues/558

Slach closed this as completed Oct 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problems with Cloudflare analytics engine Clickhouse after upgrading to 3.0.0 #558

Problems with Cloudflare analytics engine Clickhouse after upgrading to 3.0.0 #558

spelexander commented May 12, 2024 •

edited

Loading

Slach commented May 13, 2024 •

edited

Loading

caglarsayin commented May 16, 2024

Slach commented May 16, 2024

dangeredwolf commented Oct 19, 2024

Slach commented Oct 19, 2024

dangeredwolf commented Oct 19, 2024

Slach commented Oct 19, 2024 •

edited

Loading

dangeredwolf commented Oct 19, 2024

dangeredwolf commented Oct 19, 2024 •

edited

Loading

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 21, 2024

Slach commented Oct 22, 2024

Slach commented Oct 22, 2024

dangeredwolf commented Oct 22, 2024

Slach commented Oct 24, 2024

dangeredwolf commented Oct 24, 2024

Slach commented Oct 24, 2024

dangeredwolf commented Oct 24, 2024

Slach commented Oct 25, 2024

Problems with Cloudflare analytics engine Clickhouse after upgrading to 3.0.0 #558

Problems with Cloudflare analytics engine Clickhouse after upgrading to 3.0.0 #558

Comments

spelexander commented May 12, 2024 • edited Loading

Steps to reproduce:

Error message:

Slach commented May 13, 2024 • edited Loading

caglarsayin commented May 16, 2024

Slach commented May 16, 2024

dangeredwolf commented Oct 19, 2024

Slach commented Oct 19, 2024

dangeredwolf commented Oct 19, 2024

Slach commented Oct 19, 2024 • edited Loading

dangeredwolf commented Oct 19, 2024

dangeredwolf commented Oct 19, 2024 • edited Loading

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 20, 2024

Slach commented Oct 20, 2024

dangeredwolf commented Oct 21, 2024

Slach commented Oct 22, 2024

Slach commented Oct 22, 2024

dangeredwolf commented Oct 22, 2024

Slach commented Oct 24, 2024

dangeredwolf commented Oct 24, 2024

Slach commented Oct 24, 2024

dangeredwolf commented Oct 24, 2024

Slach commented Oct 25, 2024

spelexander commented May 12, 2024 •

edited

Loading

Slach commented May 13, 2024 •

edited

Loading

Slach commented Oct 19, 2024 •

edited

Loading

dangeredwolf commented Oct 19, 2024 •

edited

Loading