[bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE #5998

alextinng · 2023-12-12T12:15:57Z

Search before asking

I had searched in the issues and found no similar issues.

What happened

unfortunately I got problem connect to tdengine, I tried to find out the cause, but the reason is not printed.

part of log:

2023-12-12 19:57:29,452 ERROR org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (786924607960514561), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-TDengine-src]-SourceTask (1/1)] end with state FAILED and Exception: org.apache.seatunnel.connectors.seatunnel.tdengine.exception.TDengineConnectorException: ErrorCode:[COMMON-12], ErrorDescription:[Source reader operation failed, such as (open, close) etc...] - get TDengine connection failed:jdbc:TAOS-RS://localhost:6041/test?user=root&password=taosdata
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.open(TDengineSourceReader.java:113)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.open(SourceFlowLifeCycle.java:115)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:146)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:89)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

2023-12-12 19:57:29,452 ERROR org.apache.seatunnel.engine.server.dag.physical.SubPlan - Task TaskGroupLocation{jobId=786924607960514561, pipelineId=1, taskGroupId=50000} Failed in Job SeaTunnel_Job (786924607960514561), Pipeline: [(1/1)], Begin to cancel other tasks in this pipeline.

Next I changed the try catch block, add cause before re-throwing SQLException

I re-run the job and this time cause is printed on the screen

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:188)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.tdengine.exception.TDengineConnectorException: ErrorCode:[COMMON-12], ErrorDescription:[Source reader operation failed, such as (open, close) etc...] - get TDengine connection failed:jdbc:TAOS-RS://localhost:6041/test?user=root&password=taosdata
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.open(TDengineSourceReader.java:102)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.open(SourceFlowLifeCycle.java:115)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:146)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:89)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:TAOS-RS://localhost:6041/test?user=root&password=taosdata
        at java.sql.DriverManager.getConnection(DriverManager.java:689)
        at java.sql.DriverManager.getConnection(DriverManager.java:208)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.open(TDengineSourceReader.java:100)
        ... 9 more

        at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
        ... 2 more

this seems to be a simple problem, so I changed the source code again:

......
run the job and get another exception:

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:188)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.tdengine.exception.TDengineConnectorException: ErrorCode:[COMMON-12], ErrorDescription:[Source reader operation failed, such as (open, close) etc...] - TDengine split read error
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.lambda$pollNext$0(TDengineSourceReader.java:66)
        at java.lang.Iterable.forEach(Iterable.java:75)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.pollNext(TDengineSourceReader.java:61)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:135)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:84)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:165)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:89)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.convertDataType(TDengineSourceReader.java:145)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.read(TDengineSourceReader.java:136)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.lambda$pollNext$0(TDengineSourceReader.java:64)
        ... 12 more

        at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
        ... 2 more

......
after debug and found the cause is row columns existing null values, convertDataType will throw NPE if parameter is null

......

coding -> run the job -> and this time I succeed!

SeaTunnel Version

2.3.2

SeaTunnel Config

env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

source {
        TDengine {
          url : "jdbc:TAOS-RS://localhost:6041/"
          username : "root"
          password : "taosdata"
          database : "test"
          stable : "xxxxx"
          lower_bound : "2023-12-12 14:38:05.000"
          upper_bound : "2023-12-12 14:38:16.800"
          result_table_name = "src"
        }
}

sink {
  Console {}
}

Running Command

bin/seatunnel.sh --config a.conf -e local

Error Exception

2023-12-12 19:57:29,452 ERROR org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (786924607960514561), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-TDengine-src]-SourceTask (1/1)] end with state FAILED and Exception: org.apache.seatunnel.connectors.seatunnel.tdengine.exception.TDengineConnectorException: ErrorCode:[COMMON-12], ErrorDescription:[Source reader operation failed, such as (open, close) etc...] - get TDengine connection failed:jdbc:TAOS-RS://localhost:6041/test?user=root&password=taosdata
        at org.apache.seatunnel.connectors.seatunnel.tdengine.source.TDengineSourceReader.open(TDengineSourceReader.java:113)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.open(SourceFlowLifeCycle.java:115)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:146)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:89)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

The text was updated successfully, but these errors were encountered:

Carl-Zhou-CN · 2023-12-24T08:53:10Z

@alextinng Look forward to your contributions

github-actions · 2024-01-24T00:24:18Z

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

kulame · 2024-01-28T08:39:13Z

这个问题解决了吗？我也遇到了类似的错误。

2024-01-28 16:35:02,200 ERROR org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report error from task
org.apache.seatunnel.common.utils.SeaTunnelException: java.sql.SQLException: No suitable driver found for jdbc:TAOS-RS://localhost:6030stock?user=root&password=taosdata
        at java.sql.DriverManager.getConnection(DriverManager.java:689)
        at java.sql.DriverManager.getConnection(DriverManager.java:270)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSinkWriter.<init>(TDengineSinkWriter.java:69)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSink.createWriter(TDengineSink.java:52)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSink.createWriter(TDengineSink.java:34)
        at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:272)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:420)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:417)
        at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)

alextinng · 2024-01-29T05:53:31Z

这个问题解决了吗？我也遇到了类似的错误。

2024-01-28 16:35:02,200 ERROR org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report error from task
org.apache.seatunnel.common.utils.SeaTunnelException: java.sql.SQLException: No suitable driver found for jdbc:TAOS-RS://localhost:6030stock?user=root&password=taosdata
        at java.sql.DriverManager.getConnection(DriverManager.java:689)
        at java.sql.DriverManager.getConnection(DriverManager.java:270)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSinkWriter.<init>(TDengineSinkWriter.java:69)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSink.createWriter(TDengineSink.java:52)
        at org.apache.seatunnel.connectors.seatunnel.tdengine.sink.TDengineSink.createWriter(TDengineSink.java:34)
        at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:272)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:420)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:417)
        at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)

@kulame see #6088

alextinng added the bug label Dec 12, 2023

alextinng closed this as completed Dec 12, 2023

alextinng changed the title ~~[Improvement][tdengine] add cause while throwing TDengineConnectorException~~ [Improvement][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE Dec 13, 2023

alextinng reopened this Dec 13, 2023

alextinng changed the title ~~[Improvement][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE~~ [bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE Dec 13, 2023

Carl-Zhou-CN assigned alextinng Dec 24, 2023

alextinng mentioned this issue Dec 26, 2023

[fix] [connector-tdengine] fix sql exception and concurrentmodifyexception when connect to taos and read data #6088

Merged

4 tasks

github-actions bot added the stale label Jan 24, 2024

alextinng mentioned this issue Jan 29, 2024

[Bug] 尝试使用seatunnel同步mysql cdc数据到tdengine的时候报错，错误是找不到合适的驱动。 #6296

Open

3 tasks

Hisoka-X closed this as completed in #6088 Aug 8, 2024

alextinng mentioned this issue Oct 22, 2024

从tdengine同步数据到hdfs上一直报连接不上tdengine数据库的错误 #6747

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE #5998

[bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE #5998

alextinng commented Dec 12, 2023 •

edited

Loading

Carl-Zhou-CN commented Dec 24, 2023

github-actions bot commented Jan 24, 2024

kulame commented Jan 28, 2024

alextinng commented Jan 29, 2024 •

edited

Loading

[bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE #5998

[bug][tdengine] connector-tdengine didn't work cause no suitable driver found exception and NPE #5998

Comments

alextinng commented Dec 12, 2023 • edited Loading

Search before asking

What happened

SeaTunnel Version

SeaTunnel Config

Running Command

Error Exception

Zeta or Flink or Spark Version

Java or Scala Version

Screenshots

Are you willing to submit PR?

Code of Conduct

Carl-Zhou-CN commented Dec 24, 2023

github-actions bot commented Jan 24, 2024

kulame commented Jan 28, 2024

alextinng commented Jan 29, 2024 • edited Loading

alextinng commented Dec 12, 2023 •

edited

Loading

alextinng commented Jan 29, 2024 •

edited

Loading