Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metaclient timeout use paramter from configs #4

Merged
merged 4 commits into from
Oct 22, 2021
Merged

metaclient timeout use paramter from configs #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ff4862a
metaclient timeout use paramter from configs
Codelone Oct 18, 2021
04717b8
format code
Codelone Oct 20, 2021
f7dd897
Merge branch 'master' into liyanl-dev
Codelone Oct 22, 2021
4292d0e
metaclient, sync the test code
Codelone Oct 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions nebula-spark-connector/src/main/scala/com/vesoft/nebula/connector/nebula/MetaProvider.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ import com.vesoft.nebula.meta.{PropertyType, Schema}
import scala.collection.JavaConverters._
import scala.collection.mutable

class MetaProvider(addresses: List[Address]) extends AutoCloseable {
class MetaProvider(addresses: List[Address], timeout: Int, connectionRetry: Int, executionRetry: Int) extends AutoCloseable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameters are changed, please sync the test code to make the compile works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PropertyType.GEOGRAPHY not exists ，how to fix it

Copy link
Contributor

@Nicole00 Nicole00 Oct 22, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PropertyType.GEOGRAPHY has been supported in the latest java client, maybe your local maven repo is old. If you use IDEA, you can config it through preferences -> Build,Execution,Deployment -> Build tools -> Maven， choose Always update snapshots


val metaAddress = addresses.map(address => new HostAddress(address._1, address._2)).asJava
val client = new MetaClient(metaAddress)
val client = new MetaClient(metaAddress, timeout, connectionRetry, executionRetry)
client.connect()

def getPartitionNumber(space: String): Int = {
Expand Down
5 changes: 4 additions & 1 deletion ...k-connector/src/main/scala/com/vesoft/nebula/connector/reader/NebulaPartitionReader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@ abstract class NebulaPartitionReader extends InputPartitionReader[InternalRow] {
this()
this.schema = schema

metaProvider = new MetaProvider(nebulaOptions.getMetaAddress)
metaProvider = new MetaProvider(nebulaOptions.getMetaAddress,
nebulaOptions.timeout,
nebulaOptions.connectionRetry,
nebulaOptions.executionRetry)
val address: ListBuffer[HostAddress] = new ListBuffer[HostAddress]

for (addr <- nebulaOptions.getMetaAddress) {
Expand Down
5 changes: 4 additions & 1 deletion ...park-connector/src/main/scala/com/vesoft/nebula/connector/reader/NebulaSourceReader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,10 @@ abstract class NebulaSourceReader(nebulaOptions: NebulaOptions) extends DataSour
val returnCols = nebulaOptions.getReturnCols
val noColumn = nebulaOptions.noColumn
val fields: ListBuffer[StructField] = new ListBuffer[StructField]
val metaProvider = new MetaProvider(nebulaOptions.getMetaAddress)
val metaProvider = new MetaProvider(nebulaOptions.getMetaAddress,
nebulaOptions.timeout,
nebulaOptions.connectionRetry,
nebulaOptions.executionRetry)

import scala.collection.JavaConverters._
var schemaCols: Seq[ColumnDef] = Seq()
Expand Down
5 changes: 4 additions & 1 deletion nebula-spark-connector/src/main/scala/com/vesoft/nebula/connector/writer/NebulaWriter.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@ class NebulaWriter(nebulaOptions: NebulaOptions) extends Serializable {
val failedExecs: ListBuffer[String] = new ListBuffer[String]

val graphProvider = new GraphProvider(nebulaOptions.getGraphAddress)
val metaProvider = new MetaProvider(nebulaOptions.getMetaAddress)
val metaProvider = new MetaProvider(nebulaOptions.getMetaAddress,
nebulaOptions.timeout,
nebulaOptions.connectionRetry,
nebulaOptions.executionRetry)
val isVidStringType = metaProvider.getVidType(nebulaOptions.spaceName) == VidType.STRING

def prepareSpace(): Unit = {
Expand Down
2 changes: 1 addition & 1 deletion ...-spark-connector/src/test/scala/com/vesoft/nebula/connector/nebula/MetaProviderTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ class MetaProviderTest extends AnyFunSuite with BeforeAndAfterAll {

override def beforeAll(): Unit = {
val addresses: List[Address] = List(new Address("127.0.0.1", 9559))
metaProvider = new MetaProvider(addresses)
metaProvider = new MetaProvider(addresses, 6000, 3, 3)

val graphMock = new NebulaGraphMock
graphMock.mockStringIdGraph()
Expand Down