From 2f9f16a5c6720cb7db0c63d1cb7ba085423cbc06 Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 15:57:02 +0800 Subject: [PATCH 1/7] add: add exchange faq tips --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index df8f2bc4f1a..87c16037f03 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -116,6 +116,32 @@ nebula-exchange-2.0.0.jar \ -c application.conf ``` +### Q：Hive 数据导入时提示 schema 版本不一致 + +Spark 日志提示下面这类信息 `Hive Schema version 1.2.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt` + +这是因为 Hive 环境中配置的 metastore schema 版本和 Spark 使用的 metastore 版本不一致。 + +解决方法： + +1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本： + +假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，则需要按如下方式修改 hive.VERSION 表中的 version 字段： + +``` +update hive.VERSION set SCHEMA_VERSION="2.1.0" where VER_ID=1 +``` + +2. 在 Hive 环境的 hive-site.xml 中增加如下配置： + +``` + + hive.metastore.schema.verification + false + +``` +重启 Hive 即可。 + ## 配置问题 ### Q：哪些配置项影响导入性能？ From ccd77d4fb6d6524cec5c1ba8caa72decc294ba54 Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 19:19:44 +0800 Subject: [PATCH 2/7] Update docs-2.0/nebula-exchange/ex-ug-FAQ.md Co-authored-by: randomJoe211 <69501902+randomJoe211@users.noreply.github.com> --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index 87c16037f03..aaa0f117d58 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -118,9 +118,7 @@ nebula-exchange-2.0.0.jar \ ### Q：Hive 数据导入时提示 schema 版本不一致 -Spark 日志提示下面这类信息 `Hive Schema version 1.2.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt` - -这是因为 Hive 环境中配置的 metastore schema 版本和 Spark 使用的 metastore 版本不一致。 +Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt` 的原因是 Hive 环境中配置的 metastore schema 版本和 Spark 使用的 metastore 版本不一致。 解决方法： From 88ad67c74786aa3fd3d56928cfbe045d7b40ad19 Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 19:20:27 +0800 Subject: [PATCH 3/7] Update docs-2.0/nebula-exchange/ex-ug-FAQ.md Co-authored-by: randomJoe211 <69501902+randomJoe211@users.noreply.github.com> --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index aaa0f117d58..0bc2bf08f30 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -122,7 +122,7 @@ Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema 解决方法： -1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本： +1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本。 假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，则需要按如下方式修改 hive.VERSION 表中的 version 字段： From ad9ebfbf05924e1cee8c580cba547bf4b5b4921c Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 19:20:55 +0800 Subject: [PATCH 4/7] Update docs-2.0/nebula-exchange/ex-ug-FAQ.md Co-authored-by: randomJoe211 <69501902+randomJoe211@users.noreply.github.com> --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index 0bc2bf08f30..10d35c3a8c0 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -130,7 +130,7 @@ Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema update hive.VERSION set SCHEMA_VERSION="2.1.0" where VER_ID=1 ``` -2. 在 Hive 环境的 hive-site.xml 中增加如下配置： +2. 在 Hive 环境的 `hive-site.xml` 文件中增加如下配置： ``` From 7a05dffc35a0ca44ccfcd6d5c9b3199ba97627af Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 19:26:45 +0800 Subject: [PATCH 5/7] mod: modify format --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 29 +++++++++++++-------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index 87c16037f03..1d6d5711a6a 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -118,28 +118,27 @@ nebula-exchange-2.0.0.jar \ ### Q：Hive 数据导入时提示 schema 版本不一致 -Spark 日志提示下面这类信息 `Hive Schema version 1.2.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt` - -这是因为 Hive 环境中配置的 metastore schema 版本和 Spark 使用的 metastore 版本不一致。 +Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt` 的原因是 Hive 环境中配置的 metastore schema 版本和 Spark 使用的 metastore 版本不一致。 解决方法： -1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本： +1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本。 -假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，则需要按如下方式修改 hive.VERSION 表中的 version 字段： + 假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，则需要按如下方式修改 `hive.VERSION` 表中的 `version` 字段： -``` -update hive.VERSION set SCHEMA_VERSION="2.1.0" where VER_ID=1 -``` + ``` + update hive.VERSION set SCHEMA_VERSION="2.1.0" where VER_ID=1 + ``` -2. 在 Hive 环境的 hive-site.xml 中增加如下配置： +2. 在 Hive 环境的 `hive-site.xml` 文件中增加如下配置： + + ``` + + hive.metastore.schema.verification + false + + ``` -``` - - hive.metastore.schema.verification - false - -``` 重启 Hive 即可。 ## 配置问题 From 7bdfedbd838ff6ea7af1ce6028d62fd0fa2f955a Mon Sep 17 00:00:00 2001 From: Steam Date: Wed, 15 Dec 2021 19:28:30 +0800 Subject: [PATCH 6/7] mod: modify exchange faq format --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index 1d6d5711a6a..61d4fcf94ec 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -139,7 +139,7 @@ Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema ``` -重启 Hive 即可。 +3. 重启 Hive 即可。 ## 配置问题 From e3878b9879c7c2d5a52af5269218cbb6a50addb4 Mon Sep 17 00:00:00 2001 From: randomJoe211 <69501902+randomJoe211@users.noreply.github.com> Date: Thu, 16 Dec 2021 10:15:33 +0800 Subject: [PATCH 7/7] Update ex-ug-FAQ.md --- docs-2.0/nebula-exchange/ex-ug-FAQ.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-2.0/nebula-exchange/ex-ug-FAQ.md b/docs-2.0/nebula-exchange/ex-ug-FAQ.md index 61d4fcf94ec..40867bfd3c3 100644 --- a/docs-2.0/nebula-exchange/ex-ug-FAQ.md +++ b/docs-2.0/nebula-exchange/ex-ug-FAQ.md @@ -124,7 +124,7 @@ Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema 1. 将 Hive 环境中存储 Hive metastore 信息的 MySQL version 信息更新为 Spark 中使用的 metastore 版本。 - 假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，则需要按如下方式修改 `hive.VERSION` 表中的 `version` 字段： + 假设 Hive 在 MySQL 中存储 metastore 的数据库是`hive`，需要按如下方式修改 `hive.VERSION` 表中的 `version` 字段： ``` update hive.VERSION set SCHEMA_VERSION="2.1.0" where VER_ID=1 @@ -139,7 +139,7 @@ Spark 日志提示 `Hive Schema version 1.2.0 does not match metastore's schema ``` -3. 重启 Hive 即可。 +3. 重启 Hive。 ## 配置问题