Skip to content

Latest commit

 

History

History
58 lines (42 loc) · 1.57 KB

HDFS-14317.md

File metadata and controls

58 lines (42 loc) · 1.57 KB

HDFS-14317

In this case, the standby namenode does not trigger edit log rolling when in-progress edit log tailing is enabled.

Modify config

We assume the hostnames of 5 nodes are NODE1, NODE2, NODE3, NODE4 and NODE5. Please replace the hostnames in the commands with yours.

In experiments/reproduce/HDFS-14317/workers

NODE3
NODE4
NODE5

In experiments/reproduce/HDFS-14317/hdfs-site.xml

<property>
  <name>dfs.namenode.rpc-address.razor.nn1</name>
  <value>NODE1:8020</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.razor.nn2</name>
  <value>NODE2:8020</value>
</property>

Run commands (~15min)

Run these commands on all nodes:

experiments/reproduce/HDFS-14317/install_HDFS-14317.sh [path_to_OathKeeper] [path_to_HADOOP]
./run_engine.sh install conf/samples/hdfs-3.2.1.properties hdfs
experiments/reproduce/HDFS-14317/trigger_HDFS-14317.sh [path_to_OathKeeper] [path_to_HADOOP] NODE1 NODE2

Then you can checkout logs hadoop/hadoop-dist/target/hadoop-3.3.0-SNAPSHOT/logs/*.log, if you see NODE1 stops generating

INFO org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Triggering log roll on remote NameNode

That means reproducing succeed (you should probably check after more than 2min).

Check the hadoop/hadoop-dist/target/hadoop-3.3.0-SNAPSHOT/logs/*.out to see detection report for failed invariants. If invariants giving alerts, the detection is successful.

Shut down instances and Clean up

experiments/run/hdfs/cleanup.sh [path_to_OathKeeper] [path_to_HADOOP] NODE1