add a logging alternative #80

megazone87 · 2020-08-03T15:20:30Z

Add a lightweight alternative logger, that make it be able to use "glog-same" API on platforms that glog may be unavailable and too heavy.

csukuangfj · 2020-08-03T21:49:03Z

CMakeLists.txt

 enable_testing()

 list(APPEND CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake)
 include(cpplint)
+if(K2_USE_GLOG)


if(K2_USE_GLOG) include(glog) endif()

csukuangfj · 2020-08-03T21:49:36Z

k2/CMakeLists.txt

@@ -1,2 +1,3 @@
+add_subdirectory(util)


please put it into csrc. No need to create a new directory.

agreed RE no need for new directory.

csukuangfj · 2020-08-03T21:51:15Z

k2/csrc/arcsort.cc

@@ -11,7 +11,7 @@
 #include <utility>
 #include <vector>

-#include "glog/logging.h"
+#include "k2/util/Logging.h"


please follow the code style and sort the headers
in alphabetic order.

All filenames should be in lowercase.

It's what may differ with logging.* in glog. But on case-insensitive platforms (windows), it's useless. Thus I changed to lowercase.

Changed to the alphabetic order.

csukuangfj · 2020-08-03T21:53:15Z

k2/util/Logging.cc

@@ -0,0 +1,57 @@
+// $FILE_PATHNAME $HEADER_FILENAME $HEADER_PATHNAME


please replace this line with the one used by other k2 files.

csukuangfj · 2020-08-03T22:00:11Z

k2/util/Logging.cc

+
+#include "k2/util/Logging.h"
+
+#ifdef K2_USE_GLOG


Why not to use glog directly? K2 is a training framework and I don't think
mobile devices have the power for training. As far as I know, glog
is supported on Linux, macOS and Windows.

Actually, k2 would be used in inference as well.
Of course that will require some things to be worked on!

For mobile is one thing, the other more important reason is about glog itself actually. After investigations, I found glog is not beautiful as I thought. Even the logging system itself is a trade-off/confusing concept, that would easily be noisy, expose too much global flags/macro, takes too much responsibility (exception/error handling). So in conclusion, I found there isn't the best practice. However, glog with one minimal glog-like logging is preferred by many great projects. A minimal glog could be very simple to just support minimal functions of glog. And loguru turn out to be the expected one. So, with loguru, the two-logger thought is implemented as this PR.

csukuangfj · 2020-08-03T22:02:58Z

k2/util/Logging.cc

+#else  // !K2_USE_GLOG
+namespace k2 {
+bool InitK2Logging(int *argc, char **argv) {
+  // When doing InitCaffeLogging, we will assume that caffe's flag parser has


Why does the comment contain caffe?
Is the code copied from caffe? Please add some comment
in the code to indicate where it is from.

the argc,argv stuff may not be necessary as I don't anticipate using k2 much in command line programs.

Emm, let me explain this. Before coding, I investigated pytorch, tf, bazel, and other projects that make use of glog, and many alternative logging methods. To find one best practice for logging, some common thought is used here. I try to keep minimal and non-specific to some project codes here. As here, it declares an internal function of glog, which is a common hack as it said, so I leave it as it is. However, this comment shouldn't be here. As the loguru here does parse the arg. The comment is removed now.

To @danpovey :
This function is necessary to init both loggers. At least, to configure the log to stdout or logfile. So I prefer to keep it right now and make the changes we need later.

csukuangfj · 2020-08-03T22:04:50Z

k2/util/Logging.cc

+  if (*argc == 0)
+    return true;
+
+  loguru::init(*argc, argv);


Name of functions should be in Uppercase. init->Init.

I guess the name init is from loguru

Yes, it is from loguru.

csukuangfj · 2020-08-03T22:06:36Z

k2/util/Logging.h

@@ -0,0 +1,32 @@
+//


the first line of every file should be the path to the file.

csukuangfj · 2020-08-03T22:07:07Z

k2/util/Logging.h

@@ -0,0 +1,32 @@
+//
+// Created by songmeixu ([email protected]) on 2020/8/3.


Please follow the header style of other files.

@songmeixu And I think it's better if you declare your copyright under Xiaomi Corporation, as Dan and I did.

Sorry, it's from my IDE old setting, and I forget it in some files. Changes have been made referring to the existing style.

csukuangfj · 2020-08-03T22:09:57Z

k2/util/loguru/loguru.cc

@@ -0,0 +1,1875 @@
+#ifndef _WIN32


Please indicate the origin of this file.

qindazhu · 2020-08-04T04:26:49Z

k2/csrc/connect.cc

 #include <algorithm>
 #include <limits>
 #include <stack>
 #include <unordered_map>
 #include <vector>

-#include "glog/logging.h"
+#include "k2/csrc/connect.h"


The corresponding header file (i.e. connect.h here) should be put at the top of .cc file.

qindazhu · 2020-08-04T04:31:59Z

k2/util/Logging.cc

+#ifdef K2_USE_GLOG
+// Google glog's api does not have an external function that allows one to check
+// if glog is initialized or not. It does have an internal function - so we are
+// declaring it here. This is a hack but has been used by a bunch of others too.


RE has been used by a bunch of others, it's better if we could list some famous project here with the corresponding code links.

danpovey · 2020-08-04T06:42:45Z

Thanks! Hopefully @csukuangfj can review the changes when he has time.

danpovey · 2020-08-04T07:09:11Z

Cool... can you please also try to familiarize yourself with cub's own asserty things, CubDebug() and CubDebugExit()?
The tricky thing is that quite a lot of the code to run in CUDA will also be CPU code, so we need an assert
that can compile to something on the device and host. Perhaps that might mean creating a version of some of the
glob macros that can compile on device and that uses printf.
The compilation of this codebase is going to be a little unconventional, we'll put basically everything through nvcc,
which will pass most code through to the host compiler.
The reason is to enable lambdas that can execute on host and device. You'll see examples in compose.cc in my branch.

megazone87 · 2020-08-04T07:37:23Z

Cool... can you please also try to familiarize yourself with cub's own asserty things, CubDebug() and CubDebugExit()?
The tricky thing is that quite a lot of the code to run in CUDA will also be CPU code, so we need an assert
that can compile to something on the device and host. Perhaps that might mean creating a version of some of the
glob macros that can compile on device and that uses printf.
The compilation of this codebase is going to be a little unconventional, we'll put basically everything through nvcc,
which will pass most code through to the host compiler.
The reason is to enable lambdas that can execute on host and device. You'll see examples in compose.cc in my branch.

I see. I have started to do it.

danpovey · 2020-08-04T09:00:12Z

CMakeLists.txt

@@ -33,11 +35,16 @@ if(WIN32 AND BUILD_SHARED_LIBS)
  set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
 endif()

+# build options
+option(K2_USE_GLOG "Flag that chooses logger from [GLOG, NON-GLOG (loguru)]" OFF)


Why is this off by default? Since we install gtest by default, doesn't that already require glog?

@songmeixu
when to enable this option?

OFF is set for test, and help figure out what we need for logging that is outside a minimal one.
The gtest doesn't depend on glog, the former is for unit test, while the latter is for logging. Though the implementations are similar and very simple for some cases (*_EQ), I think there is a design principle to isolate these two.

danpovey · 2020-08-04T09:00:37Z

k2/csrc/fsa.h

 #include "k2/csrc/array.h"
 #include "k2/csrc/util.h"
+#include "k2/csrc/util/logging.h"


I think you were going to get rid of the directory util/?

mm.. there is more code than I realized, now wondering whether we should keep util/

I expect more subdirs would be here, as k2 surely needs more function ((IDK) IO, bin ..) that should distribute more organized.

And I think the dlpack related codes under python/csrc/ may be better under csrc/dlpack as dlpack is not for pybind essentially (dlpack gives a way to transfer between data structures, as here we use it transfer k2.Tensor with torch.Tensor). Pytorch put it as from torch.utils.dlpack import to_dlpack in py API. I think we would need the class k2.Tensor from current python/csrc/tensor.h.

Anyway, I hold no opinion about the right project structure🤗. Changes happen fast. So, util/ is OK for now?

danpovey · 2020-08-04T09:06:11Z

k2/csrc/util/logging.h

@@ -0,0 +1,33 @@
+// k2/csrc/util/logging_is_google_glog.h


filename wrong?

This filename is referred from pytorch, and I think it's clear (TF use default/logging.h (customized minimal glog) and google/build_config/logging.h(glog) to distinguish each other.).

qindazhu · 2020-08-04T09:13:47Z

Oh, BTW, which branch should this PR be merged into? master or cuda, or both?

danpovey · 2020-08-04T09:14:35Z

Meixu, am I correct that to use this, one is supposed to use things like CHECK_EQ and so on? (That's what I seem to see in loguru.h, for instance). I'm concerned that a method that relies on macros without prefixes may easily clash with external codebases such as PyTorch; at some point we need a .cc file that #includes both torch and k2. Or is there some reason why this is not a problem?

Also: where is one supposed to go to see what the interface of the logging is? It would be nice if logging.h explained this clearly: I tried by it goes down a rabbit hole of conditional includes.

danpovey · 2020-08-04T09:15:07Z

Mm, it's up to you guys, but maybe master first and then merge it to cuda?

csukuangfj · 2020-08-04T09:16:08Z

k2/csrc/util/loguru/loguru.cc

@@ -0,0 +1,1894 @@
+/*


I suggest downloading these two files via cmake. You can refer to k2/cmake for how pybind11 is downloaded and included into k2.

qindazhu · 2020-08-04T09:17:01Z

k2/csrc/util/logging.cc

+// Google glog's api does not have an external function that allows one to check
+// if glog is initialized or not. It does have an internal function - so we are
+// declaring it here. This is a hack but has been used by a bunch of others too.
+// (e.g. Torch).


it's better if you add the corresponding code link of PyTorch? (and I don't know what Torch here is, sorry)

qindazhu · 2020-08-04T09:27:09Z

Mm, it's up to you guys, but maybe master first and then merge it to cuda?

+1, master first and then cuda.
Note for assertion code in cuda, we may merge the those code first to cuda branch as there's no nvcc build support on master now (since we have decided to do cuda stuff on cuda branch for now and merge back into master at some time.)

danpovey · 2020-08-04T09:31:06Z

About including both (say) PyTorch and k2, or TensorFlow and k2, I'm thinking that to avoid problems we could arrange to only include a small part of k2 code that doesn't require the logging #include, in the file that we do those PyTorch/TF #includes. That way we don't have to worry so much about clashing macros. BUT this does worry me a bit. E.g. what if someone were to try to include both k2 and torch in a project? Would it even work?
I'm thinking it would cause linking issues, at least.

qindazhu · 2020-08-04T09:31:21Z

And again, I suggest wrapper glog (or loguru, but I really don't think it's necessary...) into some functions/macros with some name prefixed with K2_ or else, which will keep scalability in the future.

danpovey · 2020-08-04T10:12:03Z

How about we just #include glog as-is without anything else special, wrap it with K2_ for future compatibility via e.g.

#define K2_CHECK_EQ CHECK_EQ

and leave it till later to have this loguru stuff? I feel like that would keep things future-compatible with the minimum amount of complexity at this stage.

csukuangfj · 2020-08-04T10:39:41Z

How about we just #include glog as-is without anything else special, wrap it with K2_ for future compatibility via e.g.

#define K2_CHECK_EQ CHECK_EQ
and leave it till later to have this loguru stuff? I feel like that would keep things future-compatible with the minimum amount of complexity at this stage.

+2

megazone87 · 2020-08-04T12:25:04Z

Meixu, am I correct that to use this, one is supposed to use things like CHECK_EQ and so on? (That's what I seem to see in loguru.h, for instance).

Correct. The same macro name is undef and define in loguru with the same signature. So no changes for codes use glog.

I'm concerned that a method that relies on macros without prefixes may easily clash with external codebases such as PyTorch; at some point we need a .cc file that #includes both torch and k2. Or is there some reason why this is not a problem?

Once glog is used for library (best to be shared), the glog init/config/link.. management responsibility is transfer to the final user (application level). Worrying but unfortunately, it is the doomed usage from the official:
google/glog#83
google/glog#244
kubernetes/kubernetes#61006

You see why glog is problematic. However, pytorch/tf (what k2 would live together) and many good packages use glog and follow this way.
In case somehow the glog is misuse, unavailable, unsuitable, confusing .., there is another minimal glog-like logger for configuration. Here loguru is it.

Thus, K2_ prefix is against the rule of glog, as it would make things worse. Think about multiple libraries link together, and keep in mind glog only init once. At least, the user would lost in configurations.

To own K2_ prefix is good if exclude glog totally and uses customized one (loguru is still one promising option. Its macro would append _F to be glog-dislike if set LOGURU_REPLACE_GLOG = 0.).

Also: where is one supposed to go to see what the interface of the logging is? It would be nice if logging.h explained this clearly: I tried by it goes down a rabbit hole of conditional includes.

Yes, loguru provides some flags to tune, like set LOGURU_REPLACE_GLOG=1 to imitate glog. LOGURU_REPLACE_GLOG=1 is set to be the default, the usage is as same as glog (except only part of common features/macro of glog are supported). As to the usage of glog, it is simple to refer to some articles, e.g. http://rpg.ifi.uzh.ch/docs/glog.html

I'm not a loguru fan, a logger replacement could introduce for reasons like: to be compatible with nvcc. I mean loguru is still in consideration, IDK if it is applicable (it support printf instead of stream.), probably give my answer soon.

danpovey · 2020-08-04T12:38:11Z

OK we don't have to decide today, Meixu can keep looking and thinking about how to support CUDA. I do tend to favor a solution that would be simple and self-contained, without too many features. E.g. just write via printf, which can be made to work on CUDA as well. Generally speaking errors are going to be fatal, so it doesn't matter that much how it's handled.

…

On Tue, Aug 4, 2020 at 8:25 PM Meixu Song ***@***.***> wrote: Meixu, am I correct that to use this, one is supposed to use things like CHECK_EQ and so on? (That's what I seem to see in loguru.h, for instance). Correct. The same macro name is undef and define in loguru with the same signature. So no changes for codes use glog. I'm concerned that a method that relies on macros without prefixes may easily clash with external codebases such as PyTorch; at some point we need a .cc file that #includes both torch and k2. Or is there some reason why this is not a problem? Once glog is used for library (best to be shared), the glog init/config/link.. management responsibility is transfer to the final user (application level). Worrying but unfortunately, it is the doomed usage from the official: google/glog#83 <google/glog#83> google/glog#244 <google/glog#244> kubernetes/kubernetes#61006 <kubernetes/kubernetes#61006> You see why glog is problematic. However, pytorch/tf (what k2 would live together) and many good packages use glog and follow this way. In case somehow the glog is misuse, unavailable, unsuitable, confusing .., there is another minimal glog-like logger for configuration. Here loguru is it. Thus, K2_ prefix is against the rule of glog, as it would make things worse. Think about multiple libraries link together, and keep in mind glog only init once. At least, the user would lost in configurations. To own K2_ prefix is good if exclude glog totally and uses customized one (loguru is still one promising option. Its macro would append _F to be glog-dislike if set LOGURU_REPLACE_GLOG = 0.). Also: where is one supposed to go to see what the interface of the logging is? It would be nice if logging.h explained this clearly: I tried by it goes down a rabbit hole of conditional includes. Yes, loguru provides some flags to tune, like set LOGURU_REPLACE_GLOG=1 to imitate glog. LOGURU_REPLACE_GLOG=1 is set to be the default, the usage is as same as glog (except only part of common features/macro of glog are supported). As to the usage of glog, it is simple to refer to some articles, e.g. http://rpg.ifi.uzh.ch/docs/glog.html ------------------------------ I'm not a loguru fan, a logger replacement could introduce for reasons like: to be compatible with nvcc. I mean loguru is still in consideration, IDK if it is applicable (it support printf instead of stream.), probably give my answer soon. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/danpovey/k2/pull/80#issuecomment-668564361>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO7576A5MT6IJJE3C73R67427ANCNFSM4PTOHZ6A> .

megazone87 · 2020-08-04T12:59:45Z

Some examples of glog-used and glog-avoid packages are given here:

used:

avoid:

My point is to use K2_ prefix is fine but glog should be excluded. Just #define K2_* * just add more confusions.

danpovey · 2020-08-04T13:29:24Z

I'm fine not using glog, e.g. to use the other solution, with K2_. As glog will likely introduce linking complications.

…

On Tue, Aug 4, 2020 at 9:00 PM Meixu Song ***@***.***> wrote: Some examples of glog-used and glog-avoid packages are given here: used: - pytorch - https://github.com/pytorch/pytorch/blob/2912390662c3c50732add54c94ac9b98d17e1cba/c10/util/Logging.h#L24 - tensorflow - https://github.com/tensorflow/tensorflow/blob/44597e39fb9e6eeb614d2119d516c0d4ede084cc/tensorflow/core/platform/logging.h#L22 avoid: - protobuf - https://github.com/protocolbuffers/protobuf/blob/d0f91c863ae0fbb75b41460c8bbb786ade197a0f/src/google/protobuf/stubs/logging.h#L119 - bazel - https://github.com/bazelbuild/bazel/blob/8aaf82349cccad128c67bb83adcbb75603037d67/src/main/cpp/util/logging.h#L112 ------------------------------ My point is to use K2_ prefix is fine but glog should be excluded. Just #define K2_* * just add more confusions. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/danpovey/k2/pull/80#issuecomment-668581277>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO7NSES25E4W5IKUWPLR7AA5BANCNFSM4PTOHZ6A> .

qindazhu · 2020-08-04T13:36:33Z

I'm OK with writing those K2_CHECK by ourselves. The only thing I worry about is its robustness for complex scenarios (e.g. multiple thread，file system support, cache, etc.). But maybe we don't really need to handle those scenarios.
BTW, I like the sytle of log in protobuf, it's so lightweight and easy to understand. But it seems there's no stack trace in it?

danpovey · 2020-08-04T14:11:03Z

Mm, the multi-threaded stuff can be handled, I think, by first formatting to a string and then writing with fwrite() or whatever it is, then it becomes an atomic operation. In CUDA, I think we'd just print to stderr using fprintf directly, assuming you can get fprintf there.

…

On Tue, Aug 4, 2020 at 9:36 PM Haowen Qiu ***@***.***> wrote: I'm OK with writing those K2_CHECK by ourselves. The only thing I worry about is its robustness for complex scenarios (e.g. multiple thread，file system support, cache, etc.). But maybe we don't really need to handle those scenarios. BTW, I like the sytle of log in protobuf, it's so lightweight and easy to understand. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/danpovey/k2/pull/80#issuecomment-668601087>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLOZBQQXPXH43SEQ6EDLR7AFG7ANCNFSM4PTOHZ6A> .

qindazhu · 2020-08-04T14:37:10Z

Mm, the multi-threaded stuff can be handled, I think, by first formatting to a string and then writing with fwrite() or whatever it is, then it becomes an atomic operation. In CUDA, I think we'd just print to stderr using fprintf directly, assuming you can get fprintf there.
…
On Tue, Aug 4, 2020 at 9:36 PM Haowen Qiu @.***> wrote: I'm OK with writing those K2_CHECK by ourselves. The only thing I worry about is its robustness for complex scenarios (e.g. multiple thread，file system support, cache, etc.). But maybe we don't really need to handle those scenarios. BTW, I like the sytle of log in protobuf, it's so lightweight and easy to understand. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#80 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLOZBQQXPXH43SEQ6EDLR7AFG7ANCNFSM4PTOHZ6A .

Yes, so if we decide to write those CHECKs by ourselves (instead of based on glog or other log systems), we need to list what features (multiple-thread, stace trace, etc.) we would like to support first.. That maybe needs some time to investigate.
Maybe it's better if we can start from glog first (as we have added it in our codebase already). I mean, based on glog to write some K2_CHECK, and then we can start to use those macros to do real work. we can just update those K2_CHECK_ macros later if we would like to replace glog with something else, this will not affect those code calling K2_CHECK.

danpovey · 2020-08-04T15:46:32Z

Sounds plausible but let's give Meixu a little time to think about it, it doesn't have to be done today.

…

On Tue, Aug 4, 2020 at 10:37 PM Haowen Qiu ***@***.***> wrote: Mm, the multi-threaded stuff can be handled, I think, by first formatting to a string and then writing with fwrite() or whatever it is, then it becomes an atomic operation. In CUDA, I think we'd just print to stderr using fprintf directly, assuming you can get fprintf there. … <#m_-7677017061685835873_> On Tue, Aug 4, 2020 at 9:36 PM Haowen Qiu *@*.***> wrote: I'm OK with writing those K2_CHECK by ourselves. The only thing I worry about is its robustness for complex scenarios (e.g. multiple thread，file system support, cache, etc.). But maybe we don't really need to handle those scenarios. BTW, I like the sytle of log in protobuf, it's so lightweight and easy to understand. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#80 (comment) <https://github.com/danpovey/k2/pull/80#issuecomment-668601087>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLOZBQQXPXH43SEQ6EDLR7AFG7ANCNFSM4PTOHZ6A . Yes, so if we decide to write those CHECKs by ourselves (instead of based on glog or other log systems), we need to list what features (multiple-thread, stace trace, etc.) we would like to support first.. That maybe needs some time to investigate. Maybe it's better if we can start from glog first (as we have added it in our codebase already). I mean, based on glog to write some K2_CHECK, and then we can start to use those macros to do real work. we can just update those K2_CHECK_ macros later if we would like to replace glog with something else, this will not affect those code calling K2_CHECK. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/danpovey/k2/pull/80#issuecomment-668634927>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO3CBGFSVQFARPBUSULR7AMKNANCNFSM4PTOHZ6A> .

megazone87 added 4 commits August 3, 2020 23:12

add a logging alternative

3839c3a

Merge remote-tracking branch 'Dan/master' into mx

84e93ce

fix style as cpplint advise

b291ed3

continue above commit

154364e

megazone87 mentioned this pull request Aug 3, 2020

Assertions etc. #78

Closed

csukuangfj reviewed Aug 3, 2020

View reviewed changes

qindazhu reviewed Aug 4, 2020

View reviewed changes

megazone87 added 4 commits August 4, 2020 13:04

changes after @csukuangfj review

9ab00fb

cpplint style: change header guard

07a7774

make cxx 11 as required

882f44b

changes after reviews

ead061a

danpovey reviewed Aug 4, 2020

View reviewed changes

csukuangfj reviewed Aug 4, 2020

View reviewed changes

qindazhu reviewed Aug 4, 2020

View reviewed changes

danpovey mentioned this pull request Sep 4, 2020

add glog-like CHECK and LOG for k2. #114

Merged

megazone87 closed this Sep 4, 2020

megazone87 deleted the mx branch September 4, 2020 10:36

		@@ -0,0 +1,57 @@
		// $FILE_PATHNAME $HEADER_FILENAME $HEADER_PATHNAME

		@@ -0,0 +1,32 @@
		//
		// Created by songmeixu ([email protected]) on 2020/8/3.

add a logging alternative #80

add a logging alternative #80

Conversation

megazone87 commented Aug 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

megazone87 Aug 4, 2020 • edited Loading

Choose a reason for hiding this comment

csukuangfj Aug 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danpovey commented Aug 4, 2020

danpovey commented Aug 4, 2020

megazone87 commented Aug 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qindazhu commented Aug 4, 2020

danpovey commented Aug 4, 2020

danpovey commented Aug 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qindazhu commented Aug 4, 2020

danpovey commented Aug 4, 2020

qindazhu commented Aug 4, 2020

danpovey commented Aug 4, 2020

csukuangfj commented Aug 4, 2020

megazone87 commented Aug 4, 2020

danpovey commented Aug 4, 2020 via email

megazone87 commented Aug 4, 2020 • edited Loading

danpovey commented Aug 4, 2020 via email

qindazhu commented Aug 4, 2020 • edited Loading

danpovey commented Aug 4, 2020 via email

qindazhu commented Aug 4, 2020

danpovey commented Aug 4, 2020 via email

megazone87 Aug 4, 2020 •

edited

Loading

csukuangfj Aug 3, 2020 •

edited

Loading

megazone87 commented Aug 4, 2020 •

edited

Loading

qindazhu commented Aug 4, 2020 •

edited

Loading