feat:(logging) Added Log Format provision

[damanV5]

Added log_format feature for logger.

[damanV5]

@zachmu

[timsehn]

So, all you've done is make the configuration valid in the configuration YAML file.

Now, you have to plumb it through to change the actual logging behavior.

And finally, you should write some tests to make sure it actually logs in JSON if you have the setting enabled.

[damanV5]

@timsehn Can you take a look whether I am heading in the right direction?

[timsehn]

You are heading in the right direction. It looks like you have most of the places to actually get the value to the server. Now the server needs to read it when it wants to emit a log line and emit the right format.

@zachmu suggested that logic is here:

https://github.com/dolthub/dolt/blob/main/go/cmd/dolt/commands/sqlserver/server.go#L117

Looking at logrus (https://github.com/sirupsen/logrus), it seems they have JSON formatting capability. IN their README, they suggest

log.SetFormatter(&log.JSONFormatter{})

[damanV5]

@timsehn I have configured the logger for log_format but i am not sure how can I test the same with a logger.
So far, I am able to write a test-case to verify DefaultLogFormat.

Update:-
I have built the go folder locally and tested with log_format: json in the yaml. Following is the output:-

[timsehn]

That looks like it is working. I'm not sure the best way to write tests for this. @zachmu should answer.

[zachmu]

Hi @damanV5,

This looks really good. You should add a test for the new option in the bats file here, similar to this one:

https://github.com/dolthub/dolt/blob/main/integration-tests/bats/sql-server.bats#L123

You can check the output format is correct by redirecting the log to a file, like this test does:

https://github.com/dolthub/dolt/blob/main/integration-tests/bats/sql-server.bats#L1531

[lucasfcnunes]

@timsehn I have configured the logger for log_format but i am not sure how can I test the same with a logger.
So far, I am able to write a test-case to verify DefaultLogFormat.

Update:-
I have built the go folder locally and tested with log_format: json in the yaml. Following is the output:-

@timsehn @damanV5

Can this very first line too be in json?

[damanV5]

Need to figure something out here,

The following test is working fine for insensitivity when checked from command line but from config.yaml it is behaving sensitive. So, that I need to figure out.

Test:-

@test "sql-server: logformats are case insensitive" {
    # assert that logformat on command line is not case sensitive
    cd repo1
    PORT=$( definePORT )
    dolt sql-server --logformat jSon --port=$PORT --socket "dolt.$PORT.sock" > log.txt 2>&1 &
    SERVER_PID=$!
   wait_for_connection $PORT 8500 || { 
    echo "Server failed to start, dumping logs:"
    cat log.txt
    exit 1
}
    dolt sql -q "show databases;"
    stop_sql_server

    # assert that logformat in yaml config is not case sensitive
    cat >config.yml <<EOF
log_format: teXt
behavior:
  disable_client_multi_statements: true
listener:
  host: "0.0.0.0"
  port: $PORT
EOF
    dolt sql-server --config ./config.yml --socket "dolt.$PORT.sock" &
    SERVER_PID=$!
    wait_for_connection $PORT 8500
    dolt sql -q "show databases;"
    stop_sql_server
}

[damanV5]

@zachmu Update: I managed to add a test case for log_format.

[timsehn]

Running tests now.

[timsehn]

You need a test here to assert that the log outputted in json format.

[damanV5]

@timsehn Added.

[timsehn]

Also, you must fix the Go tests this broke.

[timsehn]

It's odd that the Go tests passed with No race but failed with race on.

[damanV5]

@timsehn I have fixed the go tests for sqlserver.

Do i also need to update minver_validation.txt with LogFormatStr *string 0.0.0 log_format,omitempty as the header says "file automatically updated by the release process". ?