Audit config section, populate advanced-config section

config/_default/menus/menus.en.toml

@@ -79,10 +79,10 @@
   url = "/storage-providers/get-started"
 
 [[storage-providers]]
-  name = "Configure"
+  name = "Setup"
   weight = 25
-  identifier = "storage-providers-configure"
-  url = "/storage-providers/configure"
+  identifier = "storage-providers-setup"
+  url = "/storage-providers/setup"
 
 [[storage-providers]]
   name = "Operate"
@@ -114,12 +114,6 @@
   identifier = "tutorials-providers"
   url = "/tutorials/lotus-miner"
 
-#[[tutorials]]
-#  name = "Store and retrieve"
-#  weight = 30
-#  identifier = "tutorials-store-and-retrieve"
-#  url = "/tutorials/store-and-retrieve"
-
 [[social]]
   name = "GitHub"
   pre = "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"20\" height=\"20\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"feather feather-github\"><path d=\"M9 19c-5 1.5-5-2.5-7-3m14 6v-3.87a3.37 3.37 0 0 0-.94-2.61c3.14-.35 6.44-1.54 6.44-7A5.44 5.44 0 0 0 20 4.77 5.07 5.07 0 0 0 19.91 1S18.73.65 16 2.48a13.38 13.38 0 0 0-7 0C6.27.65 5.09 1 5.09 1A5.07 5.07 0 0 0 5 4.77a5.44 5.44 0 0 0-1.5 3.78c0 5.42 3.3 6.61 6.44 7A3.37 3.37 0 0 0 9 18.13V22\"></path></svg>"

content/en/kb/ImportData/index.md

@@ -13,7 +13,7 @@ types: ["solution"]
 areas: ["function", "bidbot", "Lotus Node", "Markets Node"]
 ---
 
-# Problem:
+## Problem:
 
 Where does the `ImportData` function get executed? 
 
@@ -23,13 +23,13 @@ Where does the `ImportData` function get executed?
 
 The question is: How do you organize your API access/where bidbot runs? Does it expect the above to happen on the market node, or do you actual give bidbot miner-node credentials? Something else?
 
-# Environment:
+## Environment:
 
 - Mainnet 
 - Calibnet 
 - Split markets node
 
-# Resolution:
+## Resolution:
 
 The `ImportData` function is always called/processed on the market node, regardless of any split configurations between a market node and the main miner-node. 
 

content/en/kb/add-swap/index.md

@@ -0,0 +1,32 @@
+---
+title: "Add swap on Linux"
+description: "This is a guide for adding swap to a linux machine."
+date: 2022-03-17T12:00:35+01:00
+lastmod: 2021-11-16T12:00:35+01:00
+draft: false
+menu:
+  kb:
+    parent: "browse"
+toc: false
+pinned: false
+types: ["article"]
+areas: ["swap", "Linux"]
+---
+
+## Add swap on lotus miner or worker node:
+
+If you have only 128 GiB of RAM, you will need to make sure your system provides at least an extra 256 GiB of very fast swap (preferably NVMe SSD) or you will be unable to seal sectors:
+
+```sh
+sudo fallocate -l 256G /swapfile
+sudo chmod 600 /swapfile
+sudo mkswap /swapfile
+sudo swapon /swapfile
+# show current swap spaces and take note of the current highest priority
+swapon --show
+# append the following line to /etc/fstab (ensure highest priority) and then reboot
+# /swapfile swap swap pri=50 0 0
+sudo reboot
+# check a 256GB swap file is automatically mounted and has the highest priority
+swapon --show
+```

content/en/kb/failed-to-create-datastore/index.md

@@ -13,6 +13,7 @@ types: ["solution"]
 areas: ["sealing", "Lotus Worker", "Lotus Miner"]
 ---
 
+## Problem
 The following error shows up in the worker's log.
 
 ```json

content/en/kb/lotus-shed-not-installed/index.md

@@ -1,6 +1,6 @@
 ---
-title: "lotus-shed is not installed by default"
-description: "This is a solution for lotus-shed."
+title: "How to install lotus-shed"
+description: "This is a guide for installing lotus-shed."
 date: 2022-03-17T12:00:35+01:00
 lastmod: 2021-11-16T12:00:35+01:00
 draft: false
@@ -9,19 +9,19 @@ menu:
     parent: "browse"
 toc: false
 pinned: false
-types: ["solution"]
+types: ["article"]
 areas: ["lotus-shed", "Linux", "Lotus Node"]
 ---
 
-# Problem:
+## Problem:
 
 When installing lotus for mainnet, lotus-shed is not installed by default. 
 
-# Environment:
+## Environment:
 
 - Mainnet
 
-# Resolution:
+## Resolution:
 
 After lotus installs (completely) run `make install lotus-shed`
 

content/en/kb/soft-fd-limit/index.md

@@ -13,21 +13,21 @@ types: ["solution"]
 areas: ["OS", "Linux", "Lotus Node"]
 ---
 
-# Problem:
+## Problem:
 
 An alert, similar to the following, shows up in lotus or lotus-miner.
 
 `active process:fd-limit last raised at 2021-12-09 16:30:39.901 +0800 +08; reason: {"message":"soft FD limit is low","recommended_min":100000,"soft_limit":16384}`
 
 Note that this error is an example from Ubuntu, but would be similar in other OS variants. 
 
-# Environment:
+## Environment:
 
 * Mainnet
 * Calibnet
 * Devnet
 
-# Resolution:
+## Resolution:
 
 This will be host OS specific, but it should be similar on all OS types. Below is the example from Ubuntu. You will also have to reboot.
 ```

content/en/lotus/manage/troubleshooting.md

@@ -82,7 +82,7 @@ If you get a `signal killed` error, it could indcate that there was an error dur
 make: *** [Makefile:68: lotus] Error 1
 ```
 
-Double check that your computer meets the [minimum hardware requirements]({{< relref "prerequisites#minimal-requirements" >}}) for Lotus.
+Double check that your computer meets the [minimum hardware requirements]({{< relref "../../lotus/install/prerequisites#minimal-requirements" >}}) for Lotus.
 
 ## Go command not found
 

content/en/storage-providers/advanced-configurations/logging.md

@@ -0,0 +1,79 @@
+---
+title: "Logging"
+description: "This guide covers the Lotus Miner log configuration and logging usage."
+lead: "This guide describes how to capture logs to a file and search through the file for important events."
+draft: false
+menu:
+    storage-providers:
+        parent: "storage-providers-advanced-configurations"
+weight: 160
+toc: true
+---
+
+## Log file configuration
+
+The Lotus miner process generates two types of logs, namely, Go logs and Rust logs. Each of these logs can redirecting to individual files.
+
+{{< alert icon="callout" >}}
+This section is not applicable if you are running lotus-miner as a systemd service.
+{{< /alert >}}
+
+### Redirect GO logs to a file
+
+By default, lotus-miner redirect all logs to the standard output if not running as a systemd service. To change this behavior, add the following variable to the `.bashrc` file and restart the Lotus miner process to start redirecting all logs to the file.
+
+```shell
+export GOLOG_OUTPUT=FILE >> ~/.bashrc
+export GOLOG_FILE="$HOME/miner.log" >> ~/.bashrc && source ~/.bashrc
+```
+
+
+### Redirect Rust logs to a file
+By default the `fil_logger` library used by `rust-fil-proof` doesn't log anything. You can change this by setting the RUST_LOG environment variable to another level. This will show log output on stderr or in the file if Go logging is configured to a file.
+
+```shell
+export RUST_LOG=info >> ~/.bashrc && source ~/.bashrc
+```
+
+The log-level can be chose between 5 options
+- trace
+- debug
+- info
+- warn
+- error
+
+## Change logging verbosity
+
+The verbosity of the `lotus` and `lotus-miner` logs can be changed without restarting the lotus. The following command can be used to list different subsystems within the `lotus-miner` process and change the verbosity of individual subsystem to get more/less detailed logs.
+
+```shell
+lotus-miner log list
+```
+
+To change the verbosity, please run
+
+```shell
+lotus-miner log set-level --system chain debug
+```
+
+The log-level can be chose between 4 options
+- debug
+- info
+- warn
+- error
+
+You can specify multiple subsystems to change the log level of multiple subsystems at once.
+
+```shell
+lotus-miner log set-level --system chain --system chainxchg debug
+```
+
+## Searching through logs
+
+1. To look for logs related to mining a block, you can check with following commands.
+    ```shell
+    cat /path/to/log | grep "mined"
+
+2. To look for logs related to winning a block, you can check with following commands.
+    ```shell
+    cat /path/to/log | grep "isEligible"

content/en/storage-providers/advanced-configurations/market.md

@@ -0,0 +1,115 @@
+---
+title: "Market"
+description: "Lotus has a lot of advanced configurations you can tune to optimize your storage provider setup. This guide explains the advanced configuration options for lotus market"
+lead: "Lotus has a lot of advanced configurations you can tune to optimize your storage provider setup. This guide explains the advanced configuration options for lotus market"
+draft: false
+menu:
+    storage-providers:
+        parent: "storage-providers-advanced-configurations"
+weight: 120
+toc: true
+---
+
+## Dealmaking section
+
+This section controls parameters for making storage and retrieval deals:
+
+```toml
+[Dealmaking]
+  # When enabled, the miner can accept online deals
+  ConsiderOnlineStorageDeals = true
+  # When enabled, the miner can accept offline deals
+  ConsiderOfflineStorageDeals = true
+  # When enabled, the miner can accept retrieval deals
+  ConsiderOnlineRetrievalDeals = true
+  # When enabled, the miner can accept offline retrieval deals
+  ConsiderOfflineRetrievalDeals = true
+  # When enabled, the miner can accept verified deals
+  ConsiderVerifiedStorageDeals = true
+  # When enabled, the miner can accept unverified deals
+  ConsiderUnverifiedStorageDeals = true
+  # A list made of Data CIDs to reject when making deals
+  PieceCidBlocklist = []
+  # Maximum expected amount of time getting the deal into a sealed sector will take
+  # This includes the time the deal will need to get transferred and published
+  # before being assigned to a sector
+  # for more info, see below.
+  ExpectedSealDuration = "24h0m0s"
+  # When a deal is ready to publish, the amount of time to wait for more
+  # deals to be ready to publish before publishing them all as a batch
+  PublishMsgPeriod = "1h0m0s"
+  # The maximum number of deals to include in a single publish deals message
+  MaxDealsPerPublishMsg = 8
+
+  # A command used for fine-grained evaluation of storage deals (see below)
+  Filter = "/absolute/path/to/storage_filter_program"
+
+  # A command used for fine-grained evaluation of retrieval deals (see below)
+  RetrievalFilter = "/absolute/path/to/retrieval_filter_program"
+```
+
+`ExpectedSealDuration` is an estimate of how long sealing will take and is used to reject deals whose start epoch might be earlier than the expected completion of sealing. It can be estimated by [benchmarking]({{< relref "benchmarks" >}}) or by [pledging a sector]({{< relref "sector-pledging" >}}).
+
+{{< alert icon="warning" >}}
+The final value of `ExpectedSealDuration` should equal `(TIME_TO_SEAL_A_SECTOR + WaitDealsDelay) * 1.5`. This equation ensures that the miner does not commit to having the sector sealed too soon.
+{{< /alert >}}
+
+`StartEpochSealingBuffer` allows `lotus-miner` to seal a sector before a certain epoch. For example: if the current epoch is 1000 and a deal within a sector must start on epoch 1500, then `lotus-miner` must wait until the current epoch is 1500 before it can start sealing that sector. However, if `lotus-miner` sets `StartEpochSealingBuffer` to 500, the `lotus-miner` can start sealing the sector at epoch 1000. 
+
+If there are multiple deals in a sector, the deal with a start time closest to the current epoch is what `StartEpochSealingBuffer` will be based off. So, if the sector in our example has three deals that start on epoch 1000, 1200, and 1400, then we `lotus-miner` will start sealing the sector at epoch 500.
+
+### Disabling new sector for deal
+
+If `MakeNewSectorForDeals` is set to `true` then `lotus-miner` will create new sectors for incoming deals. This option can set to `false` to ensure that all new deals are sealed as snap-deals into CC sectors. This can help reduce the sealing time for the new deals as long as CC sectors are ready for the snap-deals.
+
+### Make new CC sector available for snap-deal
+
+`MakeCCSectorsAvailable` makes all the new CC sectors available to be upgraded with snap-deals. When this boolean is set to `true`, all pledged "CC" sectors from that point onwards will be converted to "Available" state after sealing. This enables sealing the incoming storage deals more quickly into these "Available" sectors compared to creating a new sector for the deals. 
+
+### Publishing several deals in one message
+
+The `PublishStorageDeals` message can publish multiple deals in a single message.
+When a deal is ready to be published, Lotus will wait up to `PublishMsgPeriod`
+for other deals to be ready before sending the `PublishStorageDeals` message.
+
+However, once `MaxDealsPerPublishMsg` is ready, Lotus will immediately publish all the deals.
+
+For example, if `PublishMsgPeriod` is 1 hour:
+
+- At 1:00 pm, deal 1 is ready to publish. Lotus will wait until 2:00 pm for other deals to be ready before sending `PublishStorageDeals`.
+- At 1:30 pm, Deal 2 is ready to publish
+- At 1:45 pm, Deal 3 is ready to publish
+- At 2:00pm, lotus publishes Deals 1, 2, and 3 in a single `PublishStorageDeals` message.
+
+If `MaxDealsPerPublishMsg` is 2, then in the above example, when deal 2 is ready to be published at 1:30, Lotus would immediately publish Deals 1 & 2 in a single `PublishStorageDeals` message. Deal 3 would be published in a subsequent `PublishStorageDeals` message.
+
+> If any of the deals in the `PublishStorageDeals` fails validation upon execution, or if the start epoch has passed, all deals will fail to be published.
+
+## Using filters for fine-grained storage and retrieval deal acceptance
+
+Your use case might demand very precise and dynamic control over a combination of deal parameters.
+
+Lotus provides two IPC hooks allowing you to name a command to execute for every deal before the miner accepts it:
+
+- `Filter` for storage deals.
+- `RetrievalFilter` for retrieval deals.
+
+The executed command receives a JSON representation of the deal parameters on standard input, and upon completion, its exit code is interpreted as:
+
+- `0`: success, proceed with the deal.
+- `non-0`: failure, reject the deal.
+
+The most trivial filter rejecting any retrieval deal would be something like:
+`RetrievalFilter = "/bin/false"`. `/bin/false` is binary that immediately exits with a code of `1`.
+
+[This Perl script](https://gist.github.com/ribasushi/53b7383aeb6e6f9b030210f4d64351d5/9bd6e898f94d20b50e7c7586dc8b8f3a45dab07c#file-dealfilter-pl) lets the miner deny specific clients and only accept deals that are set to start relatively soon.
+
+You can also use a third party content policy framework like `bitscreen` by Murmuration Labs:
+
+```shell
+# grab filter program
+go get -u -v github.com/Murmuration-Labs/bitscreen
+
+# add it to both filters
+Filter = "/path/to/go/bin/bitscreen"
+RetrievalFilter = "/path/to/go/bin/bitscreen"