Default.log doesn't rotate during replay

[abitmore]

Bug Description
Default.log doesn't rotate during replay.

Steps To Reproduce

1. With logging enabled, after synced, stop witness_node and run ./witness_node --replay.
2. Check log files during replay of after finished, will find almost all log messages are written to a big log file.

Expected Behavior
Log rotation works as defined in logging.ini.

Screenshots (optional)

Host Environment
Please provide details about the host environment. Much of this information can be found running: witness_node --version.

• Host OS: Ubuntu 16.04 LTS
• Host Physical RAM 16GB
• BitShares Version: 3.3.1
• OpenSSL Version: 1.0.2
• Boost Version: 1.58

Impacts
Describe which portion(s) of BitShares Core may be impacted by this bug. Please tick at least one box.

• API (the application programming interface)
• Build (the build process or something prior to compiled code)
• CLI (the command line wallet)
• Deployment (the deployment process after building such as Docker, Travis, etc.)
• DEX (the Decentralized EXchange, market engine, etc.)
• P2P (the peer-to-peer network for transaction/block propagation)
• Performance (system or user efficiency, etc.)
• Protocol (the blockchain logic, consensus, validation, etc.)
• Security (the security of system or user data, etc.)
• UX (the User Experience)
• Other (please add below)

CORE TEAM TASK LIST

• Evaluate / Prioritize Bug Report
• Refine User Stories / Requirements
• Define Test Cases
• Design / Develop Solution
• Perform QA/Testing
• Update Documentation

[clockworkgr]

Hunch at first glance. Log rotates by time...not size

[abitmore]

Log rotates by time...not size

True, but the issue is it doesn't rotate. See my screenshot, last write time of the T17 file is 20:58 but last write time of the T00 file is 20:59. Timezone of that machine was -4.

The underlying reason is in task management. Log rotation is implemented with fc::async(), the scheduled rotation tasks won't execute soon enough when the thread is busy running another task. By the way, I think the root cause of #1006 is the same, when a tasking is running (pushing blocks), another task (to push blocks got from one peer) is scheduled and pending, if to schedule yet another task (to push blocks got from another peer) with the same name with fc::async(), the 2nd task will be "rescheduled" (which is fair) without proper cleanup.

[pmconrad]

Waiting for fc bump

[pmconrad]

fc bumped in #1992