Allow users to manually set the number of block cache shards #1336
Comments
mdcallag
added a commit
to mdcallag/mysql-5.6
that referenced
this issue
Jul 19, 2023
This fixes facebook#1336 The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case.
facebook-github-bot
pushed a commit
that referenced
this issue
Jul 20, 2023
Summary: This fixes #1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: #1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
|
Fixed |
luqun
pushed a commit
to luqun/mysql-5.6
that referenced
this issue
Aug 10, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this issue
Oct 3, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this issue
Oct 18, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Dec 20, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Dec 20, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Dec 21, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to percona/percona-server
that referenced
this issue
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Feb 8, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Feb 9, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Feb 9, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Feb 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this issue
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jun 21, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jul 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Jul 25, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Jul 30, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jul 31, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 6, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Aug 21, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Aug 28, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Aug 30, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to percona/percona-server
that referenced
this issue
Sep 23, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to percona/percona-server
that referenced
this issue
Sep 25, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Sep 25, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Oct 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Oct 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this issue
Oct 22, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Oct 28, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to percona/percona-server
that referenced
this issue
Oct 30, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Nov 11, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Nov 14, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Nov 14, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 23, 2025
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 23, 2025
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
lukin-oleksiy
pushed a commit
to lukin-oleksiy/percona-server
that referenced
this issue
Jan 27, 2025
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this issue
Jan 27, 2025
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
lukin-oleksiy
pushed a commit
to lukin-oleksiy/percona-server
that referenced
this issue
Jan 29, 2025
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
While RocksDB allows for num_shard_bits to be set for the block cache, MyRocks does not expose this.
This is a feature request for MyRocks to expose an option that allows me to set it.
There can be serious perf problems when the block cache is too small. With intra-L0 compaction there can be large SSTs in the L0, large SSTs imply large filter blocks and reading a large object into a small cache shard can evict everything from that shard.
A blog post on this is here.
The text was updated successfully, but these errors were encountered: