All memcached tests fail with tarantool 2.10.0-beta1-376-gd2a012455 (master branch) #96
Assignees
Labels
bug
Something isn't working
Comments
Tarantool 2.10.0-beta1-376-gd2a012455 |
|
Looks related to #59, we had libsmall update in tarantool recently: tarantool/tarantool#6678. However after tarantool/tarantool#5932 it should not hit us. At least, not due to symbols clash. I grepped $ ag cord_slab_cache
memcached/internal/network.c
34: mempool_create(&ibuf_pool, cord_slab_cache(), sizeof(struct ibuf));
35: mempool_create(&obuf_pool, cord_slab_cache(), sizeof(struct obuf));
51: ibuf_create((struct ibuf *)ibuf, cord_slab_cache(), iobuf_readahead);
60: obuf_create((struct obuf *)obuf, cord_slab_cache(), iobuf_readahead);
memcached/internal/memcached.c
250: region_create(&con.gc, cord_slab_cache());
memcached/internal/alloc.c
9: struct slab *slab = slab_get(cord_slab_cache(), len);
31: if (ptr) slab_put(cord_slab_cache(), slab_from_data(ptr));At least we try to use module's |
ylobankov
added a commit
that referenced
this issue
Dec 23, 2021
This patch skips tests run for tarantool 2.10 in reusable_testing.yml workflow due to the bug [1]. After [1] is fixed, the skip condition should be removed from the workflow. [1] #96
ylobankov
added a commit
that referenced
this issue
Dec 24, 2021
This patch skips tests run for tarantool 2.10 in reusable_testing.yml workflow due to the bug [1]. After [1] is fixed, the skip condition should be removed from the workflow. [1] #96
ylobankov
added a commit
that referenced
this issue
Dec 27, 2021
This patch skips tests run for tarantool 2.10 in reusable_testing.yml workflow due to the bug [1]. After [1] is fixed, the skip condition should be removed from the workflow. [1] #96
ylobankov
added a commit
that referenced
this issue
Dec 28, 2021
This patch skips tests run for tarantool 2.10 in reusable_testing.yml workflow due to the bug [1]. After [1] is fixed, the skip condition should be removed from the workflow. [1] #96
ligurio
added a commit
that referenced
this issue
Feb 10, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Feb 10, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new method slab() was introduced. Output of slab() is
similar to output from `box.slab.info()`, but without items_size,
items_used, items_used_ratio and arena_used is not implemented.
tarantool> require('memcached'):slab()
---
- info:
quota_used: 4194304
arena_used_ratio: 0
quota_used_ratio: 0.097656249999998
arena_size: 4194304
quota_size: 4294967296
arena_used: 0
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Feb 10, 2022
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new method slab() was introduced. Output of slab() is
similar to output from `box.slab.info()`, but without items_size,
items_used, items_used_ratio and arena_used is not implemented.
tarantool> require('memcached'):slab()
---
- info:
quota_used: 4194304
arena_used_ratio: 0
quota_used_ratio: 0.097656249999998
arena_size: 4194304
quota_size: 4294967296
arena_used: 0
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new method slab() was introduced. Output of slab() is
similar to output from `box.slab.info()`, but without items_size,
items_used, items_used_ratio and arena_used is not implemented.
tarantool> require('memcached'):slab()
---
- info:
quota_used: 4194304
arena_used_ratio: 0
quota_used_ratio: 0.097656249999998
arena_size: 4194304
quota_size: 4294967296
arena_used: 0
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new method slab() was introduced. Output of slab() is
similar to output from `box.slab.info()`, but without items_size,
items_used, items_used_ratio and arena_used is not implemented.
tarantool> require('memcached'):slab()
---
- info:
quota_used: 4194304
arena_used_ratio: 0
quota_used_ratio: 0.097656249999998
arena_size: 4194304
quota_size: 4294967296
arena_used: 0
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new method slab() was introduced. Output of slab() is
similar to output from `box.slab.info()`, but without items_size,
items_used, items_used_ratio and arena_used is not implemented.
tarantool> require('memcached'):slab()
---
- info:
quota_used: 4194304
arena_used_ratio: 0
quota_used_ratio: 0.097656249999998
arena_size: 4194304
quota_size: 4294967296
arena_used: 0
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Feb 11, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Name of internal function for creating a new memcached instance is memcached_init(). It may be confusing - name can be mixed up with name of function for module initialization. In the following commit we will need an explicit function for module initialization, memcached_init() was renamed to memcached_create_instance() for clarity. This commit changes internal function's name, no impact on public API. Needed for #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. In tests function is_port_open() has been replaced to memcached minimalistic client that do set value, get value with the same key and compare both values. This check is more rigorous than check port openess. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new module function memcached.slab.info() was
introduced. Output of memcached.slab.info() is similar to output
produced by `box.slab.info()`, but without items_size, items_used,
items_used_ratio and arena_used.
tarantool> mc = require('memcached')
creating arena with 4294967295 bytes...
allocate slab cache with slab size 4194304
---
...
tarantool> mc.slab.info()
---
- quota_used: 0
quota_size: 4294967296
quota_used_ratio: 0%
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Name of internal function for creating a new memcached instance is memcached_init(). It may be confusing - name can be mixed up with name of function for module initialization. In the following commit we will need an explicit function for module initialization, memcached_init() was renamed to memcached_create_instance() for clarity. This commit changes internal function's name, no impact on public API. Needed for #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. In tests function is_port_open() has been replaced to memcached minimalistic client that do set value, get value with the same key and compare both values. This check is more rigorous than check port openess. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new module function memcached.slab.info() was
introduced. Output of memcached.slab.info() is similar to output
produced by `box.slab.info()`, but without items_size, items_used,
items_used_ratio and arena_used.
tarantool> mc = require('memcached')
creating arena with 4294967295 bytes...
allocate slab cache with slab size 4194304
---
...
tarantool> mc.slab.info()
---
- quota_used: 0
quota_size: 4294967296
quota_used_ratio: 0%
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Name of internal function for creating a new memcached instance is memcached_init(). It may be confusing - name can be mixed up with name of function for module initialization. In the following commit we will need an explicit function for module initialization, memcached_init() was renamed to memcached_create_instance() for clarity. This commit changes internal function's name, no impact on public API. Needed for #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Historically memcached module uses slab cache created by Tarantool. Fibers get access to it via call of function cord_slab_cache() that is a part of Tarantool C API. The problem is happen when memcached and Tarantool use different versions of Small library where ABI is broken. Issues #59 and #96 are examples of such problems. We decided use independent slab arena in memcached module to avoid such problems in future and this patch implements it. Below I'll describe what is important in switching to own arena. memcached module uses different types of Small's allocators (ibuf and obuf used for every network connection, one slab cache used per fiber). There are two important things with used Small's allocators: 1. some allocators depends on other and sequence of allocations should be correct. README in small source code repository has a clear description of relationships between allocators [1]. 2. some allocators are thread-safe and others not. slab arena is shared between all memcached instances, when slab cache allocated on top of slab arena is not thread-safe and must be used in scope of fiber. memcached performs memory allocations and deallocations on different stages and I would list all these stages explicitly: - memcached module initialization - memcached module deinitialization - memcached instance gc - memcached instance initialization - memcached instance start - memcached instance stop - memcached instance deinitialization Most part of these cases covered in tests test/common/instance_api.test.lua and test/common/module_api.test.lua that were added in previous commits. The exception here is gc that was checked manually. In tests function is_port_open() has been replaced to memcached minimalistic client that do set value, get value with the same key and compare both values. This check is more rigorous than check port openess. 1. https://github.com/tarantool/small Fixes #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Before switching to our own slab arena user can control usage and
information about slab arena using Tarantool Lua API (specifically
box.slab.{check,info,stat}() functions), see [1]. Example of output
box.slab.info() is below:
tarantool> box.slab.info()
---
- items_size: 146896
items_used_ratio: 26.05%
quota_size: 268435456
quota_used_ratio: 12.50%
arena_used_ratio: 3.7%
items_used: 38264
quota_used: 33554432
arena_size: 33554432
arena_used: 1234296
...
With our own slab arena we need a way to obtain information about arena.
It's a reason why new module function memcached.slab.info() was
introduced. Output of memcached.slab.info() is similar to output
produced by `box.slab.info()`, but without items_size, items_used,
items_used_ratio and arena_used.
tarantool> mc = require('memcached')
creating arena with 4294967295 bytes...
allocate slab cache with slab size 4194304
---
...
tarantool> mc.slab.info()
---
- quota_used: 0
quota_size: 4294967296
quota_used_ratio: 0%
...
1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/slab_info/
Follows up #96
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
ligurio
added a commit
that referenced
this issue
Apr 6, 2022
Merged
CuriousGeorgiy
added a commit
to CuriousGeorgiy/tarantool
that referenced
this issue
Jun 22, 2023
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in tarantool#7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes tarantool#7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
CuriousGeorgiy
added a commit
to CuriousGeorgiy/tarantool
that referenced
this issue
Jun 22, 2023
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in tarantool#7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes tarantool#7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
CuriousGeorgiy
added a commit
to CuriousGeorgiy/tarantool
that referenced
this issue
Jun 22, 2023
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in tarantool#7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes tarantool#7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
CuriousGeorgiy
added a commit
to CuriousGeorgiy/tarantool
that referenced
this issue
Jun 23, 2023
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in tarantool#7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes tarantool#7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
locker
pushed a commit
to tarantool/tarantool
that referenced
this issue
Jun 23, 2023
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in #7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes #7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Host:
Ubuntu 20.04Tarantool:
Try to run tests:
However, tests work fine with
tarantool 2.8.2-80-g69fc3de62.The text was updated successfully, but these errors were encountered: