Behavior Test Framework for Bindings

[Xuanwo]

OpenDAL's behavior test greatly enhances the stability and reliability of the openal rust core. I used to think bindings can use rust core directly and don't need to perform their own test frameworks. However, some users blocked by binding issues, for example: #3037

So I think it's better to build a behavior test framework for all bindings instead. This test framework should reuse all our existing setup and workflows as much as possible. And our binding should only need to implement test case once and can run upon different services.

Design

• Binding read .env like core does to init different services.
• Binding implement behavior test like core does, for example, write a file and read it.

Tasks

• Move all fixtures from core/src/services/{service} to top-level fixtures/{service}
• Java
  • Implement functions to read .env (and env) to init operator
  • Implement test_write / test_read
  • Add java test in service_test_s3.yml
  • Migrate redis test to service_test_redis.yml
• Python
  • Implement functions to read .env (and env) to init operator
  • Implement test_write / test_read
  • Add python test in service_test_s3.yml
• Node.js
  • Implement functions to read .env (and env) to init operator
  • Implement test_write / test_read
  • Add node.js test in service_test_s3.yml
• Remove Cucumber based BDD Test

Other bindings can follow those work based on their own progress.

Notes

This issue is solely for bootstrapping the test framework. We will create separate tracking issues to address different bindings in the future.

[Xuanwo]

cc @tisonkun @messense @suyanhanx for comments.

[laipz8200]

I'd like to try the Python part and I found that already has a behavior test based on memory, should I update it to the s3 service?

[suyanhanx]

I'd like to try the Python part and I found that already has a behavior test based on memory, should I update it to the s3 service?

The behavioral tests mentioned in this issue are not quite the same as the parts we did before. The new tests will be a bit closer to real-world usage.
The test in Java may give you some help as an example.

[Xuanwo]

I'd like to try the Python part and I found that already has a behavior test based on memory, should I update it to the s3 service?

The old behavior test will be replaced with the new test framework mentioned here. In the new test framework, we will:

• reading .env files to bootstrap different test services.
• support running tests based on service and test names.
• implement tests like what we do in rust core and java binding.

[eryue0220]

I'd like to solve the Node.js part. Can I have a try?

[suyanhanx]

I'd like to solve the Node.js part. Can I have a try?

Welcome!

[Xuanwo]

Most work has been done. Let's close.