Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dubbo-python] Service Registration Implementation and Documentation Construction #36

Merged
merged 39 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
cef3abc
feat: do something about service reference
cnzakii May 27, 2024
1e36ffd
fix: fix ci
cnzakii May 27, 2024
4db92d7
feat: define UnaryUnaryMultiCallable
cnzakii May 27, 2024
81e22e9
feat: update applicationConfig
cnzakii May 30, 2024
1b38707
feat: do some work related to service reference
cnzakii Jun 2, 2024
2f48f0e
feat: add ci
cnzakii Jun 2, 2024
ca61726
fix: fix ci
cnzakii Jun 2, 2024
31172fc
fix: fix ci
cnzakii Jun 2, 2024
7f3ee01
feat: add logger feat
cnzakii Jun 2, 2024
9bc8bdd
fix: fix ci
cnzakii Jun 2, 2024
d66a40e
perf: Extension Manager becomes a singleton
cnzakii Jun 2, 2024
8ad1133
fix: fix ci
cnzakii Jun 2, 2024
351c0a2
refactor: Make the code more standardized and robust
cnzakii Jun 13, 2024
8f0a155
feat: add logger extension
cnzakii Jun 13, 2024
2def13b
feat: add url
cnzakii Jun 14, 2024
4bb4f8a
fix: fix ci
cnzakii Jun 14, 2024
81a06e6
feat: finish logger part
cnzakii Jun 15, 2024
89ae477
perf: update something about logger
cnzakii Jun 16, 2024
1e73977
style: Becoming more regulated
cnzakii Jun 17, 2024
9206c5a
fix: fix ci
cnzakii Jun 17, 2024
345eafe
fix: fix ci
cnzakii Jun 17, 2024
05ec4db
feat: update something about client
cnzakii Jun 29, 2024
2fb6d89
fix: fix ci
cnzakii Jun 29, 2024
c4f8d52
fix: Delete some invalid files
cnzakii Jun 29, 2024
952541d
feat: Complete the network transmission part
cnzakii Jul 1, 2024
dd83710
perf: Optimization of the network transmission part
cnzakii Jul 5, 2024
cd3c39e
feat: Complete the client's call link
cnzakii Jul 8, 2024
7355cd8
feat: Complete the basic functions of the client
cnzakii Jul 14, 2024
7608afe
feat: Refactored and refined rpc calling capabilities
cnzakii Aug 4, 2024
d17a8ff
docs: Comment completely using reStructuredText style
cnzakii Aug 4, 2024
5fce7fe
feat: Completion of the service registration function
cnzakii Aug 14, 2024
11b723e
fix:fix ci
cnzakii Aug 14, 2024
7d599a6
docs: update README.md
cnzakii Aug 20, 2024
242b51c
docs: update README.md
cnzakii Aug 20, 2024
17135ef
feat: Implement service discovery and registration functionality, and…
cnzakii Aug 20, 2024
8c92912
feat: Adding Samples and Documentation
cnzakii Aug 21, 2024
20e846b
fix: fix ci
cnzakii Aug 21, 2024
070c1c2
docs: update some documents
cnzakii Aug 21, 2024
8516358
Merge remote-tracking branch 'origin/main'
cnzakii Aug 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/python-lint-and-license-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
run: |
# fail if there are any flake8 errors
pip install flake8
flake8 .
flake8 ./dubbo

check-license:
runs-on: ubuntu-latest
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up Python 3.10
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: '3.11'

- name: Install dependencies
run: |
Expand Down
2 changes: 2 additions & 0 deletions .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,15 @@ header: # `header` section is configurations for source codes license header.

paths-ignore: # `paths-ignore` are the path list that will be ignored by license-eye.
- '**/*.md'
- '**/*.proto'
- 'LICENSE'
- 'NOTICE'
- '.asf.yaml'
- '.gitignore'
- '.github'
- '.flake8'
- 'requirements.txt'
- 'samples/**'
comment: on-failure # on what condition license-eye will comment on the pull request, `on-failure`, `always`, `never`.

# license-location-threshold specifies the index threshold where the license header can be located,
Expand Down
159 changes: 107 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,107 @@
## Python Client For Apache Dubbo
## Achieve load balancing on the client side、auto discovery service function with Zookeeper
### Python calls the Dubbo interface's jsonrpc protocol
Please use dubbo-rpc-jsonrpc and configure protocol in Dubbo for jsonrpc protocol
*Reference* [https://github.com/apache/incubator-dubbo-rpc-jsonrpc](https://github.com/apache/incubator-dubbo-rpc-jsonrpc)

### Installation

Download code
python setup.py install
pip install
pip install dubbo-client==1.0.0b5
Git install
pip install git+[http://git.dev.qianmi.com/tda/[email protected]](http://git.dev.qianmi.com/tda/[email protected])
or
pip install git+[https://github.com/qianmiopen/[email protected]](https://github.com/qianmiopen/[email protected])

### Load balancing on the client side, service discovery

Get the registration information of the service through the zookeeper of the registry.
Dubbo-client-py supports configuring multiple zookeeper service addresses.
"host":"192.168.1.183:2181,192.168.1.184:2181,192.168.1.185:2181"
Then the load balancing algorithm is implemented by proxy, and the server is called.
Support Version and Group settings.
### Example
config = ApplicationConfig('test_rpclib')
service_interface = 'com.ofpay.demo.api.UserProvider'
#Contains a connection to zookeeper, which needs caching.
registry = ZookeeperRegistry('192.168.59.103:2181', config)
user_provider = DubboClient(service_interface, registry, version='1.0')
for i in range(1000):
try:
print user_provider.getUser('A003')
print user_provider.queryUser(
{u'age': 18, u'time': 1428463514153, u'sex': u'MAN', u'id': u'A003', u'name': u'zhangsan'})
print user_provider.queryAll()
print user_provider.isLimit('MAN', 'Joe')
print user_provider('getUser', 'A005')

except DubboClientError, client_error:
print client_error
time.sleep(5)

### TODO
Optimize performance, minimize the impact of service upper and lower lines.
Support Retry parameters
Support weight call
Unit test coverage
### Licenses
Apache License
### Thanks
Thank @jingpeicomp for being a Guinea pig. It has been running normally for several months in the production environment. Thank you!
# Apache Dubbo for python

![License](https://img.shields.io/github/license/apache/dubbo-python)

---

<p align="center">
<img src="https://cn.dubbo.apache.org/imgs/nav_logo2.png" alt="Logo" width="40%" />
</p>

Apache Dubbo is an easy-to-use, high-performance WEB and RPC framework with builtin service discovery, traffic management, observability, security features, tools and best practices for building enterprise-level microservices.

Dubbo-python is a Python implementation of the [triple protocol](https://dubbo.apache.org/zh-cn/overview/reference/protocols/triple-spec/) (a protocol fully compatible with gRPC and friendly to HTTP) and various features designed by Dubbo for constructing microservice architectures.

Visit [the official website](https://dubbo.apache.org/) for more information.

### 🚧 Early-Stage Project 🚧

> **Disclaimer:** This project is in the early stages of development. Features are subject to change, and some components may not be fully stable. Contributions and feedback are welcome as the project evolves.

## Features

- **Service Discovery**: Zookeeper
- **Load Balance**: Random
- **RPC Protocols**: Triple(gRPC compatible and HTTP-friendly)
- **Transport**: asyncio(uvloop)
- **Serialization**: Customizable(protobuf, json...)


## Getting started

Before you begin, ensure that you have **`python 3.11+`**. Then, install Dubbo-Python in your project using the following steps:

```shell
git clone https://github.com/apache/dubbo-python.git
cd dubbo-python && pip install .
```

Get started with Dubbo-Python in just 5 minutes by following our [Quick Start Guide](https://github.com/apache/dubbo-python/tree/main/samples).

It's as simple as the following code snippet. With just a few lines of code, you can launch a fully functional point-to-point RPC service :

1. Build and start the Server

```python
import dubbo
from dubbo.configs import ServiceConfig
from dubbo.proxy.handlers import RpcServiceHandler, RpcMethodHandler


def handle_unary(request):
s = request.decode("utf-8")
print(f"Received request: {s}")
return (s + " world").encode("utf-8")


if __name__ == "__main__":
# build a method handler
method_handler = RpcMethodHandler.unary(handle_unary)
# build a service handler
service_handler = RpcServiceHandler(
service_name="org.apache.dubbo.samples.HelloWorld",
method_handlers={"unary": method_handler},
)

service_config = ServiceConfig(service_handler)

# start the server
server = dubbo.Server(service_config).start()

input("Press Enter to stop the server...\n")
```

2. Build and start the Client

```python
import dubbo
from dubbo.configs import ReferenceConfig


class UnaryServiceStub:

def __init__(self, client: dubbo.Client):
self.unary = client.unary(method_name="unary")

def unary(self, request):
return self.unary(request)


if __name__ == "__main__":
reference_config = ReferenceConfig.from_url(
"tri://127.0.0.1:50051/org.apache.dubbo.samples.HelloWorld"
)
dubbo_client = dubbo.Client(reference_config)

unary_service_stub = UnaryServiceStub(dubbo_client)

result = unary_service_stub.unary("hello".encode("utf-8"))
print(result.decode("utf-8"))
```



## License

Apache Dubbo-python software is licensed under the Apache License Version 2.0. See
the [LICENSE](https://github.com/apache/dubbo-python/blob/main/LICENSE) file for details.
7 changes: 7 additions & 0 deletions dubbo/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,10 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from .bootstrap import Dubbo
from .client import Client
from .server import Server
from .__version__ import __version__

__all__ = ["Dubbo", "Client", "Server"]
17 changes: 17 additions & 0 deletions dubbo/__version__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

__version__ = "1.0.0b1"
134 changes: 134 additions & 0 deletions dubbo/bootstrap.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import threading
from typing import Optional

from dubbo.classes import SingletonBase
from dubbo.configs import (
ApplicationConfig,
LoggerConfig,
ReferenceConfig,
RegistryConfig,
)
from dubbo.constants import common_constants
from dubbo.loggers import loggerFactory


class Dubbo(SingletonBase):
"""
Dubbo class. This class is used to initialize the Dubbo framework.
"""

def __init__(
self,
application_config: Optional[ApplicationConfig] = None,
registry_config: Optional[RegistryConfig] = None,
logger_config: Optional[LoggerConfig] = None,
):
"""
Initialize a new Dubbo bootstrap.
:param application_config: The application configuration.
:type application_config: Optional[ApplicationConfig]
:param registry_config: The registry configuration.
:type registry_config: Optional[RegistryConfig]
:param logger_config: The logger configuration.
:type logger_config: Optional[LoggerConfig]
"""
self._initialized = False
self._global_lock = threading.Lock()

self._application_config = application_config
self._registry_config = registry_config
self._logger_config = logger_config

# check and set the default configuration
self._check_default()

# initialize the Dubbo framework
self._initialize()

@property
def application_config(self) -> Optional[ApplicationConfig]:
"""
Get the application configuration.
:return: The application configuration.
:rtype: Optional[ApplicationConfig]
"""
return self._application_config

@property
def registry_config(self) -> Optional[RegistryConfig]:
"""
Get the registry configuration.
:return: The registry configuration.
:rtype: Optional[RegistryConfig]
"""
return self._registry_config

@property
def logger_config(self) -> Optional[LoggerConfig]:
"""
Get the logger configuration.
:return: The logger configuration.
:rtype: Optional[LoggerConfig]
"""
return self._logger_config

def _check_default(self):
"""
Check and set the default configuration.
"""
# set default application configuration
if not self._application_config:
self._application_config = ApplicationConfig(common_constants.DUBBO_VALUE)

if self._registry_config:
if not self._registry_config.version and self.application_config.version:
self._registry_config.version = self.application_config.version

def _initialize(self):
"""
Initialize the Dubbo framework.
"""
with self._global_lock:
if self._initialized:
return

# set logger configuration
if self._logger_config:
loggerFactory.set_config(self._logger_config)

self._initialized = True

def create_client(self, reference_config: ReferenceConfig):
"""
Create a new Dubbo client.
:param reference_config: The reference configuration.
:type reference_config: ReferenceConfig
"""
from dubbo import Client

return Client(reference_config, self)

def create_server(self, config):
"""
Create a new Dubbo server.
:param config: The service configuration.
:type config: ServiceConfig
"""
from dubbo import Server

return Server(config, self)
Loading
Loading