Skip to content

Commit

Permalink
[improve][pip] PIP-362: Add admin APIs for batch updating and deletin…
Browse files Browse the repository at this point in the history
…g bookie rack information.
  • Loading branch information
ruihongzhou committed Jun 17, 2024
1 parent f3d4d5a commit f372ea6
Showing 1 changed file with 135 additions and 0 deletions.
135 changes: 135 additions & 0 deletions pip/pip-362.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# PIP-362: Add admin APIs for batch updating and deleting bookie rack information

# Background knowledge

Each time the rack information of a bookie is updated or deleted, the mapping information of the bookie rack
will be updated in the `/bookies` directory on Zookeeper. At the same time, all bookies observing this path will be
notified to fetch the latest bookie rack mapping information from Zookeeper.

# Motivation

Currently, we only provide single update or delete admin APIs for bookie rack information (`updateBookieRackInfo`
and `deleteBookieRackInfo`). If we want to update the rack information for n bookies, we need to call
`updateBookieRackInfo` n times, which will result in a large number of write and read operations on Zookeeper,
thereby affecting Zookeeper's stability.

Therefore, we want to introduce admin APIs for batch updating and deleting bookie rack information to reduce
the number of times Zookeeper is accessed when batch setting bookie rack information.

# Goals

## In Scope

Add admin APIs for batch updating and deleting bookie rack information.

# High Level Design

Add admin APIs for batch updating and deleting bookie rack information.

# Detailed Design

## Design & Implementation Details

Add admin APIs for batch updating and deleting bookie rack information.

## Public-facing Changes

### Public API

```java
@DELETE
@Path("/racks-info")
@ApiOperation(
value = "Removed the rack placement information for a batch of bookies in the cluster",
notes = "If the 'deleteAll' parameter is set to true, it will remove the rack placement "
+ "information for all bookies in the cluster, ignoring the 'bookieAddresses' parameter"
)
@ApiResponses(value = {
@ApiResponse(code = 204, message = "Operation successful"),
@ApiResponse(code = 403, message = "Don't have admin permission")
})
public void batchDeleteBookiesRackInfo(@Suspended final AsyncResponse asyncResponse,
@ApiParam(value = "List of bookie addresses")
@QueryParam("bookieAddresses") List<String> bookieAddresses,
@ApiParam(value = "Whether to delete all bookies rack info",
defaultValue = "false")
@QueryParam("deleteAll") @DefaultValue("false")
boolean deleteAll) throws Exception {
...
}

@POST
@Path("/racks-info")
@ApiOperation(value = "Updates the rack placement information for a batch of bookies in the cluster")
@ApiResponses(value = {
@ApiResponse(code = 204, message = "Operation successful"),
@ApiResponse(code = 403, message = "Don't have admin permission")}
)
public void batchUpdateBookiesRackInfo(@Suspended final AsyncResponse asyncResponse,
@ApiParam(value = "List of bookie info", required = true)
List<ExtBookieInfo> extBookieInfos) throws Exception {
...
}
```

```java
/**
* Remove rack placement information for a batch of bookies in the cluster.
*/
void batchDeleteBookiesRackInfo(List<String> bookieAddresses) throws PulsarAdminException;

/**
* Remove rack placement information for a batch of bookies in the cluster asynchronously.
*/
CompletableFuture<Void> batchDeleteBookiesRackInfoAsync(List<String> bookieAddresses);

/**
* Clears the rack placement information for all bookies int the cluster.
*/
void clearAllBookiesRackInfo() throws PulsarAdminException;

/**
* Clears the rack placement information for all bookies int the cluster asynchronously.
*/
CompletableFuture<Void> clearAllBookiesRackInfoAsync();

/**
* Updates the rack placement information for a batch of bookies in the cluster.
*/
void batchUpdateBookiesRackInfo(List<ExtBookieInfo> extBookieInfos) throws PulsarAdminException;

/**
* Updates the rack placement information for a batch of bookies in the cluster asynchronously.
*/
CompletableFuture<Void> batchUpdateBookiesRackInfoAsync(List<ExtBookieInfo> extBookieInfos);
```

### Binary protocol

### Configuration

### CLI

### Metrics

# Monitoring

# Security Considerations

# Backward & Forward Compatibility

## Revert

## Upgrade

# Alternatives

# General Notes

# Links

<!--
Updated afterwards
-->
* Mailing List discussion thread:
* Mailing List voting thread:

0 comments on commit f372ea6

Please sign in to comment.