Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentinel Support #2664

Merged
merged 80 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
59cf984
redis client socket changes needed for sentinel
sjpotter Oct 26, 2023
cd254b1
Sentinel Implementation [EXPERIMENTAL]
sjpotter Nov 13, 2023
18fc971
add pooling
sjpotter Nov 13, 2023
29d4da0
Merge remote-tracking branch 'leibale/v5' into sentinel
sjpotter Nov 15, 2023
27d7f4e
improve typing with SENTINEL_ client members
sjpotter Nov 15, 2023
1b7cfc6
cleanup - remove unused comments / commented code
sjpotter Nov 16, 2023
13a18bd
small sendCommand change + revert change to tsconfig
sjpotter Nov 16, 2023
8f59b35
add more sentinel commands needed for testing.
sjpotter Nov 19, 2023
b7e59d5
lots of fixups and a reasonable first pass test suite
sjpotter Dec 1, 2023
8ae7d1f
add a timer option to update topology in background
sjpotter Dec 3, 2023
d3505ce
format all the things
sjpotter Dec 3, 2023
300889f
more progress
sjpotter Dec 5, 2023
6f14410
Merge remote-tracking branch 'upstream/v5' into sentinel
sjpotter Dec 5, 2023
9e3cda6
small cleanup
sjpotter Dec 5, 2023
18bb2d0
try to group promises together to minimize the internal await points
sjpotter Dec 5, 2023
6bfe7b7
redo events, to keep a single topology event to listen on
sjpotter Dec 6, 2023
5e722ef
nits + readme
sjpotter Dec 6, 2023
f0be3a8
add RedisSentinelFactory to provide lower level access to sentinel
sjpotter Dec 7, 2023
45e64f1
Merge remote-tracking branch 'upstream/v5' into sentinel
sjpotter Dec 7, 2023
1e9e65b
nit
sjpotter Dec 7, 2023
1731059
update
sjpotter Dec 11, 2023
c74c4ba
add RedisSentinelClient/Type for leased clients
sjpotter Dec 12, 2023
c8f74f3
add self for private access + improve emitting
sjpotter Dec 12, 2023
882c1c7
nit
sjpotter Dec 12, 2023
9a59887
nits
sjpotter Dec 12, 2023
84fa90b
improve testing
sjpotter Dec 14, 2023
93cd57a
ismall nit for typing
sjpotter Dec 14, 2023
43a40a1
bunch of changes
sjpotter Dec 15, 2023
57e9889
improve pub sub proxy.
sjpotter Dec 17, 2023
2e6091d
wrap the passed through RedisClient error to make clear where its com…
sjpotter Dec 17, 2023
02bbb13
refactor sentinel object / factory tests apart
sjpotter Dec 17, 2023
53bd858
harden tests a little bit more
sjpotter Dec 18, 2023
a661d89
add pipeline test
sjpotter Dec 18, 2023
6bd1c67
add scripts/function tests + fixups / cleanups to get them to work
sjpotter Dec 18, 2023
173fcc4
change to use redis-stack-server for redis nodes to enable module tes…
sjpotter Dec 19, 2023
c150bfc
fix test, forgot to return in use function with module
sjpotter Dec 19, 2023
65f543e
rename test
sjpotter Dec 19, 2023
5f83c09
improve tests to test with redis/sentinel nodes with and withput pass…
sjpotter Dec 19, 2023
ce35310
cleanup for RedisSentinel type generic typing in tests
sjpotter Dec 19, 2023
0f331dc
remove debugLog, just rely on traace mechanism
sjpotter Dec 19, 2023
9a77d5e
added multi tests for script/function/modules
sjpotter Dec 19, 2023
3da082c
don't emit errors on lease object, only on main object
sjpotter Dec 19, 2023
73e0536
improve testing
sjpotter Dec 20, 2023
89f33f9
extract out common code to reduce duplication
sjpotter Dec 20, 2023
cea648e
nit
sjpotter Dec 20, 2023
8dcd811
nits
sjpotter Dec 20, 2023
60974ca
nit
sjpotter Dec 20, 2023
1f3e19d
remove SENTINEL_... commands from main client, load them via module i…
sjpotter Dec 20, 2023
3dea400
missed adding RedisSentinelModule to correct places in RedisSentinelF…
sjpotter Dec 20, 2023
9ec1838
nits
sjpotter Dec 21, 2023
873014e
fix test logging on error
sjpotter Dec 22, 2023
c567741
invalidate watches when client reconnects
sjpotter Jan 3, 2024
5259124
remove WATCH and UNWATCH command files, fix WATCH and UNWATCH return …
leibale Jan 3, 2024
4f0b040
missing file in last commit :P
leibale Jan 3, 2024
19288ef
support for custom message in `WatchError`
leibale Jan 3, 2024
91b27cd
setDirtyWatch
leibale Jan 3, 2024
7bf97ec
update watch docs
leibale Jan 3, 2024
4567564
Merge branch 'watch-invalidate' into sentinel
sjpotter Jan 8, 2024
8945cf5
fixes needed
sjpotter Jan 8, 2024
c289366
Merge branch 'watch-invalidate' into sentinel
sjpotter Jan 8, 2024
26f72ae
wip
sjpotter Jan 8, 2024
4968e4c
get functions/modules to work again
sjpotter Jan 8, 2024
2554c0c
reuse leased client on pipelined commands.
sjpotter Jan 8, 2024
cc0768b
test tweaks
sjpotter Jan 8, 2024
2b865f1
nit
sjpotter Jan 9, 2024
6959b17
change how "sentinel" object client works, allow it to be reserved
sjpotter Jan 18, 2024
54e8e5e
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 29, 2024
28d6587
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 29, 2024
bb9b466
review
leibale Jan 29, 2024
1920858
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 29, 2024
a3d198e
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 29, 2024
5df6069
fixes to get more tests to pass
sjpotter Jan 29, 2024
838a6f1
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 31, 2024
27cb970
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale Jan 31, 2024
bd5a820
handle dirtyWatch and watchEpoch in reset and resetIfDirty
leibale Jan 31, 2024
07e62f3
"fix", but not correct, needs more work
sjpotter Feb 1, 2024
d917967
fix pubsub proxy
leibale Feb 1, 2024
4cb9564
remove timeout from steadyState function in test, caused problems
sjpotter Feb 1, 2024
8da0970
improve restarting nodes
sjpotter Feb 1, 2024
0aefb77
fix pubsub proxy and test
sjpotter Feb 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/clustering.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ await cluster.close();
| modules | | Included [Redis Modules](../README.md#packages) |
| scripts | | Script definitions (see [Lua Scripts](../README.md#lua-scripts)) |
| functions | | Function definitions (see [Functions](../README.md#functions)) |

## Auth with password and username

Specifying the password in the URL or a root node will only affect the connection to that specific node. In case you want to set the password for all the connections being created from a cluster instance, use the `defaults` option.
Expand Down
100 changes: 100 additions & 0 deletions docs/sentinel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Redis Sentinel

The [Redis Sentinel](https://redis.io/docs/management/sentinel/) object of node-redis provides a high level object that provides access to a high availability redis installation managed by Redis Sentinel to provide enumeration of master and replica nodes belonging to an installation as well as reconfigure itself on demand for failover and topology changes.

## Basic Example

```javascript
import { createSentinel } from 'redis';

const sentinel = await createSentinel({
name: 'sentinel-db',
sentinelRootNodes: [{
host: 'example',
port: 1234
}]
})
.on('error', err => console.error('Redis Sentinel Error', err));
.connect();

await sentinel.set('key', 'value');
const value = await sentinel.get('key');
await sentinel.close();
```

In the above example, we configure the sentinel object to fetch the configuration for the database Redis Sentinel is monitoring as "sentinel-db" with one of the sentinels being located at `example:1234`, then using it like a regular Redis client.

## `createSentinel` configuration

| Property | Default | Description |
|-----------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| name | | The sentinel identifier for a particular database cluster |
| sentinelRootNodes | | An array of root nodes that are part of the sentinel cluster, which will be used to get the topology. Each element in the array is a client configuration object. There is no need to specify every node in the cluster: 3 should be enough to reliably connect and obtain the sentinel configuration from the server |
| maxCommandRediscovers | `16` | The maximum number of times a command will retry due to topology changes. |
| nodeClientOptions | | The configuration values for every node in the cluster. Use this for example when specifying an ACL user to connect with |
| sentinelClientOptions | | The configuration values for every sentinel in the cluster. Use this for example when specifying an ACL user to connect with |
| masterPoolSize | `1` | The number of clients connected to the master node |
| replicaPoolSize | `0` | The number of clients connected to each replica node. When greater than 0, the client will distribute the load by executing read-only commands (such as `GET`, `GEOSEARCH`, etc.) across all the cluster nodes. |
| reserveClient | `false` | When `true`, one client will be reserved for the sentinel object. When `false`, the sentinel object will wait for the first available client from the pool. |
## PubSub

It supports PubSub via the normal mechanisms, including migrating the listeners if the node they are connected to goes down.

```javascript
await sentinel.subscribe('channel', message => {
// ...
});
await sentinel.unsubscribe('channel');
```

see [the PubSub guide](./pub-sub.md) for more details.

## Sentinel as a pool

The sentinel object provides the ability to manage a pool of clients for the master node:

```javascript
createSentinel({
// ...
masterPoolSize: 10
});
```

In addition, it also provides the ability have a pool of clients connected to the replica nodes, and to direct all read-only commands to them:

```javascript
createSentinel({
// ...
replicaPoolSize: 10
});
```

## Master client lease

Sometimes multiple commands needs to run on an exclusive client (for example, using `WATCH/MULTI/EXEC`).

There are 2 ways to get a client lease:

`.use()`
```javascript
const result = await sentinel.use(async client => {
await client.watch('key');
return client.multi()
.get('key')
.exec();
});
```

`.getMasterClientLease()`
```javascript
const clientLease = await sentinel.getMasterClientLease();

try {
await clientLease.watch('key');
const resp = await clientLease.multi()
.get('key')
.exec();
} finally {
clientLease.release();
}
```
2 changes: 1 addition & 1 deletion docs/transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ await multi.execTyped(); // [string]

## [`WATCH`](https://redis.io/commands/watch/)

You can also [watch](https://redis.io/docs/interact/transactions/#optimistic-locking-using-check-and-set) keys by calling `.watch()`. Your transaction will abort if any of the watched keys change.
You can also [watch](https://redis.io/docs/interact/transactions/#optimistic-locking-using-check-and-set) keys by calling `.watch()`. Your transaction will abort if any of the watched keys change or if the client reconnected between the `watch` and `exec` calls.

The `WATCH` state is stored on the connection (by the server). In case you need to run multiple `WATCH` & `MULTI` in parallel you'll need to use a [pool](./pool.md).

Expand Down
Loading