Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Nebulex v2 #23

Merged
merged 1 commit into from
Oct 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .credo.exs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@
## Design Checks
{Credo.Check.Design.AliasUsage, priority: :low},

# Deactivate due to they're not compatible with current Elixir version
{Credo.Check.Refactor.MapInto, false},
{Credo.Check.Warning.LazyLogging, false},

## Readability Checks
{Credo.Check.Readability.MaxLineLength, priority: :low, max_length: 100},

Expand Down
1 change: 1 addition & 0 deletions .dialyzer_ignore.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
74 changes: 74 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: CI

on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
nebulex_test:
name: 'NebulexRedisAdapter Test (Elixir ${{ matrix.elixir }} OTP ${{ matrix.otp }})'
runs-on: ubuntu-latest
strategy:
matrix:
include:
- elixir: 1.10.x
otp: 23.x
- elixir: 1.10.x
otp: 22.x
- elixir: 1.9.x
otp: 22.x
env:
GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
MIX_ENV: test
NBX_TEST: true
REDIS_CLUSTER_IP: '0.0.0.0'
steps:
- uses: actions/checkout@v2
- name: Start Redis
run: docker-compose up -d
- uses: actions/setup-elixir@v1
with:
otp-version: '${{ matrix.otp }}'
elixir-version: '${{ matrix.elixir }}'
- uses: actions/cache@v1
with:
path: deps
key: >-
${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-mix-${{
hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
restore-keys: |
${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-mix-
- uses: actions/cache@v1
with:
path: _build
key: >-
${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-build-${{
hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
restore-keys: |
${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-build-
- name: Install Dependencies
run: mix deps.get
- name: Compile Code
run: mix compile --warnings-as-errors
- name: Check Format
run: mix format --check-formatted
- name: Check Style
run: mix credo --strict
- name: Run Tests
run: |
epmd -daemon
mix coveralls.github
- uses: actions/cache@v1
with:
path: priv/plts
key: '${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-plt-v1'
restore-keys: |
${{ runner.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-plt-v1
- name: Run Dialyzer
run: mix dialyzer --format short
- name: Report Doc Coverage
run: MIX_ENV=docs mix inch.report
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ erl_crash.dump
.elixir*
.vs*
mix.lock
/priv
39 changes: 0 additions & 39 deletions .travis.yml

This file was deleted.

91 changes: 67 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# NebulexRedisAdapter
> ### Nebulex adapter for Redis with cluster support.

[![Build Status](https://travis-ci.org/cabol/nebulex_redis_adapter.svg?branch=master)](https://travis-ci.org/cabol/nebulex_redis_adapter)
![CI](https://github.com/cabol/nebulex_redis_adapter/workflows/CI/badge.svg)
[![Coverage Status](https://coveralls.io/repos/github/cabol/nebulex_redis_adapter/badge.svg?branch=master)](https://coveralls.io/github/cabol/nebulex_redis_adapter?branch=master)
[![Inline docs](http://inch-ci.org/github/cabol/nebulex_redis_adapter.svg)](http://inch-ci.org/github/cabol/nebulex_redis_adapter)
[![Hex Version](https://img.shields.io/hexpm/v/nebulex_redis_adapter.svg)](https://hex.pm/packages/nebulex_redis_adapter)
Expand Down Expand Up @@ -71,12 +71,11 @@ There are different ways to support distributed caching when using

### Redis Cluster

Redis can be setup in distributed fashion by means of [Redis Cluster][redis_cluster],
which is a built-in feature since version 3.0 (or greater). The adapter provides
the `:redis_cluster` mode to setup **Redis Cluster** from client-side
automatically and be able to use it transparently.

[redis_cluster]: https://redis.io/topics/cluster-tutorial
[Redis Cluster](https://redis.io/topics/cluster-tutorial) is a built-in feature
in Redis since version 3, and it may be the most convenient and recommendable
way to set up Redis in a cluster and have a distributed cache storage out-of-box.
This adapter provides the `:redis_cluster` mode to set up **Redis Cluster**
from the client-side automatically and be able to use it transparently.

First of all, ensure you have **Redis Cluster** configured and running.

Expand Down Expand Up @@ -124,11 +123,10 @@ configured by the adapter once it gets the cluster slots info.
> This one could be the easiest and recommended way for distributed caching
using Redis and **NebulexRedisAdapter**.

### Client-side Cluster based on Sharding (and consistent hashing)
### Client-side Cluster based on Sharding

**NebulexRedisAdapter** also brings with a simple client-side cluster
implementation based on Sharding as distribution model and consistent
hashing for node resolution.
implementation based on Sharding distribution model.

We define our cache normally:

Expand All @@ -140,13 +138,31 @@ defmodule MyApp.ClusteredCache do
end
```

The Keyslot module using consistent hashing:

```elixir
defmodule MyApp.ClusteredCache.Keyslot do
use Nebulex.Adapter.Keyslot

@impl true
def hash_slot(key, range) do
key
|> :erlang.phash2()
|> :jchash.compute(range)
end
end
```

And then, within the config:

```elixir
config :my_app, MayApp.ClusteredCache,
# Enable client-side cluster mode
mode: :cluster,

# Keyslot with consistent hashing
keyslot: MyApp.ClusteredCache.Keyslot,

# Nodes config (each node has its own options)
nodes: [
node1: [
Expand Down Expand Up @@ -175,32 +191,29 @@ config :my_app, MayApp.ClusteredCache,
]
```

> **NOTE:** It is highly recommendable to provide a consistent hashing
implementation for `Nebulex.Adapter.Keyslot`.

That's all, the rest of the work is done by **NebulexRedisAdapter**
automatically.

### Using `Nebulex.Adapters.Dist`

Another simple option is to use the `Nebulex.Adapters.Dist` and set as local
cache the `NebulexRedisAdapter`. The idea here is that each Elixir node running
the distributed cache (`Nebulex.Adapters.Dist`) will have as local backend or
cache a Redis instance (handled by `NebulexRedisAdapter`).
Another simple option is to use the `Nebulex.Adapters.Partitioned` and set as
local cache the `NebulexRedisAdapter`. The idea here is each Elixir node running
the distributed cache (`Nebulex.Adapters.Partitioned`) will have as local
backend or cache a Redis instance (handled by `NebulexRedisAdapter`).


This example shows how the setup a distributed cache using
`Nebulex.Adapters.Dist` and `NebulexRedisAdapter`:
`Nebulex.Adapters.Partitioned` and `NebulexRedisAdapter`:

```elixir
defmodule MyApp.DistributedCache do
use Nebulex.Cache,
otp_app: :nebulex,
adapter: Nebulex.Adapters.Dist,
local: MyApp.DistributedCache.RedisLocalCache

defmodule RedisLocalCache do
use Nebulex.Cache,
otp_app: :nebulex,
adapter: NebulexRedisAdapter
end
adapter: Nebulex.Adapters.Partitioned,
primary_storage_adapter: NebulexRedisAdapter
end
```

Expand All @@ -213,6 +226,37 @@ Instead of connect the adapter against the Redis nodes, we connect it against
the proxy nodes, this means, in the config, we just setup the pools with the
host and port for each proxy.

## Using the cache for executing a Redis command or pipeline

Since `NebulexRedisAdapter` works on top of `Redix` and provides features like
connection pools and "Redis Cluster" support, it may be seen also as a sort of
Redis client, but it is meant to be used mainly with the Nebulex cache API.
However, Redis API is quite extensive and there are a lot of useful commands
we may want to run taking advantage of the `NebulexRedisAdapter` features.
Therefore, the adapter injects two additional/extended functions to the
defined cache: `command!/3` and `pipeline!/3`.

```elixir
iex> MyCache.command!("mylist", ["LPUSH", "mylist", "world"])
1
iex> MyCache.command!("mylist", ["LPUSH", "mylist", "hello"])
2
iex> MyCache.command!("mylist", ["LRANGE", "mylist", "0", "-1"])
["hello", "world"]

iex> cache.pipeline!("mylist", [
...> ["LPUSH", "mylist", "world"],
...> ["LPUSH", "mylist", "hello"],
...> ["LRANGE", "mylist", "0", "-1"]
...> ])
[1, 2, ["hello", "world"]]
```

**NOTE:** The `key` is required when used the adapter in mode `:cluster` or
`:redis_cluster`, for `:standalone` no needed (optional). And the `name` is
in case you are using a dynamic cache and you have to pass the cache name
explicitly.

## Testing

To run the **NebulexRedisAdapter** tests you will have to have Redis running
Expand Down Expand Up @@ -289,7 +333,6 @@ Before to submit a PR it is highly recommended to run:

* `export NBX_TEST=true` to fetch Nebulex from GH directly and be able to
re-use shared tests.
* `mix test` to run tests
* `mix coveralls.html && open cover/excoveralls.html` to run tests and check
out code coverage (expected 100%).
* `mix format && mix credo --strict` to format your code properly and find code
Expand Down
8 changes: 3 additions & 5 deletions config/test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,15 @@ use Mix.Config

# Redis Standalone
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.Standalone,
version_generator: Nebulex.Version.Timestamp,
conn_opts: [
host: "127.0.0.1",
port: 6379
]

# Redis test cache
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.Cluster,
version_generator: Nebulex.Version.Timestamp,
mode: :cluster,
keyslot: NebulexRedisAdapter.TestCache.Keyslot,
nodes: [
node1: [
conn_opts: [
Expand All @@ -35,7 +34,6 @@ config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.Cluster,

# Redis test clustered cache
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisCluster,
version_generator: Nebulex.Version.Timestamp,
mode: :redis_cluster,
master_nodes: [
[
Expand Down Expand Up @@ -68,9 +66,9 @@ config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisClusterConnErr
]

# Redis test clustered cache
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisClusterWithHashSlot,
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisClusterWithKeyslot,
mode: :redis_cluster,
hash_slot: NebulexRedisAdapter.TestCache.HashSlot,
keyslot: NebulexRedisAdapter.TestCache.Keyslot,
pool_size: 2,
master_nodes: [
[
Expand Down
4 changes: 1 addition & 3 deletions coveralls.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
{
"skip_files": [
"test/*",
"lib/nebulex_redis_adapter/cluster/hash_slot.ex",
"lib/nebulex_redis_adapter/list.ex"
"test/*"
],

"coverage_options": {
Expand Down
Loading