Skip to content

Commit

Permalink
[Refactor] Change the project/repo name to EmbedKV/embed-kv
Browse files Browse the repository at this point in the history
  • Loading branch information
rhdong committed Oct 19, 2022
1 parent e8cad7d commit f43cea8
Show file tree
Hide file tree
Showing 11 changed files with 54 additions and 54 deletions.
6 changes: 3 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ link_directories(
file(GLOB_RECURSE merlin_hkvs_src RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} *.cpp *.cu)

# TODO:
# add_library(merlin_kv STATIC ${merlin_kv_src})
# target_compile_features(merlin_kv PUBLIC cxx_std_14)
# target_link_libraries(merlin_kv PUBLIC ...)
# add_library(embed_kv STATIC ${embed_kv_src})
# target_compile_features(embed_kv PUBLIC cxx_std_14)
# target_link_libraries(embed_kv PUBLIC ...)


add_executable(merlin_hashtable_benchmark benchmark/merlin_hashtable_benchmark.cc.cu)
Expand Down
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Contributing

## About Merlin-KV
## About EmbedKV

Merlin-KV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements.
EmbedKV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements.

The key capability of Merlin-KV is to store key-value (feature-embedding) on high-bandwidth memory (HBM) of GPUs and in host memory.
The key capability of EmbedKV is to store key-value (feature-embedding) on high-bandwidth memory (HBM) of GPUs and in host memory.

You can also use the library for generic key-value storage.

## Maintainership

Merlin-KV is co-maintianed by [NVIDIA Merlin Team](https://github.com/NVIDIA-Merlin) and NVIDIA product end-users,
EmbedKV is co-maintianed by [NVIDIA Merlin Team](https://github.com/NVIDIA-Merlin) and NVIDIA product end-users,
and also open for public contributions, bug fixes, and documentation. This project adheres to NVIDIA's Code of Conduct.

## Contributing

We’re grateful for your interest in Merlin-KV and value your contributions.
We’re grateful for your interest in EmbedKV and value your contributions.
We welcome contributions via pull requests(PR).

Before sending out a pull request for significant change on the end-user API, we recommend you open an issue and
Expand All @@ -24,7 +24,7 @@ All submissions require review by project reviewers.

### Coding Style

Refer to the [Style Guide](http://github.com/NVIDIA-Merlin/merlin-kv/STYLE_GUIDE.md)
Refer to the [Style Guide](http://github.com/NVIDIA-Merlin/embed-kv/STYLE_GUIDE.md)

### Additional Requirements

Expand All @@ -34,7 +34,7 @@ In addition to the above requirements, contribution also needs to meet the follo

## Community

* Merlin-KV code (https://github.com/NVIDIA-Merlin/merlin-kv)
* EmbedKV code (https://github.com/NVIDIA-Merlin/embed-kv)

## Licence
Apache License 2.0
Expand Down
40 changes: 20 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# [NVIDIA Merlin-KV](https://github.com/NVIDIA-Merlin/merlin-kv)
# [NVIDIA EmbedKV](https://github.com/NVIDIA-Merlin/embed-kv)

[![Version](https://img.shields.io/github/v/release/NVIDIA-Merlin/merlin-kv?color=orange)](https://github.com/NVIDIA-Merlin/merlin-kv/releases)
[![GitHub License](https://img.shields.io/github/license/NVIDIA-Merlin/merlin-kv)](https://github.com/NVIDIA-Merlin/merlin-kv/blob/master/LICENSE)
[![Documentation](https://img.shields.io/badge/documentation-blue.svg)](https://nvidia-merlin.github.io/merlin-kv/master/README.html)
[![Version](https://img.shields.io/github/v/release/NVIDIA-Merlin/embed-kv?color=orange)](https://github.com/NVIDIA-Merlin/embed-kv/releases)
[![GitHub License](https://img.shields.io/github/license/NVIDIA-Merlin/embed-kv)](https://github.com/NVIDIA-Merlin/embed-kv/blob/master/LICENSE)
[![Documentation](https://img.shields.io/badge/documentation-blue.svg)](https://nvidia-merlin.github.io/embed-kv/master/README.html)

## About Merlin-KV
## About EmbedKV

Merlin-KV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements.
EmbedKV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements.

The key capability of Merlin-KV is to store key-value (feature-embedding) on high-bandwidth memory (HBM) of GPUs and in host memory.
The key capability of EmbedKV is to store key-value (feature-embedding) on high-bandwidth memory (HBM) of GPUs and in host memory.

You can also use the library for generic key-value storage.

## Benefits of Merlin-KV
## Benefits of EmbedKV

When building large recommender systems, machine learning (ML) engineers face the following challenges:

Expand All @@ -21,44 +21,44 @@ When building large recommender systems, machine learning (ML) engineers face th
- It is difficult to efficiently control consumption growth of limited HBM with customized strategies.
- Most generic key-value libraries provide low HBM and host memory utilization.

Merlin-KV alleviates these challenges and helps the machine learning engineers in RecSys with the following benefits:
EmbedKV alleviates these challenges and helps the machine learning engineers in RecSys with the following benefits:

- Supports training large RecSys models on **HBM and host memory** at the same time.
- Provides better performance by **full bypassing CPUs** and reducing the communication workload.
- Implements table-size restraint strategies that are based on **LRU or customized strategies**.
The strategies are implemented by CUDA kernels.
- Operates at a high working-status load factor that is close to 1.0.

Merlin-KV makes NVIDIA GPUs more suitable for training large and super-large models of ***search, recommendations, and advertising***.
EmbedKV makes NVIDIA GPUs more suitable for training large and super-large models of ***search, recommendations, and advertising***.
The library simplifies the common challenges to building, evaluating, and serving sophisticated recommenders models.

## API Documentation

The main classes and structs are below, and it's recommended to read the comments in the source code directly:

- [`class HashTable`](https://github.com/NVIDIA-Merlin/merlin-kv/blob/master/include/merlin_hashtable.cuh#L101)
- [`class EvictStrategy`](https://github.com/NVIDIA-Merlin/merlin-kv/blob/master/include/merlin_hashtable.cuh#L106)
- [`struct HashTableOptions`](https://github.com/NVIDIA-Merlin/merlin-kv/blob/master/include/merlin_hashtable.cuh#L34)
- [`Struct HashTable::Vector`](https://github.com/NVIDIA-Merlin/merlin-kv/blob/master/include/merlin_hashtable.cuh#L106)
- [`class HashTable`](https://github.com/NVIDIA-Merlin/embed-kv/blob/master/include/merlin_hashtable.cuh#L101)
- [`class EvictStrategy`](https://github.com/NVIDIA-Merlin/embed-kv/blob/master/include/merlin_hashtable.cuh#L106)
- [`struct HashTableOptions`](https://github.com/NVIDIA-Merlin/embed-kv/blob/master/include/merlin_hashtable.cuh#L34)
- [`Struct HashTable::Vector`](https://github.com/NVIDIA-Merlin/embed-kv/blob/master/include/merlin_hashtable.cuh#L106)

For regular API doc, please refer to [API Docs](https://nvidia-merlin.github.io/merlin-kv/master/api/index.html)
For regular API doc, please refer to [API Docs](https://nvidia-merlin.github.io/embed-kv/master/api/index.html)

## Usage restrictions

- The `key_type` and `meta_type` must be `uint64_t`.

## Contributors

Merlin-KV is co-maintianed by [NVIDIA Merlin Team](https://github.com/NVIDIA-Merlin) and NVIDIA product end-users,
EmbedKV is co-maintianed by [NVIDIA Merlin Team](https://github.com/NVIDIA-Merlin) and NVIDIA product end-users,
and also open for public contributions, bug fixes, and documentation. [[Contribute](CONTRIBUTING.md)]

## How to build

Basically, Merlin-KV is a headers only library, the commands below only create binaries for benchmark and unit testing.
Basically, EmbedKV is a headers only library, the commands below only create binaries for benchmark and unit testing.

```shell
git clone --recursive https://github.com/NVIDIA-Merlin/merlin-kv.git
cd merlin-kv && mkdir -p build && cd build
git clone --recursive https://github.com/NVIDIA-Merlin/embed-kv.git
cd embed-kv && mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -Dsm=80 .. && make -j

// for Benchmark
Expand Down Expand Up @@ -117,7 +117,7 @@ Your environment must meet the following requirements:

### Support and Feedback:

If you encounter any issues or have questions, go to [https://github.com/NVIDIA-Merlin/merlin-kv/issues](https://github.com/NVIDIA-Merlin/merlin-kv/issues) and submit an issue so that we can provide you with the necessary resolutions and answers.
If you encounter any issues or have questions, go to [https://github.com/NVIDIA-Merlin/embed-kv/issues](https://github.com/NVIDIA-Merlin/embed-kv/issues) and submit an issue so that we can provide you with the necessary resolutions and answers.

### Acknowledgment
We are very grateful to external initial contributors [@Zhangyafei](https://github.com/zhangyafeikimi) and [@Lifan](https://github.com/Lifann) for their design, coding, and review work.
Expand Down
2 changes: 1 addition & 1 deletion STYLE_GUIDE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#### C++
C++ code should conform to [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).

Merlin-KV uses [clang-format](https://clang.llvm.org/docs/ClangFormat.html)
EmbedKV uses [clang-format](https://clang.llvm.org/docs/ClangFormat.html)
to check your C/C++ changes. Sometimes you have some manually formatted
code that you don’t want clang-format to touch.
You can disable formatting like this:
Expand Down
12 changes: 6 additions & 6 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Documentation

This folder contains the scripts necessary to build the documentation for Merlin-KV.
You can view the generated [Merlin-KV documentation](https://nvidia-merlin.github.io/merlin-kv/master/README.html).
This folder contains the scripts necessary to build the documentation for EmbedKV.
You can view the generated [EmbedKV documentation](https://nvidia-merlin.github.io/embed-kv/master/README.html).

## Contributing to Docs

Expand All @@ -20,7 +20,7 @@ pip install -r docs/requirements-doc.txt

The preceding command runs Sphinx in your shell and outputs to build/html/index.html.

The build process for Merlin-KV is unique among the Merlin projects because it
The build process for EmbedKV is unique among the Merlin projects because it
uses Doxygen, Breathe, and Exhale to create API documentation from the C++ source.

## Preview the changes
Expand All @@ -45,7 +45,7 @@ for the `master` branch and the six most recent tags. The job runs daily,
but you can trigger it manually by going to the following URL and clicking
the *Run workflow* button.

<https://github.com/NVIDIA-Merlin/merlin-kv/actions/workflows/docs-sched-rebuild.yaml>
<https://github.com/NVIDIA-Merlin/embed-kv/actions/workflows/docs-sched-rebuild.yaml>

### Source management: README and index files

Expand Down Expand Up @@ -82,7 +82,7 @@ You can use a relative path for the link. For example--both the `README.md` fil
directory, you could add a link to a heading in the `README.md` file like this:

```markdown
To build Merlin-KV from scratch, refer to
To build EmbedKV from scratch, refer to
[How to Build](./README.md#how-to-build) in the `README` file.
```

Expand All @@ -99,7 +99,7 @@ the repository:

```markdown
## Coding Style
Refer to the [Style Guide](http://github.com/NVIDIA-Merlin/merlin-kv/STYLE_GUIDE.md)
Refer to the [Style Guide](http://github.com/NVIDIA-Merlin/embed-kv/STYLE_GUIDE.md)
in the GitHub repository for more details.
```

Expand Down
8 changes: 4 additions & 4 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,16 +133,16 @@
source_suffix = [".rst", ".md"]

breathe_projects = {
"Merlin-KV": "/tmp/doxygen/xml"
"EmbedKV": "/tmp/doxygen/xml"
}
breathe_default_project = "Merlin-KV"
breathe_default_project = "EmbedKV"

exhale_args = {
"containmentFolder": "./api",
"rootFileName": "index.rst",
"doxygenStripFromPath": "../../include",
"rootFileTitle": "Merlin-KV C++ API Documentation",
"fullApiSubSectionTitle": "Complete Merlin-KV API",
"rootFileTitle": "EmbedKV C++ API Documentation",
"fullApiSubSectionTitle": "Complete EmbedKV API",
"createTreeView": False,
"exhaleExecutesDoxygen": True,
"exhaleDoxygenStdin": """
Expand Down
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Related Resources
-----------------

Merlin Key-Value Storage GitHub Repository
`<https://github.com/NVIDIA-Merlin/merlin-kv>`_
`<https://github.com/NVIDIA-Merlin/embed-kv>`_

About Merlin
Merlin is the overarching project that brings together the Merlin projects.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/toc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ subtrees:
- file: api/index.rst
title: API Documentation
- file: CONTRIBUTING.md
title: Contributing to Merlin-KV
title: Contributing to EmbedKV


# The multi-modal data example uses several notebooks to demonstrate how to use of multi-modal data (text and images)
Expand Down
2 changes: 1 addition & 1 deletion include/merlin/flexible_buffer.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class FlexPinnedBuffer {
try {
if (!ptr_) CUDA_CHECK(cudaFreeHost(ptr_));
} catch (const nv::merlin::CudaException& e) {
cerr << "[merlin-kv] Failed to free FlexPinnedBuffer!" << endl;
cerr << "[embed-kv] Failed to free FlexPinnedBuffer!" << endl;
}
}

Expand Down
2 changes: 1 addition & 1 deletion include/merlin/utils.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ inline void merlin_check_(bool cond, const std::string& msg, const char* file,
int line) {
if (!cond) {
throw CudaException(std::string(file) + ":" + std::to_string(line) +
": Merlin-KV error " + msg);
": EmbedKV error " + msg);
}
}

Expand Down
18 changes: 9 additions & 9 deletions include/merlin_hashtable.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ enum class EvictStrategy {
};

/**
* @brief The options struct of Merlin-KV.
* @brief The options struct of EmbedKV.
*/
struct HashTableOptions {
size_t init_capacity = 0; ///< The initial capacity of the hash table.
Expand Down Expand Up @@ -103,7 +103,7 @@ using EraseIfPredict = bool (*)(
);

/**
* A Merlin-KV hash table is a concurrent and hierarchical hash table that is
* A EmbedKV hash table is a concurrent and hierarchical hash table that is
* powered by GPUs and can use HBM and host memory as storage for key-value
* pairs. Support for SSD storage is a future consideration.
*
Expand All @@ -129,7 +129,7 @@ template <class K, class V, class M, size_t D>
class HashTable {
public:
/**
* @brief The value type of a Merlin-KV hash table.
* @brief The value type of a EmbedKV hash table.
*/
struct Vector {
using value_type = V;
Expand Down Expand Up @@ -194,7 +194,7 @@ class HashTable {
options_.block_size = SAFE_GET_BLOCK_SIZE(options_.block_size);
reach_max_capacity_ = (options_.init_capacity * 2 > options_.max_capacity);
MERLIN_CHECK((!(options_.io_by_cpu && options_.max_hbm_for_vectors != 0)),
"[merlin-kv] `io_by_cpu` should not be true when "
"[embed-kv] `io_by_cpu` should not be true when "
"`max_hbm_for_vectors` is not 0!");
initialized_ = true;
CudaCheckError();
Expand Down Expand Up @@ -308,7 +308,7 @@ class HashTable {

{
static_assert(sizeof(value_type*) == sizeof(uint64_t),
"[merlin-kv] illegal conversation. value_type pointer "
"[embed-kv] illegal conversation. value_type pointer "
"should be 64 bit!");

const size_t N = n;
Expand Down Expand Up @@ -455,7 +455,7 @@ class HashTable {

if (!is_fast_mode()) {
static_assert(sizeof(value_type*) == sizeof(uint64_t),
"[merlin-kv] illegal conversation. value_type pointer must "
"[embed-kv] illegal conversation. value_type pointer must "
"be 64 bit!");

const size_t N = n;
Expand Down Expand Up @@ -552,7 +552,7 @@ class HashTable {

{
static_assert(sizeof(value_type*) == sizeof(uint64_t),
"[merlin-kv] illegal conversation. value_type pointer "
"[embed-kv] illegal conversation. value_type pointer "
"must be 64 bit!");

const size_t N = n;
Expand Down Expand Up @@ -750,7 +750,7 @@ class HashTable {

MERLIN_CHECK(
(block_size > 0),
"[merlin-kv] block_size <= 0, the K-V-M size may be too large!");
"[embed-kv] block_size <= 0, the K-V-M size may be too large!");
const size_t shared_size =
(sizeof(key_type) + sizeof(vector_type) + meta_size) * block_size;
const int grid_size = (n - 1) / (block_size) + 1;
Expand Down Expand Up @@ -849,7 +849,7 @@ class HashTable {

MERLIN_CHECK(
(block_size > 0),
"[merlin-kv] block_size <= 0, the K-V-M size may be too large!");
"[embed-kv] block_size <= 0, the K-V-M size may be too large!");
const size_t shared_size =
(sizeof(key_type) + sizeof(vector_type) + meta_size) * block_size;
const int grid_size = (n - 1) / (block_size) + 1;
Expand Down

0 comments on commit f43cea8

Please sign in to comment.