Skip to content

Commit

Permalink
Adding the ability to find the chunks of a document given its name (#42)
Browse files Browse the repository at this point in the history
* DocumentName in search and sourcereference

* Updated SDKs README

---------

Co-authored-by: Your Name <[email protected]>
  • Loading branch information
IsisChameleon and Your Name authored Sep 26, 2024
1 parent 9227a7b commit ad96a8c
Show file tree
Hide file tree
Showing 15 changed files with 700 additions and 32 deletions.
2 changes: 2 additions & 0 deletions protos/chunks.proto
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ message SourceReference {
string document_version = 5;
// Document path in the source system e.g. "My Drive/document.txt", "slack-channel-name"
optional string document_path = 6;
// Document name in the source system e.g. "document.txt"
optional string document_name = 7;
}

message ChunkReference {
Expand Down
23 changes: 23 additions & 0 deletions protos/search.proto
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import "chunks.proto";
service Search {
// Query the index for relevant chunks
rpc QueryChunks(QueryRequest) returns (QueryResponse);
// Query the index for all chunks of a specific document
rpc QueryChunksByDocumentName(QueryByDocumentNameRequest) returns (QueryByDocumentNameResponse);
// Get chunks by URL
rpc GetChunksByUrl(GetChunksByUrlRequest) returns (GetChunksByUrlResponse);
}
Expand All @@ -19,6 +21,11 @@ message Query {
string semantic_query = 1;
}

message DocumentNameQuery {
// Document name to search for
string document_name = 1;
}

message TimeSpan {
optional google.protobuf.Timestamp after = 1;
optional google.protobuf.Timestamp before = 2;
Expand Down Expand Up @@ -68,3 +75,19 @@ message GetChunksByUrlResponse {
// List of chunks
repeated Chunk chunks = 3;
}

message QueryByDocumentNameRequest {
// The query to execute
DocumentNameQuery query = 2;
// Filters to apply to query
optional Filters filters = 3;
}

message QueryByDocumentNameResponse {
// Query was successful
bool success = 1;
// Error message if query failed
optional google.protobuf.Struct error = 2;
// List of relevant chunks
repeated Chunk chunks = 3;
}
16 changes: 15 additions & 1 deletion sdks/node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,29 @@ const response = await client.exchangeTokens("OAUTH2-AUTH-CODE");

### SearchClient

With the Redactive access_token, User can search documents with Redactive Search service.
With the Redactive access_token, you can perform three types of searches using the Redactive Search service:

1. **Semantic Query Search**: Retrieve relevant chunks of information that are semantically related to a user query.
2. **URL-based Search**: Obtain all the chunks from a document by specifying its URL.
3. **Document Name Search**: Query for all the chunks from a document based on the name of the document.

```javascript
import { SearchClient } from "@redactive/redactive";

const client = new SearchClient();
const accessToken = "REDACTIVE-ACCESS-TOKEN";

// Semantic Search: retrieve text extracts (chunks) from various documents pertaining to the user query
const semanticQuery = "Tell me about AI";
await client.queryChunks({ accessToken, semanticQuery });

// URL-based Search: retrieve all chunks of the document at that URL
const url = "https://example.com/document";
await client.getChunksByUrl({ accessToken, url });

// Document Name Search : retrieve all chunks of a document identified by its name
const documentName = "AI Research Paper";
await client.queryChunksByDocumentName({ accessToken, documentName });
```

## Development
Expand Down
45 changes: 36 additions & 9 deletions sdks/node/src/grpc/chunks.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 29 additions & 5 deletions sdks/node/src/grpc/google/protobuf/struct.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions sdks/node/src/grpc/google/protobuf/timestamp.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit ad96a8c

Please sign in to comment.