Skip to content

Commit

Permalink
Lower fragment creation size to 128 MB, and leave maximum fragment si…
Browse files Browse the repository at this point in the history
…ze check at 256 MB.
  • Loading branch information
drobison00 committed Oct 23, 2024
1 parent f8d6572 commit 8c77365
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/nv_ingest/modules/sinks/redis_task_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ def create_json_payload(message: ControlMessage, df_json: Dict[str, Any]) -> Lis
df_json_size = sys.getsizeof(df_json_str)

# 256 MB size limit (in bytes)
size_limit = 256 * 1024 * 1024
size_limit = 128 * 1024 * 1024

# If df_json is larger than the size limit, split it into chunks
if df_json_size > size_limit:
Expand Down

0 comments on commit 8c77365

Please sign in to comment.