Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modules/zstd: Add initial ZSTD Encoder procs #1846

Draft
wants to merge 105 commits into
base: main
Choose a base branch
from

Conversation

lpawelcz
Copy link
Contributor

This PR adds initial procs of the ZSTD Encoder. It's based on #1654.

Changes in this PR consist of:

  • Adding AlignedParallelRam proc - Responsible for handling Read/Write operations on multiple RAM models in parallel.
  • Adding HashTable proc - Generic Hash Table implementation. It uses underlying RAM models for operation and is controlled with a Read/Write Request/Response channels interface. In the ZSTD Encoder, it is used for fast lookup of repeating symbols in the input data buffer.
  • Adding HistoryBuffer proc - Data buffer capable of storing historical data and outputting said data from arbitrary chosen offset up to the top of the buffer. Uses AlignedParallelRam proc underneath.
  • Adding MatchFinder proc - Processes data stream from the input buffer in the system memory. Searches for repeating symbol patterns in the HashTable proc. Symbols that are not found are stored directly as literals in the literals buffer, while those that have matches are stored as sequences in the sequence buffer, using the matched offset and length information. Sequence and Literals Buffers are implemented in the system memory accessible via the AXI interface.

This PR is currently a WIP.
The following work needs to be finished to mark this PR as ready for review:

  • Fix MatchFinder codegen
  • Enable back the IR benchmarks, codegen, synthesis and P&R targets in the CI

koblonczek and others added 30 commits January 20, 2025 11:46
- XLSStruct for easier handling and serializing/deserializing XLS structs
- XLSChannel that serves as a dummy receiving channel
- XLSMonitor that monitors transactions on an XLS channel
- XLSDriver that can send data on an XLS channel
- LatencyScoreboard that can measure latency between corresponding transactions on input and output buses
- File-backed AXI memory python model

Internal-tag: [#64075]
Signed-off-by: Krzysztof Obłonczek <[email protected]>
Co-authred-by: Pawel Czarnecki <[email protected]>
Co-authred-by: Robert Winkler <[email protected]>
Signed-off-by: Michal Czyz <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Remove references to buffer structs as those are not used anywhere

Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Robert Winkler <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Oblonczek <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Robert Winkler <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
Internal-tag: [#67096]
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Maciej Torhan <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
Internal-tag: [#66955]
Signed-off-by: Pawel Czarnecki <[email protected]>
@rw1nkler rw1nkler force-pushed the zstd_encoder branch 2 times, most recently from 6899691 to 20c221b Compare February 27, 2025 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants