Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rekordbox file format specs. #116

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Add rekordbox file format specs. #116

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d8944be
Add rekordbox file format specs.
Nov 20, 2018
108c283
Rename KSY structures, per @KOLANICH
Nov 21, 2018
84a3455
No longer fail if the first four bytes are not zero.
Nov 21, 2018
7019417
Rename files to match top-level ids
Nov 21, 2018
3c7635d
Abandon use of enum for FourCC until issue #300 is resolved
Nov 21, 2018
f4a44d4
Attempt #2 at renaming file, weird!
Nov 25, 2018
00ea701
Replace file accitentally moved.
Nov 25, 2018
7ed34f0
Simplify storiage of heap position per @GreyCat
Nov 25, 2018
c49dea2
Improved understanding of waveform headers.
Nov 27, 2018
ad378db
Loosen up constraints on magic row header bytes.
Dec 6, 2018
97a5b49
Handle longer-string variant of artist rows.
Dec 11, 2018
a2898a0
Improve long-offset artist name handling.
Dec 12, 2018
ed80bf6
Incorporate tweaks made while finishing writeup article.
Mar 8, 2019
52bd8ab
Add recent discoveries.
May 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 47 additions & 5 deletions database/rekordbox_pdb.ksy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -376,10 +376,10 @@ types:
doc: |
A row that holds an artist name and ID.
seq:
- type: u2
- id: subtype
type: u2
doc: |
Some kind of magic word? Usually 0x60, 0x00 but have seen
0x64, 0x00.
Usually 0x60, but 0x64 means we have a long name embedded in the row.
- id: index_shift
type: u2
doc: TODO name from @flesniak, but what does it mean?
Expand All @@ -390,19 +390,34 @@ types:
and linked from other rows (such as tracks).
- type: u1
doc: |
@flesniak says: "alwayx 0x03, maybe an unindexed empty string"
@flesniak says: "always 0x03, maybe an unindexed empty string"
- id: ofs_name
type: u1
doc: |
The location of the variable-length name string, relative to
the start of this row.
the start of this row, unless subtype ix 0x64.
instances:
name:
type: device_sql_string
if: subtype == 0x60
pos: _parent.row_base + ofs_name
doc: |
The name of this artist.
-webide-parse-mode: eager
ofs_long_name:
type: u2
if: subtype == 0x64
pos: _parent.row_base + 0x0a
Copy link
Contributor

@KOLANICH KOLANICH Dec 11, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about

    ofs_real_name:
        value: _parent.row_base + (subtype == 0x64 ? ofs_long_name : ofs_name)
    name:
        pos: ofs_real_name
        type: device_sql_string

Copy link
Contributor

@KOLANICH KOLANICH Dec 11, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please follow the convention pos [io] type desc if

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Around the time you were writing this, I was at my lunchtime swim, and in the pool is often when I review recent code and make improvements and optimizations, and I came up with the same idea, it will make dealing with the value much easier. Thanks! I will fix the order of the fields as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rats, when I try that I am getting a KSC error in the generate phase: mapping values are not allowed here in 'reader', line 410, column 66: ... (subtype == 0x64)? ofs_name_far : ofs_name_near) (column 66 is right after the :. Any ideas?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, that’s a FAQ, need to escape it as a string for the silly YAML format.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up doing it a slightly different way, but it has the same net effect, the struct is easier to understand and much easier to use; the name is always accessed using the same instance regardless of how it is actually stored.
brunchboy@a2898a0

doc: |
For Names longer than 0xff bytes, this holds the position
relative to the start of the row.
long_name:
type: device_sql_long_string
if: subtype == 0x64
pos: _parent.row_base + ofs_long_name
doc: |
Names longer than 0xff bytes will be found here.


artwork_row:
doc: |
Expand Down Expand Up @@ -839,6 +854,24 @@ types:
-webide-parse-mode: eager
-webide-representation: '{body.text}'

device_sql_long_string:
doc: |
A string longer than 255 bytes that is embedded next to a row.
seq:
- id: kind
type: u1
doc: |
Indicates whether the encoding is ASCII or UTF16-BE.
- id: body
type:
switch-on: kind
cases:
0x40: device_sql_long_ascii
0x90: device_sql_long_utf16be
_: device_sql_unknown
-webide-parse-mode: eager
-webide-representation: '{body.text}'

device_sql_short_ascii:
doc: |
An ASCII-encoded string up to 127 bytes long.
Expand Down Expand Up @@ -895,6 +928,15 @@ types:
doc: |
The content of the string.

device_sql_unknown:
doc: |
A string type we do not yet recognize, but want to avoid crashing.
seq:
- id: length
type: u2
doc: |
Contains the length of the string in bytes, we think?

enums:
page_type:
0:
Expand Down