Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow longer traces #1569

Merged
merged 2 commits into from
Feb 5, 2025
Merged

Allow longer traces #1569

merged 2 commits into from
Feb 5, 2025

Conversation

ltratt
Copy link
Contributor

@ltratt ltratt commented Jan 28, 2025

This PR is as much a RFC as "let's merge this". Right now, we reject a lot of traces as being "too long". There are two current causes for this:

  1. We exceeded the PT buffer.
  2. We exceeded yk's willingness to compile the trace.

It's easy to think these two cases are the same, but they're not. With YK_LOG=3 the first case will be detected as stop-tracing time; the second case at compilation time.

This PR only tackles the second case: it allows yk to compile longer traces. First, I think we should give up on the 16/15-bit InstIdxs, at least for now: it's a very small type, and probably too restrictive. 04d3e8a changes this to 32/31 bits. Note: Insts now become two machine words big. I think this is fine for now.

That then allows us to increase the size of traces we'll compile. How high is too high? We don't yet know, but it seems worth it to me to substantially increase the size 20,000 is still, I suspect, a fairly conservative limit efad158. But we're going to have to experiment with longer traces in order to know "how long is too long", so perhaps we should go higher, at least temporarily.

@@ -3143,7 +3201,7 @@ mod tests {
/// Ensure that any given instruction fits in 64-bits.
#[test]
fn inst_size() {
assert!(mem::size_of::<Inst>() <= mem::size_of::<u64>());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring above needs updating.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me may as well say our instructions are u128 sized rather than "two 64-bit sized"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the force push.

@vext01
Copy link
Contributor

vext01 commented Jan 28, 2025

I don't object to this because we do indeed reject a lot of traces.

@ptersilie
Copy link
Contributor

Works for me too.

@ltratt ltratt force-pushed the allow_longer_traces branch from efad158 to 278a83a Compare February 3, 2025 13:58
@ltratt ltratt marked this pull request as ready for review February 3, 2025 13:58
@vext01 vext01 added this pull request to the merge queue Feb 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 3, 2025
@ltratt ltratt force-pushed the allow_longer_traces branch from 278a83a to a48307a Compare February 3, 2025 14:29
@ltratt
Copy link
Contributor Author

ltratt commented Feb 3, 2025

With luck the force push fixes the test that breaks in swt.

@vext01 vext01 added this pull request to the merge queue Feb 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 3, 2025
@ltratt
Copy link
Contributor Author

ltratt commented Feb 3, 2025

OK so it seems after some local testing that the only way to get this to explode is to make swt take a very long time. I am inclined to ignore the test. OK to push on that basis?

@ltratt
Copy link
Contributor Author

ltratt commented Feb 3, 2025

Ah, hang on, it's ignored on swt, so my measuring that was pointless 🤦‍♂️ Let me see if I can find a "big enough" value that's not too slow...

Although `InstIdx` looks to be 16 bits big, `PackedOperand` forces it to
be 15 bits and that just isn't enough (32,767) for some of the traces we
now see, particularly after peeling.

Probably 19 bits would be big enough, but right now that level of bit
fiddling isn't worth our effort. Instead, this commit brute forces
`InstIdx` to 32 bits. That also forces `ConstIdx` to be 32 bits, to keep
code churn to a minimum.
@ltratt ltratt force-pushed the allow_longer_traces branch from a48307a to 928a06e Compare February 5, 2025 09:48
@ltratt
Copy link
Contributor Author

ltratt commented Feb 5, 2025

I finally worked out what I was doing wrong: the trace_too_long test needed two constants updating. Fixed in the force push.

@vext01 vext01 added this pull request to the merge queue Feb 5, 2025
Merged via the queue into ykjit:master with commit de8dd26 Feb 5, 2025
2 checks passed
@ltratt ltratt deleted the allow_longer_traces branch February 6, 2025 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants