Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SPI Abstraction #264

Open
wants to merge 5 commits into
base: rust-next
Choose a base branch
from
Open

Conversation

Skallwar
Copy link

@Skallwar Skallwar commented May 12, 2021

Hi,

@CohenArthur @n1tram1 and myself are working on a Rust chrdev module for the MFRC522 RFID reader for a class. In order to do so, we needed a new SPI abstraction so we created it.

It is not finished yet but we would like to have some feedback on our work and we are hoping we can merge it in the future.

Currently our driver is compiling correctly using our abstraction but we have are having some Kernel OOPS when our SPI probe function returns (fixed by e55c21a)

[   36.266704] mfrc522: [MFRC522-RS] Init
[   36.267087] mfrc522: [MFRC522-RS] SPI Registered
[   36.272868] Unable to handle kernel paging request at virtual address ffff80001087b7d0
[   36.282778] Mem abort info:
[   36.286541]   ESR = 0x96000007
[   36.290281]   EC = 0x25: DABT (current EL), IL = 32 bits
[   36.296322]   SET = 0, FnV = 0
[   36.300027]   EA = 0, S1PTW = 0
[   36.303836] Data abort info:
[   36.307375]   ISV = 0, ISS = 0x00000007
[   36.311887]   CM = 0, WnR = 0
[   36.315514] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000013f1000
[   36.322967] [ffff80001087b7d0] pgd=0000000001810003, p4d=0000000001810003, pud=0000000001811003, pmd=00000000039b9003, pte=0000000000000000
[   36.337034] Internal error: Oops: 96000007 [#1] PREEMPT SMP
[   36.343378] Modules linked in: mfrc522 hci_uart btqca btbcm bluetooth ecdh_generic ecc 8021q mrp garp stp llc spidev brcmfmac vc4 cec brcmutil drm_kms_helper cfg80211 drm raspberrypi_cpufreq rfkill clk_raspberrypi raspberrypi_hwmon crct10dif_ce i2c_bcm2835 bcm2835_thermal bcm2835_rng rng_core spi_bcm2835 aes_neon_bs aes_neon_blk ip_tables x_tables ipv6
[   36.378161] CPU: 3 PID: 539 Comm: systemd-udevd Tainted: G        W         5.12.0-rc4+ #25
[   36.388189] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[   36.395343] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--)
[   36.402250] pc : dev_uevent+0x138/0x1e8
[   36.406966] lr : uevent_show+0x90/0x118
[   36.411677] sp : ffff80001079bba0
[   36.415849] x29: ffff80001079bbc0 x28: ffff670b41b9d400
[   36.422066] x27: 0000000000000000 x26: 0000000000000001
[   36.428281] x25: 0000000000000000 x24: ffff670b49b42700
[   36.434496] x23: ffffdc99f37a4168 x22: ffff670b4191a880
[   36.440711] x21: ffff670b42475800 x20: ffff670b47f59000
[   36.446900] x19: ffff670b42475800 x18: 0000000000000000
[   36.453060] x17: 0000000000000000 x16: 0000000000000000
[   36.459216] x15: ffff670b4342f0d0 x14: 0000000000000000
[   36.465374] x13: 0000000000000001 x12: 0000000000000000
[   36.471522] x11: 0000000000000001 x10: ffff670b47f58000
[   36.477660] x9 : 0de4e88efe7dd600 x8 : ffff80001087b7d0
[   36.483806] x7 : 0000000000000000 x6 : 000000000000003f
[   36.489957] x5 : 0000000000000040 x4 : ffff80001079bb80
[   36.496107] x3 : 0000000000000001 x2 : ffff670b47f59000
[   36.502254] x1 : ffff670b42475800 x0 : ffff670b4191a880
[   36.508389] Call trace:
[   36.511582]  dev_uevent+0x138/0x1e8
[   36.515824]  uevent_show+0x90/0x118
[   36.520048]  dev_attr_show+0x20/0x58
[   36.524339]  sysfs_kf_seq_show+0xa0/0x110
[   36.529046]  kernfs_seq_show+0x2c/0x9c
[   36.533465]  seq_read_iter+0x11c/0x3b4
[   36.537858]  kernfs_fop_read_iter+0x68/0x188
[   36.542758]  vfs_read+0x290/0x2bc
[   36.546666]  ksys_read+0x74/0xe0
[   36.550464]  __arm64_sys_read+0x1c/0x28
[   36.554867]  el0_svc_common+0x90/0x110
[   36.559170]  do_el0_svc+0x24/0x80
[   36.563029]  el0_svc+0x28/0x88
[   36.566622]  el0_sync_handler+0x84/0xe4
[   36.571006]  el0_sync+0x154/0x180
[   36.574856] Code: aa1403e0 97f8c37e f9403668 b40000c8 (f9400102)
[   36.581549] ---[ end trace 556d98a75f645ca9 ]---
[   36.588678] Unable to handle kernel paging request at virtual address ffff80001087b7d0
[   36.597285] Mem abort info:
[   36.600580]   ESR = 0x96000007
[   36.604092]   EC = 0x25: DABT (current EL), IL = 32 bits
[   36.609969]   SET = 0, FnV = 0
[   36.613503]   EA = 0, S1PTW = 0
[   36.617115] Data abort info:
[   36.620451]   ISV = 0, ISS = 0x00000007
[   36.624759]   CM = 0, WnR = 0
[   36.628167] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000013f1000
[   36.635407] [ffff80001087b7d0] pgd=0000000001810003, p4d=0000000001810003, pud=0000000001811003, pmd=00000000039b9003, pte=0000000000000000
[   36.649039] Internal error: Oops: 96000007 [#2] PREEMPT SMP
[   36.655146] Modules linked in: mfrc522 hci_uart btqca btbcm bluetooth ecdh_generic ecc 8021q mrp garp stp llc spidev brcmfmac vc4 cec brcmutil drm_kms_helper cfg80211 drm raspberrypi_cpufreq rfkill clk_raspberrypi raspberrypi_hwmon crct10dif_ce i2c_bcm2835 bcm2835_thermal bcm2835_rng rng_core spi_bcm2835 aes_neon_bs aes_neon_blk ip_tables x_tables ipv6
[   36.688923] CPU: 0 PID: 181 Comm: systemd-udevd Tainted: G      D W         5.12.0-rc4+ #25
[   36.698488] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[   36.705416] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--)
[   36.712093] pc : dev_uevent+0x138/0x1e8
[   36.716577] lr : uevent_show+0x90/0x118
[   36.721052] sp : ffff8000104e3ba0
[   36.724989] x29: ffff8000104e3bc0 x28: ffff670b43875400
[   36.730970] x27: 0000000000000000 x26: 0000000000000001
[   36.736956] x25: 0000000000000000 x24: ffff670b47de2e00
[   36.742944] x23: ffffdc99f37a4168 x22: ffff670b4191a880
[   36.748923] x21: ffff670b42475800 x20: ffff670b434c0000
[   36.754895] x19: ffff670b42475800 x18: 0000000000000000
[   36.760851] x17: 0000000000000000 x16: 0000000000000000
[   36.766787] x15: ffff670b4342f0d0 x14: 0000000000000000
[   36.772723] x13: 0000000000000000 x12: 0000000000000000
[   36.778653] x11: 0000000000000002 x10: 0000000080080000
[   36.784572] x9 : 0de4e88efe7dd600 x8 : ffff80001087b7d0
[   36.790498] x7 : 0000000000000000 x6 : 000000000000003f
[   36.796428] x5 : 0000000000000040 x4 : ffff8000104e3b80
[   36.802356] x3 : 0000000000000001 x2 : ffff670b434c0000
[   36.808284] x1 : ffff670b42475800 x0 : ffff670b4191a880
[   36.814202] Call trace:
[   36.817189]  dev_uevent+0x138/0x1e8
[   36.821226]  uevent_show+0x90/0x118
[   36.825254]  dev_attr_show+0x20/0x58
[   36.829352]  sysfs_kf_seq_show+0xa0/0x110
[   36.833874]  kernfs_seq_show+0x2c/0x9c
[   36.838115]  seq_read_iter+0x11c/0x3b4
[   36.842337]  kernfs_fop_read_iter+0x68/0x188
[   36.847072]  vfs_read+0x290/0x2bc
[   36.850820]  ksys_read+0x74/0xe0
[   36.854464]  __arm64_sys_read+0x1c/0x28
[   36.858717]  el0_svc_common+0x90/0x110
[   36.862874]  do_el0_svc+0x24/0x80
[   36.866589]  el0_svc+0x28/0x88
[   36.870038]  el0_sync_handler+0x84/0xe4
[   36.874278]  el0_sync+0x154/0x180
[   36.877984] Code: aa1403e0 97f8c37e f9403668 b40000c8 (f9400102)
[   36.884527] ---[ end trace 556d98a75f645caa ]---

Here is the code to our driver if you want to see how we are using it: https://github.com/ks0n/mfrc522-linux

@jeyhun1991

This comment was marked as spam.

Copy link
Collaborator

@TheSven73 TheSven73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work !!

To review this properly, I believe that we will probably want to see a skeleton spi client driver as part of this PR. The out-of-tree driver you linked to might be difficult to build. And how would we know we're checking out the right version? For example, the logfile of your crash contains log messages

[   36.267087] mfrc522: [MFRC522-RS] SPI probed

that I can't find in your linked-to mfrc522 driver.

rust/kernel/spi.rs Outdated Show resolved Hide resolved
rust/kernel/spi.rs Outdated Show resolved Hide resolved
rust/kernel/spi.rs Outdated Show resolved Hide resolved
rust/kernel/spi.rs Outdated Show resolved Hide resolved
rust/kernel/spi.rs Outdated Show resolved Hide resolved
probe: Option<SpiMethod>,
remove: Option<SpiMethod>,
shutdown: Option<SpiMethodVoid>,
) -> Self {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't your driver require driver data, i.e. the private memory you usually kzalloc() in probe(), and attach to your driver using spi_set_drvdata()? I noticed that in your driver you've sidestepped this issue by storing the driver data as a static. But this will stop working once you instantiate multiple instances (devices) of the same class (driver). E.g. you could have two spi memory chips of the same type. On different SPI busses, or even on the same SPI bus, using a different chip-select.

We currently have a PR in-flight that's trying to solve the problem of driver data, you could help review?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a problem we are indeed facing. While it wasn't necessary to complete on the project we have, it's definitely something that we have to improve on later on. Supporting multiple devices is indeed an application that makes sense, and a shortcoming of our driver as well as this PR.

We could allow the user to pass their own data as an Option<T> and pass a pointer to this instance to the function spi_set_drvdata. As long as the DriverRegistration lives, this pointer will be valid and therefore safe. Another solution would be to add a wrapper around spi_set_drvdata and to let the user use it as they wish.

We currently have a PR in-flight that's trying to solve the problem of driver data, you could help review?

Of course!

rust/kernel/spi.rs Outdated Show resolved Hide resolved
@CohenArthur
Copy link

CohenArthur commented May 12, 2021

Nice work !!

To review this properly, I believe that we will probably want to see a skeleton spi client driver as part of this PR. The out-of-tree driver you linked to might be difficult to build.

You are right. We will provide a sample to accompany this PR. Should it simply contain an example of SPI driver registration or should it contain some more functionality? How would you view it? Since we cannot get probed simply without the necessary hardware resources.

And how would we know we're checking out the right version? For example, the logfile of your crash contains log messages

[   36.267087] mfrc522: [MFRC522-RS] SPI probed

that I can't find in your linked-to mfrc522 driver.

Sorry about that, we edited the backtrace to reflect the actual message in the code. We updated the code in the meantime and forgot to update the backtrace.

Skallwar,
n1tram1,
CohenArthur

rust/kernel/spi.rs Outdated Show resolved Hide resolved
rust/kernel/spi.rs Outdated Show resolved Hide resolved
@ksquirrel

This comment has been minimized.

Copy link
Collaborator

@TheSven73 TheSven73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of the v2 changes you pushed.

I am not convinced that safety is correct here. Or that macros are the best way to call SPI functions.

Speaking more generally, I suspect that you are running into the same problem I encountered myself with #254 : our first instinct is to do a straightforward conversion from C to Rust APIs. While that may be great as a proof-of-concept ("look, we can get this to work in Rust. cool!") it's probably not good enough to convince the upstream kernel community that Rust-as-a-second-language is a worthwhile thing to have. For that to happen, we will probably need to create Rust APIs/abstractions which are significantly better than C for driver writers: safe(r), more intuitive, easier to use. This will likely be the hard part.

(This is only my personal hunch as a hobby contributor - the people in charge @ojeda @wedsonaf may be more aware of the actual situation, and wish to proceed differently)

Can I invite you to help review the PR series starting with #276? It would be great if you could think how this could apply to your spi driver. For example, will it need to bind to devicetree? acpi? How could this be done in Rust in a way that is easier, safer, more intuitive than C?

pub struct SpiDevice(*mut bindings::spi_device);

impl SpiDevice {
pub unsafe fn from_ptr(dev: *mut bindings::spi_device) -> Self {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I understand the use of unsafe here. Or the // SAFETY: guarantee in the macros below.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done because of bjorn3's comment

Copy link

@CohenArthur CohenArthur May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I understand the use of unsafe here. Or the // SAFETY: guarantee in the macros below.

What bjorn said is that to the user, this function is unsafe as it uses a raw pointer which could be NULL or invalid. If we mark it as unsafe then we show the user that it holds no guarantees. Regarding the SAFETY comment: When our Spi driver gets probed (or removed, or shut down), the spi_device pointer is provided by the kernel. It is a pointer to the SPI client probing interacting with our device. Therefore, it is safe to pass to this function as it's just a valid pointer that the kernel gives us.

This is a bit similar to the PointerWrapper trait defined here.

Copy link
Member

@ojeda ojeda May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What bjorn said is that to the user, this function is unsafe as it uses a raw pointer which could be NULL or invalid. If we mark it as unsafe then we show the user that it holds no guarantees.

Not exactly -- to clarify a bit: the function does not use the pointer, and so it could potentially be a safe one. The problem is that other functions later on actually use it, and they are not unsafe. Since it is best to require that the pointer is valid just once, it is best to do it here, and then SpiDevice gains a type invariant, which then Spi can use to not need to be unsafe.

Speaking of which: you are missing the documentation and the # Safety section on the function, etc. The SpiDevice should also have documentation, including this type invariant. See other examples in rust/kernel (you can also wait a bit if you do not mind, since I have a PR pending with more docs explaining this).

impl Drop for DriverRegistration {
fn drop(&mut self) {
unsafe { bindings::driver_unregister(&mut self.spi_driver.as_mut().unwrap().driver) }
// FIXME: No unwrap? But it's safe?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does bindings::spi_driver need to live inside an Option? If you could get rid of the Option, you would not need unwrap() here (which could theoretically panic).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does bindings::spi_driver need to live inside an Option

I think not. I made a PR but I need to see if it works correctly before merging it

@CohenArthur
Copy link

CohenArthur commented May 20, 2021

I am not convinced that safety is correct here. Or that macros are the best way to call SPI functions.

Definitely agree with this. We implemented something similar to the FileOperations's virtual table that is going to land soon! It's here and being reviewed/rebased

Speaking more generally, I suspect that you are running into the same problem I encountered myself with #254 : our first instinct is to do a straightforward conversion from C to Rust APIs. While that may be great as a proof-of-concept ("look, we can get this to work in Rust. cool!") it's probably not good enough to convince the upstream kernel community that Rust-as-a-second-language is a worthwhile thing to have. For that to happen, we will probably need to create Rust APIs/abstractions which are significantly better than C for driver writers: safe(r), more intuitive, easier to use. This will likely be the hard part.

What parts do you have in mind specifically? But this is definitely something that we might be running into, we did go from our C driver to write this abstraction.

Can I invite you to help review the PR series starting with #276? It would be great if you could think how this could apply to your spi driver. For example, will it need to bind to devicetree? acpi? How could this be done in Rust in a way that is easier, safer, more intuitive than C?

For sure! We'll review it all three of us in the morning in a few hours.

Is there anything you'd like to see in a small driver example to showcase the usage of our SPI abstraction?

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@Skallwar Skallwar marked this pull request as ready for review May 21, 2021 13:31
@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@ksquirrel

This comment has been minimized.

@Skallwar
Copy link
Author

Skallwar commented Jun 1, 2021

Hi. I have rebase on top of the rust branch

rust/kernel/spi.rs Outdated Show resolved Hide resolved
this.registered = true;
Ok(())
}
_ => Err(Error::from_kernel_errno(res)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better just to have a if for the error case, and the fallthrough to the Ok case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, please always try to keep the happy path non-indented.

rust/kernel/spi.rs Outdated Show resolved Hide resolved
@ksquirrel

This comment has been minimized.

@Skallwar
Copy link
Author

Skallwar commented May 9, 2022

I rebased this. The only notable changes are the use of error::code and the fact that the remove function of the struct spi_device now returns void instead of int

use core::pin::Pin;

/// Wrapper struct around the kernel's `spi_device`.
#[derive(Clone, Copy)]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this allow me to clone a valid SpiDevice only to have it being deleted eventually while I am still holding a SpiDevice to a dangling pointer which would result in unsafe behaviour when calling the safe write functions below?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think that's a problem. I will try to find some time to fix this PR as Rust for Linux has evolved quite a lot and I'm not sure if this is still even compiling

@Skallwar
Copy link
Author

I rebased this. The only notable changes is the use of core::ffi::* instead of c_types::*.
I intend to read all the conversations to address all the remaining issues in the near future

Skallwar and others added 4 commits March 19, 2024 11:00
This add support for the SPI API for Rust modules.
Features:
- Register/Unregister
- Probe, Remove and Shutdown
- write_then_read, write, read

Co-developed-by: Martin Schmidt <[email protected]>
Signed-off-by: Martin Schmidt <[email protected]>
Co-developed-by: Arthur Cohen <[email protected]>
Signed-off-by: Arthur Cohen <[email protected]>
Signed-off-by: Esteban Blanc <[email protected]>
Signed-off-by: Esteban Blanc <[email protected]>
@Skallwar Skallwar force-pushed the feature/spi branch 2 times, most recently from af38138 to 8b1321d Compare March 19, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

9 participants