File transfer over BLE

[tannewt]

Should feel like USB MSC but wireless including auto-reload

[ladyada]

just a note, there doesnt seem to be a standard file transfer protocol - we'd have to hand-implement. not too hard but will need implementing per-platform
https://www.bluetooth.com/specifications/gatt

[arturo182]

There is however a Object Transfer Service that could potentially be used for this https://www.bluetooth.com/specifications/gatt/viewer?attributeXmlFile=org.bluetooth.service.object_transfer.xml

And the Nordic SDK has experimental support for it: http://infocenter.nordicsemi.com/topic/com.nordic.infocenter.sdk5.v15.0.0/group__nrf__ble__ots__c.html?cp=4_0_0_6_3_35

It mentions Apple Notification Center Service (ANCS) but it seems to be a copy-paste error.

[tannewt]

@ladyada I realize that. Its just too handy to not have.

@arturo182 Thanks for the link. That looks promising.

[arturo182]

I can have a look at this at some point, I think we should look into incorporating some parts of Nordic's BLE SDK as that would ease development and reduce the number of bugs we introduce when writing everything from scratch.

[ladyada]

i totally think we should have it, just wanted to make a note that it wouldnt be as elegant as it ought to be if the BT consortium had come up with a profile :D

[dhalbert]

I did some more looking at Bluetooth File transfer. There are some pre-existing specs for classic Bluetooth. There's OBEX, a general object exchange protocol which is used for IrDA, Bluetooth, USB, and other protocols. GOEP (Generalized Object Exchange Protocol) is a Bluetooth profile based on on OBEX. There's a File Transfer Profile (FTP).

https://en.wikipedia.org/wiki/OBject_EXchange
https://en.wikipedia.org/wiki/OBject_EXchange#Protocols
https://en.wikipedia.org/wiki/List_of_Bluetooth_profiles#File_Transfer_Profile_(FTP)

[ladyada]

@hathach do you have any thoughts on which one of these would be most universal or best to implement?

[ladyada]

https://www.howtogeek.com/208365/how-to-use-bluetooth-file-transfer-between-os-x-and-android-5.0-devices/

[hathach]

So far I didn't know/work with a way to transfer file via ble yet. Haven't got any request for this in Arduino

[alexwhittemore]

I might be inclined to work on this a bit if I can't easily satisfy my use case with a hackier system, so I'd love to collect some more thoughts about how this could/should work.

In my mind I've got two clear use cases: 1) something like image transfer, where a device with a camera needs to backhaul a small .jpg or something for processing on a gateway. 2) (my current need) in-the-field live firmware update. A gateway advertises some kind of FW update service where a device with a lower version can request the new code.py, copy it over, and restart.

I'm working on a many-node sensor prototype. I'm not sure how long I'll stick with CP for iterating (eventually, code size will be a serious constraint), but if I make it to tens of units, I'll definitely need to put together something where I can push out a code.py update without manually plugging in each sensor for reprogramming.

For that use case, it might even be fine to use nordic uart, write the text block out to disk, then do a rename and exit - no idea how that works in practice but a full file transfer service might be overkill. If it's not though, how do y'all envision such a service working? I know @tannewt mentioned in #3707 wanting to integrate this into the circuitpython core so it's available outside of the VM. One thing I'm very much NOT familiar with is the overall architecture and strategy of circuitpython implementation. Is there any under-the-hood documentation for would-be contributors?

Alternately, writing it all in Python and packaging it up as a library seems more my speed. But the questions then become 1) how useful is that, 2) how practical is that (i.e. memory constraints), 3) what comparable BLE service implementations exist to use as a style guide?

[tannewt]

In my mind I've got two clear use cases: 1) something like image transfer, where a device with a camera needs to backhaul a small .jpg or something for processing on a gateway. 2) (my current need) in-the-field live firmware update. A gateway advertises some kind of FW update service where a device with a lower version can request the new code.py, copy it over, and restart.

I'm picturing that the device will have a BLE Server that allows a Client to read and write its files. The "hey I have a new image" would have to be done separately.

For that use case, it might even be fine to use nordic uart, write the text block out to disk, then do a rename and exit - no idea how that works in practice but a full file transfer service might be overkill. If it's not though, how do y'all envision such a service working?

I'm picturing a Service with two characteristics. One is a simple number that is the version number. The second is a bidirectional service used to transfer the file info. It would be used similar to UART where it carries a custom protocol. (I don't want to overload UART directly though.) The client then supplies an action and a full path. Paths are always complete because you don't need to maintain current directory state.

If the client is reading it'll then send the number of bytes it can buffer. The server will return the total number of bytes, the number of bytes in response and then the bytes themselves. The client can then request more bytes by sending another number of bytes it can read or zero to stop reading. The server will return zeroes once it is at the end of the file.

Writing is similar except the server replies with it's free space and the client provides the length to be written.

We'll also need a way to list a directory. I think a json list is simplest but I'm open to ideas. Basically it'd work like file reading and writing but have a specific form for a directory.

I know @tannewt mentioned in #3707 wanting to integrate this into the circuitpython core so it's available outside of the VM. One thing I'm very much NOT familiar with is the overall architecture and strategy of circuitpython implementation. Is there any under-the-hood documentation for would-be contributors?

It's not super detailed. I began sketching this out ages ago: https://github.com/adafruit/circuitpython/blob/main/supervisor/shared/bluetooth.c

Alternately, writing it all in Python and packaging it up as a library seems more my speed. But the questions then become 1) how useful is that, 2) how practical is that (i.e. memory constraints), 3) what comparable BLE service implementations exist to use as a style guide?

You could definitely prototype the service in pure Python. You'll be ok until a code.py update breaks the updater.

I'd probably model it after the MIDI service because it uses one bidirectional characteristic and a packet buffer for buffering.

[alexwhittemore]

Tons of good food for thought, thanks!

[tannewt]

This was added in #4918 and uses the protocol defined here: https://github.com/adafruit/Adafruit_CircuitPython_BLE_File_Transfer

[alexwhittemore]

So rad. Thank you!