Moving to UR2.0

[craigraw]

Several projects (e.g. Specter, Bluewallet, Cobo) are still using the draft UR1.0 'spec' while others (e.g. FN, Sparrow, Gordian) have moved to UR2.0. @ChristopherA and @wolfmcnally have noted that the UR1.0 version was for comment only, while UR2.0 is closer to be considered "ready for use". As the user bases of all these projects grow, users are starting to have issues with two different standards in encoding and decoding QRs.

The major difference lies in the fountain encoding used when encoding the UR into QR parts. This is not strictly necessary to implement however - if the minimum number of parts is specified, no fountain encoding needs to be performed. Nevertheless there seems to be some misunderstanding around UR2.0, so this discussion is an attempt to clear that up and determine whether there is broad support to move to UR2.0.

For those projects still on UR1.0:

1. Is there an intent to move to UR2.0 soon?
2. If not, are there specific concerns with UR2.0?

Reference:
UR1.0
UR2.0

[Overtorment]

for bluewallet, our main goal was compatibility with cobo. and to be precise, cobo team was the one who sent a pr to add UR support, so I personally didnt dig too much into specs.
quickly grepping our code, we mostly use decodeUR, and it almost always comes from cobo. and practicly always bytes. example in our test suite: https://github.com/BlueWallet/BlueWallet/blob/01aed8d00fff4a91224cc50261254dc9aa952665/tests/unit/multisig-hd-wallet.test.js#L139
we use implementation by cobo team, this one: https://github.com/CoboVault/cobo-vault-blockchain-base/tree/master/packages/bc-ur
but we forked it a long time ago so we use a pretty dated version, so Im waiting for cobo team to ping me that its time to update the fork.

im in touch with @stepansnigirev (specter), so he is basically closely watching what cobo (as animated qr firstmover) is doing to keep interoperability.

and from our talks with foundation devices team (im really excited about theit forthcoming devices) they are going to have interoperability with bw (and possibly cobo?) at the start, so that means UR encoding as well. which version - good question.

so my main question is, what about backward compatibility? say, cobo team updates their UR lib, and we update from their upstream. will we have to write hacks, some iffs or smth to determine which UR version we have just scanned? because on the other end, it might be really dated cobo device with URv1 qrs, or potentially a fresh one, with URv2 qrs.

and vice versa. say, a bit dated bw version, and user has a neat fresh cobo device on him, with potentially URv2 qrs. i assume this just wont work?

[aaronisme]

when we integrate with bluewallet and specter in June. we found UR, version 1.0 at that time. we think it is a great way to encoding the data to QR codes so we adopt it during our integration with bluewallet and specter. actually, we are planning to adopt UR on our own companion app.

currently, UR has been upgraded to UR2.0, which introduce fountain encoding, I think it can provide lossy transmission, but it also introduces redundancy which will probably large the QR images number. this is one of our concern.

and another question is about backward compatiblity, which @Overtorment metioned above.

Techically, from implemenation, UR2.0 scaner can read UR1.0 data which @craigraw have metioned in the above.
for now and for Cobo Vault, I think we will stick to our UR1.0. for UR2.0 we will keep eyes on it and we will plan to move to it in the future. Meanwhile we are happy and liseten to commnuties idea.

[ChristopherA]

@aaronisme wrote:

currently, UR has been upgraded to UR2.0, which introduce fountain encoding, I think it can provide lossy transmission, but it also introduces redundancy which will probably large the QR images number. this is one of our concern.

for now and for Cobo Vault, I think we will stick to our UR1.0. for UR2.0 we will keep eyes on it and we will plan to move to it in the future. Meanwhile we are happy and liseten to commnuties idea.

I'm really hoping you'd reconsider this decision. There is no 1.0 version — it was intended as a demo to see if wallet teams were interested, and we are quite pleased there was interest, thus the official UR version.

I would really like all of us to be on the same page for animated QRs. Having some people on the demo version vs the final version of the UR code will cause confusion with other companies, and with getting adoption of UR QR codes in web-based coordinators like Caravan, River, etc. which means they might not support either.

Also, there seems to be some misunderstanding — you don't have to turn on fountain codes if you don't want to. But we do want to apply Postel's Law of internet protocols here

Postel's law, after Jon Postel, who wrote in an early specification of TCP: TCP implementations should follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others.

Given that, I am are fine if you don't want to send with fountain codes for some reason. But you should always accept them if they are required by the other device.

Separately, there seems to be some concern that there is too much "redundancy". There is some redundancy when using the fountain codes, but I don't think it is that significant as compared to the cost of missing the first frame and having to wait until the end and repeat to get all the frames you need. I can have @wolfmcnally offer more details on the "redundancy" costs if you are still concerned.

But again, if you find you don't need fountain codes, allow users to turn them off. But please do use the current standard.

-- Christopher Allen

[wolfmcnally]

There is nothing "lossy" about fountain codes in UR 2.0. "Lossy" encodings like JPEG literally throw away data. UR encoding is always lossless.

If you want to use the minimum number of codes, the API tells you when you have generated them. The minimum number of codes encodes the entire message without actually using fountain codes. Only if you continue to generate codes beyond the minimum does fountain encoding actually start. The intent here was that if you want to let the user advance through each QR code manually, you don't really need fountain codes, so we support this directly. But if you want to animate the QR codes and let the user point their camera at them at any point in the stream and always receive the full message in a reasonable amount of time (allowing for missed codes) then fountain codes are also there.

The UR 2.0 implementation can not read UR 1.0 codes and is not in any way backwards compatible. UR 2.0 uses Bytewords, while UR 1.0 uses BC32. They are entirely different encodings.

[zachherbert]

I want to express our agreement with @ChristopherA; we have a unique opportunity here to define the standard for the Bitcoin industry around transmitting airgapped data. The first UR version was intended as a demo, and UR2.0 is the official version supported by Blockchain Commons and many different projects.

If we as an industry can converge around UR2.0, then it will be a no-brainer for every Bitcoin service, wallet, and device to introduce support. We can approach exchanges like River, banks like Avanti, wallets like BlueWallet, services like Unchained Capital and Casa, and device makers – and get everyone on board.

If we are already fractured into two groups, with one supporting a pre-release demo and the other the official version, this will create a lot of confusion and significantly slow adoption. Many will likely wait on the sidelines to see what standard wins in the market.

Let's figure out how to get the apps and devices using UR1.0 transitioned to UR2.0 right away, so that we can introduce this standard to the broader industry.

Please share your thoughts @aaronisme @Overtorment @stepansnigirev. Thanks!

[ChristopherA]

Let's figure out how to get the apps and devices using UR1.0 transitioned to UR2.0 right away, so that we can introduce this standard to the broader industry.

If there are missing library issues preventing people from moving to UR, let us know.

For instance, there is a new Java/C++ for Android library from Bitmark's @jollyjoker992 needing review at
https://github.com/jollyjoker992/bc-ur-java

[Overtorment]

is that correct, that when scanning QRs its not a problem to distinguish between ver1 & ver2 ? so backwards compatibility here is a nobrainer

[wolfmcnally]

There is no version field to check, so it's not that simple, but version 1 uses BC32 to encode while version 2 uses Bytewords. Bytewords uses only the 26 English letters, so the absence of numerals is a strong sign that you're dealing with version 2. It's impossible that a version 1 decoder can read a version 2 message and vice versa, so if you just try both, the one that works is the version you're dealing with.

[Overtorment]

so basically, apps can update their readers to support both, and wait for everyone else (like cobo vault, who is usually the one to produce qrs) to update.
once everyone including cobo are up to date it should be safe to switch off v1

[craigraw]

Sounds like a plan!

[craigraw]

I've added a UR1.0 LegacyUREncoder and LegacyURDecoder to the Hummingbird Java UR library. Sparrow thus now supports Cobo Vault, but these classes are marked as deprecated and will be removed in a future release.

The code it uses to detect whether the fragment is UR1.0 or not is below:

    public static boolean isLegacyURFragment(String fragment) {
        String[] components = fragment.split("/");

        //Multi-part legacy encoding must include digest which adds an extra component
        if(components.length > 3) {
            return true;
        }

        //Last component is always fragment payload in both legacy and current
        String payload = components[components.length-1];

        //BC32 will never contain the following characters
        if(payload.indexOf('b') > -1 || payload.indexOf('i') > -1 || payload.indexOf('o') > -1) {
            return false;
        }
        
        //Bytewords and BC32 strings can usually be distinguished because the latter includes digits as permissible characters
        return (payload.matches(".*\\d.*"));
    }

[ChristopherA]

To be clear, Blockchain Commons does not plan on supporting the old version, but does plan on the current version long-term. We also will likely be working toward a BIP, and we will not be documenting the old version.

[aaronisme]

After Internal discussions and Consideration, For Cobo Vault, we have added the upgrading to UR2.0 in our RoadMap and plan to do it in April 2021, but there is indeed some work we need to coordinate with other applications like BlueWallet and Specter.
we are willing to work with the whole community to push UR forward :)

[Overtorment]

nice! dont forget to ping me when you update your js lib, we use it!

[aaronisme]

Updates: The keystone wallet has fully upgraded to UR2.0. Let's move UR forward

[Overtorment]

dont forget to test bluewallet compatibility :-)

[Fonta1n3]

@Overtorment @craigraw The other day I was adding better compatibility between various wallets and Fully Noded. I had raised issue #4032 on Blue Wallets repo because I ran into what I thought was a misuse of crypto-account. Its not really a BW issue so I closed it, it is more an issue with crypto-account itself.

My main confusion here was that crypto-account (which is what you see when you scan the BW QR in the above issue) is, as per the spec, supposed to consist of an array of crypto-output; however cosigners are not output descriptors and are certainly not crypto-output (if one follows the spec). The entire purpose of an output descriptor is to describe an entire wallet not just a cosigner.

I understand the button that produces the crypto-account in Blue Wallet is designed to export a public key cosigner for that multisig quorum, in my opinion a crypto-hdkey is much better suited for this. And I would suggest that crypto-account drop all usage of crypto-output for multisig cosigners and instead utilize crypto-hdkey which is perfectly suited for cosigners.

It would be nice to see more complete adoption of UR, its not meant to be just for cosigners. Its also meant to be a cross wallet compatibility for backups/imports of entire wallets. solving the issue of wallet developers having to cater to every single wallet standard (total nightmare) and for users only having one backup that works with one wallet. Using UR only to share cosigners is a very narrow use case.

We (as an industry) need excellent cross wallet compatibility for wallets. The ideal UR type for this is (in my opinion) crypto-output (e.g. an output descriptor in UR).

It would be great to see all wallets implement a plaint text output descriptor export option and the UR crypto-output equivalent for each wallet (can be public or private key based). Seeing as output descriptors can unambiguously describe every wallet type imaginable we ought to take advantage of them.

[Fonta1n3]

IMO I think nothing needs to be changed as far as what has been implemented in existing apps. But we ought to clarify in the crypto-account spec that handling of the multi-sig scripts types should be a special case and is not a complete output descriptor.

I think we should create a new crypto-cosigner type for this specific use case as it does come in handy.

Generally when someone implements UR they will parse each UR type independently, so if crypto-output is nested in crypto-account the current msig derivations can be misinterpreted as single sig descriptor with a multi-sig script type. Just another point the term "account" to me implies you can derive an address and receive btc to the account, but you can't with cosigners as they are incomplete in nature.

crypto-hdkey is script type agnostic. Developers could either deduce the script type from the derivation path or ask the user which script type they'd like to use for that cosigner.

[craigraw]

@Fonta1n3 your objection seems to be mainly that you dislike the use of crypto-output in it's role within crypto-account since a single cosigner does not represent a full output descriptor - I understand this view. The alternative is however to define an entirely new format to hold exactly the same information, which duplicates the spec and the code and seems redundant to me. For example, the hummingbird UR implementation reuses the code for crypto-output within crypto-account.

I would suggest that crypto-account drop all usage of crypto-output for multisig cosigners and instead utilize crypto-hdkey which is perfectly suited for cosigners.

When using crypto-hdkey alone, there is no way to indicate which HD key corresponds to which script type. This means choosing and then sharing key information for one script type only, and for that choice of script type to then be lost in the encoding. Generally we are sharing from a limited cosigner to a more capable coordinator, so this is the wrong way around from a UX PoV. Allowing the script type to specified within the data format gives a great deal of flexibility to the UX design - crypto-account can used to contain information for one key, all single sig keys, all multisig keys etc - this makes it suitable for a wide variety of applications and interface designs, and to be forward compatible in a way that allows both sides to change which script types they support independently.

It would be nice to see more complete adoption of UR, its not meant to be just for cosigners.

Indeed, one all the cosigners have been added, crypto-output should be used to share the full wallet description with the cosigners. The use of the format within crypto-account certainly does nothing to limit that.

I will also note that Keystone have already implemented crypto-account, and Passport have indicated they intend to do so soon.

I think we should create a new crypto-cosigner type for this specific use case as it does come in handy.

Again, this seems redundant to me. It is difficult enough to get adoption of standards - increasing the number of them unnecessarily (in my view) only makes it harder.

[craigraw]

But we ought to clarify in the crypto-account spec that handling of the multi-sig scripts types should be a special case

Happy to do this.

[Fonta1n3]

I guess what it boils down to is HWW and software wallets which use crypto-account need to know that they ought to handle the single sig descriptors with multi-sig script types as cosigners and not actual accounts. Clarifying that in the docs would be prudent. Thanks @craigraw 🙏

[craigraw]

PR added: BlockchainCommons/Research#90