Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor SC - balance_hash, batch unlock #39

Merged
merged 16 commits into from
Apr 26, 2018

Conversation

loredanacirstea
Copy link
Contributor

@loredanacirstea loredanacirstea commented Apr 12, 2018

fixes raiden-network/spec#35
fixes raiden-network/spec#31

fixes #28
fixes #27
fixes #26

fixes #30

Summary

(most tests are happy cases, more will be added in a separate PR)

Gas costs

Gas costs now

TokenNetworksRegistry DEPLOYMENT 3872498
TokenNetworksRegistry.createERC20TokenNetwork 2576500
TokenNetwork DEPLOYMENT 3355362
TokenNetworksRegistry createERC20TokenNetwork 2576500

TokenNetwork.openChannel 115230
TokenNetwork.setDeposit 53475
TokenNetwork.closeChannel 93951
TokenNetwork.updateNonClosingBalanceProof 88168
TokenNetwork.settleChannel 83636  # 2 token transfers
TokenNetwork.unlock 1 locks 30716
TokenNetwork.unlock 6 locks 64605
SecretRegistry.registerSecret 45414

Initial gas costs (current master branch)

TokenNetwork.openChannel 110826
TokenNetwork.setDeposit 70862
TokenNetwork.closeChannel 106721
TokenNetwork.updateTransfer 67077   # updateNonClosingBalanceProof
TokenNetwork.settleChannel 63701  # 2 token transfers

TokenNetworkRegistry DEPLOYMENT 3224456
TokenNetwork DEPLOYMENT 2734177
TokenNetworkRegistry.createERC20TokenNetwork 2100525
SecretRegistry.registerSecret 45508

- refactored data structures
- batch `unlock` after `settleChannel`
- balance hash in the balance proof
- refactored `settleChannel` logic

raiden-network/spec#35
raiden-network/spec#31
@LefterisJP
Copy link
Contributor

Performing a review of this PR now.

@czepluch
Copy link
Contributor

I am also reviewing right now. Forgot to write when I started(sorry), but I think two people taking a look at this is okay.

Copy link
Contributor

@czepluch czepluch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me. Only have some small comments, but nothing important for now. I approve, but let's wait for Lefteris' review too.


/*
* Events
*/

event SecretRevealed(bytes32 secret);
event SecretRevealed(bytes32 indexed secrethash);

function registerSecret(bytes32 secret) public returns (bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a docstring to this function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 0634caa

bytes32 balance_hash_or_locksroot;

// Nonce used in updateNonClosingBalanceProof to compare balance hashes during the
// settlement window. This is replace in `settleChannel` by the total amount of tokens
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace -> replaced

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 0634caa

uint248 settle_block_number;

// Channel state
// 1 = open, 2 = closed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess we could use an Enum here to make it more verbose, but I am fine with either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, was also thinking about this but I decided to leave it like this for now. I am planning to review the data structures again after implementing the rest of the features & writing some more tests.

Copy link
Contributor

@LefterisJP LefterisJP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big PR so lots of comments from my side

// The latest known merkle root of the pending hash-time locks, used to
// validate the withdrawn proofs.
bytes32 locksroot;
uint240 deposit;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why uint240 here and in the other places where deposit is used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was thought as an optimization for gas usage. The other two variables initialized and is_closer are packed with deposit into a single slot of 32 bytes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this optimizes for gas succesfully and is it worth the discrepancy of using a different uint type? How much gas does this save?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened #46
not part of this PR imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed but this PR already makes an optimization by introducing a uint240 for deposits and a uint248 for block numbers.

And just because I agree it would make more sense to move this change, as per your issue, out of this PR.

// This is a value set to true after the channel has been closed, only if this is the
// participant who closed the channel.
// This is bytes1 and it gets packed with the rest of the struct data.
bool is_closer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you rename the variable to is_the_closer? is_closer can have a very different meaning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed a2e4768

bytes32 balance_hash_or_locksroot;

// Nonce used in updateNonClosingBalanceProof to compare balance hashes during the
// settlement window. This is replace in `settleChannel` by the total amount of tokens
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace -> replaced

// and settling the channel.
// After the channel has been uncooperatively closed, this value represents the
// block number after which settleChannel can be called.
uint248 settle_block_number;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why uint248 here and in all the other places where you count block durations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reason as for deposit.
settle_block_number gets packed with state into a single 32 bytes storage slot -> less gas.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we decide to use uint256 for block numbers on issue #2 to avoid packing/unpacking?

0634fa0#diff-5d5d8d0b58307a4b1624f680cbd59cbcR11 (btw, the commit message is lacking)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hackaugusto The case is different - in the referenced issue there was no reason to downcast and it actually increased gas usage. Here, it actually saves gas - a lot more than it consumes for type casting

Copy link
Contributor

@hackaugusto hackaugusto Apr 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, so I misunderstood your test on the issue then.

Just to clarify: In solidity values of a mapping are rounded up to an EVM word? Meaning that we cannot save space for primitive types, but we can save space for compound types?

Copy link
Contributor Author

@loredanacirstea loredanacirstea Apr 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(prev comment was incorrect or things changed recently)
Referencing the Warning text from http://solidity.readthedocs.io/en/v0.4.23/miscellaneous.html#layout-of-state-variables-in-storage

You can save space for primitive types storage if you have multiple small size primitive types. But they need to be defined one after another.

E.g.

bytes2 a;
bytes2 b;

defined in the common contract storage will get packed together.

But mapping(bytes32 => uint64) public secret_to_block;
will save the uint64 in a storage slot defined by the mapping key -> values cannot be packed together because they can be all over the place.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if it's worth optimizing that much for gas as we end up optimizing for the current solidity version. The way things are packed/stored can change in the future as has happened before.

How much gas do we actually save with this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #46
Imo, don't think this should be part of this PR.

uint256 channel_identifier,
address participant,
uint256 unlocked_amount,
uint256 returned_tokens);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closing paragraph in its own line

from functools import reduce
from collections import namedtuple
from web3 import Web3
from ..utils.config import SETTLE_TIMEOUT_MIN
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same thing here on relative imports as before


def get_packed_transfers(pending_transfers, types):
packed_transfers = [encode_abi(types, x[:-1]) for x in pending_transfers]
return reduce((lambda x, y: x + y), packed_transfers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could also use the aforementioned helper function here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That helper function did something else. This is (should be) just used here.

return reduce((lambda x, y: x + y), packed_transfers)


def get_settlement_amounts(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small docstring explaining what this helper function returns would be helpful here

return MerkleTree(layers=[[EMPTY_MERKLE_ROOT]])

if not len(leaves) == len(set(leaves)):
raise ValueError('The items must not cointain duplicate items')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The items -> The leaves items andcointain -> contain


tree = [leaves]
layer = leaves
while len(layer) > 1:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recognize this pattern. This is the one we were discussing in the chat in the morning and we have it in the raiden code. Can you make a note somewhere to get it abstracted it and put it in the common raiden libs and use it both in raiden and here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already in raiden-libs but without the comment you added yesterday, so I opened raiden-network/raiden-libs#50
I really wanted to not have dependencies on that repo unless necessary. So, I know about it and I will look to see if I need to reuse anything else. If yes, I'll do it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also opened #42 to track common components with raiden-libs (which I will probably end up using)

// the channel is open
uint256 settle_timeout;
mapping(address => Participant) participants;
struct BalanceData {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not two struct definitions? the combination balance_hash locked_amonut seems invalid. Does solidity have ADTs/tagged union types?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what about renaming it to BalanceProofData?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not two struct definitions?

In terms of gas cost, you will end up with 10k more gas when calling settleChannel
now: 5k * 2 (2 SSTORE for modifying the 2 variables in BalanceData) = 10k
2 structs: - 10k * 2 (deleting 1 struct with balance_hash, nonce) + 20k * 2 (adding the 2nd struct) = 20k

But if the logic is not clear enough, it can be considered.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what about renaming it to BalanceProofData?

It is BalanceProofData for balance_hash and nonce and BalanceData for locksroot and locked_amount. At least this is how we agreed in raiden-network/spec#54 .
If we combine them in 1 struct, as it is now BalanceData seems more general to me + it's shorter. But if we get another opinion saying otherwise, I'll change.

Copy link
Contributor

@hackaugusto hackaugusto Apr 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms of gas cost, you will end up with 10k more gas when calling settleChannel
now: 5k * 2 (2 SSTORE for modifying the 2 variables in BalanceData) = 10k
2 structs: - 10k * 2 (deleting 1 struct with balance_hash, nonce) + 20k * 2 (adding the 2nd struct) = 20k

What is adding the 2nd struct? I thought that would be the two SSTOREs.

Edit: I'm okay with the gas optimization, it's an understandable trade-off. My suggestion makes more sense if the language suports tagged unions, which would just reuse the same address space.

@@ -579,74 +585,112 @@ contract TokenNetwork is Utils {

}*/

function updateBalanceProofData(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateBalanceHashData or updateBalanceProofHashData are better names IMO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are updating balance_hash and nonce = balance proof data in my opinion

updateBalanceHashData - confusing because we have balance_hash
updateBalanceProofHashData - technically, possibly more correct because signing does hashing before, but I still think updateBalanceProofData is ok and shorter. And again, I don't want confusion with balance_hash

If I say updateBalanceProofData - what does it mean to you?

Copy link
Contributor

@hackaugusto hackaugusto Apr 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I say updateBalanceProofData - what does it mean to you?

I think about the whole balance proof transferred amount, merkleroot, nonce, etc.

And again, I don't want confusion with balance_hash

It's exactly because the function is updating the balance_hash that I gave that suggestion (and because I would call it balanceproof_hash :P ):

balance_state.balance_hash_or_locksroot = balance_hash;

What about updateBalanceHashAndNonce? But your naming makes sense, it's mainly my problem when I think balanceproof_hash

nonce,
transferred_amount,
locksroot,
additional_hash,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we also add the closing_signature here? (havent thought much about it, not sure what are the benefits/drawbacks)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not see the point in including the closing_signature here.
Not including it keeps the code more dry (1 function for all balance proof address recoveries)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both signatures are required. The first signature proves the balance proof was sent by the channel participant and it's valid, the second proves the balance proof was validated and allows delegate calls. If the closing signature is not included this attack is possible:

  • The attacker makes sure to never send a balance proof with a nonce higher than the counter party's latest balance proof, and to exhaust its capacity from the channel (this is really easy to do, just wait for the counter party to use the channel twice and send a single payment that uses all the available balance).
  • Then the attacker closes the channel. It will provide the latest valid balance proof from the counter party to properly close it.
  • Then the attacker gets the counter party balance proof with a nonce equal to its latest balance proof nonce + 1, signs it, and calls updateNonClosingBalanceProof like it was a monitoring service. The balance proof and signature are valid, but for the wrong direction, and because the attacker took care to never produce a balance proof with a higher nonce the other node cant overwrite it.

The wrapping of the signatures is what proves the direction of the balance proof.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, yes, agreed. Will be changed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here 40388b3

// Make sure the second signature is from the non-closing participant
require(closing_requests[channel_identifier].closing_participant != non_closing_participant);
// Make sure the signature is from a channel participant
require(non_closing_participant_state.initialized);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a signature of a channel participant is not sufficient, it needs to be a signature of the non_closing party, otherwise the closer can double sign and call this with forged data.

please add require(!closing_participant_state.is_closer);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

humm, well, this is enforced by require(closing_participant != non_closing_participant);, so perhaps just change the comment?

Also, have you considered keeping the invariants grouped? I think it is easier to verify them if we don't need to jump around in the contract code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have:

        // Make sure the signatures are from different accounts
        require(closing_participant != non_closing_participant);
        [....]
        // Make sure the first signature is from the closing participant
        require(closing_participant_state.is_the_closer);

Isn't this enough?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keeping the invariants grouped

Would definitely be nice.
I try to do that whenever possible, but with require (uses REVERT opcode), the address that calls the function receives back the gas corresponding to the rest of the code after the failed require. So this means placing the require as soon as you can helps to not burn gas/eth.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this should not take priority over clarity if the input is invalid, only in cases where the input is valid but a race happened (e.g. two monitoring services called update at the same time, only one of the transactions will work)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to change it in this PR but I agree with Augusto. Clarity should be what we aim for in smart contract code rather than providing tiny bit more gas back with REVERT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keeping track of this here #47

address participant2)
isClosed(channel_identifier)
timeoutOver(channel_identifier)
uint256 participant1_transferred_amount,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cant we use a compound type here? something like:

struct BalanceProof { address, uint256, uint256, bytes32 }
function settleChannel(uint256, BalanceProof, BalanceProof)

I guess this would be useful only if it helps with the stack too deep error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know. I cannot use struct yet with the current solc version that we are using. It is possible after 0.4.22, but we are waiting for it to be stabilized & the new ABI encoder changes to be introduced in web3 (neither js nor py have integrated dynamic types or structs as function arguments)

I will revisit this anyway after functionality & features are finished.

require(token.transfer(participant, unlocked_amount));

// Transfer the rest of the tokens back to the partner
if (returned_tokens > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please remove this guard or add it to the settle function too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I noticed I did this. I will add it to the settle function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha! :)

// and settling the channel.
// After the channel has been uncooperatively closed, this value represents the
// block number after which settleChannel can be called.
uint248 settle_block_number;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we decide to use uint256 for block numbers on issue #2 to avoid packing/unpacking?

0634fa0#diff-5d5d8d0b58307a4b1624f680cbd59cbcR11 (btw, the commit message is lacking)

// and settling the channel.
// After the channel has been uncooperatively closed, this value represents the
// block number after which settleChannel can be called.
uint248 settle_block_number;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// @notice Called on a closed channel, the function allows the non-closing participant to
// provide the last balance proof, which modifies the closing participant's state. Can be
// called multiple times by anyone, as long as they provide signatures from both participants.
// called multiple times by anyone.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/consisten/consistent/

reviews on comment reviews are also a thing?

el := mload(add(merkle_proof, i))
bytes32[] memory merkle_layer = new bytes32[](length / 96 + 1);

for (i = 32; i < length; i += 96) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bytes is a variable length type, it's more likely the length of the merkle_tree_leaves input.

As requested in raiden-network#39 (comment)
Note: these optimizations went hand in hand with other data structure decisions, this will need to be assessed in raiden-network#46
added_deposit = total_deposit - participant_state.deposit;

// Change the state
// Update the participant's channel deposit
participant_state.deposit += added_deposit;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not update this directly? participant_state.deposit = total_deposit ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of it more like a measure of security - increase with the actual amount used in the token transfer. If proper checks are in place, either way should be fine.

Copy link
Contributor

@LefterisJP LefterisJP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my comments have been addressed.

As mentioned in raiden-network#39 (comment), we need the non-closing participant to also sign the closing participant's signature, to avoid replay attacks on updateNonClosingBalanceProof.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants