Discussion: Use plaintext TCP connection instead of STCP

[crypto-ape]

I would like to start a discussion about the need for secure connections.

We are using a custom 'Secure sockets (STCP socket class)' implementation that performs EC Diffie–Hellman key exchange and sets up an AES-encrypted stream between two peers using the shared secret.

If I am not missing anything, this approach does nothing but hides the traffic between the two peers.
This seems to me like something we don't need, because the data that flows between the peers is all public information.
Neither does this protect the peers against MITM attacks.

All this at some performance cost of encrypting/decrypting the data. Sure, this may be alleviated on platforms that support hardware AES instructions.

Removing one level of abstraction in the code base may be also an appealing motivation.

To sum it up,
We could replace STCP connections with pure TCP connections.

Reasons for removing STCP:

• looks like it is not needed
  • the data that flows through is already publicly available
• removes the performance cost associated with AES crypto
  • makes running lightweight nodes on embedded platforms interesting
• simpler network debugging (inspecting the flow on the network)

Reasons for further discussion:

• backward compatibility with clients expecting STCP connections (I already have an idea)
• maybe I am missing some reasons why we need STCP

[clockworkgr]

@dnotestein you're probably the best person to add some insight on this

[jmjatlanta]

We're crypto people we should not be removing crypto, we should be adding more More MORE! :-D

There have been "light" discussions about optimizations that could take place if peers in the p2p layer were authenticated. Off-loading the encrypt/decrypt to a separate thread has also been talked about recently ( note @pmconrad's presentation at Bitfest about performance spoke of it as an idea ). From his testing, crypto uses a good chunk of resources, so would be a candidate to be off-loaded.

Here are some ideas to ponder. Note: These are off the top of my head, so not fully baked:

A busy API node wants to hear P2P traffic, but only from witness nodes he trusts.

A "premium" node wants to accept P2P traffic from nodes that pay for "premium" access.

How can they assure the node on the other end is legit, and not an imitation or MITM?

Now, I agree that the crypto used at the moment serves little purpose other than to make it difficult to read as it goes over the wire. And I hope the points of the OP spurs useful discussion. My utopia is to let nodes decide what kind of traffic they want to send and receive, expand the usefulness of p2p connections, and give node operators the tools to let them do what they want.

[pmconrad]

We're crypto people we should not be removing crypto, we should be adding more More MORE! :-D

This!

One thing the P2P encryption is good for that it makes traffic analysis harder for an opponent who is listening to the network. Traffic analysis could help identify block signing nodes for example.

Peers don't currently authenticate because we don't have a secure means for sharing keys, but they could.

Performance is not that much of a problem. The initial key exchange is expensive, but the AES stuff used afterwards is comparatively cheap (and I think it is handled by separate threads, but not sure).

[ryanRfox]

Thank you @crypto-ape for raising this topic. The P2P layer is too often overlooked. It is the basis for which trust in the protocol and the transactions traversing the network are established.

I feel the BitShares P2P code should be reviewed and explored for potential optimizations, including security, efficiency an node operation. My understanding of the current BitShares P2P implementation using STCP connections does mitigate against potential DDoS vectors currently existing within Bitcoin Core nodes as documented in "An evaluation of the security of the Bitcoin Peer-to- Peer Network" (2018 Tapsell, et al. PDF). The paper concludes with two suggested topics for further research:

• Protocol hardening
• Management of network infrastructure

A series of BIPs define Peer Services improvements that would mitigate the first:

• Peer-to-Peer Communication Encryption (2016 Schnelli BIP151)
• Peer Authentication (2016 Schnelli BIP150)
• Other BIPs in the range (150-159)

I feel BitShares should become familiar with the P2P specifications offered in the BIPs and consider adopting certain aspects thereof. Authenticated BitShares peers, starting with the seed nodes, would provide the basis for a more robust P2P network.

[crypto-ape]

First of all,
thank you guys for your reactions.

I have identified some shared themes among your replies, let me go through them:

Premium nodes / Trusted nodes

Don't see how encrypted AES channel would help here.
The data is public, the network is public.
If anything, you would like to put trust into an entity (which would be able sign the data).

Talking about premium nodes. You would want to sign the request,
so the node would know that the request came from an authorized peer.
Maybe I am missing something, but I assume that the data is again public (no need for channel encryption),
but the resource is at premium (prioritized access to the node).

In case of trusted nodes, you as the node would sign your outgoing data, so that the receiver knows
that you can be trusted.
Still, the same principle holds - we don't need an encrypted channel because the data is public.

You have to keep in mind that right now we don't have a support for any trust model.
In current implementation an interested party could perform MITM attack.
Even if we have implemented some trust model,
the encrypted channel would not be a necessary part of it.

The data we transfer is public. There's nothing to encrypt. We are transparent.

Performance / Offloading crypto to some other thread

The more interesting question is how to parallelize efficiently.

We have to discriminate between two uses of crypto in our system.
The symmetric AES that encrypts the network channel,
and the asymmetric crypto doing all the signing and verification of items.

I haven't tested it rigorously yet, but I would make a qualified guess
that the AES part has smaller performance impact on the system
than the asymmetric crypto.
On the other side, the AES encryption/decryption seems not to be parallelizable.

The network channel AES crypto is already performed asynchronously.

The asymmetric crypto is performed synchronously and parallelizing this part makes
a lot of sense (replaying blockchain, verification of items, ...).

The performance of the network channel decryption/encryption
could not be improved and may be a bottleneck for lightweight systems.

Is @pmconrad's presentation about performance available for download?

Easier to do traffic analysis

Sure, this is the other side of the ability to debug it easier :)

But the interesting question is if the plaintext network channel would
introduce some new potential attacks or identification schemes.

I don't think so. Using our current encryption scheme does not protect against anything.
If you could do some identification attack in plaintext,
the same vulnerability could be exploited right now.

@pmconrad

Traffic analysis could help identify block signing nodes for example

Could you please elaborate on this?

• How would this be performed?
• Isn't it a problem already in our current implementation?
  • Don't forget, that any interested party could do a MITM at the moment
    and the same info would be revealed as if it was sent in plaintext.

Comparison to BitCoin

I have read the article that @ryanRfox mentioned and I don't think
we can apply the conclusions from the article to our case.
The article analyzes weaknesses in BitCoin's protocol design to perform a targeted DDoS using an amplification scheme.

Regarding the BitCoin Proposals 150 and 151.

They try to create an encrypted communication (BIP-151) that allows for peer authentication (BIP-150).

Taking from the proposals,

The Bitcoin network does not encrypt communication between peers today. This opens up security issues (eg: traffic manipulation by others) and allows for mass surveillance / analysis of bitcoin users. Mostly this is negligible because of the nature of Bitcoin's trust model, but...

Analyzing the type of p2p communication would still be possible because of the characteristics (size, sending-interval, etc.) of the encrypted messages.

Encrypting traffic between peers is already possible with VPN, ...

It looks like even their mode of encryption would not solve the issues we are discussing.

Both of the proposals were introduced in 2016-03 and are still Drafts.

@ryanRfox

Authenticated BitShares peers, starting with the seed nodes, would provide the basis for a more robust P2P network.

This might be a good idea, but even though we decided to implement
some authentication scheme, the network channel does not have to be encrypted.

@ryanRfox

I feel the BitShares P2P code should be reviewed and explored for potential optimizations, including security, efficiency an node operation.

This is something I am interested in and may contribute to, if planned.

Finally,

I want to stress that all the data that flows between the nodes is public data.
All the blocks, transactions, peer lists. Everything is publicly available and
peers should provide the same data to everyone.

I like @jmjatlanta's 'utopistic' idea, that the peer would be able to decide
what kind of communication does it want to have. This is doable.

Once again, thank you guys,

I hope we will continue having a fruitful discussion.

[pmconrad]

Is @pmconrad's presentation about performance available for download?

http://dev.bitshares.works/en/master/references/info_bts_events.html

Could you please elaborate on this? [traffic analysis]

Suppose an adversary can monitor all network traffic. Obviously he can determine which node was the first to announce a given block, and consequently that node must be the one running the witness who signed that block.

I'm not claiming that P2P encryption provides perfect protection against this, but it makes this much harder.
Your argument about MITM is correct in principle, but an attacker would have to place a MITM between all node connections, which is much more difficult to pull off than "only" snooping on the network.

I want to stress that all the data that flows between the nodes is public data.
All the blocks, transactions, peer lists. Everything is publicly available and
peers should provide the same data to everyone.

That is only correct as far as the blocks and transactions are concerned. But the data flowing across a connection contains more than that - for example, the fact that two given nodes exchanged a certain piece of data at a certain time. This part is not public and deserves protection.

I really don't see the point of this ticket. Most people would agree that internet communications should be encrypted. Encryption doesn't hurt, we're not going to remove it.

[abitmore]

I'm not claiming that P2P encryption provides perfect protection against this, but it makes this much harder.
Your argument about MITM is correct in principle, but an attacker would have to place a MITM between all node connections, which is much more difficult to pull off than "only" snooping on the network.

I agree with this.

Actually I think it should not be too hard to get rid of the MITM factor by modifying the protocol, for example, require every default seed node (a node in the default peer list) to provide a public key and hard code them. With this it's guaranteed that nobody can do MITM between a peer and a default seed node, so all peers that connected to the seed nodes might got "verified". When advertising peer lists, advertise their IP/port and the verified public keys together, so all peers will be able to verify each other, thus a trusted network can be built.

Of course there are other factors threatening the network, E.G. an interested party can

• DDoS the default peers;
• run a node and do data analysis e.g. reveal IP addresses of all peers then interrupt connections among them when possible;
• refuse to relay certain data,
• even get into the default seed node list.

But removing encryption among peers is not in the way of strengthening the network.

[pmconrad]

Another option would be to leverage DANE and DNSSEC for key provisioning.

[knaperek]

Peers don't currently authenticate because we don't have a secure means for sharing keys, but they could.

They don't and it is perfect that way. Have you even considered how much work it would be to add a reliable authentication scheme to a trustless decentralized system like this? We haven't even invented a workable solution for web yet. PKI really sucks, no doubt about that.

But why even bother with this all? I really hate to see comments like "we are crypto people, we should be adding more crypto" (I know it was a joke, but many people really think this way and it is just so sad). Adding crypto without even considering the actual threat model is just ridiculous.

So why do we not have to be concerned with encryption or authentication of the p2p sockets? It's dead simple - we already have (and need to have anyway) a robust trustless system on the application layer. Proper operation of a node is definitely not affected by wrong messages being sent to it via p2p connections. The only way (aside from DoS; but authentication nor encryption has nothing to do with it) to cripple a node is to cut off all of his connections. As long as there's at least one working connection to an honest node, it will work fine.

So what can an attacker do?
1.) "cut the cable" - no encryption or authentication helps here
2.) sniff on the data - yeah...so what? Everything is public.
3.) attempt MITM attack. Now, let's stop here for a second. If an attacker is technically capable of carrying out an MitM attack, he can already cut us off - which is already the worst thing that can happen. What else could he do? If he starts sending wrong data, we can simply ignore it (which we do already) or even drop the connection (to save resources). That's it.

Now, what remains are some concerns about traffic analysis, so let's discuss that briefly.
DPI and traffic analysis is a fast evolving field (together with AI) and there may certainly be some ISPs or transit firewalls that (try to) block certain communications. This is definitely not a standard thing on the Internet, but it may be adopted on some edge networks depending on their policy. The problem is that the techniques for traffic identification are advancing fast and it may simply not be worth it to try to keep the pace ourselves and change the protocol everytime.
If our users find themselves behind a restrictive firewall and don't have any other option, they can and should use a VPN (IPSec, OpenVPN, WireGurad, SSH,...).
It's a separate network layer that is responsible for this and it doesn't make much sense to mix it all together (it only increases the complexity and limits flexibility).

Let's take a look at Bitcoin, for example. Not only does it not use encrypted p2p communication, they even decided to drop the support for encrypted RPC connections. You could wonder why - RPC connection are not trustless so they need to be sufficiently secured if performed over a public network. Yet they dropped it. The thing is, Bitcoin Core is the reference implementation and it needs to be good at what it does. This is just a distraction. There are tons of other ways to secure a TCP communication that are specifically designed and still being actively developed for that single purpose.

One big misunderstanding about security is that adding more bloat and layers is always better. Often it is the opposite.

[pmconrad]

Stop setting up straw men. Nobody said we were going to add a reliable authentication scheme. Nobody said we're trying to keep pace with AI traffic analysis techniques. In fact the P2P layer hasn't really changed since the beginning. Nobody said we're going to add "more bloat".

Why are you so eager to have the STCP layer removed? The reasons given in the OP have already been refuted. I have not seen any convincing arguments to remove it and I'd rather not waste further time on this discussion.

[knaperek]

Hey, hold on with the emotions. I'm not setting up anything, I'm trying to hold a rational discussion. In fact I was referring to comments posted above, including yours just above my comment :-) Now your words seem contradictory to me:

Another option would be to leverage DANE and DNSSEC for key provisioning.

Nobody said we were going to add a reliable authentication scheme.

You are right that SCTP has been there from the very beginning, I know that. I just don't see the reason for it. So quite naturally I'm trying to open up a discussion in order to either find a good enough reason or else propose to remove it.

If you think this is a waste of time then I am very sorry to have bothered you.