Node Roles and Handshake Protocol

9 minute read
Blockchain
, , ,
Subscribe to the mailing list!

Project: Let's build Bitcoin

In the previous blog post, we implemented the API to send and receive data from the blockchain network.

In this post, we will learn about the following concepts:

  • What are node roles
  • What is a network routing role
  • What is a full node role
  • What is a miner role
  • What is a wallet role
  • How Bitcoin handshake process works

We will also implement our first messages to validate the version compatibility. Furthermore, we will start two nodes and see them exchange messages for the first time.

Node roles

A node role identifies the logic that runs on a node. There are four roles in the LearnCoin P2P network:

  • Network routing – a function of the routing node is to relay information about blocks and transactions to the blockchain network. All nodes have this role.
  • Full blockhain – a function of the full blockchain node is to validate blocks and transactions. They do so by maintaining a full blockchain with all blocks and transactions. In the early years of Bitcoin, all nodes were full nodes. The Bitcoin Core client is a full node.
  • Miner – a function of the mining node is to produce new blocks with unconfirmed transactions. Some mining nodes are also full nodes, but this doesn’t have to be the case.
  • Wallet – a function of a wallet is to send and receive coins. It may be part of the full node, which is usually the case with desktop clients. However, there are light wallet implementations that are good for mobile phones. We will talk about wallets in more detail in the future chapters.

Learn Coin node

Before we talk about various kinds of messages, let’s introduce a component that will send and receive messages and maintain the state of the blockchain. We will call this component a LearnCoin node. It will implement the network routing and full node roles. We will implement the miner role as a separate process in future blog posts.

The node must connect to the network first.

// learncoin/src/learncoin_node.rs

pub struct LearnCoinNode {
    network: LearnCoinNetwork,
    version: u32,
}

impl LearnCoinNode {
    pub fn connect(network_params: NetworkParams, version: u32) -> Result<Self, String> {
        let network = LearnCoinNetwork::connect(network_params)?;
        Ok(Self {
            network,
            version,
        })
    }
}

After connecting to the network, the node repeatedly runs the event loop, which accepts new peers and processes messages from the network.

// learncoin/src/learncoin_node.rs

impl LearnCoinNode {
    // ...

    pub fn run(mut self) -> Result<(), String> {
        loop {
            let _new_peers_ignored_for_now = self.network.accept_new_peers()?;

            // Receive data from the network.
            let all_messages = self.network.receive_all();
            for (peer_address, messages) in all_messages {
                for message in messages {
                    self.on_message(&peer_address, message);
                }
            }

            // Waiting strategy to avoid busy loops.
            thread::sleep(Duration::from_millis(1));
        }
    }

    fn on_message(&mut self, peer_address: &str, message: PeerMessagePayload) {
        unimplemented!()
    }
}

Great, we have some initial logic to communicate with the blockchain network. However, nobody is sending any messages yet, so let’s fix that.

Handshake protocol

The node establishes a TCP connection at a specified port (8333 is known as the bitcoin port) to connect to a known peer. Once connected, the nodes start the “handshake” process by transmitting a version message. The version message has basic information about the node.

In our implementation, we will only ensure that both nodes are running the same version.

The version message is always the first message sent by any node. The local peer receiving the message examines the version and decides if the peer is compatible. If the peer is compatible, the local node responds with a verack message. Otherwise, the node closes the connection to the peer.

Protocol messages

Let’s begin by introducing the handshake messages to our protocol.

// learncoin/src/peer_message.rs

// ...

#[derive(Serialize, Deserialize, Clone)]
pub struct VersionMessage {
    version: u32,
}

/// Payload sent to and received from the peers.
#[derive(Serialize, Deserialize, Clone)]
pub enum PeerMessagePayload {
    Version(VersionMessage),
    VersionAck,
}

// ...

Send initial version message

A peer that initiates a connection must send the version message and wait for the verack message to complete the handshake. During the bootstrap process, the local node connects to the list of initial peers, so it must send the version message to all of them. Additionally, the node keeps track of which peers should send the verack message back in the peers_to_receive_verack_from.

// learncoin/src/learncoin_node.rs

impl LearnCoinNode {
    // ...

    pub fn run(mut self) -> Result<(), String> {
        // A peer that initiates a connection must send the version message.
        // We broadcast the version message to all of our peers before doing any work.
        self.network
            .broadcast(PeerMessagePayload::Version(VersionMessage::new(
                self.version,
            )));

        // We expect a verack message from the peers if their version is compatible.
        self.peers_to_receive_verack_from
            .extend(self.network.peer_addresses().iter().map(|s| s.to_string()));


        // ... event loop comes next
    }
}

Remember the new peers in the event loop implementation that we ignored? The node expects these peers to respond with the version message, so let’s add them topeers_to_receive_version_from.

// learncoin/src/learncoin_node.rs

pub fn run(mut self) -> Result<(), String> {
    // ...

    loop {
        let new_peers = self.network.accept_new_peers()?;
        // The local node expects the peers that initiated a connection to send the version
        // messages.
        self.peers_to_receive_version_from.extend(new_peers);

        // ...
    }
}

Finally, let’s add the vectors to the LearnCoinNode struct.

// learncoin/src/learncoin_node.rs

pub struct LearnCoinNode {
    // ... existing fields above
    // A list of peers for which the local node expects a verack message.
    peers_to_receive_verack_from: Vec<String>,
    // A list of peers from which the local node expects a version message.
    peers_to_receive_version_from: Vec<String>,
}

Process messages

All that’s left to do is process the handshake messages. The on_message function delegates messages to the message-specific handlers to keep the code organized.

// learncoin/src/learncoin_node.rs

impl LearnCoinNode {
    // ...
    fn on_message(&mut self, peer_address: &str, message: PeerMessagePayload) {
        match message {
            PeerMessagePayload::Version(version) => self.on_version(peer_address, version),
            PeerMessagePayload::Verack => self.on_version_ack(peer_address),
        }
    }
}

Version

Upon receiving the version message, the node validates the version compatibility and either responds with a verack message or closes the connection.

// learncoin/src/learncoin_node.rs

impl LearnCoinNode {
    // ...

    fn on_version(&mut self, peer_address: &str, peer_version: VersionMessage) {
        // We don't expect the version message from this peer anymore.
        let is_version_expected_from_peer = self.peers_to_receive_version_from.remove(peer_address);
        if !is_version_expected_from_peer {
            println!(
                "Received redundant version message from the peer: {}",
                peer_address
            );
            return;
        }

        let is_version_compatible = peer_version.version() == self.version;
        if !is_version_compatible {
            self.close_peer_connection(
                peer_address,
                &format!(
                    "Version is not compatible. Expected {} but peer's version is: {}",
                    self.version,
                    peer_version.version()
                ),
            );
            return;
        }

        // The version is compatible, send the verack message to the peer.
        self.network
            .unicast(peer_address, &PeerMessagePayload::VersionAck);
    }
}

Verack

Upon receiving the verack message, the handshake process with the peer is complete. The node doesn’t explicitly track the peers for which the handshake is complete. Instead, if the peer is not present in both peers_to_receive_verack_from and peers_to_receive_version_from then the handshake is complete.

// learncoin/src/learncoin_node.rs

fn on_version_ack(&mut self, peer_address: &str) {
    self.verack_message_peers.remove(peer_address);
}

Close peer connection

Let’s also provide an implementation for the function to close a peer connection. Closing the connection includes removing it from the network and cleaning up any resources allocated for that peer.

// learncoin/src/learncoin_node.rs

fn close_peer_connection(&mut self, peer_address: &str, reason: &str) {
    self.network.close_peer_connection(peer_address);
    // Free any resources allocated for the peer.
    self.verack_message_peers.remove(peer_address);
    self.version_message_peers.remove(peer_address);
    eprintln!(
        "Closed a connection to the peer {}. Reason: {}",
        peer_address, reason
    );
}

Running the blockchain for the first time

This is very exciting. We get to run the blockchain servers for the first time. Let’s start with two servers. We expect them to exchange handshake messages and agree on the version.

Let’s introduce a simple logging API that would dump the debug output for the given message.

// learncoin/src/peer_connection.rs

pub struct MessageLogger {}

impl MessageLogger {
    pub fn log<T: Debug>(prefix: &str, message: &T) {
        println!("{} {:#?}", prefix, message);
    }
}

Let’s log the payload in the send and receive functions.

// learncoin/src/peer_connection.rs

pub fn send(&mut self, payload: &PeerMessagePayload) -> Result<bool, String> {
  // ...
  MessageLogger::log("Send:", &payload);
  // ...
}

pub fn receive(&mut self) -> Result<Option<PeerMessagePayload>, String> {
  // ...
  MessageLogger::log("Recv:", &payload);
  // ...
}

Also, let’s log whenever new peers connect in the main event loop.

// learncoin/src/learncoin_node.rs

impl LearnCoinNode {
    // ...

    pub fn run(mut self) -> Result<(), String> {
        // ...
        loop {
            // ...
            if !new_peers.is_empty() {
                println!("New peers connected: {:#?}", new_peers);
            }
            // ...
        }
    }
    // ...
}

Let’s finally run server #1.

./learncoin server --address 127.0.0.1:8333

New peers connected: [
    "127.0.0.1:60401",
]
Recv: Version(
    VersionMessage {
        version: 1,
    },
)
Send: Verack

As expected, nothing has happened yet. As soon as the peer connects, we expect them to exchange version and verack messages. The peer initiating the connection should send the version message, and the peer accepting the connection should respond with the verack.

Let’s now run server #2.

./learncoin server --address 127.0.0.1:8334 --peers "127.0.0.1:8333"

Send: Version(
    VersionMessage {
        version: 1,
    },
)
Recv: Verack

The server running at port 8334 sends the version message and receives the verack message back. The exchange of messages completes the handshake process.

You may notice that the port of the connected peer is 60401 rather than 8334 in the first output, even though we know that the connecting peer is running (listening) at 8334. However, when we initiate a new TCP connection, it has nothing to do with the address at which the server is listening. They are independent. What happens here is that the loopback device assigns some port to each incoming connection.

This is very cool. Our nodes can talk to each other successfully. However, they don’t do much at this point, but we will be addressing this soon.

Peer state

Before we conclude with this post, let’s refactor our code to improve maintainability. We’ve introduced a state to keep track of which peers we expect version and verack messages, and we’ve had to remember to clean up the entries when the peer connection is closed. We would introduce more states similar to this one, so it would be error-prone to remember to clean it up, and the code would become less readable.

To avoid that, let’s introduce a new concept called PeerState which tracks everything that the local node knows about a peer.

// learncoin/src/peer_state.rs

pub struct PeerState {
    pub expect_version_message: bool,
    pub expect_verack_message: bool,
}

impl PeerState {
    pub fn new() -> Self {
        Self {
            expect_version_message: false,
            expect_verack_message: false,
        }
    }
}

We’re keeping it simple for the time being, but we will improve this code in future blog posts.

The next step is to integrate the new concept in the LearnCoinNode. I’ll leave this to you as an exercise, but here’s the code diff showing the refactor step.

Summary

In this blog post, we’ve learned about the handshake protocol. We’ve introduced new protocol messages and implemented version checks. Finally, we’ve started two servers for the first time and were able to see nodes exchange messages with each other.

Here’s the full source code.

What’s next?

In the next post, we will discuss and implement the initial block download protocol to sync a node’s local state with the network.

comments powered by Disqus

Join my mailing list

Subscribe to get new notifications about new content. It takes time to write quality posts, so you can expect at most 1-2 emails per month.