Mining Process

21 minute read
Blockchain
, , ,
Subscribe to the mailing list!

Project: Let's build Bitcoin

In the previous blog post, we’ve talked about …

In this blog post, we will implement the mining process, and learn:

  • What is a JSON-RPC protocol
  • How getblockstemplate mining protocol works
  • How miners submit blocks to a node
  • How miners know which block to mine

What is JSON-RPC?

JSON-RPC is an RPC protocol encoded as JSON. A client sends a single request to the server implementing the JSON-RPC protocol and receives a response.

A request is a call to a method provided by the server, and it contains three members:

  • method – the name of the method to be invoked.
  • params – an object or an array of values to be passed as parameters to the method.
  • id – an identifier that is included in the response, used to match it with the corresponding response.

The server replies with a response to all received requests, which contains the following members:

  • result – the data returned by the invoked method, formatted as JSON. This field is not set if the response includes an error.
  • error – an error object formatted as JSON if there was an error while invoking the method. This field is not set if the response has no error.
  • id – an identifier of the corresponding request.

Bitcoin daemon can expose JSON-RPC, but it is only safe to use by trusted machines, for example, the bitcoin CLI tool or a trusted miner.

Implementation plan

We will implement some form of JSON-RPC in this post. However, instead of encoding the data as JSON, we will be using Rust’s bin_io library.

The obvious advantage is that we can use the existing approach, i.e. peer messages, which makes it very easy to implement the above protocol. However, the downside is that the encoding depends on Rust internals and it requires the client to respect the other parts of the protocol, e.g. initial handshake. In any case, we will write code in a way that allows us to decouple the JSON RPC part in the future.

If there is a demand to implement an actual JSON encoding, please let me know in the comments.

Extend peer message protocol

Let’s extend the peer message protocol to support JSON-RPC.

// learncoin/src/peer_message.rs

// ...
pub enum PeerMessagePayload {
    // ... existing messages
    JsonRpcRequest(JsonRpcRequest),
    JsonRpcResponse(JsonRpcResponse),
}

JSON-RPC request

JSON-RPC requests have three fields: id, method and params. Our implementation uses binary encoding which allows us to encode params as part of the variant, so we don’t need explicit params field.

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub struct JsonRpcRequest {
    pub id: u64,
    pub method: JsonRpcMethod,
}

We will introduce first JSON-RPC methods when we implement the getblocktemplate mining protocol. For now, the JsonRpcMethod remains empty.

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub enum JsonRpcMethod {
    Placeholder,
}

JSON-RPC response

JSON-RPC responses also have three fields: id, result and error. However, given that exactly one of the result or error fields may be set, we can use a single field called response to represent both.

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub struct JsonRpcResponse {
    pub id: u64,
    pub response: Result<JsonRpcResult, String>,
}

The response field has a JsonRpcResult if the reequest was successful, or an error of type String describing the failure.

JSON-RPC responses may have data associated with the response, or they may be a notification. Notifications are confirmations of a successfully executed method. We don’t support any methods yet, so the JsonRpcResult only contains a Notification.

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub enum JsonRpcResult {
    Notification,
}

Process JSON-RPC messages

Similar to what we’ve done so far, we extend the on_message functino to handle JSON-RPC requests and responses.

// learncoin/src/learncoin_node.rs

// ...
impl LearnCoinNode {
    // ...

    fn on_message(
        &mut self,
        peer_address: &str,
        message: PeerMessagePayload,
        current_time: Instant,
    ) {
        match message {
            // ...
            PeerMessagePayload::JsonRpcRequest(request) => {
                self.on_json_rpc_request(peer_address, request)
            }
            PeerMessagePayload::JsonRpcResponse(response) => {
                self.on_json_rpc_response(peer_address, response)
            }
        }
}

The server never sends JSON-RPC requests, so it doesn’t expect to receive any responses. If so, the peer is misbehaving.

fn on_json_rpc_response(&mut self, peer_address: &str, response: JsonRpcResponse) {
    // Node doesn't use JSON-RPC to invoke methods on any node, so it doesn't expect any
    // responses in return.
    // The peer is misbehaving.
    self.close_peer_connection(
        peer_address,
        &format!("Received JSON-RPC response: {:#?}", response),
    );
}

There are many methods that can be executed via the JSON-RPC, so we will handle each one in a separate function. The request handler only dispatches the underlying data to the appropriate function.

fn on_json_rpc_request(&mut self, peer_address: &str, request: JsonRpcRequest) {
    let JsonRpcRequest { id, method } = request;
    match method {
        JsonRpcMethod::Placeholder => {}
    }
}

We have now implemented the basic handling of JSON-RPC protocol. Here’s the source code.

Miner

In today’s world, mining takes on two forms:

  • Solo mining, where the miner attempts to generate new blocks on its own, with the proceeds from the block reward and transaction fees going directly itself. However, the time it takes to find a block is much larger, with so many miners around the world it is practically impossible to be the first one to find a block.
  • Pooled mining, where multiple miners connect to the same pool and try to find the solution to the next block. In this approach, the miners have more hashing power which makes it more likely to find the block, but they share the reward.

Get block template protocol

Get block template is the new decentralized Bitcoin mining protocol.

The original mining protocol, called get block protocol, issues block headers for a miner to solve. The miner is kept in the dark as to what is actually in the block, and has no influence over it. All decisions are made by the pool operator or a node.

The new protocol moves the block creation to the miner, while still allowing pools and nodes to define rules for participation.

There are two methods in the get block template protocol:

  • getblocktemplate used by miner to request a block template from the node. The node responds with the JSON-RPC block template message which includes previous block hash, list of pending transactions, difficulty target, public key address to which the reward is sent, and current time since epoch. Bitcoin implementation includes a couple of more fields, but we won’t need them for our implementation.
  • submitblock used by miner to notify the node about newly mined blocks. The node responds with JSON-RPC notification message.

We will discuss public-key address in more detail when we implement locking and unlocking scripts. For now, let’s introduce a new empty struct to represent it, but we won’t be using it until later in the project.

// learncoin/src/public_key_address.rs
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)]
pub struct PublicKeyAddress {}

The methods are implemented on top of the JSON RPC protocol, so let’s introduce new messages.

// learncoin/src/peer_message.rs

// ...

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub struct BlockTemplate {
    pub previous_block_hash: BlockHash,
    pub transactions: Vec<Transaction>,
    pub difficulty_target: u32,
    pub public_key_address: PublicKeyAddress,
    pub current_time: u64,
}

#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub enum JsonRpcMethod {
    // Expects BlockTemplate in response.
    GetBlockTemplate,
    // Expects Notification in response.
    SubmitBlock(Block),
}

// ...
#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub enum JsonRpcResult {
    Notification,
    BlockTemplate(BlockTemplate),
}

Implementation plan

In this blog post, we will implement a simple solo miner that uses the get block template protocol to request the information from the node about the next block to mine. At the high level, the miner connects to the server, completes the handshake protocol because it uses the peer connection API, and then runs the never ending loop. In each loop iteration, the miner requests a block template from the node, searches for the nonce that is a solution to the next block, and submits the block if it finds one.

The miner has to make sure that it requests new block templates from the server often to avoid mining old block that may not be relevant anymore. For example, another miner may find a block while our miner is still searching for a valid nonce, in which case it will take a while before it realises that there is a new block. Also, there may be new transactions that should be included in the next block, which would result in a higher fee for the miner.

Therefore, the miner sends the get block template request as soon as it receives a response, so that there is always one request in-flight. The node would usually wait for new relevant information to arrive before sending a response, but the miner must assume that the node will respond immediately, potentially with the same information.

Proof of Work checkpoints

Let’s first extend the API of the Proof-of-Work algorithm that allows the miner to specify the range of nonce values to test. Instead of testing all 2^32 possibilities before checking if a new block template is available, the miner can test a smaller batch of nonce values (e.g. one million values). This approach reduces the time it takes miner to learn about a new block template. We don’t want the range of tested nonce values in a single batch to be too small though because the miner would be checking for new block templates too often and introduce additional overhead.

// learncoin/src/proof_of_work.rs

// ...
pub fn compute_nonce_with_checkpoint( // replaced previous line: `pub fn compute_nonce(`
        previous_block_hash: &BlockHash,
        merkle_root: &MerkleHash,
        timestamp: u64,
        difficulty: u32,
        start_nonce: u32, // new parameter which specifies the initial nonce value
        stop_nonce: u32,  // new parameter which specifies the last nonce value to test
    ) -> Option<u32> {
    let target_hash = Self::target_hash(difficulty);
    let mut nonce = start_nonce; // replaced previous line: `let mut nonce = 0 as u32`
    loop {
        // ...
        if nonce == u32::MAX || nonce == stop_nonce {  // replaced previous line: `if nonce == u32::MAX {`
            // We have run out of nonce values, stop the computation.
            break;
        }
        // ...
    }
}

// We implement the old API using the above function.
pub fn compute_nonce(
        previous_block_hash: &BlockHash,
        merkle_root: &MerkleHash,
        timestamp: u64,
        difficulty: u32,
    ) -> Option<u32> {
        let start_nonce = 0;
        let stop_nonce = u32::MAX;
        Self::compute_nonce_with_checkpoint(
            previous_block_hash,
            merkle_root,
            timestamp,
            difficulty,
            start_nonce,
            stop_nonce,
        )
    }

Connect to the server

To get the information about the next block, the miner must connect to the server and complete the handshake protocol. The miner takes the address at which the server listens for new connections.

pub struct MinerParams {
    // Address at which TCP server runs (listens for peer connections).
    pub server_address: String,
    // Size of the receive buffer for each peer connection.
    pub recv_buffer_size: usize,
}

pub struct Miner {
    connection: PeerConnection,
    is_handshake_complete: bool,
}

impl Miner {
    pub fn new(params: MinerParams) -> Result<Self, String> {
        let connection = PeerConnection::connect(params.server_address, params.recv_buffer_size)?;
        Ok(Self {
            connection,
            is_handshake_complete: false,
        })
    }

    pub fn run(mut self) -> Result<(), String> {
        unimplemented!();
    }
}

Handle handshake and JSON-RPC messages

We use the same pattern as with the LearnCoinNode, i.e. the miner reads the new messages from the server in each loop iteration. To implement the handshake protocol, the miner must send the version message first. The handshake is complete upon receiving the verack message.

Additionally, let’s handle the JSON-RPC messages too. The miner only understands handshake and JSON-RPC response messages. All other messages are ignored. The miner doesn’t know how to respond to the JSON-RPC requests.

    pub fn run(mut self) -> Result<(), String> {
        self.connection
            .send(&PeerMessagePayload::Version(VersionMessage::new(
                MINER_VERSION,
            )))?;
        loop {
            for message in self.connection.receive_all()? {
                self.on_message(message);
            }
        }
    }

    fn on_message(&mut self, message: PeerMessagePayload) {
        match message {
            PeerMessagePayload::Version(_) => {
                // The miner sends the version message, so it doesn't expect this from the server.
                eprintln!("Unexpected message from the server: {:#?}", message);
            }
            PeerMessagePayload::Verack => {
                self.is_handshake_complete = true;
            }
            PeerMessagePayload::JsonRpcRequest(request) => {
                // Miner doesn't respond to JSON RPC requests.
                eprintln!(
                    "Unexpected JSON RPC request from the server: {:#?}",
                    request
                );
            }
            PeerMessagePayload::JsonRpcResponse(response) => self.on_json_rpc_response(response),
            _ => {
                // Miner understands only JSON RPC protocol.
                eprintln!("Unexpected message from the server: {:#?}", message);
            }
        }
    }

    fn on_json_rpc_response(&mut self, response: JsonRpcResponse) {
        match &response.result {
            Ok(JsonRpcResult::Notification) => {
                // Submit request has succeeded.
            }
            Ok(JsonRpcResult::BlockTemplate(block_template)) => {
                self.on_block_template(response.id, block_template);
            }
            Err(e) => {
                // None of the requests should fail.
                eprintln!("Unexpected failed result for: {:#?}: {}", response, e);
            }
        }
    }

The miner doesn’t try very hard to deal with the malicious requests because in practice it would connect to a trusted server node.

Start mining blocks

Let’s introduce an ActiveBlockTemplate struct to store the current block template used to find a solution. The struct contains the precomputed merkle root to avoid constructing it multiple times. Additionally, the transactions are extended to include the coinbase transaction.

// learncoin/src/miner.rs

// ...

struct ActiveBlockTemplate {
    // We store the corresponding block template as a convenience 
    // mechanism to compare it against the new block templates.
    block_template: BlockTemplate,
    previous_block_hash: BlockHash,
    difficulty_target: u32,
    height: u32,
    public_key_address: PublicKeyAddress,
    current_time: u64,
    merkle_root: MerkleHash,
    transactions: Vec<Transaction>,
}

pub struct Miner {
    // ... existing fields
    active_block_template: Option<ActiveBlockTemplate>,
    in_flight_get_block_template: Option<u64>,
    checkpoint_nonce: u32,
    next_json_rpc_request_id: u64, // used later to send JSON-RPC requests
}

impl Miner {
    pub fn new(params: MinerParams) -> Result<Self, String> {
        // ...
        Ok(Self {
            // ...
            active_block_template: None,
            in_flight_get_block_template: None,
            checkpoint_nonce: 0,
            next_json_rpc_request_id: 0,
        })
    }
}

Each loop iteration starts with the miner processing reading messages from the server, which we’ve already implemented above.

After the initial handshake is complete, the miner requests new block template.

loop {
    // ...
    if !self.is_handshake_complete {
        continue;
    }

    if self.in_flight_get_block_template.is_none() {
        self.send_get_block_template()?;
    }
}

Upon receiving the new block template, the miner starts searching for the valid nonce value. If the active block template is unavailable, then the miner has nothing to do. Otherwise, the miner computes the range of nonce values to test with the Proof-of-Work algoirthm.

loop {
    // ...
    match &self.active_block_template {
        None => {
            // No block template available, keep trying.
            std::thread::sleep(Duration::from_millis(1000));
        }
        Some(active_block_template) => {
            // Ensure the stop nonce doesn't overflow.
            let start_nonce = self.checkpoint_nonce;
            let stop_nonce = if u32::MAX - NONCE_BATCH_SIZE < start_nonce {
                u32::MAX
            } else {
                start_nonce + NONCE_BATCH_SIZE
            };
            let nonce = ProofOfWork::compute_nonce_with_checkpoint(
                &active_block_template.block_template.previous_block_hash,
                &active_block_template.merkle_root,
                active_block_template.block_template.current_time,
                active_block_template.block_template.difficulty_target,
                start_nonce,
                stop_nonce,
            );

            // ... to be continued below
        }
    }
}

If the PoW finds a valid nonce, the miner submits the block and clears the active block template. i.e. removes it and resets the checkpoint nonce to 0. Otherwise, the miner stores the next nonce value as a checkpoint used to continue in the next loop, assuming there are more nonce values left.

// ... continuation

match nonce {
    None => {
        // No valid nonce has been found.
        if stop_nonce == u32::MAX {
            // The miner has exhausted all possible nonce values.
            // Drop the current active block.
            self.reset_active_block_template();
        } else {
            self.checkpoint_nonce = stop_nonce + 1;
        }
    }
    Some(valid_nonce) => {
        self.submit_block(valid_nonce)?;
        self.reset_active_block_template();
    }
}

This approach allows the miner to check for new block templates frequently without testing the same nonce for the same block template twice. In practice, our miner implementation is won’t be able to test all 2^32 nonce values before it receives the new block template even if there are no new blocks or transactions. It can test about a million nonce values every second, at which point the timestamp of the next block can be updated.

Now that we have implemented the core logic, let’s provide implementations for the remaining functions as well.

Let’s begin with the send_get_block_template function which requests the get block template from the server, keeping track of the in-flight request.

fn send_get_block_template(&mut self) -> Result<(), String> {
    assert!(self.in_flight_get_block_template.is_none());
    let id = self.generate_next_json_rpc_request_id();
    let method = JsonRpcMethod::GetBlockTemplate;
    let json_rpc_request = JsonRpcRequest { id, method };
    self.connection
        .send(&PeerMessagePayload::JsonRpcRequest(json_rpc_request))?;
    self.in_flight_get_block_template = Some(id);
    Ok(())
}

fn generate_next_json_rpc_request_id(&mut self) -> u64 {
    let id = self.next_json_rpc_request_id;
    self.next_json_rpc_request_id += 1;
    id
}

The miner updates the active block template and resets the checkpoint nonce if it receives a new block template. It expects that the response ID is the same as the in-flight request ID because the server is trusted.

fn on_block_template(&mut self, id: u64, block_template: &BlockTemplate) {
    assert_eq!(self.in_flight_get_block_template == Some(id));
    match &self.active_block_template {
        None => self.update_active_block_template(block_template),
        Some(active_block_template) => {
            if active_block_template.block_template != *block_template {
                self.update_active_block_template(block_template);
            }
        }
    }
    self.in_flight_get_block_template = None;
}

To update the active block template, the miner computes the merkle root and creates the coinbase transaction with the appropriate reward.

fn update_active_block_template(&mut self, block_template: &BlockTemplate) {
    let mut transactions = vec![Self::make_coinbase_transaction(
        Self::calculate_block_reward(block_template.height),
        &block_template.public_key_address,
    )];
    transactions.extend_from_slice(block_template.transactions.as_slice());
    let merkle_root = MerkleTree::merkle_root_from_transactions(&transactions);
    self.checkpoint_nonce = 0;
    self.active_block_template = Some(ActiveBlockTemplate {
        block_template: block_template.clone(),
        previous_block_hash: block_template.previous_block_hash,
        difficulty_target: block_template.difficulty_target,
        height: block_template.height,
        public_key_address: block_template.public_key_address.clone(),
        current_time: block_template.current_time,
        merkle_root,
        transactions,
    });
}

fn make_coinbase_transaction(
        block_reward: i64,
        _public_key_address: &PublicKeyAddress,
    ) -> Transaction {
        let inputs = vec![TransactionInput::new_coinbase()];
        let outputs = vec![TransactionOutput::new(block_reward)];
        // Safety: The constructed transaction is always valid.
        Transaction::new(inputs, outputs).unwrap()
    }

Note that we don’t make a proper use of the public-key address yet. We will do that in future blog posts.

Finally, let’s calculate a reward for the given block height. The reward for the genesis block is 50 coins, and it is halved every 2016 blocks.

const NUM_BLOCKS_AFTER_REWARD_IS_HALVED: u32 = 2016;

// ...

fn calculate_block_reward(height: u32) -> i64 {
    INITIAL_BLOCK_REWARD >> (height / NUM_BLOCKS_AFTER_REWARD_IS_HALVED)
}

And finally, let’s provide an implementation for the submit_block function. The function constructs the mined block using the valid nonce value and the metadata from the active block template.

fn submit_block(&mut self, valid_nonce: u32) -> Result<(), String> {
    let id = self.generate_next_json_rpc_request_id();
    // Safety: Active block template always exists when calling submit_block function.
    let active_block_template = self.active_block_template.as_ref().unwrap();
    let block = Block::new(
        active_block_template.previous_block_hash,
        active_block_template.current_time,
        active_block_template.difficulty_target,
        valid_nonce,
        active_block_template.transactions.clone(),
    );
    let method = JsonRpcMethod::SubmitBlock(block);
    let json_rpc_request = JsonRpcRequest { id, method };
    self.connection
        .send(&PeerMessagePayload::JsonRpcRequest(json_rpc_request))?;
    Ok(())
}

Here’s the source code that implements a miner.

Get block template support in Learn Coin Node

Learn Coin Node already supports the JSON-RPC messages, but it doesn’t support the ones specific to the get block template protocol.

Let’s start with dispatching the messages to the corresponding handlers.

// learncoin/src/learncoin_node.rs

// ...

impl LearnCoinNode {
    // ...

    fn on_json_rpc_request(
        &mut self,
        peer_address: &str,
        request: JsonRpcRequest,
        unix_timestamp: u64,
    ) {
        let JsonRpcRequest { id, method } = request;
        match method {
            JsonRpcMethod::GetBlockTemplate(get_block_template) => {
                self.on_get_block_template(peer_address, id, get_block_template, unix_timestamp)
            }
            JsonRpcMethod::SubmitBlock(block) => self.on_submit_block(peer_address, id, block),
        }
    }
}

Upon receiving the submit block request, the node responds with the notification JSON-RPC response. The node ensures that the submitted block contains a valid parent and the block hasn’t been seen before. Additionally, it relays the block to other peers.

fn on_submit_block(&mut self, peer_address: &str, id: u64, block: Block) {
    let result = Ok(JsonRpcResult::Notification);
    let response = JsonRpcResponse { id, result };
    self.network
        .send(peer_address, &PeerMessagePayload::JsonRpcResponse(response));

    if !self
        .block_index
        .exists(&block.header().previous_block_hash())
    {
        self.close_peer_connection(
            peer_address,
            &format!(
                "Miner submitted a block without a valid parent: {:#?}",
                block
            ),
        );
    } else if !self.block_index.exists(block.id()) {
        self.block_index.insert(block.header().clone());
        self.block_storage.insert(block.clone());
        self.relay_block(block);
    }
}

fn relay_block(&mut self, block: Block) {
    // TODO: Implemented in future blog posts.    
    println!("Submitted block: {:#?}", block);
}

We will implement block relaying in future blog posts. For now, the relay_block function only prints the submitted block.

The last function to implement is on_get_block_template which constructs the block template from the tip of the active chain, and responds with the block template message. We set some hardcoded values for now, but we will fix that in future blog posts. For example, we choose a difficulty of 28 since it’s not too easy for a single machine to find blocks, but not too hard either. However, the difficulty should be adjusted over time.

fn on_get_block_template(
    &mut self,
    peer_address: &str,
    id: u64,
    unix_timestamp: u64,
) {
    let tip = self.active_chain.tip();
    let block_template = BlockTemplate {
        previous_block_hash: *tip.id(),
        // TODO: Keep track of pending transactions.
        transactions: vec![],
        // TODO: Calculate difficulty target.
        difficulty_target: 28,
        height: self
            .block_index
            .get_block_index_node(tip.id())
            .unwrap()
            .height as u32 + 1,
        // TODO: Choose appropriate PublicKeyAddress.
        public_key_address: PublicKeyAddress {},
        current_time: unix_timestamp,
    };
    let result = Ok(JsonRpcResult::BlockTemplate(block_template));
    let response = JsonRpcResponse { id, result };
    self.network
        .send(peer_address, &PeerMessagePayload::JsonRpcResponse(response));
}

Note that the function takes unix_timestamp parameter, but we don’t pass it.

The unix timestamp is different from the Instant that we’ve used so far. Instant is an opaque measurement of a monotonically nondescreasing clock. As such, it is useful for tasks such as measuring benchmarks or timing how long an operation takes. In this case, the opaque means that instants can only be compared to one another, but they do not provide methods to get e.g. “the number of seconds” since epoch. Therefore, we have to use a system clock to get current time since epoch, i.e. the unix timestamp.

// learncoin/src/learncoin_node.rs

// ...
impl Miner {
    // ...

    pub fn run()mut self) -> Result<(), String> {
        // ...
        loop {
            let unix_timestamp = SystemTime::now()
                .duration_since(UNIX_EPOCH)
                .unwrap()
                .as_secs();
            // ...
        }
    }
}

Make sure to pass the unix_timestamp to the on_get_block_template function.

Here’s the source code that extends the learncoin node to support the get block template protocol.

Running the miner

Let’s start a single server node and the miner.

I’ve set the NONCE_BATCH_SIZE to 100_000_000. In practice, a better value would be something like 1_000_000 but that would emit a lot of logging.

./learncoin server --address 127.0.0.1:8334

New peers connected: [
    "127.0.0.1:56919",
]
[127.0.0.1:56919] Recv: Version(
    VersionMessage {
        version: 1,
    },
)
[127.0.0.1:56919] Send: Verack
[127.0.0.1:56919] Recv: JsonRpcRequest(
    JsonRpcRequest {
        id: 0,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:56919] Send: JsonRpcResponse(
    JsonRpcResponse {
        id: 0,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431756,
                },
            ),
        ),
    },
)
[127.0.0.1:56919] Recv: JsonRpcRequest(
    JsonRpcRequest {
        id: 1,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:56919] Send: JsonRpcResponse(
    JsonRpcResponse {
        id: 1,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431758,
                },
            ),
        ),
    },
)
[127.0.0.1:56919] Recv: JsonRpcRequest(
    JsonRpcRequest {
        id: 2,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:56919] Send: JsonRpcResponse(
    JsonRpcResponse {
        id: 2,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431829,
                },
            ),
        ),
    },
)

Here’s the miner output that’s been running in parallel.

./learncoin miner --server 127.0.0.1:8334

[127.0.0.1:8334] Send: Version(
    VersionMessage {
        version: 1,
    },
)
[127.0.0.1:8334] Recv: Verack
[127.0.0.1:8334] Send: JsonRpcRequest(
    JsonRpcRequest {
        id: 0,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:8334] Recv: JsonRpcResponse(
    JsonRpcResponse {
        id: 0,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431756,
                },
            ),
        ),
    },
)
[127.0.0.1:8334] Send: JsonRpcRequest(
    JsonRpcRequest {
        id: 1,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:8334] Recv: JsonRpcResponse(
    JsonRpcResponse {
        id: 1,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431758,
                },
            ),
        ),
    },
)

The miner requests the block template, and the server responds. Eventually, the miner asks the server again for a new block template. There is no new data, but the server responds with the block template with the updated timestamp. The process repeats itself a couple of times, but the miner hasn’t found the valid nonce yet.

The following output is the continuation from the miner process.

[127.0.0.1:8334] Send: JsonRpcRequest(
    JsonRpcRequest {
        id: 2,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:8334] Recv: JsonRpcResponse(
    JsonRpcResponse {
        id: 2,
        result: Ok(
            BlockTemplate(
                BlockTemplate {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    transactions: [],
                    difficulty_target: 28,
                    height: 0,
                    public_key_address: PublicKeyAddress,
                    current_time: 1637431829,
                },
            ),
        ),
    },
)
[127.0.0.1:8334] Send: JsonRpcRequest(
    JsonRpcRequest {
        id: 3,
        method: GetBlockTemplate,
    },
)
[127.0.0.1:8334] Send: JsonRpcRequest(
    JsonRpcRequest {
        id: 4,
        method: SubmitBlock(
            Block {
                id: BlockHash(
                    0000000dce519a9bdbe248a6e6a371b638c514af7d69a65d0acab8e4567edd39,
                ),
                header: BlockHeader {
                    previous_block_hash: BlockHash(
                        00440e2238ebe96bf7013d7b86b4ec8293bb1bc0294444607817e73d8eecf1d1,
                    ),
                    merkle_root: MerkleHash(
                        c6d96f868d2c9068c0865d38d14873a0b62a3d03af1add517af1b494a527baf8,
                    ),
                    timestamp: 1637431829,
                    difficulty_target: 28,
                    nonce: 29852218,
                },
                transactions: [
                    Transaction {
                        id: TransactionId(
                            69bce1ce26f67aa8244741000b3ae00dfacb1f20169bb0f3b677ddf29a1da0f8,
                        ),
                        inputs: [
                            TransactionInput {
                                utxo_id: TransactionId(
                                    0000000000000000000000000000000000000000000000000000000000000000,
                                ),
                                output_index: OutputIndex(
                                    -1,
                                ),
                                unlocking_script: UnlockingScript,
                            },
                        ],
                        outputs: [
                            TransactionOutput {
                                locking_script: LockingScript,
                                amount: 50,
                            },
                        ],
                    },
                ],
            },
        ),
    },
)

Eventually, the miner successfully finds the block with a valid nonce 29852218, and submits it to the node.

Note that the node doesn’t update the active chain yet, so the block template will always contain the same previous block hash. We will fix this in one of the future blog posts.

Summary

In this blog post, we have:

  • Learned how JSON-RPC protocol works.
  • Implemented JSON-RPC protocol via the peer connection protocol.
  • Learned how a miner uses the “get block template” protocol.
  • Implemented a simple miner that frequently checks for new block templates.
  • Implemented support for the “get block template” protocol in the Learn Coin Node.
  • Run the server and the miner, and observed miner finding new blocks.

Here’s the full source code produced in this blog post.

What’s next?

In the next blog post, we will extend the protocol to relay mined blocks.

comments powered by Disqus

Join my mailing list

Subscribe to get new notifications about new content. It takes time to write quality posts, so you can expect at most 1-2 emails per month.