Visualize Blockchain With Graphwiz
Project: Let's build Bitcoin
In the previous blog post, we explained how blocks are relayed and we implemented the algorithm to update the active blockchain.
In this post, we will visualize the blockchain using Graphwiz. To implement this, we will implement our first client-side command that requests the full blockchain from the server using via the JSON-RPC protocol. In practice, visualizing the full blockchain may not be practical when there are hundreds of thousands of blocks. However, our blockchain network doesn’t have that many blocks yet, so this would be a good starting point.
Extend the JSON RPC protocol
Let’s first extend the JSON RPC protocol to allow clients to request full blockchain. The response should include all the blocks, but also tell the client which blocks belong to the active chain.
// learncoin/src/peer_message.rs
// ...
pub enum JsonRpcMethod {
// ...
// Expects Blockchain in response.
GetBlockchain,
}
pub enum JsonRpcResult {
// ...
// Blockchain(list of all blocks headers, list of active blocks).
Blockchain(Vec<BlockHeader>, Vec<Block>),
}
Let’s now implement the new protocol.
// learncoin/src/learncoin_node.rs
// ...
impl LearnCoinNode {
// ...
fn on_get_blockchain(&mut self, peer_address: &str, id: u64) {
let all_headers = self.block_index.all_headers();
let active_blocks = self.active_chain.all_blocks().clone();
let result = Ok(JsonRpcResult::Blockchain(all_headers, active_blocks));
let response = JsonRpcResponse { id, result };
self.network
.send(peer_address, &PeerMessagePayload::JsonRpcResponse(response));
}
}
Nothing special here.
We have also introduced new methods to BlockIndex and ActiveChain for retrieving all the data.
These are trivial so I’ll leave that to you as an exercise.
Don’t forget to delegate the GetBlockchain request in the on_json_rpc_request.
Client command to request full blockchain
Let’s implement a client command that connects to the local node and requests the full blockchain.
./learncoin client get-blokchain --format graphvwiz --output-file ./diagram.dot
We’d like the above command to print the full blockchain to the standard output in the format that Graphwiz can understand.
Additionally, the get-blockchain command may support other formats in the future, e.g. JSON.
Introduce get-blockchain command
Let’s begin with introducing a new command to the client, which takes format, suffix-length, and output-file.
suffix-length is the length of the block hash suffix used in the diagram.
Shorter hashes are nicer to read.
// learncoin/src/commands/client_command.rs
fn get_blockchain() -> App<'static> {
App::new("get-blockchain")
.version("0.1")
.about("Requests the full blockchain from the local node and prints it to the output file.")
.arg(
Arg::new("format")
.short('f')
.long("format")
.about("Output format of the printed blockchain.")
.takes_value(true)
.default_value("graphwiz")
.required(false),
)
.arg(
Arg::new("suffix-length")
.long("suffix-length")
.about("Length of the block hash suffix that is printed.")
.takes_value(true)
.default_value("8")
.required(false),
)
.arg(
Arg::new("output-file")
.long("output-file")
.about("File to which the output is printed.")
.takes_value(true)
.required(true),
)
}
pub fn client_command() -> App<'static> {
// ... high-level command declaration
.subcommand(get_blockchain())
}
Hopefully the above code snippet is self-describing. Let me know if there is anything unclear.
Parse the inputs
Let’s also implement the client command.
pub fn run_client_command(matches: &ArgMatches) -> Result<(), Box<dyn Error>> {
let options = ClientCliOptions::parse(matches)?;
let mut client = Client::connect_with_handshake(options.server, options.timeout)?;
if let Some(ref matches) = matches.subcommand_matches("get-blockchain") {
let format = matches.value_of_t("format")?;
let hash_suffix = matches.value_of_t("suffix-length")?;
let output_file = matches.value_of("output-file").unwrap();
client.execute_get_blockchain(format, hash_suffix, output_file)?;
Ok(())
} else {
panic!("No command has been specified")
}
}
Similar to what we did with the miner, the client uses the peer connection under the hood to talk to the server.
The Client would expose API for each command that we want to implement, and the code in this file is responsible for parsing the arguments and deciding which function to call.
Implement the client
Before we can implement the execute_get_blockchain command, we have to come up with a framework to allow us to send JSON RPC requests to the local server.
The implementation will be fully synchronous.
As mentioned before, the client uses the PeerConnection to communicate with the server.
Therefore, let’s start with the handshake protocol.
// learncoin/src/learncoin_node.rs
const MAX_RECV_BUFFER_SIZE: usize = 10_000_000;
pub struct Client {
peer_connection: PeerConnection,
timeout: Duration,
next_json_rpc_id: u64,
}
impl Client {
pub fn connect_with_handshake(server: String, timeout: Duration) -> Result<Self, String> {
let mut peer_connection = PeerConnection::connect(server, MAX_RECV_BUFFER_SIZE)?;
let mut client = Self {
peer_connection,
timeout,
next_json_rpc_id: 0,
};
client.send_message(&PeerMessagePayload::Version(VersionMessage::new(VERSION)))?;
match client.wait_for_response()? {
PeerMessagePayload::Verack => Ok(client),
unexpected => Err(format!("Received unexpected message: {:?}", unexpected)),
}
}
}
The Client sends the Version message and waits for the Verack for some time.
The handshake is complete when the Client receives the Verack message.
If the client doesn’t receive the Verack message within the given time, the command fails.
Let’s now implement the helper functions that send the message and wait for the response.
The wait_for_response function attempts to read messages from the connection.
It keeps trying until there is a message or the timeout expires.
fn wait_for_response(&mut self) -> Result<PeerMessagePayload, String> {
let instant = Instant::now();
while instant.elapsed().lt(&self.timeout) {
match self.peer_connection.receive()? {
None => continue,
Some(message) => return Ok(message),
}
}
Err(format!(
"Timed out after {}ms while waiting for the handshake to complete.",
self.timeout.as_millis()
))
}
The send function in the PeerConnection returns whether the flow-control has pushed back.
However, the client doesn’t care about this, so to keep things simple, let’s introduce a send_message API that fails if the flow control kicks in.
fn send_message(&mut self, message: &PeerMessagePayload) -> Result<(), String> {
let is_sent = self.peer_connection.send(&message)?;
if !is_sent {
Err(format!(
"Failed to send the version message due to flow-control."
))
} else {
Ok(())
}
}
Great, we have all the boilerplate code that allows the client to send messages and wait for responses.
Since the client would mostly be using the JSON RPC messages, let’s also provide helper functions to easily send the JSON RPC request and wait for the response.
fn send_json_rpc_request(&mut self, method: JsonRpcMethod) -> Result<u64, String> {
let id = self.next_json_rpc_id();
self.send_message(&PeerMessagePayload::JsonRpcRequest(JsonRpcRequest {
id,
method,
}))?;
Ok(id)
}
fn next_json_rpc_id(&mut self) -> u64 {
let id = self.next_json_rpc_id;
self.next_json_rpc_id += 1;
id
}
fn wait_for_json_rpc_response(&mut self, expected_id: u64) -> Result<JsonRpcResponse, String> {
match self.wait_for_response()? {
PeerMessagePayload::JsonRpcResponse(response) if response.id == expected_id => {
Ok(response)
}
unexpected => Err(format!("Received unexpected message: {:?}", unexpected)),
}
}
Execute get blockchain command
We’re almost done with the client-side code, and soon we’ll be implementing the Graphwiz generator.
Before we move to the next step, let’s implement the final function execute_get_blockchain to execute the command.
The function requests the blockchain from the server and waits for the response.
It formats the response based on the given parameters (we only supprot Graphwiz format for now), and prints the contents to the file.
pub fn execute_get_blockchain(
&mut self,
format: GetBlockchainFormat,
suffix_length: usize,
output_file: &str,
) -> Result<(), String> {
let id = self.send_json_rpc_request(JsonRpcMethod::GetBlockchain)?;
match self.wait_for_json_rpc_response(id)? {
JsonRpcResponse { id, result } => match result? {
JsonRpcResult::Blockchain(blocks, active_block_hashes) => {
let data = match format {
GetBlockchainFormat::Graphwiz => {
Graphwiz::blockchain(blocks, &active_block_hashes, suffix_length)
}
};
fs::write(output_file, data).map_err(|e| e.to_string())?;
Ok(())
}
unexpected => Err(format!("Received unexpected message: {:?}", unexpected)),
},
}
}
Graphwiz blockchain generator
The last bit that’s left for us to do is generate the contents that the Graphwiz tool called dot can process and generate the .svg of the blockchain.
We will not spend time on explaining the syntax of the dot files.
You can read documentation about the Graphwiz here.
What’s important is that we’d like to generate the following code:
digraph G {
subgraph cluster_0 {
style=filled;
color=lightgrey;
node [style=filled,color=white];
a0 -> a1 -> a2 -> a3;
label = "Active";
}
s0 -> s1;
s1 -> s2;
}
The subgraph represents the hashes in the active chain, i.e. a0 -> a1 -> ... -> an represent the block hashes in the active chain.
On the other hand, s0 -> s1 and s1 -> s2 are examples of parent-child relationships that are not part of the active chain.
// learncoin/src/graphwiz.rs
pub struct Graphwiz {}
impl Graphwiz {
pub fn blockchain(
all: Vec<BlockHeader>,
active_blocks: &Vec<Block>,
suffix_suffix: usize,
) -> String {
let mut code = String::new();
let genesis_block_parent = BlockHash::new(Sha256::from_raw([0; 32]));
let active_blocks_code = active_blocks
.iter()
.map(|block| format!(r#""{}""#, Self::hash_suffix(block.id(), suffix_suffix)))
.collect::<Vec<String>>()
.join(" -> ");
let all_blocks_code = all
.iter()
.filter(|block_header| !Self::is_active(block_header, active_blocks))
.map(|block_header| {
let parent = Self::hash_suffix(&block_header.previous_block_hash(), suffix_suffix);
let child = Self::hash_suffix(&block_header.hash(), suffix_suffix);
// Don't print the parent of the genesis block.
if block_header.previous_block_hash() == genesis_block_parent {
format!(r#""{}""#, child)
} else {
format!(r#""{}" -> "{}";"#, parent, child)
}
})
.collect::<Vec<String>>()
.join("\n");
writeln!(&mut code, "digraph G {{").unwrap();
writeln!(&mut code, " subgraph cluster_0 {{").unwrap();
writeln!(&mut code, " style=filled;").unwrap();
writeln!(&mut code, " color=lightgrey;").unwrap();
writeln!(&mut code, " node [style=filled,color=white];").unwrap();
writeln!(&mut code, " {};", active_blocks_code).unwrap();
writeln!(&mut code, " label = \"Active\";").unwrap();
writeln!(&mut code, " }}").unwrap();
writeln!(&mut code, " {}", all_blocks_code).unwrap();
writeln!(&mut code, "}}").unwrap();
code
}
}
Let’s look at the above code more closely.
We generate the code as in the above example by writing to the string code with the writeln! macro.
Note that {{ is a way to escape character { because {} has a special meaning in Rust string formatting, i.e. it’s a placeholder for the to_string value of the corresponding argument.
The active_blocks_code represents the relationships in the active blockchain, which is achieved by joining all block hashes on the string ->.
Finally, we construct all_blocks_code by generating a parent and a child relationship per line for all blocks that do not exist in the active chain.
Note that the genesis block doesn’t have a parent, so we don’t print it.
All that’s left now is to implement the remaining helper functions.
fn hash_suffix(hash: &BlockHash, suffix_length: usize) -> String {
// Safety: BlockHash string representation matches the ASCII reprsentation, so it's safe
// to unwrap the UTF-8 string slice.
let s = hash.to_string();
s.as_str()
.get((s.len() - suffix_length)..)
.unwrap()
.to_string()
}
fn is_active(header: &BlockHeader, active_blocks: &Vec<Block>) -> bool {
active_blocks
.iter()
.any(|active| *active.id() == header.hash())
}
hash_suffix returns the last n characters of the block hash, while the is_active checks if a block is part of the active chain or not.
Generate the SVG of the blockchain
Let’s run two nodes and two miners for a while. Initially, I’ve left them running over night. However, I found a bug and didn’t have patience to wait for overnight results again, so I reduced the difficulty and made it more likely for forks to happen at any point.
./learncoin server --address 127.0.0.1:8333
./learncoin server --address 127.0.0.1:8334 --peers 127.0.0.1:8333
./learncoin miner --server 127.0.0.1:8333
./learncoin miner --server 127.0.0.1:8334
# ... Wait a bit
./learncoin client --server 127.0.0.1:8333 -get-blockchain --output-file ./8333.dot
cat ./8333.dot | dot -Tsvg > 8333.svg
We use the dot tool to generate the SVG from the generated code.
We can see in the above diagram that there are 7 blocks in the active chain, with two forks happening after 8eecf1d1 and 6bb35b4a.
I have also started a third node much later and confirmed that all graphs have the same active chain, while secondary chains were different on the third node because the IBD only downloads blocks from the active chain.
Summary
In this blog post, we have implemented the client as a CLI tool that allows us to send JSON RPC requests to the local server.
We’ve also implemented the get-blockchain command that requests the full blockchain from the local server and saves the contents to a file in the given format.
Finally, we generated the contents as a Graphwiz diagram which uses the dot language.
Here’s the full source code.
What’s next?
We almost have a functioning blockchain. The main goal of this blockchain is to allow users to exchange money via transactions. In the next blog post, we will extend the client to send transactions and show balances (first wallet-like functionality).