What is Shred

Shred is Solana's fundamental unit for segmenting, transmitting, and processing block data. Solana significantly improves data transmission efficiency by dividing block data into Shreds and broadcast them with Turbine. The data structure of Shred is shown in the figure below

Shred is mainly divided into two parts: Payload and Header.

Payload plays the role of data storage and is divided into data shred and code shred.

• data shred: contains the actual ledger entries of the block, with each entry consisting of a set of transactions.
• code shred: contains redundant data generated based on Reed-Solomon erasure codes, which is used to recover lost data shards in response to network packet loss or malicious behavior.

The Header contains basic identification and verification information for the shred. The slot and index are used to identify the position of the shred within its slot, and the fec_set_index is used to identify the FEC set to which the shred belongs, allowing Solana to group data shreds and code shreds together for data recovery.

What can Shred do

Block Reconstruction: Validator nodes, RPC nodes, and regular Solana nodes all need to receive shreds for reconstruct blocks and updating node states. For validators, receiving shred with low latency ensures that vote transactions can be submitted promptly, thereby increasing voting credits and securing a higher share of staking rewards; for RPC nodes, receiving shreds with low latency enables immediate block reconstruction, allowing dApps to query the latest on-chain data in real-time.

Signal Capture: For Sniping Bots and Copy Trading Bots, shreds can be used to capture trading signals. Receiving shreds with low latency enables them to capture trading signals faster than competitors, improving win rate by securing first-mover advantage.

Transaction Analysis: For DEX projects, shreds help to execution results of transactions and generation of price K-lines for trading pairs. Low-latency shred reception ensures these operations can be executed with minimal delay, ultimately delivering an exceptional trading experience for users.

Based on the above analysis, although different users have varying scenarios for using Shred, low latency is a rigid requirement across all scenarios.

How to Make Low-Latency Shred the Key to Win

This article uses Sniper Bot as an example to introduce best practices for improving performance through low-latency Shred.

Sniper Bot is an automated trading program designed to quickly "snipe" newly listed tokens (typically meme coins). They monitor new liquidity pools on decentralized exchanges (such as Raydium or Pump.fun) and aim to buy in before price surges, pursuing profits through high-frequency trading.

The conventional method for monitoring newly activated liquidity pool transactions relies on WebSocket subscriptions to Solana RPC nodes using JSON-RPC. However, this approach has a critical limitation: it can only access data after block reconstruction, resulting in higher latency. In contrast, the solution with lower latency involves real-time subscription to raw Shreds emitted by Validators during block reconstruction, which are then parsed into actionable data.

Step1: Identifying Low-Latency Shred Sources

Validator Node: High-staked validator nodes can forward shreds with fewer hops in Turbine.

Jito Block Engine: Forward shreds emitted by Jito-Solana(Jito Validator client).

Shred Service: Currently, there are multiple vendors in the market providing Shred services. We will soon release a benchmark to compare the performance of different Shred services.

Step2: Receive Raw Shreds in Low Latency

It is recommended to deploy shredstream-proxy for low-latency Shred reception. Shredstream-proxy is a Shred subscription proxy service provided by Jito Labs, capable of receiving native Shred streams via the UDP protocol. The shredstream-proxy operates in two modes:

Shredstream Mode: An active connection mode where the proxy actively connects to the Jito Block Engine to retrieve Shreds. It requires an authentication keypair to communicate with the Jito service and maintains the connection with the Block Engine through a heartbeat mechanism.

ForwardOnly Mode: A passive listening mode where the proxy simply forwards any received data packets without interacting with the Jito service, allowing it to listen to specified low-latency Shred sources.

The shredstream-proxy supports receiving native Shreds from multiple channels. In ForwardOnly mode, the shredstream-proxy achieves multi-source data reception through multiple UDP sockets (each UDP socket independently receives data packets) and supports tracking packet statistics by source IP address. The figure is as follows:

To minimize transmission latency between shredstream-proxy and Shred sources, it is recommended to deploy shredstream-proxy in a geographic location close to the Shred sources. Currently, Frankfurt, Amsterdam, Tokyo, and New York are regions with relatively concentrated Shred sources.

Step3: Fast Shred Parsing

After receiving Shreds with low latency, it is necessary to quickly parse the Shreds into data that meets the requirements of specific business scenarios. The goal of Sniper Bot is to obtain transactions related to the activation of new liquidity pools, so it only needs to parse Shreds into transactions. Currently, there are two methods:

shredstream-proxy

The shredstream-proxy provides an optional gRPC reconstruction service, supporting the reconstruction of native Shreds into entries. Sniper Bot can deserialize these entries into transactions to capture new liquidity pool activation transactions. The figure is as follows:

Third-Party Open-Source Tools

Sniper Bot can directly integrate third-party open-source tools in its project, such as ☞ https://github.com/BlockRazorinc/shreds-subscribe.git 和 ☞ https://github.com/dyodx/unshred, to more efficiently parse received Shreds into transactions. The diagram is as follows:

The two methods mentioned above can basically meet the business requirements of Sniper Bot. For scenarios requiring complete block data, it is necessary to deploy a Solana node on a local server while integrating the Yellowstone plugin.

As an open-source project, Sniper Bot can subscribe to publicly available Yellowstone gRPC services. For local deployment, the figure is as follows:

As shown above, Sniper Bot can deploy a Solana node locally with the integrated Yellowstone plugin, opening the TVU port to receive Shreds with low latency and reconstruct blocks. The Yellowstone plugin receives data emitted during the block reconstruction process, processes it through the gRPC service, and pushes it to clients in real-time via a gRPC Stream.

Yellowstone provides client libraries in multiple languages to simplify the integration process, including Rust, Node.js, and TypeScript. Sniper Bot can import these client libraries into project, and after creating a client instance and connecting to the gRPC server, it can receive the gRPC stream.

Summary

Low-latency Shreds are a critical requirement in scenarios such as block reconstruction, transaction signal capture, and transaction analysis. Fully leveraging low-latency Shreds can enhance business performance, secure a competitive edge, and become a key factor in achieving success.

Compared to traditional subscription based on WebSocket, this article uses Sniper Bot as an example to introduce best practices for enhancing business performance through low-latency Shreds. Users can flexibly tailor and combine best practices based on their business scenarios, data requirements, and integration convenience to create the most suitable solution for their needs.