How to stream real-time Pump.fun updates on Solana

Harness the power of real-time data with gRPCs— a well-detailed guide on Pump.fun

Accessing real-time on-chain data is crucial for developers, researchers, and trading bots. While there are different ways to stream this data, gRPC is the fastest and most efficient, offering low latency and simpler code.

Pump.fun is a popular Solana platform where users can launch coins that are instantly tradable for under $2. With millions of meme coins launched and thousands more every minute, developers need real-time updates to stay ahead. gRPC provides these updates in milliseconds. In this article, you’ll learn how to stream Pump.fun events in real-time using gRPC.

Before Getting Started

To get started, we will need a few things.

Authentication: Your Shyft API Key, gRPC endpoint and gRPC token

You can get your own Shyft API Key(an auth parameter used by Shyft) from the Shyft website. You can also find your region-specific gRPC endpoint and access token on your Shyft Dashboard.

A server-side backend (like NodeJS) to receive gRPC data

As gRPC services are unsupported in web browsers, you would need a backend application to receive gRPC data. For this example, we have used NodeJS, but any other backend server-side environments such as C#, Go, Java, Kotlin, Python or PHP can also be used.

Introduction — Topics to be covered

In this article, we will look at streaming trading transactions on Pump.fun using gRPC. We will be covering the following topics,

• Cloning and installing dependencies
• Setting up a gRPC connection to the Solana network
• Subscribing to a streaming endpoint for trading transactions on Pump.fun
• Processing and deserializing the streaming data in real-time

Cloning & Installation

To get started, we clone the following and install the dependencies:

$ git clone https://github.com/Shyft-to/solana-defi.git

cd grpc-pump

npm install

All the code related to this article is available on our GitHub here. Please feel free to clone it and follow along.

Setting up a gRPC connection to the Solana network

Our very first step to streaming data from Pump.fun using gRPC is setting up our client service. It requires your gRPC URL and your X_TOKEN which looks like this

const client = new Client(
    'Your Region specific Shyft gRPC URL',
    'Shyft gRPC Access Token',
    undefined,
  );

now we can go ahead and obtain our data from the blockchain.

Subscribing to a streaming endpoint for trading transactions on pumpfun

Our goal is to stream specific data from the Solana blockchain using gRpc. To accomplish this, we will use the handleStream() function, which takes in two parameters:

1. client: Client: This parameter is used to subscribe to on-chain events and establish a connection to the Solana blockchain.
2. arg: SubscriberRequest: This parameter specifies the data that we want to fetch from the blockchain.

By using the handleStream() function in this way, we can efficiently and conveniently stream real-time on-chain data on the Solana blockchain.

interface SubscribeRequest {
    accounts: { [key: string]: SubscribeRequestFilterAccounts };
    slots: { [key: string]: SubscribeRequestFilterSlots };
    transactions: { [key: string]: SubscribeRequestFilterTransactions };
    transactionsStatus: { [key: string]: SubscribeRequestFilterTransactions };
    blocks: { [key: string]: SubscribeRequestFilterBlocks };
    blocksMeta: { [key: string]: SubscribeRequestFilterBlocksMeta };
    entry: { [key: string]: SubscribeRequestFilterEntry };
    commitment?: CommitmentLevel | undefined;
    accountsDataSlice: SubscribeRequestAccountsDataSlice[];
    ping?: SubscribeRequestPing | undefined;
  }

async function handleStream(client: Client, args: SubscribeRequest) {
    // Subscribe for events
    const stream = await client.subscribe();
  
    // Create `error` / `end` handler
    const streamClosed = new Promise<void>((resolve, reject) => {
      stream.on("error", (error) => {
        console.log("ERROR", error);
        reject(error);
        stream.end();
      });
      stream.on("end", () => {
        resolve();
      });
      stream.on("close", () => {
        resolve();
      });
    });
  
    // Handle updates
    stream.on("data", async (data) => {
      try{
     console.log(data);
  }catch(error){
    if(error){
      console.log(error)
    }
  }
});
  
    // Send subscribe request
    await new Promise<void>((resolve, reject) => {
      stream.write(args, (err: any) => {
        if (err === null || err === undefined) {
          resolve();
        } else {
          reject(err);
        }
      });
    }).catch((reason) => {
      console.error(reason);
      throw reason;
    });
  
    await streamClosed;
  }

After defining the SubscriberRequest parameter for the handleStream() function, the next step is to specify the data that we want to fetch from the Solana blockchain. In this tutorial, we are focusing on fetching token transactions on pump.fun. To do this, we need to define a SubscriberRequest object that specifies the parameters for our data request. This object should include the following information:

• The type of data that we want to fetch (in this case, token transactions)
• The address of pump.fun
• Any additional filters or parameters that are needed to narrow down the data that we want to fetch

Here is an example of what the SubscriberRequest object for fetching token transactions on pump.fun might look like:

const req = {
    accounts: {},
    slots: {},
    transactions: {
      bondingCurve: {
        vote: false,
        failed: false,
        signature: undefined,
        accountInclude: ['6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P'], //Address 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P
        accountExclude: [],
        accountRequired: [],
      },
    },
    transactionsStatus: {},
    entry: {},
    blocks: {},
    blocksMeta: {},
    accountsDataSlice: [],
    ping: undefined,
    commitment: CommitmentLevel.CONFIRMED, //for receiving confirmed txn updates
  };

The address 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P represents the on-chain address for the “pump.fun” platform on the Solana blockchain. However, this address alone is not sufficient to fetch the data that we need. To access the real-time on-chain data, we need to define the SubscribeCommand function. This function will be used to specify the parameters for our data request and subscribe to a streaming endpoint on the Solana blockchain. The SubscribeCommand function should take the SubscriberRequest object as a parameter and use it to establish a connection to the Solana blockchain and subscribe to the desired streaming endpoint.

async function subscribeCommand(client: Client, args: SubscribeRequest) {
    while (true) {
      try {
        await handleStream(client, args);
      } catch (error) {
        console.error("Stream error, restarting in 1 second...", error);
        await new Promise((resolve) => setTimeout(resolve, 1000));
      }
    }
  }

and call it

subscribeCommand(client, req);

our output should look like this

{
  filters: [ 'bondingCurve' ],
  account: undefined,
  slot: undefined,
  transaction: {
    transaction: {
      signature: <Buffer 18 f2 41 ca 0d c3 2d 24 a3 4c 25 0b 0f d5 e0 bd fd 75 35 a1 ed a3 43 8a f8 fc 86 c0 01 08 70 32 f9 ac 53 3c 46 e8 e2 62 22 4c 79 ab 3f f5 4b d5 81 0c ... 14 more bytes>,
      isVote: false,
      transaction: [Object],
      meta: [Object],
      index: '315'
    },
    slot: '283417510'
  },
  block: undefined,
  ping: undefined,
  pong: undefined,
  blockMeta: undefined,
  entry: undefined
}

Although this is the data we requested for, it is not the data we hope to get. We have come this close, we only need to deserialize our output to make it comprehensible.

Processing and Deserializing the streaming data in real-time

In order to convert the data received from the streaming endpoint into a more readable and usable format, a process known as deserialization is required. Here’s an example of how you could write a function to deserialize the data

import base58 from "bs58";
export function decodeTransact(data){
    const output = base58.encode(Buffer.from(data,'base64'))
    return output;
}

The function decode any Buffered data to more usable format.

Next we would want to defined our output in a more formatted manner

export function tOutPut(data){
    const dataTx = data.transaction.transaction
    const signature = decodeTransact(dataTx.signature);
    const message = dataTx.transaction?.message
    const header = message.header;
    const accountKeys = message.accountKeys.map((t)=>{
        return  decodeTransact(t)
    })
    const recentBlockhash =  decodeTransact(message.recentBlockhash);
    const instructions = message.instructions
    const meta = dataTx?.meta
    return {
        signature,
        message:{
           header,
           accountKeys,
           recentBlockhash,
           instructions
        },
        meta
    }
}

Then we rewrite our handleStream()function to fit into the updated codes

async function handleStream(client: Client, args: SubscribeRequest) {
    // Subscribe for events
    const stream = await client.subscribe();
  
    // Create `error` / `end` handler
    const streamClosed = new Promise<void>((resolve, reject) => {
      stream.on("error", (error) => {
        console.log("ERROR", error);
        reject(error);
        stream.end();
      });
      stream.on("end", () => {
        resolve();
      });
      stream.on("close", () => {
        resolve();
      });
    });
  
    // Handle updates
    stream.on("data", async (data) => {
      try{
     const result = await tOutPut(data);
     console.log(result);
  }catch(error){
    if(error){
      console.log(error)
    }
  }
});
  
    // Send subscribe request
    await new Promise<void>((resolve, reject) => {
      stream.write(args, (err: any) => {
        if (err === null || err === undefined) {
          resolve();
        } else {
          reject(err);
        }
      });
    }).catch((reason) => {
      console.error(reason);
      throw reason;
    });
  
    await streamClosed;
  }

Finally we make our call with the SubscribeCommand Function

subscribeCommand(client, req);

our updated output should look like this

signature: '4aTaVifyJNwaxrqP1939p6Cwi7doEJ627eo7XV1dC6cHHDLVf5JJ8qSdsVzQbmFRd2vWDK7sDZRHWWvuWwsnaCKj',
  message: {
    header: {
      numRequiredSignatures: 1,
      numReadonlySignedAccounts: 0,
      numReadonlyUnsignedAccounts: 8
    },
    accountKeys: [
      'Dn2sbLk6cMYyRu9D5h1G66vUGFTE7BRgqho9X6c52wLE',
      'CebN5WGQ4jvEPvsVU4EoHEpgzq1VV7AbicfhtW4xC9iM',
      'Gf72ajaDxUPSgXbRCnYjAXpPQXofam6FmedepfYJNY4B',
      '7GXjNDdNkSnsmaXmWUy4Woh6NAQgMCABBEPWtHyLBJ9J',
      'CcCAgQcArc6jjF2zAaDUzQ1FKgisqDjubCNJvVmPHVMU',
      'ZG98FUCjb8mJ824Gbs6RsgVmr1FhXb2oNiJHa2dwmPd',
      'ComputeBudget111111111111111111111111111111',
      '6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P',
      '4wTV1YmiEkRvAtNtsSGPtUrqRYQMe5SKy2uB4Jjaxnjf',
      'C1Fabt6rjQGcS68obCEH6zuywWfzpfvCCfKBJntCpump',
      '11111111111111111111111111111111',
      'ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL',
      'TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA',
      'Ce6TQqeHC9p8KetsN6JsjHK7UTZk7nasjjnr7XxXp9F1'
    ],
    recentBlockhash: 'EGc4SgYd4xmex8ZS4LQcjtvzBW8aU3yafphxscM9hBTa',
    instructions: [ [Object], [Object], [Object] ]
  },
  meta: {
    err: undefined,
    fee: '405000',
    preBalances: [
      '822302413',  '272648835990597',
      '1191944363', '2039280',
      '2039280',    '13219204698',
      '1',          '1141440',
      '2500000',    '1461600',
      '1',          '731913600',
      '934087680',  '0'
    ],
    postBalances: [
      '843092001',  '272648836206849',
      '1170319136', '2039280',
      '2039280',    '13219419085',
      '1',          '1141440',
      '2500000',    '1461600',
      '1',          '731913600',
      '934087680',  '0'
    ],
    innerInstructions: [ [Object] ],
    innerInstructionsNone: false,
    logMessages: [
      'Program ComputeBudget111111111111111111111111111111 invoke [1]',
      'Program ComputeBudget111111111111111111111111111111 success',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P invoke [1]',
      'Program log: Instruction: Sell',
      'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA invoke [2]',
      'Program log: Instruction: Transfer',
      'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA consumed 4645 of 381291 compute units',
      'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA success',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P invoke [2]',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P consumed 2003 of 373161 compute units',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P success',
      'Program data: vdt/007mYe6jgQqFQndU+0Owx6KwKFXj5LXaRYNPQ2h7er5FKWGLH4v5SQEAAAAAL/LStqYAAAAAvdV/G53p8Qe4ABBpTvcEEXUBRn8h6KfMWZcLzg+Qi3tazrtmAAAAAPCG0kEHAAAAuu/G0EirAwDw2q5FAAAAALpXtIS3rAIA',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P consumed 30418 of 399850 compute units',
      'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P success',
      'Program 11111111111111111111111111111111 invoke [1]',
      'Program 11111111111111111111111111111111 success'
    ],
    logMessagesNone: false,
    preTokenBalances: [ [Object], [Object] ],
    postTokenBalances: [ [Object], [Object] ],
    rewards: [],
    loadedWritableAddresses: [],
    loadedReadonlyAddresses: [],
    returnData: undefined,
    returnDataNone: true,
    computeUnitsConsumed: '30718'
  }
}

Conclusion

Streaming data on the Solana blockchain using dedicated nodes can indeed be a complex and resource-intensive task, requiring a larger and more intricate code base. However, with gRPC, developers can now conveniently stream on-chain data with ease, simplifying their code and allowing them to focus on building robust applications that tap into the power of the Solana blockchain. At Shyft, we are thrilled to introduce this innovative solution to the developer community. Our team is committed to providing the necessary tools and support to make this process seamless. We encourage developers to join our Discord Server or follow us on Twitter for updates and further assistance. With gRPCs, developers can take advantage of the low latency and reduced code base to build complex projects with ease. For further assistance, join our Discord Server or follow us on Twitter for updates.

All the code related to this article is available on our GitHub here. Please feel free to clone it and follow along. To know more about data streaming on Solana, checkout our other articles on streaming real-time data on Solana or Tracking new Pools on Raydium.

Resources

• SHYFT API Documentation
• Shyft Website
• Get API Key
• GitHub
• Join our Discord
• Javascript SDK
• Try out our APIs on Swagger UI