Introduction
The TON Indexer project introduces a sophisticated system designed to enhance data access and interaction with the TON blockchain. By leveraging a PostgreSQL database, the TON Indexer transforms the way blocks, transactions, messages, NFTs, Jettons, and DNS domains are stored and queried, surpassing the limitations of the native RocksDB storage used by TON nodes. This document outlines the architecture, setup, and operational guidelines of the TON Indexer, highlighting its significance in the TON ecosystem.
Motivation
The native key-value database, RocksDB, utilized by TON nodes, poses challenges for developers and applications requiring complex queries and efficient data retrieval. The TON Indexer addresses these challenges by providing a more flexible and query-friendly SQL database solution, facilitating improved data accessibility and manipulation. This shift not only optimizes data handling but also opens up new possibilities for application development on the TON blockchain.
Implementation Overview
The TON Indexer comprises several components, each serving a distinct role in the data indexing and access process:
- Postgres: Hosts the PostgreSQL server, acting as the central storage for indexed data.
- Index-API: A FastAPI server offering a range of endpoints for database interaction.
- Alembic: Manages database migrations to ensure schema consistency.
- Index-Worker: Reads and parses data from the TON node database, requiring proximity to a functioning TON Node.
How to Run
Prerequisites
- Docker and Docker Compose
- A running TON full node (archival mode recommended for comprehensive indexing)
- Recommended hardware specs vary based on the component and the depth of indexing required.
Setup Steps
- Clone the repository and initialize submodules.
- Configure the environment using
./configure.shand customize the.envfile with relevant parameters. - Build Docker images for the necessary services.
- Launch the stack with Docker Compose, starting with database services followed by the index worker.
Configuration Parameters
A comprehensive list of available parameters allows for detailed customization of PostgreSQL settings, API behavior, and index worker operation. These include database host, port, user credentials, database name, API root path, and worker-specific settings such as the TON node database path and starting block sequence number.
Comparative Analysis
| Feature | RocksDB (Native TON Storage) | TON Indexer (PostgreSQL) |
|---|---|---|
| Data Query Flexibility | Limited | High (SQL-based queries) |
| Setup Complexity | Low (native to TON nodes) | Moderate (requires additional setup and configuration) |
| Data Access Efficiency | High for key-value access | High for complex queries |
| Cross-Application Use | Limited | Enhanced (API endpoints for diverse applications) |
| Storage Requirements | High (for archival nodes) | Variable (depends on indexing depth and hardware) |
Conclusion
The TON Indexer revolutionizes data access and manipulation on the TON blockchain by bridging the gap between the inherent storage limitations of RocksDB and the versatile querying capabilities of PostgreSQL. With its comprehensive stack and flexible configuration, the TON Indexer not only improves development efficiency but also paves the way for innovative blockchain applications. As the TON ecosystem continues to grow, the TON Indexer stands out as a pivotal tool for developers seeking to leverage the full potential of blockchain data.