The Goku database is a time-series database developed by Pinterest to store, query and manage large volumes of monitoring metrics. It is not related to Goku, the central character of the Dragon Ball franchise. In Pinterest’s technology context, Goku refers to an internal infrastructure system built for observability.
Pinterest originally relied on OpenTSDB and HBase for time-series metrics. As monitoring traffic grew, Pinterest reported problems including expensive infrastructure, slower queries, garbage-collection overhead and operational maintenance requirements. The company began developing Goku around 2018, with the goal of maintaining compatibility with its existing observability stack while improving scalability and efficiency.
The distinction matters because “Goku database” can produce two very different types of search intent. One concerns enterprise infrastructure and databases; another may concern Dragon Ball character databases. For technology readers, the Pinterest system is the relevant interpretation.
How Pinterest’s Goku Architecture Works
Goku is not simply one database server. The published architecture describes a collection of components that work together.
The principal layers include Goku Short Term, or GokuS, which keeps recent metrics in memory; Goku Long Term, or GokuL, which stores older metrics on persistent storage; Goku Compactor, which handles aggregation and conversion work; and Goku Root, which manages query routing.
| Component | Primary role | Storage or function |
| GokuS | Recent metrics | In-memory storage |
| GokuL | Older metrics | SSD/HDD persistent storage |
| Goku Compactor | Data transformation | Aggregation and conversion |
| Goku Root | Query coordination | Routing and aggregation |
| Kafka ingestion | Data pipeline | High-throughput metric ingestion |
This architecture reflects a practical principle: frequently accessed data should be close to computation, while older information can move to cheaper storage.
Goku’s Time-Series Data Model
Goku uses a schemaless time-series model. A series is identified through a metric name combined with tag-value pairs. For example, a CPU metric might include tags identifying a host, operating system, cluster and availability zone.
Each datapoint consists of a timestamp and numeric value. Queries can specify a time range, metric, tags, filters and aggregation requirements. Supported operations include functions such as sum, maximum, minimum, mean, count, p50 and p99. Downsampling can also reduce the granularity of returned data.
This model is important for observability because infrastructure changes constantly. New hosts, containers and services can appear without requiring a rigid database schema for every possible metric dimension.
Data and Query Characteristics
| Area | Goku approach |
| Data model | Schemaless |
| Metric identity | Metric name + tag values |
| Query protocol | Apache Thrift |
| Recent data | In memory |
| Older data | SSD/HDD |
| Aggregation | Sum, min, max, mean, p50, p99, count |
| Downsampling | Supported |
| Retention controls | Configurable |
| Backup | AWS S3 |
Why Pinterest Built Goku
Pinterest’s published research explains that OpenTSDB and HBase became increasingly difficult to operate as monitoring traffic expanded. The company identified slow queries caused by secondary-storage access and aggregation patterns, high infrastructure costs and substantial maintenance requirements.
Goku was therefore designed around several measurable requirements. Pinterest’s original targets included millions of datapoints per second, billions of time series per day, 24 hours of in-memory data, retention of up to one year and p99 query targets of five seconds for in-memory data and ten seconds for secondary storage.
One important architectural decision was to perform more computation closer to where data resides. Instead of transferring large quantities of raw metrics to a central machine before aggregation, Goku can perform partial processing at storage nodes and combine results later. This reduces unnecessary network traffic and central processing pressure.
Storage, Compression and Cost Control
Goku’s evolution shows that time-series database design is not simply a race for maximum query speed. Storage economics matter just as much.
The system uses tiered storage. Recent data is kept in memory, while older information moves to persistent storage. Pinterest introduced GokuL to support data beyond the original one-day window, using RocksDB for long-term storage.
The later architecture also incorporates AWS S3 for backup and data movement. Pinterest has described features such as namespaces, rollups, pre-aggregation and configurable retention as ways to control storage requirements and improve query efficiency.
A particularly useful insight is that retention does not have to mean keeping every datapoint at identical resolution. Older data can be rolled up into lower-granularity representations. That preserves historical trends while reducing the storage and processing burden.
Performance and Operational Trade-Offs
Goku’s design solves specific problems, but its complexity is also a trade-off. A distributed observability database requires ingestion pipelines, storage tiers, routing, replication, compaction and configuration management.
Pinterest’s engineering work illustrates this continuing optimisation. In 2023, the company reported that its Goku ingestion system processed approximately 2.5 billion to 5 billion datapoints per minute. The same engineering report described reliability problems associated with high garbage-collection overhead in the older ingestion architecture.
Pinterest later reported improvements including reduced index memory usage and a 37% reduction in stored time series through mechanisms designed to identify unnecessary write-heavy metrics.
These examples demonstrate that database performance is broader than query latency. Ingestion reliability, memory consumption, storage footprint and recovery time can all affect the usefulness of an observability platform.
Three Practical Insights From Goku
1. Recent data deserves different treatment. Monitoring systems frequently need immediate access to current metrics, while historical data is usually queried for trends, capacity planning or investigations. Tiered storage aligns infrastructure costs with those access patterns.
2. Cardinality is a hidden cost. Every unique combination of metric names and tags creates another time series. Pinterest’s documentation specifically identifies high cardinality as a factor that can hurt query performance.
3. Compatibility can accelerate infrastructure migration. Maintaining OpenTSDB-compatible query behaviour allowed Pinterest to develop a new backend without completely redesigning the surrounding observability workflow.
The Future of Goku Database in 2027
Goku’s documented trajectory suggests continued emphasis on cost-efficient observability rather than a single headline performance metric. Pinterest’s 2025 VLDB paper describes tiered storage, configurable TTLs, rollup granularity, backfilling, replication and S3-backed fault tolerance as established parts of the system.
The likely technical direction is greater automation around retention, aggregation and resource allocation. However, public documentation does not establish a specific 2027 product roadmap, so stronger claims would be speculative.
The infrastructure challenge will remain familiar: monitoring volumes grow alongside services, while teams still expect fast queries and reliable alerts. Goku’s architecture shows one way of balancing those competing requirements.
Key Takeaways
- Goku was developed by Pinterest for large-scale observability.
- It uses a schemaless metric-and-tag model.
- Recent data is prioritised for fast access through in-memory storage.
- Long-term information uses persistent storage and rollups.
- Kafka supports the ingestion pipeline.
- RocksDB supports long-term metric storage.
- Cost optimisation has remained a major part of Goku’s development.
Conclusion
The Goku database is best understood as an evolving observability infrastructure rather than a conventional general-purpose database. Pinterest built it around the practical demands of monitoring a large distributed technology platform.
Its architecture combines time-series modelling, tiered storage, sharding, query routing, aggregation and configurable retention. The result is a system designed around the reality that not every metric requires the same storage cost, resolution or access speed.
The most significant lesson from Goku is architectural rather than brand-specific. Large monitoring systems need to balance ingestion volume, query latency, storage economics, reliability and operational complexity simultaneously. Pinterest’s documented development of Goku demonstrates how those priorities can shape a database over several years.
FAQ
What is the Goku database?
Goku is Pinterest’s in-house time-series database for storing and querying observability metrics. It was developed to address scaling and operational limitations encountered with OpenTSDB and HBase.
Is Goku database related to Dragon Ball?
No. Pinterest’s Goku is a software system. The name is shared with Goku, the fictional Dragon Ball character, but the two have no technological relationship.
What type of database is Goku?
Goku is a schemaless time-series database designed for large-scale monitoring and observability workloads.
Why did Pinterest create Goku?
Pinterest developed Goku after experiencing performance, infrastructure-cost and maintenance challenges with its previous OpenTSDB and HBase-based monitoring system.
Does Goku use long-term storage?
Yes. Goku includes a long-term storage layer called GokuL, which uses persistent storage for metrics older than the short-term in-memory window.
What technology does Goku use for long-term storage?
Pinterest’s published architecture describes RocksDB as the storage engine for GokuL, while AWS S3 is also used as part of the broader backup and data-storage architecture.
Methodology
This article was based on Pinterest Engineering publications from 2018, 2019, 2021, 2023 and 2024, together with the peer-reviewed Proceedings of the VLDB Endowment paper on Goku. The analysis prioritises Pinterest’s own engineering documentation and the published academic paper for architectural and performance claims. Independent reporting was used only for contextual comparison.
The main limitation is that Goku is an internal Pinterest system. Public documentation does not provide complete implementation details, current production configuration or a guaranteed 2027 roadmap. Forward-looking statements are therefore limited to documented architectural trends.
References
Sanghavi, M. M., Hu, M.-M., Luo, Z., Li, X., & Bajaj, K. (2024). Goku: A schemaless time series database for large scale monitoring at Pinterest. Proceedings of the VLDB Endowment, 18(2), 503–515. DOI: 10.14778/3705829.3705862.
Pinterest Engineering. (2018, September 13). Goku: Building a scalable and high performant time series database system.
Pinterest Engineering. (2019, December 12). GokuL: Extending time series data storage to serve beyond one day.
Pinterest Engineering. (2021, December 17). Cost reduction in Goku.
Sanghavi, M. M., Bajaj, K., Hu, M.-M., Li, X., & Luo, Z. (2023, November 22). Improving efficiency of Goku time series database at Pinterest, Part 1.
Sanghavi, M. M., Li, X., Hu, M.-M., Luo, Z., & Bajaj, K. (2024, March 12). Improving efficiency of Goku time series database at Pinterest, Part 2.
Pinterest Engineering. (2024, September 9). Improving efficiency of Goku time series database at Pinterest, Part 3.
AI Editorial Disclosure: This article was drafted with AI assistance and should be reviewed by a human editor before publication. Named claims, statistics and references should be independently checked against the original sources.
