Strategic Shift: From Go Framework to Rust Runtime
Encore began as a Go-based framework, equipped with a Go runtime, CLI, parser, and compiler. Supporting TypeScript applications posed a critical architectural decision. While extending the Go runtime with a bridge or writing the runtime directly in TypeScript were straightforward options, the team opted for Rust. This choice was driven by two factors: the potential for multi-language scalability and performance gains through multithreading. Rust's precedence in projects like Prisma and Pydantic demonstrated its effectiveness in binding with high-level languages such as Node.js and Python.
Implementing core infrastructure logic in Rust, which could then interface with various language runtimes, avoided the need for reimplementation across different environments. Furthermore, Node.js's inherent single-threaded nature made it clear that Rust's multithreaded capabilities could significantly enhance performance, particularly for tasks like database connection management and HTTP lifecycle processing.
Performance Bottlenecks in the Go Sidecar Prototype
The initial approach to integrate TypeScript involved prototyping a Go runtime as a sidecar process alongside Node.js. Communication between the two relied on IPC mechanisms. However, this design introduced substantial latency overhead. Every database query, pubsub message, and trace event had to traverse a process boundary, amplifying response times. The latency issues arose from the serialization and deserialization required for inter-process communication.
This bottleneck highlighted the limitations of IPC-based designs for high-performance applications. It underscored the need for a runtime that could operate natively within the same process space as Node.js, eliminating the inefficiencies caused by cross-process communication.
Optimizing Performance through Multithreading
Rust's compatibility with Tokio, an asynchronous runtime for multithreaded tasks, enabled a transformative approach. By delegating infrastructure concerns-such as HTTP request handling, database pooling, pubsub, and tracing-to Rust, the runtime achieved a level of parallelism unattainable within the single-threaded confines of Node.js. This architecture allowed TypeScript applications to focus purely on business logic, while Rust handled the underlying complexities.
Multithreading provided measurable performance gains. Tasks that traditionally bottleneck Node.js applications, including database connection management and object storage, could now execute concurrently, drastically improving throughput and resource utilization.
Scope of the Rust Runtime Implementation
Two years and 67,000 lines of Rust later, the runtime now fully supports the HTTP request lifecycle, including routing, request parsing, validation, and response serialization. It integrates database connection pooling, querying, pubsub across three cloud providers, distributed tracing, metrics collection, object storage, caching, and an API gateway powered by Pingora. This comprehensive infrastructure forms the backbone of TypeScript applications, abstracting away the complexities of cloud-native operations.
The runtime's scope demonstrates Rust's capability to handle extensive infrastructure concerns while maintaining high performance. By centralizing these operations, developers are freed from managing low-level details, enabling them to focus on building scalable TypeScript applications.
Lessons and Reflections
The decision to build a Rust runtime brought to light several nonobvious challenges. From balancing multithreading complexity to ensuring seamless integration with Node.js, the process required iterative problem-solving. The team recognized that while the Go runtime excelled for Go applications, its design was not directly transferable to TypeScript without introducing significant latency overheads.
Reflecting on the journey, the team noted areas for improvement, including refining error handling mechanisms and optimizing bindings for better developer experience. These insights will guide future expansions to additional languages, ensuring a scalable and efficient architecture.