Picture one endpoint whose latency graph is not flat. Most of the time it is calm, then it rises sharply, drops again, and rises again. You stare at that graph over coffee, and what comes to mind is not a profiler. What comes is the sentence, "maybe we should just switch languages." 😅
That urge is completely reasonable. Some large companies really did switch languages and got good results. But there is something we rarely notice: the two most famous stories did not start with a new language. They started with numbers, with measuring until it was clear exactly where things jammed. Joel Spolsky once wrote that rewriting from scratch is "the single worst strategic mistake that any software company can make", and we will see why that sentence still holds.
Let's break down three real cases, plus one counterexample. The point is not to say a language switch is wrong. The point is to know when the cheap fixes are exhausted, and when you may start thinking about a rewrite.
Key takeaways
Moving languages only has a basis when the bottleneck provably comes from the runtime or from an architecture that cannot share resources, and the component is isolated enough to work on alone.\n- What you actually buy from a language switch is not "faster code", but the disappearance of GC in the Discord case and the ability to share resources across threads in the Cloudflare case.\n- Discord only moved to Rust after two tuning attempts failed, and Cloudflare evaluated its options every quarter for several years before finally building Pingora.\n- Instagram instead just turned off GC in its Python and immediately became 10% more efficient, so a language considered slow is not a problem when used correctly.\n- As long as a query, an index, or a configuration is still unfixed, the same problem just moves to the next language, along with the learning cost.
Why did Discord eventually move from Go to Rust?
What they moved is called Read States. Its only job is to record which channel and which message you have read. Simple, except this service is called every time you connect, every time you send a message, and every time you open one. So it touches almost everything you do there.
The scale is large. There are billions of Read States, one per user per channel. Each cache instance holds millions of users and tens of millions of Read States. Every second there are hundreds of thousands of cache updates and tens of thousands of database writes.
The symptom was odd but consistent: latency and CPU spikes every 2 minutes or so. The cause is that Go forces the GC to run at least every 2 minutes, regardless of how much the heap grows. From the graph on their blog, CPU rose from around 20% to 30–40%. Average response time went from around 0–2 ms to around 25 ms. p95 jumped from around 0–10 ms to around 300 ms. Note that this is p95, not p99, and all these numbers came from reading a graph, so they are estimates.
Now the important part: they did not switch right away. They tried tuning first, and both attempts failed. First, changing the GC percent at the endpoint. It made no difference, because their allocation rate was not fast enough to trigger GC more often. Second, shrinking the LRU cache. The spikes did shrink, but p99 went up because the cache missed more often and the load moved to the database.
So they had hit a wall. Not a lazy wall, but one they had measured and tried to break twice. Only after that did they port to Rust, and by their account the latency is as good as the Go version, and there is no spike at all. With basic optimization alone the Rust version already beat a hard-tuned Go version. After profiling again, Rust won on latency, CPU, and memory.
A few days later the cache capacity was raised to 8 million Read States, something impossible in the Go version because its GC grew heavier over time. As a result, average response time came down to microseconds, and Max @mention came down to milliseconds. But read their own footnote: they do not think you should rewrite everything in Rust just because you can.
Cloudflare: faster code, or architecture?
The second story comes from Cloudflare, which replaced NGINX with Pingora. According to their write-up on Pingora, the new system serves more than 1 trillion requests per day. It needs only a third of the CPU and memory of the previous proxy infrastructure. In production, that saves 70% CPU and 67% memory at the same load. TTFB also dropped by 5 ms at the median and 80 ms at p95.
Now the part that gets missed most. Those savings are not because the code is faster. Their old service was already sub-millisecond. What got faster is the way it shares connections. 🤯
The problem is in NGINX's architecture. One request is served by one worker only, which is one OS process. Load is not balanced across cores, and the connection pool is per worker. So the more workers you add, the worse the connection reuse ratio gets. It is like everyone having a water cooler at their own desk. You need a drink, the cooler next to you is still full, but you cannot take it, so you buy another one. Adding people does not make the existing coolers more used; it just keeps adding new ones.
Pingora was built with multithreading, the async Tokio runtime, and work stealing. They also wrote their own HTTP library because much of the traffic on the internet does not obey the RFC. As a result, new connections created per second dropped to a third. For one large customer, the connection reuse ratio rose from 87.1% to 99.92%, and new connections to origin fell 160x. Their blog puts it as: "we save 434 years of handshake time every day."
There were other reasons they moved. C is not memory safe, Lua is not performant enough and has no static typing, and the NGINX community moves slowly. After serving several hundred trillion requests, Pingora has never crashed because of its service code. The system was then open-sourced on 28 February 2024.
So what do you actually buy when you switch languages?
The two wins above came from two different mechanisms. At Discord, the GC disappeared. At Cloudflare, the gain was sharing resources across threads. That is it. Not "Rust is faster" by magic, but one specific constraint finally vanishing.
If your problem is not one of those two, switching languages will change nothing. A slow query stays slow in Rust. N+1 stays N+1 in Go. You will just have the same problem with different syntax, plus a learning cost that is not small.
Knuth's quote often gets cut into a slogan. The full version reads: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." The slogan gets used to avoid measuring, when Knuth actually asks you to know which code sits in that 3%. The only way to know is a profiler.
What if GC really is your problem, but the language is Python?
Here is some calming news. If GC really is your problem, you don't always need a new language. In January 2017, Instagram turned off GC in its Python and immediately became 10% more efficient. No language switch, no framework change, just one feature turned off. 🙌
The context is this. Instagram runs multi-process Django on a uWSGI pre-fork model. One master forks itself into dozens of workers that serve requests. Ideally, memory shared between workers stays shared.
But the finding was different. Shared memory per worker dropped from around 250 MB to around 140 MB, roughly a third, within seconds of a worker being born. The reason is that Python uses reference counting. Every time an object is read, its counter is written. That write triggers copy-on-write in the kernel, so pages that were shared become owned by each process.
Think about that for a second. The symptom "GC makes latency bad" shows up in three different places: in Go (Discord), in the JVM running Cassandra (Discord again), and in Python (Instagram). The answers differ. Discord switched languages, Instagram changed a setting. What was wrong was not the language, but the decisions taken around it.
Sometimes the bottleneck isn't even the language
This is the counterexample, and again from Discord. In 2023 they moved from Cassandra to ScyllaDB, and 177 nodes became 72 nodes. P99 for fetching historical messages dropped from 40–125 ms to 15 ms. Message inserts dropped from 5–70 ms to 5 ms.
The real root cause was hot partitions, uncontrolled concurrency, and GC in the Cassandra JVM. What solved it was request coalescing plus consistent hash routing (written in Rust), and a database free of GC. So even in a "we moved to Rust" story, most of the win came from design and from choosing the right components.
As a bonus, the migrator was written in Rust and its estimated migration time changed from 3 months to 9 days. The peak ran at 3.2 million messages per second. So Rust still has a clear place, but that place is narrow and specific.
When can a rewrite be considered?
Spolsky's reasoning holds up: old code is already in use, already tested, and hundreds of bugs have been found and fixed in it. Rewriting means throwing all that knowledge away. He said of performance too, "1% of the work gets you 99% of the bang." If you have not looked for that 1% yet, don't jump to 100%.
So a rewrite is the last decision, not the first step. There are three questions you need data for.
- Have the cheaper options run out: query, index, caching, N+1, and connection pool? Remember, Cloudflare evaluated its choices every quarter for several years.
- Is the bottleneck proven to come from the runtime or the architecture, not from your own code?
- Is the component isolated enough? Discord moved one service, and Cloudflare replaced only the proxy layer.
If you cannot answer all three with data, the answer today is not yet. And that is a healthy answer, not a sign you are a coward.
If you still have to move, which one?
This is only an opinion. Pick Go when your load is dominated by I/O and you need to be productive fast. Pick Rust when your load is CPU-bound, you hold a large cache in RAM, you build network or storage infrastructure, or you need a flat p99 with no tolerance for GC pauses.
And if your Python or Node still runs fine after every cheap fix has been tried, nothing forces you to move. A language considered slow is not a problem when it is used correctly.
This week, open the profiler first
So back to that endpoint that spikes. Before writing the first line of Rust, open the profiler this week. Find out whether the spike comes from GC, from a query, or from a connection pool that is not shared. If the answer is one of the first two and it can only be fixed by moving runtime, you have the data to convince your team. If not, congratulations, you just saved months of work. 😄