Solana Labs Reveals New Resiliency Measures After Outage

After an irreversible outage a couple of days ago which saw bots swarming the Candy Machine NFT minting tool built on Solana, the protocol's core developers have released a set of new resiliency measures that are slated for implementation on its mainnet.

The outage, which happened at approximately 20:30 UTC on April 30th saw Solana's Mainnet Beta cluster experiencing a temporary stall for its block production. This went on for around seven hours, with a massive number of inbound transactions flooding the network for over 6 million TPS (transactions per second), effectively crippling the capacity of individual nodes.

Rumors of a threat actor releasing a denial of service (DoS) attack on the protocol were firmly denied by Solana Labs, who then claimed that the post-event analysis reveals how bots were programmed to overload the Candy Machine NFT minting tool in order to seize or acquire new NFTs.

"The specific reason why consensus stalled was due to validators running out of memory and crashing. The root cause of the high memory usage was insufficient votes landing to finalize earlier blocks, preventing abandoned fork cleanup. The number of forks validators had to evaluate exceeded their capacity to do so, even after a reboot, necessitating manual intervention," stated Solana Labs.

The outage promptly caused $SOL prices across the board to decline by roughly 13%. For this reason, Solana Labs has decided to reveal details and insights on how such a similar case of network instability can be mitigated in future versions of the protocol.

According to Solana Labs, their devs are currently building an interoperability protocol based on QUIC, a general-purpose transport layer network protocol built by Google's Jim Roskind in 2012. Currently, Solana's mainnet is layered over a UDP-based protocol (User Datagram Protocol), which is prone to threats and abuse from threat actors and malicious groups such as those who launched bots on the Candy Machine NFT minting tool. The new interoperability protocol will open more options to adapt and optimize data ingestion for the Solana mainnet, while also making transactions more effective over time.

Alongside this new development, there is also a stake-weighted Quality of Service (QoS) parameter being developed by Solana Labs, one that would prioritize users based on the value of their staked tokens. With this new parameter, the current rule of a first-come-first-served basis for queueing transactions will no longer be implemented. Fee prioritization is also in the pipeline, according to Solana Labs, with the full implementation planned for release on v1.11 of the Solana Mainnet.

As crypto grows in popularity and adoption, networks like Solana must continue to develop and implement new resiliency measures to ensure the stability of their systems. The recent outage, which saw Solana's Mainnet Beta network experiencing a temporary stall for block production, has somewhat forced Solana Labs to unveil features that are still being developed, in order to protect the protocols integrity and retain confidence from Solana users and investors.

Disclaimer: This article is provided for informational purposes only. It is not offered or intended to be used as legal, tax, investment, financial, or other advice.