Easy way to assess DeFi protocol risks

We're not here to be degens. If you're in DeFi, recognize that you're playing in a frontier space with lots of hidden risks, and it pays to be as attuned as possible to these. Let's get you started!

While the benefits of DeFi proclaim increased transparency, accessibility, and decentralization — it does not mean that the protocols themselves are free from underlying risks. While the premise that “code is law” is meant to guard against the inherent corruptibility of human nature — the irony is that code is not of itself anti-corruptible. Code can be good-intentioned, but still be hacked, manipulated, and tampered with outside of its jurisdiction.

While the benefits of crypto (specifically the fact that everything is on-chain and transparent) entail that the public is able to assess and understand the vulnerabilities in an open-source manner, it doesn’t mean that crypto is invulnerable. There have been many hacks that have occurred in the space — from a $600M hack from an NFT platform Ronin Network, to DeFi protocols like Cream Finance and Badger suffering >$100M vulnerabilities — that show us that there is still immense value from being savvy users and investors, especially in such an early-stage space.

And it’s not just about code risk too. While the premise of DeFi is all about decentralization — there are many centralization vectors that could be exploited as well. Just because it’s on the blockchain doesn’t mean it eliminates centralization altogether. Don’t be naive. DeFi protocols could have their administrative controls dictated by anonymous multi-signature wallets that could rug-pull you without reputation consequences (Convex, for example, had a centralization vulnerability that could have resulted in $15B being rug-pulled if the owners of the then majority anonymous multi-sig followed a sequence of steps), or have prices be dictated by a single source of centralized truth — an oracle — that can easily be manipulated.

At the end of the day, we’re not here to advocate using crypto to be complete degenerates. We’re playing in a frontier space with lots of hidden risks, and it pays to be as attuned as possible to the possible risks out there. You should absolutely invest in doing your homework, rather than ape into anything that offers crazy yield. As the term goes: “If it’s too good to be true, it probably is.”

In this article, I will go over the important risk signals that I personally consider when investing, or using any DeFi protocol. There are other resources out there I like (such as DeFi Safety) — but I honestly haven’t found a good source that breaks it down in an easily understandable way. As much as possible, I try to quantify this with the protocols I utilize, and backwards test this with protocols that have went through critical vulnerabilities. This way, I’m able to gauge an “upper and lower bound” on how various protocols perform, and have a like-for-like manner for assessing risk. We’re likely unable to eliminate all risks, but when we do this, we become more analytical and rigorous in our approach. In a world that does not have much protection to shield you — this is a fundamental skill. Let me get you started.

Smart contract risks

When we assess protocol risks, perhaps the most important thing to start off with is to pick up signals of potential smart contract vulnerabilities. If the smart contracts aren’t safe — nothing else really matters. Remember that you’re deploying your capital for a period of time, and while the protocol may have not been hacked in the past — it doesn’t necessitate that it won’t in the future. DeFi is ripe with bad actors, and within this we try to assess the signals of how to determine the integrity of a protocol’s smart contracts. It’s important to note however, that none of these individual vectors are enough to fully determine risk. Risk comes from a combination of these factors interplaying with one another.

i) Audits

Before engaging with a DeFi protocol, it’s always crucial to look at their audit history (which should all be publicly available and easily accessible; if they’re hiding it — it’s a sign to watch out for). In particular, we can look at 4 main factors: the quality of auditors, the # of audits done (and within that, assessing how many man-weeks committed), the recency of audits, and specifically whether a smart contract has been audited or not (in the case that a new mechanism was launched, and the protocol has not yet audited it before launching to the public).

Quality of auditors: Much of this is qualitative and based on track record and involvement within the communities.

Number of audits / number of man-weeks: We should try to assess the length that the protocols were audited for and how many audits there are (to avoid reliance on say a single audit).

Recency of audits: Audit recency is a signal of whether or not a protocol has continued to maintain focus on ensuring that their smart contracts are as secure as possible even post-launch. This consistency is important — as it’s rather easy to have a protocol approach an audit as an administrative process of something to do prior to launch, but then completely let it go after the protocol has started to run.

Specificity of audits: This is more binary than the rest (and depends on your risk tolerance as well), but especially for lower-risk investors, you might want to drill down on specifically whether the audits covered the specific DeFi mechanism you’re trying to engage with within the protocol.

ii) Bug bounty programs

The open-source nature of the crypto world entails that anyone (from individuals to institutional auditors) can help ensure the security of the protocol. Protocols can try to promote this behavior in a good-natured way (called white-hat hacking) by incentivizing hackers to expose vulnerabilities to them in exchange for a sum of money, and also public recognition from the community. When bug bounty programs are attractive, strong, and secure — there are more vulnerabilities that can get caught before malicious attackers get to them (ultimately creating a much safer protocol). There are two signals that I personally look at:

Payout amounts: Protocols vary in their bug bounty programs in terms of how much they pay out. In particular — I look at the payouts for the most critical vulnerabilities uncovered.

Dynamic vs. static: Bug bounties can be either managed internally (called “static bug bounty programs”), or outsourced to a provider like ImmuneFi that provides an external layer of communication between white-hat hackers and the protocol (called “dynamic bug bounty programs”). The risk of bug bounties being statically managed is two-fold: first (and most important), is that there might be a risk of white-hat attackers disclosing something that the internal team then exploits for themselves. The second risk that exists for static bounty programs is also that they tend to be slower. A dynamic intermediary has the management of white-hat disclosures as their first and upmost priority. However, a static program tends to be managed by an internal development team who might not have this as their main priority (among all the other things they do).

iii) Historical vulnerabilities

While history may not necessarily predict the future (as Nassim Taleb would say), we should look at a protocol’s historical vulnerabilities as a lens to see whether there have been any systemic indicators of risk present.

Number of maliciously exploited vulnerabilities: The number of malicious vulnerabilities — especially if they are frequent and occur over and over again — is an indicator of systemic risk (no matter how big, or how small they are).

Nature of vulnerabilities: While the number of malicious vulnerabilities measures a sense of “breadth”, the nature of vulnerabilities (both white-hat and malicious) tries to measure a sense of “depth”. While white-hat vulnerabilities are bound to be uncovered — ideally none should be terrible enough to cause such a significant wipe-out of funds.

Response post-vulnerability: This is mostly a qualitative judgment call — but for protocols that have suffered public vulnerabilities or fallouts, I like to see how they’ve handled the response. Were they quick to admit their mistake, respond fast, and handle all risk vectors immediately — or on the other side of the spectrum, were they slow and blaming others?

iv) Time on main-net / since launch

For non-physical items, the Lindy Effect states that the length of time it will last going forward is proportional to the length of time it has existed thus far. It’s important to capture this when measuring signals of smart contract risks — as it’s likely that just by managing to exist successfully for a longer period of time, a protocol is more secure. While there are assumptions baked into this, I see time on main-net as one of the critical signals of smart contract risk to measure.

Centralization risks

Beyond smart contract risks — we must also pay attention to risks caused by centralization both on the administrative and mechanistic side of these protocols. Even if the underlying code itself is safe, there are potential risks that could materialize by bad actors or overly centralized dependencies.

i) Protocol administration

Protocol administration covers ownership functionality and risks that occur if operators act out of bad faith (by intention or by hackers), or decisions are made out of a centralized source that may — for example — be made against a community’s best interests. We want to gauge the potential for this to potentially happen, as it has in the past.

Number of unique multi-sig admins: It’s not only about the fact whether or not a protocol has a multi-sig control that matters, it’s about making sure that the multi-sig is actually working as intended (aka the keys are owned by unique people). This was shown through the Ronin Chain hack as although the multi-sig had 9 keys — 5 of the keys were actually owned at time of the hack with one owner (crazy!).

Publicness of admins: This is important as “publicness” protects against bad faith actions, given the reputational risks that someone incurs if they do something against the community’s will.

Time-lock: A time-lock for decision making is a way to institute a “cool-off period” in order to similarly prevent “bad faith” actors from doing damage in a sudden way. However, many DeFi protocols choose not to adopt a time-lock just because their mechanisms require a lot of changes made through admin controls on a day-to-day basis. As long as they state this upfront and it’s clear – it should be fine. In the case of community-owned protocols (where admin controls are governed by a vote) — the “vote” essentially acts as a time-lock mechanism so we can treat the vote duration in the same manner as the time-lock duration.

ii) Oracle centralization

One core flaw of smart contracts is that they are deterministic by nature — and by that I mean they need clear rules to execute in order to be fully trustless. This is intentional design. However, this deterministic nature also means that blockchains cannot inherently connect to the real world. Oracles aim to get “real world data” (in the DeFi case, price) through various mechanisms: either on-chain or off-chain. We’ll talk about two mechanisms: TWAP oracles and off-chain oracles.

TWAP oracles (or time-weighted average-price oracles), determine price with on-chain data by calculating the mean price of an asset during a specified period of time. This oracle is internal to an AMM — and relies on the liquidity pool within the AMM itself to determine price. Curve, for instance, uses a TWAP internal oracle to determine prices in their AMM.

Off-chain oracle networks like Chainlink allow you to connect to real-world data by requesting data from nodes off-chain. The key word here is that these off-chain oracle networks need to be decentralized, as using a centralized node defeats the entire purpose of using blockchains in the first place. Decentralized networks reach consensus with off-chain data by utilizing aggregated data from multiple trusted sources and posting that to the smart contract, whereas centralized ones well… rely on one source (you can imagine how this is prone to centralization vulnerabilities).

This is where you can be qualitative in judgment based on needs. In the case of Curve, due to the fact that they mostly deal with stable assets, having a TWAP oracle works good enough (and limits any vulnerabilities with dealing with external oracles). However, TWAP oracles are not necessarily the best when dealing with assets of high volatility — and do not have the benefit of taking in the aggregation of multiple sources of data.

In most cases, I’d rank a decentralized, trusted external oracle provider as highest (e.g. using Chainlink), completely centralized oracle as lowest, and make qualitative judgment calls on what’s in the middle.

Next steps: How to put this into action

Now that you understand the key risk vectors when it comes to DeFi protocols — you should develop your own framework for being able to quantitatively compare protocols you’re using like-for-like. I leave this open-ended because some of these depend on your individual needs.

Finally — once you have this done — use the framework systematically to assess risk when you engage with protocols. At this point, because you understand the fundamental principles, it should be easy to instill this as a habit. Refresh and revise based on new occurrences (e.g. vulnerabilities) that happen. This is how you develop a strong data-based intuition. As Ray Dalio says:

“Most people do not look thoughtfully at the facts & draw their conclusions by objectively weighing the evidence. Instead, they make their decisions based on what their deep-seated subconscious mind wants and then they filter the evidence to make it consistent with those desires.”

Don’t be a subconscious thinker. Use the evidence in the world to make strong decisions, and refine your thinking over time. This is how you become a first principles thinker instead of being swayed by opinions or internal biases.

Conclusion

If you found this helpful, I’d love to hear from you! Follow me on Substack, Twitter @playmatenate, or LinkedIn (or all 3) to hear more. And if you have any thoughts or additional considerations for risk vectors that I did not cover that you feel are extremely important — please feel free to hit me up as well. Totally down to chat and add this in so that this can be a public resource for anyone venturing into DeFi.

All the best!