The Reputation Value Chain

Photo by Stephen Dawson on Unsplash — (aka. obligatory Medium cover image)

Introduction

Web3 ecosystems can benefit hugely from establishing a robust and credibly neutral reputation layer. There are a number of ways we can think about what this looks like, but lets start with what an ecosystem needs in terms of what a reputation layer could provide:

• People need a way to find high quality dapps, DAOs and communities that are relevant to them, based on their interests and values.
• Dapps need a way to find product-market-fit, and to do this they need some analytics to tell them what sort of customers are out there, and what those customers want, what communities their customers are members of, where they are based etc.
• Dapps need a way to convince individuals they are trustworthy, and individuals need a way to establish some level of trust in a dapp, without relying on a review system run by a centralized intermediary that will likely develop mis-aligned incentives (as we’ve seen in web2).
• Dapps and DAOs need a way to prevent sybils and bots, which just end up pushing out real users.
• Dapps and DAOs need a way to reward loyal customers / members without incentivizing airdrop-hunters who aren’t going to stick around and who just create noise.

And as well as that:

• We need all of the above without needing someone to store a giant database of people’s data, or to to change the order of search results or alter trust scores in return for payment, or to censor results based on requests from the government in some jurisdiction. Basically, it needs to be credibly neutral.
• We also need a way to try to protect people from having their PII put on an immutable public ledger without their consent by unscrupulous actors.

How do we get everything we need from a robust reputation layer without repeating the problems that we’ve seen with reputation systems in web2? e.g. lack of transparency, mis-aligned incentives, ulterior motives, and mis-information.

The Reputation Value Chain

A reputation layer will likely be composed of a number of different actors in a value chain. A mental model for the value chain for reputation can viewed as thus:

To break down this value chain a bit more:

• Users are individuals or organizations that derive some value fom their reputation. Reputation can mean many things, but we could think of it like an individual’s LinkedIn Profile, a company’s Glassdoor profile, a service or product’s TrustPilot score etc. There is a wide variety of ways in which reputation can be valuable and many form it can take.
• Issuers enable data points to be created, so for example, a provider of an online course is an issuer of a certificate that attests to completion of the course. In web3 this can be a dapp that provides proof-of-humanity, a dapp review, a badge for completing a quest etc.
• Reputation Protocols provide the algorithms that derive some summary data or score from the aggregate of the data created by the issuer’s of attestations. They focus on the inbound logistics of acquiring raw data from many sources. Their “manufacturing process” involves applying custom algorithms to the aggregate data to derive valuable reputational data, and then distributing that reputational data to customers (which sometimes involves them being an issuer).
• Dapps / DAOs harness the summary reputation data created by RPs to understand their customers and members better, to develop product-market-fit and attract customers that align with them, to improve customer retention and customer loyalty, and to reward members that contribute and add value and to make their UX better.

The following section goes into more detail about the various actors in this value chain and the roles they play in the reputation layer of an ecosystem.

Actors in the Reputation Value Chain

Credential / Attestation issuers

These are the teams that are issuing credentials / claims / attestations. These can be broken down into to broad groups:

Trusted Issuers

These are issuers that attest to something and sign off on it. They do this through the form of a Verifiable Credential, or some sort of SBT or on-chain attestation. I’ve called them trusted issuers because it usually requires the verifying party to trust the issuer to some degree. An example of this is a KYC service, or it could be as simple as an online course that offers a qualification or certification, for which is issued a credential or attestation.

Trustless Issuers

There are a number of teams that have developed solutions based on sophisticated cryptography, that can create credentials / attestations based on a zero-knowledge proof, usually derived from a TLS session with some other web2 service. They can attest to anything from your Twitter handle, to your bank balance, to the number of Uber rides you’ve taken. The verifying party doesn’t trust the issuer directly, but rather they trust the original datasource and the fact that the technology that the issuer has developed is open-source and peer-reviewed, and leverages MPC or TEEs so that the user is in full control and maintains complete privacy.

Projects that have developed this sort of technology include, Clique, PADO Labs, zkPass, Reclaim Protocol, Chainlink (via DECO), and of course the original TLS Notary by the PSE group in the Ethereum Foundation.

Reputation Protocols (RPs)

These are actors that index data from both on-chain and off-chain sources and compute a reputation score or reputation profile based on some algorithm. These reputation scores are focused on a variety of use cases, ranging from credit scores, to developer profiles, to web3 analytics, to smart contract / dapp trust scores etc. Below are some examples:

• Trusta Labs is an RP that provides very sophisticated reputation management for individuals, and analytics for dapps. Individuals can analyze their on-chain behavior and assess their value score across five dimensions (Monetary, Engagement, Diversity, Identity, Age). They can also visualize your progress on their web3 journey and discover more.
• {r}elinkd is an RP that has developed a sophisticated blockchain addresses scoring protocol and dapp, that has been built for creating a trustworthy digital identity layer. They also issue these reputation scores as both Verifiable Credentials and SBTs.
• Aspecta is an RP that allows individuals to link their valuable data across GitHub, Twitter, Stack Overflow, MetaMask, etc. and to unlock a unique Aspecta ID. This allows individuals to aggregate Web2 & Web3 footprints into an AI-generated reputation profile. They are one of the few RPs to leverage AI, and are arguably the leader in this regard.
• HAPI is an RP (and much more) specializing in security and risk profiling. Their decentralized security protocol that detects and prevents malicious activity in the blockchain space. It utilizes external and on-chain data to provide accurate information on compromised wallets and malicious activities and can be integrated into DEXes and DeFi protocols to prevent money laundering and enhance cybersecurity.
• dappSheriff is an innovative RP that provides trust scores to dapps based on user feedback and reviews. Each review incorporates the number of transactions and number of contracts that the reviewing wallet has interacted with, with respect to the dapp being reviewed.
• ReputeX — web3 analytics platform that helps developers to understand users and user behavior based on on-chain activity, as well as aggregating reputation data from other RPs.
• RociFi is an RP and issuer of credit score attestations as NFCS (non-fungible credit scores).
• Masa are issuers of the Masa Credit Score Soulbound Tokens (MCS SBTs, in and uses Cred Protocol’s web3 credit scoring technology as it’s RP.
• Morfyus are both an RP and an issuer . They issue attestations called “Kreds”, which have a reputation score that grows as users connect, complete work, and receive recommendations.
• Kleoverse is a similar RP and issuer. They provide “Skill Scores” which aggregate a user’s relevant achievements, including GitHub contributions, courses taken, and projects built. Kleoverse then issues attestations called Badges which are issued as SBTs.
• SOURC3 is a decentralized, on-chain platform for reputation management, that allows developers to build on-chain reputation by giving provenance to GitHub contributions.
• Nomis is a multichain identity protocol that helps users build, manage, and leverage their on-chain reputation and web3 projects to make data-driven decisions. Nomis scores are derived from a wallet’s on-chain activity and are issued as attestation via NFTs. Nomis enables users to leverage their on-chain reputation to get personalized web3 experiences. It also helps dapp developers to offer custom terms, rates and powers to their end-users based on the reputation scores.
• digitalsocial.id is building trust through non-transferable reputation profiles for digital identities in web3. Individuals initiate some activity such as checking out in a store, making a donation, or undertaking some work. They then connect their wallet before they check out or before making a donation etc., and the protocol issues an attestation as an SBT.

The methods that these RPs use to generate scores or profiles is also quite diverse, from classical reputation algorithms such as Eigentrust to newer machine learning algorithms.

The sources can also be quite diverse, ranging from on-chain to off-chain data, for example, Omnid lists 78 different sources for the reputation score. This is an area where registries can complement the data pipelines that reputation protocols build, and can allow them to derive even more powerful reputation data.

Attestation Registries

Attestation registries play an important role in the reputation ecosystem as verifiable data registries, but are well suited for use in web3 because they are decentralized. Furthermore, on-chain attestation registries are directly interoperable with other dapps via smart contracts.

Taken from the W3C Verifiable Credential specification
The difference with on-chain registries is the attestations are issued directly to the registry, not the holder

Public attestation registries allow for reputation protocols to build powerful models by being able to discover and index more data faster. They allow for recursive reputation, whereby reputation profiles and scores can be issued as attestations that can be consumed by other RPs.

Currently RPs index data from various on-chain and off-chain data sources. This is quite an overhead considering every on-chain data source has its own idiomatic smart contract with its own specific data structures and function calls. Off-chain data is even more of an overhead, with API authentication and limits and different protocols. Many reputation protocols either maintain their own database of data points or scrape the data on-demand, but developing an integration for every data point requires development effort. This is where being able to retrieve data from consolidated attestation registries makes things much easier, and leads to more powerful reputation algorithms.

Whether a credential / attestation should be on-chain or off-chain is a tricky question. See below for more details.

Dapps, DAOs, Users

In the end, issuers and RPs exist to serve the needs of both individuals and of dapps, DAOs, and protocols, as described at the start of this post. The issuers and RPs create a flywheel of self-reinforcing reputational value, whereby attestation issuers can create data points that help individuals and organizations to find each other, and establish trust using credibly neutral mechanisms.

Using these credibly neutral mechanisms such as on-chain registries, RPs can compose claims / credentials / attestations from an ever increasing number of sources via issuers, and can create more robust and sophisticated reputation profiles and scores. Reputation Protocols can themselves become issuers as they feed back reputation scores as attestations into the registries, creating meta-attestations. Issuers can themselves have reputation profiles / scores, which can help discern the quality of attestations, which in turn makes the reputation protocols stronger.

A note on wallets

In terms of managing reputation, most crypto wallets support NFTs, and the vast majority of on-chain attestations are made via NFTs / SBTs. So it could be argued that users can manage their reputation in this manner. But this is a rather crude approach to management of reputation. For instance, how can I group NFTs into multiple different reputation contexts, such as my professional reputation context, or my social context, or my gaming context, or my credit score context? How can I compose NFTs together to form an aggregate view of my reputation, which is more accurate of who I am? Some wallets will allow you to store Verifiable Credentials which are obviously far more powerful than NFTs in many regards and which address most, but not all of these issues and of course cannot be read directly by smart contracts.

Wallet support for Verifiable Credentials

Currently there are a limited number of wallets that allow a user to fully manage their reputation through Verifiable Credentials (Korir et al. have done some good research, and there is a good list here). There are a number of wallets that have been developed that allow users to store Verifiable Credentials, and to respond to requests to selectively disclose the data in them as Verifiable Presentations.

The wallets that support Verifiable Credentials allow for privacy and control of the individual’s personal data. Users self-custody the data, hopefully backing it up to an encrypted data store. For the most part this is enough. For example, a GitCoin passport can give you a Verifiable Credential, and you can easily store this in a wallet.

The trade-off here is that the personal data is not publicly discoverable, insofar as the individual needs to manually disclose the data to each verifying party, which is exactly the way it should be in most cases, but also means that there is a barrier to developing valuable analytics without falling back to invasive web2 analytics platforms, or more rudimentary on-chain analytics.

Reputation protocols can become much stronger if they can read multiple levels of interpersonal data from a public registry. Take the example of the reputation score for a smart contract module:

A user wants to install a module into their smart contract wallet. The user checks the reputation score for this module’s GitHub repo or on-chain factory contract from one or more RPs. The RPs generate a score based on data points including a smart contract audit, and the number of instances deployed on-chain, perhaps the age of contracts on-chain. The more data they have, the stronger and more robust the reputation score is, for example:

• Have there been any previous incidents with this smart contract? Having an audit is a pretty good data point, and perhaps that’s enough.
• But what if you could incorporate the reputation of the developers? You could look at how many projects they contributed to, what their activity looks like, and how long they have been a developer.
• What about the reputation of auditors? How long have they been around, how big are they? How many projects have they audited?
• What about reviews? Maybe the smart contract is secure, and they have great marketing, so lots of people have installed it, only to find out it’s really basic, or poorly designed. How many reviews does it have?
• Are the reviewers real people? Maybe somebody spun up a thousand wallet addresses, and created a thousand on-chain reviews that were all positive.
• What’s the reputation of the addresses of the reviews? How long have those addresses been around? How many transactions have they broadcast? How many different types of dapps have they interacted with?

Ok, I may have lost the run of myself there, but hopefully you get the picture. Deriving a reputation score from all those data points is very valuable, but very challenging to do if you’re relying on the contract developers to disclose all that information via a Verifiable Presentation. This is where attestation registries make sense. Pulling this information from consolidated data sources instead of individual data sources, or requiring all the data to be disclosed by individuals, allows RPs to push coordination and trust mechanisms to a whole new level. It also means that the data points used in the reputation score are easily verifiable.

In summary, there are a number of wallets that currently support Verifiable Credentials, and there are wallets that offer good support for NFTs / SBTs, but there are only a handful that do both (Ontology are building something quite cool, and Tuum Tech are building a very interesting MetaMask snap). However, there is yet no sign of wallets building functionality for attestation registries. Until wallets support credential / attestation registries, it’s hard to see how users will be able to fully manage their web3 reputations easily.

On-chain vs. Off-chain

By this point, I think we covered the value of having attestation registries. To summarize: they provide a credibly neutral consolidated directory of credentials and attestations, that allows for discoverability, and for reputation protocols to build powerful profiles and scores based on publicly verifiable interpersonal and transpersonal data. On-chain attestation registries provide all of the properties of verifiability and credible neutrality and are a simple primitive that are easy to interact with. It was also one of the first areas in web3 reputation space to be explored (uPort wrote the Ethereum Claims Registry over 5 years ago!).

However . . . we can’t just put everything on-chain. In fact, most data will still live off-chain, and there are several good reasons for this:

• Privacy — personally identifiable information shouldn’t really go on-chain if the individual isn’t explicitly putting it directly on-chain themselves. There’s no hard and fast rule about this (well except for GDPR I suppose), but it’s a good rule of thumb.
• Data-correlation — even if there is no PII involved, and all data is pseudonymous, the more data points there are, the easier it becomes to correlate them, revealing more information about an individual than they have consented to disclosing.
• Cost and Scale — it is prohibitively expensive to store all attestations on-chain. End users will be disincentivized to create some attestations, (e.g. a review of a dapp), if they have to pay for it. If organizations pay on behalf of users, the cost can be huge at scale.

We can use off-chain attestations such as Verifiable Credentials for example, but as we discussed before, this sacrifices discoverability.

Ceramic offers an alternative that is sort of a hybrid approach, where you can store the Verifiable Credential on the Ceramic Network, which is decentralized (it uses a IPFS for storage and Ethereum for notarization), and allows anyone that is running an anchor node (the equivalent of a blockchain archive node) to discover any new schemas / models that are created, and / or any credentials that are published to the network.

This solves the discoverability problem for RPs, and it avoids the cost associated with putting everything on-chain. This won’t suit all use cases of course, as it lacks the ability for smart contracts to interact directly with registry, either when creating attestations, or reading from the registry. In the end, it seems inevitable that we will have a number of different registries with different strengths and characteristics and trade offs.

Reputational Lego

If public on-chain registries for credentials and attestations start to see significant adoption, then we can start to do some really cool things. For instance, we can start to think of attestations as being “reputational legos”, that can be composed together to create reputation constructs from which we can derive value that is greater than the sum of its parts.

As a thought experiment: imagine building LinkedIn from composable attestations. One could imagine taking attestations from an RP that creates profiles based on a user’s GitHub activity (check out Aspecta for how far you can go with this), and perhaps issues them as attestations. Let’s say we combine these developer profile attestations with attestations from their DAO activity made by an RP like KarmaHQ, SourceCred, or Govrn, or attestations made by Devfolio to show their hackathon contributions? What if there was a platform to allow companies to issue attestations to wallet addresses or to DIDs controlled by their employees? What if there was simple attestation that allowed individuals to issue endorsements, (e.g. “0x123… is proficient in Solidity”).

Similarly, you can build TripAdvisor, TrustPilot or Glassdoor, but with complete transparency and verifiability of reputation data. The larger and more diverse the data points in the registry are, the more innovative the reputational constructs will start to emerge from composing those data points together.

Economic Sustainability

Having public goods such as on-chain registries that issuers can put credentials and attestations into can open up a whole new space of possibilities, but what is going to incentivize issuers to populate these registries with data? After all, the most valuable data is the data that takes time and effort to acquire. Issuers are not going to share all this data publicly for free (at least not after they go bankrupt anyway), and RPs will struggle to keep competing on data acquisition without a market of issuers. So how does everyone make a living and keep the whole thing going? There are a few business models currently being explored:

• Attestation issuers can charge for issuing attestations. This simple business has already proven effective for some teams, and there is definitely a market for it. Individuals want to be able to demonstrate things about themselves, from simply being human, to being proficient in a certain skill, to being an expert in a certain area.
• Freemium model: Issuers and RPs can give some of their data away for free, while keeping the more valuable data beyond a paywall, use an on-chain registry as a method of advertising. The more their attestations are adopted, the more widely known they become as a source of valuable data.
• Encrypted attestations: this is actually an exchange of information between two parties. It is likely an encrypted link to an off-chain datastore, or a password protected data store. Why this would be put on-chain is simple: reputation. It allows for a company to show off how many other dapps are consuming their data in a publicly verifiable, sybil resistant manner on a credibly neutral platform. The number of consumers of their data becomes a selling point.
• Scaling services: RPs can charge organizations that use off-chain registries or data stores, to create aggregate proofs and reputation scores based on that data, which can be pushed on-chain and are fully verifiable.
• RPs can create recursive reputation attestations, which are based on attestations that are already in the registry. They can charge for these aggregate attestations, in turn becoming issuers themselves.

What are the risks?

When building any new technology, we need to ask ourselves: “are we able to think about the potential for this to go wrong?”.

We need to ask ourselves what happens when we start building systems for reputation that sit on public, permissionless, immutable ledgers. We’ve seen very problematic and harmful content and discourse appear on web2 social media networks, which has resulted in platforms being compelled to intervene based on both legislative reasons, and because of public opinion. We’ve seen websites and communities that have turned toxic, and that have caused enormous stress to the individuals that have found themselves as the subject of harassment. How do we manage the risks of something like that occurring on a permissionless and censorship resistant public ledger?

Maybe it’s an intractable problem and there are some trade-offs we have to accept, but have we done enough to think about this topic as a community, and to try to quantify what the effects will be?

Moreover, if we fail to give people the tools to properly manage their on-chain reputation contexts, then we could end up actually eroding people’s privacy even further? Orthogonally related to this is the concept of negative attestations: what are the externalities that develop when we allow unsolicited negative attestations? Buterin et al. talked about this in their paper Decentralized Society: Finding Web3’s Soul. Unfortunately immutable ledgers make this data available in perpetuity, and also add a more open-ended and permissionless aspect to the issuance of negative attestations. What sort of negative externalities will this present?

Then there are the security considerations: if we don’t build systems that are secure, then we are opening the door for identity theft on a potentially larger scale than we have seen previously. Fortunately there has been great progress in terms of account abstraction and progressive self-custody, which should in theory help to make people’s wallets more secure, but will developers do their part to educate users on the risks and responsibilities that self-custody entails?

Open Research Questions

I’m still exploring the identity and reputation space in web3. I can honestly say at this stage that I have more question and answers. These are but some of them:

• How does a user know how much their attestations are worth, where can they use them, and what for? How do they understand the value of individual attestations, and also what they are worth in aggregate?
• How can an issuer receive payments as its attestations are consumed? Currently issuers change the subject at the point at which the attestation is issued, but how do you charge based on the amount of times the credential is consumed, without sacrificing the properties of a public registry?
• Verifiability of cross-chain and cross-registry composability. It’s pretty easy for RPs to index data from a number of registries and from registries on different networks, but how easy is it for consumers to verify reputation scores that are derived from multiple registries / networks. On-chain registries improve on the current status-quo of aggregating data from different APIs and various NFTs on numerous networks, but it still might not be easy enough to verify it.
• How do reputation protocols manage profiles / scores that are based on multiple different attestations, that each have different expiration dates? Is a reputation profile only valid until the earliest expiration date among all the attestations used? Or is there a polynomial decay function that can be used, expressed as a series of coefficients across the attestations?
• How can LLMs leverage on-chain attestations? By leveraging linked data to imbue on-chain attestations with semantic context, by linking them to shared vocabularies and ontologies, LLMs can better understand what all these ostensibly random attestations actually mean. This adds a whole new layer of accessibility, by allowing LLMs to index, and to understand, the data that is stored in the attestation registry. By linking attestations together using RDF triples for example, attestations can be easily be ported to a graph DB, and then a vector DB, making them very easy to train LLMs with. This opens a whole design space for interfacing with publicly verifiable repution data.
• How do RPs broadcast a “request-for-attestations” (RFAs)?
  If we think of on-chain registries as the infrastructure for a marketplace for data via attestations, then what form do the bids and asks in this marketplace look like? How does an RP submit and ask for specific data that it doesn’t have or can’t access? Does it submit an RFQ? Are there mechanisms for issuers to provide bids? Does the data come from an issuer single or multiple sources, perhaps individuals? In the latter case, what does the incentivization look like? Similarly, can dapps or DAOs broadcast a “request-for-reputation” (an RFR)?
• Can dapps use something like the DIF’s Presentation Exchange standard to specify what credentials / attestations they require? This would be a valuable mechanism to have in terms of allowing wallets and dapps to easily ascertain if an address can meet the specified criteria based on it’s attestations, and easily identity what is missing.
• How do you bridge off-chain data storage / off-chain attestations, with on-chain discoverability? Where is the intersection and what does it look like?
• Voting and governance of DAOs by leveraging reputation: at what point will we have a sufficiently large and diverse range of data points within public registries to allow for sophisticated voting mechanisms that go beyond quadratic voting, to reputation weighted voting systems such as connection oriented cluster matching?
• How does an issuer qualify an attestation in a standardized way e.g. “I attest to X, but only so long as it’s used for Y purpose, otherwise I’m not comfortable standing over the claim” (e.g. POH but not KYC).
• How do you prevent vendor lock-in with RPs? How can I navigate away from one RP to another RP? If a developer decides to build their dapp based on a specific RP, what happens if they need to move away from the RP at some point in the future, will this be possible? Pinyal et al. have done interesting research in this regard, in which they explore implementing reputation using a common ontology.
• Can we create a DB as well as an analytics dashboard that tracks RPs and issuers and the growth of the ecosystem over time in order to prove our hypotheses about the directions of market growth? We can track SBTs from various issuers, as well as track attestations into EAS, Verax and Ceramic.

Conclusion

Even though Identity and Reputation was one of the first areas of exploration for blockchain and web3, it’s still one of the most complex and hardest to predict areas. There seems to be growing interest in the area of reputation in web3, and hopefully this will result in value chains of specialist teams that are able to build and rich and robust identity layer in the broader web3 ecosystem. I am keenly interested to see how it evolves.

Big thanks to Adam Corrado from Ceramic for review and feedback.