CIP 118: Too much solution for not enough need

Abstract

CIP 118 fails to demonstrate sufficient need for the ledger changes it proposes. The use cases CIP 118 cites can already be supported. No evidence is provided that the specific contexts which cannot be supported are critical, nor is there an attempt to quantify the claimed performance improvements. Accepting CIP 118 will not only add to ledger complexity, it is an open invitation for a series of subsequent ledger changes, both tweaks and fixes of missteps, and as more of the inexhaustible list of intents gets shoe horned in. Efforts and energies would be much better spent making accessible and utilizing what we already have.

Context

Of the CIP

CIP 118 proposes ledger changes to accommodate “nested transactions”. At present, a participant of a transaction must sign the transaction body, complete byte for byte. With CIP 118, a participant could provide witness to an “intent” or “subtransaction”. A subtransaction does not need to follow the same ledger rules as a transaction. For example, a subtransaction does not need to be balanced.

Of the author

All solutions worth the time of day ought to be solving actual problems. It may be highly subjective as to whether or not something is a problem. Some say feature others say bug. Even if we agree that something is a problem, we may still disagree on its relative severity. What to me is a mild annoyance, may to you be an absolute deal breaker.

The problems I think are the ones most worth solving are generally the ones that have caused me the most trauma. While it is not so hard to extrapolate from personal experience and otherwise sympathize with other people’s problems, it is harder to judge their relative severity, and thus a solution’s relative importance.

While working on Cardano, designing dapps, I have no shortage of pain points to point at. I have spent relatively little time architect-ing systems that might be described as something along the lines of “singular, many to many”. Token swaps would be an example: a participant wishes to partake in a singular event, potentially with any other participant(s). By the end of the transaction, a user has no further expectation of, nor obligations to, other users. I have spent much more time thinking about “on-going, few to few” systems such as Lightning Networks, where engagements are p2p enduring and with only a small subset of the overall network, or Subbit.xyz, where a consumer pays their provider repeatedly over a long period of time.

Of the piece

This piece came about in preparation to participating in this roundtable discussion.

Main

Making ledger changes is not something to be undertaken lightly. One should ask:

1. Is the problem addressed sufficiently important?
2. Does the proposal adequately address the problem(s) claimed?
3. What are other consequences of the proposal?
4. What are the alternatives?

The following is mainly concerned with question one, with three and four considered in passing. We do not dwell on question two.

Use cases can already be accommodated

It is immediate that any unsafe behavior found in the ledger is a problem sufficiently important that the ledger ought to be changed. For example, a way to submit transactions that can result in a DDOS style attack. The case for a set of ledger changes that only add or augment features and functionality is much less immediate, and should be demonstrated with receipts.

The use cases cited by the proposal can be feasibly accommodated with the current ledger rules. The proposed use cases are outlined in a CPS, referenced in CIP 118. They are: atomic swaps including limit orders, market orders, and babel fees; dapp fee sponsorship; and access to yet to be determined data. Some implementations, albeit nascent, are or will soon be available (Aquarium, Bullet).

CIP 118 is not sufficiently explicit which characteristics of the use cases the authors deem necessary to support. For example, if it includes that a user must be able to spend a UTXO at a payment key address with only a partial witness, then a ledger change is of course necessary. If this requirement is relaxed by, say, we can assume it is a validator address, or there can be some multi step communication between parties, then such functionality can be supported without ledger changes, as it is in the aforementioned examples.

At the time of writing the proposal states:

There are off-chain solutions being developed for the purpose of achieving the same (major) goals, including fee coverage, atomic swaps, DApp fee sponsorship, collateral sponsorship, etc.

This seems reasonable, but its continuation is more moot:

This gives us indication that it is a worthy pursuit to solve this problem at the ledger level, in a way that is cheaper, more convenient, maintains formal guarantees, requires less off-chain communication, and does not require deposits or interaction with smart contracts.

This is insufficient. All dapp logic can be cheaper, more convenient, etc if it is written into the ledger.

Token swapping is dominant because we can already do it relatively well, and we still find other applications very hard to build. In some respect the “singular, many to many” paradigm is what L1s are designed to handle. You are free to send funds to anyone and they to you, but its expensive and slow - a necessary consequence of a secure, decentralised system. (Insert joke here about mockchain that solves the blockchain trilemma.) Token swapping is one of the few problems for which we’ve actually got some solutions. Not perfect (too centralised, too siloed, too expensive, etc), but they exist. Moreover, any such problem could be solved with vanilla transaction signatures, tooling permitting. Meanwhile, “on-going, few to few” paradigms have so much potential that is being stifled by other problems.

Neither of us are providing receipts for our contradictory interpretation of the dominance of token swapping. The difference is, this is not a CIP.

The onus is on the proposers to make a strong case that either there is some specific formulation of the use case that is not currently supported and is critical, or that the proposal will make game changing improvements to what is already feasible. Neither of these are presented.

Complexity is the root of some evil

When building software, the primary questions are commonly “what happens if …”. The difficulty to address these questions is directly related to the scope/ complexity of the execution context. For Cardano dapps, the Plutus script context has been kept intentionally narrow (ie simple) so that answering “what happens if …” questions remains manageable. It is still wide enough to build many many things.

Each change to the ledger generally increases complexity. Backwards compatibility is supposedly a must: if the Plutus Code runs today, it should run “forever”. There are many terms and conditions applied to this. The legacy of Byron has been a ball and chain on many a tool developers’ foot. Two different types of address, different rules on transaction logic etc. Since Vasil, we have two different ways to handle datum; with Chang, three. The more cases that need to be handled, the more code is needed to the handle them, the higher the maintenance burden and the higher cognitive load when reasoning about them.

We do not always have the luxury to restrict the aspects of the ledger we are concern with. When building from scratch and no need to interop with existing scripts, we likely do have this luxury. We’ll use the newest version of Plutus, and have no concern about what came before. But what happens if the tool we want to build is a general transaction composer to reduce user fees and chain traffic? What happens if we’re trying to build an application, that will consume funds from an existing validator address? These are not very exotic examples, and the differing versions and behaviours are now our problems. What happens if someone did compose a Plutus V3 and Plutus V4 script in a single transaction? It couldn’t be a nested transaction, but a normal one? An intent might cover burning tokens, but now the end user, and/ or the tooling, need to know to handle these different cases. Complexity can grow exponentially, and erodes confidence that we have made the right assumptions or reasoned correctly.

A ledger change proposal must justify the complexity cost they will “forever” leave with maintainers and developers.

Justifying previous changes

There have been multiple previous changes to the ledger. How do previous changes compare to CIP 118 in terms of scale of problem, and scale of ledger change?

Let’s consider Vasil. Three CIPs implemented in Vasil include 31 Reference Inputs, 32 Inline Datums, and 33 Reference Scripts. Claim: The change was demand led, it addressed a fundamental and widely shared problem, and it was a relatively small perturbation of the current design.

The changes were user demand led. Transaction size was the bottleneck and it was for everyone. Transaction size still is frequently the bottleneck in applications. Of course, one can produce scripts that exhaust ex units, but in my experience once other factors are optimized for, the hard bound is often the transaction size. It is still the case that a medium complexity script will easily be three quarters of the max transaction size.

CIP 31 purportedly addressed the problem of broadcasting state. Plutus script execution is narrow. When an application needed data from the chain, it has to be included in the form of spending an input. This was awkward when it came to making the same state available to concurrent transactions. CIP 31 provided a much easier and cheaper mechanism for broadcasting state in this sense. The transaction referencing the input pays in fees and bytes only a fraction of what it would otherwise cost, particularly with CIP 32.

CIP 32 purports to address an inconvenience caused by the former design’s support only of datum hashes at tip. Quote: “This is also inconvenient for the spending party, who must watch the whole chain to spot it”. I find this argument incredibly weak. To create a transaction, a user already needs access to the chain via functions getUtxosAtTip and submitTx. Having another getDatumByHash is not a big deal - it is just a tooling issue. A more compelling case is that it reduces the size of transactions. Prior to the change and in normal application an output datum hash would need to be validated, and so the datum would be present in the witness set, adding to the transaction size. The consuming transaction would then need to include the same data. There are some contexts where this can be improved upon, but CIP 32 essentially halves this problem by paying for the data just once. The impact of CIP 32 is compounded when used with CIP 31: any referencing transaction does not pay in fees nor bytes for the inclusion of the datum.

CIP 33 allowed scripts to be referenced, rather than having to be included in the transaction. As mentioned, scripts can easily exhaust most if not all the transaction size limit. Referencing scripts dramatically reduces the size of script heavy transactions. Generally speaking, of the three CIPs, this was the most significant.

The ledger changes in Vasil were a small perturbation of what had come before. CIPs 32 and 33 did not introduce new functionality in and of themselves. They shifted the need to provide data from the transaction submitter, to the verifying Cardano node. Nodes already did recall for transaction inputs. Adding recall for datums and scripts adds a little extra workload to the nodes. It unlocked new functionality, only in the sense that the same transaction could be much smaller. It also had network performance benefits with respect to potentially higher effective throughput, and/or lower bandwidth and storage requirements. CIP 31, is arguably a slightly larger perturbation than the other two with respect to ledger design change. It is still, however, small.

A brief comparison with Chang and the changes introducing governance: Chain governance has long been a key milestone in the vision of Cardano. In many ways Cardano’s focus on governance is the USP in an oversaturated sea of competitor cryptocurrencies. Governance could not have been reasonably implemented without substantial changes to ledger rules. These changes are justified because of the significance of the capability they introduced.

Ledger change record is wanting

It is a direct trade-off: additional functionality (complexity); additional ways to shoot yourself in the foot. We do not have a stellar record when it comes to ledger changes. There are multiple cases in which ledger designs were introduced only to be deemed incorrect. We list a few.

The Alonzo hardfork brought Plutus V1. Plutus V1 permitted a mint value that had a zero entry for ada. This is confusing because ada is impossible to mint by smart contract. A correction was later pushed through.

Plutus V1 permitted 0 value withdraws. It was proposed by the ledger maintainers that this mistake ought to be fixed, only for it to be pointed out that such a change would have broken a bunch of dapps.

Reference scripts were introduced in Vasil. They left open a moderate attack vector resulting in a DDOS attempt. A fix was pushed through in which the new fee calculation was, afaics, whatever the Haskell code said it was. Reference inputs were introduced in Vasil. In either Chang 1 or Plomin, these were “fixed” to prevent an input being both referenced and spent. This broke a dapp I had previously written.

It is unrealistic to believe there won’t ever be mistakes. It is moot whether these mistakes, or “mistakes”, were preventable had more consideration been given to the changes. Perhaps if there was more demand and more demand driven development, then these problems might have been addressed before they reached mainnet.

Current ledger assumptions are still un- or underdocumented

It is a misdiagnosis to read the lack of products and solutions as the lack of supported functionality at the ledger level. Cardano development is notorious for the fraught difficulty it entails. The current ledger assumptions are still un- or underdocumented. This is a massive hole, down which developers sink their valuable time. Time that could be spent making better tooling and better product.

Cardano developers seemingly must perform a rite of passage. Through their own painful journey each will learn: inputs are lexicographically ordered and lists are encoded as indefinite length. They will mull:

• Is this fixed behaviour I can rely on or an implementation quirk?
• Where are the docs on the datum integrity hash computation, for which version?
• Which version of Plutus and the ledger was it that the mint value stop including ada?

There are probably a few dozen people who would be able to answer these. They would do so with ease since the knowledge gained though trauma is written into cerebral scar tissue. For everyone else, bon courage: the answers are in Github issues, dangling code comments, and code itself. Before we dig this hole still deeper by adding further to ledger complexity, we ought to prioritise documenting its current behaviour, in a way developers can self serve.

Systems of intents are always incomplete

CIP 118 would open us up to an never ending drip of ledger changes,
each bringing additional intents yet supported. CIP 118, does not limit us: any intent not supported can be expressed in an invoked Plutus script. However, I’m not confident that interested parties would not lobby to have their particularly favoured intents entrenched in the ledger. Were CIP 118 to be accepted, the bar for inclusion would seem to be relatively low.

Intents are often an excellent abstraction. The code can clearly communicate the consequences of the logic they represent, even to non-experts, which is a particularly admirable goal in relation to smart contracts. A key reason why I used Plutus Apps long after many others had abandoned it was that its system of intents (constraints in their vernacular) resulted in code that was far clearer to a keen reader compared to the few alternatives we had at the time (eg Plutarch).

Intents are never complete. Until it was abandoned, the list of supported intents in Plutus Apps kept growing. Worse still, it was not designed to be extensible. If the logic you required was not supported by the intents exposed, you were left to fork the repo and add your own. In the long ago era, when I used Plutus Apps this became my problem when the intents did not support addresses with stake credentials. It was a massive time sink. I am highly wary of libraries that use system of intents that aren’t extensible via, say, the use of Rust traits or Haskell classes pattern. Every system of intents I have seen has grown the list of things it originally intended to support. As addressed, CIP 118 is extensible via Plutus but in this way it creates a two tier system. It seems inevitable there would forever be pressure to grow the list of ledger supported intents.

Conclusion

CIP 118 needs to make a stronger case that it’s a solution worth implementing. I am skeptical this can be made, but would be happily proved wrong. It would be far more beneficial to put the efforts and energies that would be spent on implementing CIP 118 on improving the tooling and accessibility around the functionality we already have.