BUIP037: Hardfork SegWit
Proposer: Amaury SÉCHET
Submitted: 2016-11-12
Status: draft

Summary

There are various problem with the current transaction format, including
malleability and quadratic hashing. SegWit proposes to add a new
transaction format, which solves this problem but does so in a way that
does not allow to spend existing UTXO. As a result, SegWit doesn’t
deliver on its promise. FlexTrans proposes an alternative way to solve
these problems in a way that is compatible with existing UTXO, which
allows us to eventually weed out the old transaction format.

This BUIP proposes to adopt a strategy similar to FlexTrans but using
implementation details more similar to SegWit. Doing so should allow
actors in the ecosystem, who have already implemented SegWit, to support
this BUIP with minimal effort.

Specification

A new transaction format is introduced. It is recognized by its use of
version 7.

The binary format is similar to existing transaction format. However,
the signature script for each input is replaced by witness data, the
output is limited to a version and a hash, and options and metadata
fields are added for extensibility.

For compactness all integer are represented via a variable length
encoding, except the version field. The leading byte of the format
indicate how large the representation is. The representation is big
endian.

It has the following binary format:

Transaction id (aka txid) is computed by hashing the transaction,
skipping over input’s witness and metadata. See the inputs section for
more details.

The option field is interpreted as follow:

The metadata field contains a set of entries. Each entry is made of one
varuint tag, followed by a value, which size and representation depends
on the tag. One tag value cannot appear twice.

Because the size of the metadata field is known by the parser, the
parser can skip over remaining metadata when it encounter a tag with a
value it doesn’t know how to interpret.

This BUIP defines the following tags for transaction metadata:

Inputs

TxIn consists of the following fields:

The OutPoint structure consists of the following fields:

Witness consist of the following fields:

The witness field represent element that are on the stack before the
redeem script starts to run.

This BUIP defines the following tags for input metadata:

Metadata tags are a BUIP039 extension point. The can be signaled with
the InMD prefix.

The signature process compute sighash as double_sha256(txid +
previous_output + TxOut + metadata + hashType)

TxOut needs to use the new format. When spending UTXO from older
transaction, please refers to the conversion procedure in the Outputs
section.

Outputs

TxOut consists of the following fields:

The 3 least significant bits of the version field indicate the size of
the output hash as follow:

This BUIP define 4 valid versions:

For both version, we’ll define OP_HASH as OP_HASH160 is the hash size
is 20 bytes and OP_HASH256 if the hash size is 32bytes.

If we have P2KH version, the following redeem script is executed to
verify the signature:

OP_DUP OP_HASH <pk_script> OP_EQUALVERIFY OP_CHECKSIG

If we have P2SH version, the topmost element of the stack popped and
hashed using OP_HASH, and the result compared to hash. If the
comparison fails, the transaction is invalid. If the comparison succeed
the popped element is executed as a script to validate the spend.

Out version are a BUIP039 extension point. They can be signaled with the
OutV prefix.

Legacy UTXO can be converted to this new format using the following
procedure:

NB: this may require output script to be duplicated in the witness to
spend legacy UTXO. This will happen in a very minimal number of cases
and is on purpose. Witness data can be put in cold storage while UTXO
data need to be kept hot as normal node needs to query these data
frequently when operating. Because of this, it is desirable to shift as
much data as possible from the UTXO set to the witness data.

Semantic

- Signature malleability are prohibited as per BIP146 -
OP_CHECKMULTISIG and OP_CHECKMULTISIGVERIFY dummy argument must be a
null vector as per BIP147.

Rationale

This BUIP keeps thing close to SegWit to minimize sunk cost for actor
supporting it. It also reuse the tag system from FlexTrans to ensure the
format is extensible, but limit this to the metadata and option field in
order to enable BUIP039.

In conclusion, This combine the best part of SegWit (extensibility and
privacy using a version/hash pair as output) and FlexTrans
(extensibility, compatible UTXO) and allow for BUIP039. In addition,
this limits the sunk cost for actors who prepared for SegWit.

Revisions

EDIT: Revision, removing the lock_time field and reintroducing the
sequence one. More revision to come as this converge toward FlexTrans.

EDIT2: Revision.

Output script are now just a hash. This is also an idea from SegWit and
allow to shift data from the UTXO set to the witness data. Because this
is a hard fork, it can be done in a way that is backward compatible.
Inputs now have a metadata field containing optional data identified via
a tag. This is a natural extension point for future extension and an
ideal place to store optional data such as the sequence field.

EDIT3: Use BUIP039 to extend this transaction format.

EDIT4: Add a global option field which respects the metadata. Leverage
it to implement lock_time.

EDIT5: Rework the Out structure to be more compact. Add a rationale
section to compare to FlexTrans and SegWit.

EDIT6: Remove comparison to FT and SW as post length is limited.

EDIT7: Use variable size encoding for int all over the place. Define the
format.