Implement c2sp.org/tlog-proof

[Hayden-IO]

This adds support for constructing a log proof formatted per the specification, along with verifying a proof for a given leaf hash and log verifier.

[Hayden-IO]

@AlCutter @mhutchinson I'm looking for early feedback on where this should live. I was debating whether it made more sense under transparency-dev/formats or transparency-dev/merkle (or another repo entirely?). The construction of the proof could live under /formats, but verification of the proof seemed better aligned with /merkle, and I'd rather keep the two in the same package. This would mean that the merkle module will now depend on the formats module, and I wasn't sure if you wanted to keep these separate.

In order to verify a proof, I needed the witness policy implementation, so I've temporarily duplicated it here. Would you be open to moving it to /formats?

Tests are still a WIP, so please ignore those if you start looking over any code.

[Hayden-IO]

Just bumping, any preferences on where this should live under transparency-dev?

[mhutchinson]

@AlCutter and I discussed this earlier, and then the day got away from me. Sorry about that, and thanks for the reminder! (and happy new year 🎉 )

We have the opinion that merkle should be usable without a dep on formats, because this is pure merkle tree calculation code. And formats shouldn't require a dep on merkle, because it's just encoding/decoding.

I know you said that you'd "rather keep the two in the same package", but how much of a problem would it be to structure the cod according to these projects, e.g.:

• formats contains a struct, Marshal, Unmarshal (similar to https://github.com/transparency-dev/formats/blob/main/log/checkpoint.go)
• merkle might not need changing, the client would just unpack the struct from formats and pass it into the existing VerifyInclusion

[Hayden-IO]

Happy new year as well :) Definitely would be fine with keeping encoding and verification separate! I'll refactor this and send out PRs shortly, thanks.

[Hayden-IO]

transparency-dev/formats#228 and transparency-dev/formats#229.

As for whether or not we need to update merkle, personally I'd find it useful if there was a method to verify a tlog-proof without needing to call the three methods to verify the inclusion proof, checkpoint and witness policy separately, to make sure that none of those steps are missed. Is there another place this could live so that merkle doesn't depend on formats?

[mhutchinson]

I'm opinionated that merkle isn't the right home for it, but we can find another home for it. Let's see how much code is left after putting the proof struct and marshal/unmarshal in formats?

Some amount of my hesitation is that the function you're proposing isn't universal. e.g. the Tessera client winds checkpoints forwards locally, and does checkpoint validity & witnessing checks at that point. Inclusion proofs are done separately against that checkpoint. That said, this way of working might change when witnessing is taken for granted...