multistream-select-v2: Initial draft

[MarcoPolo]

closes #700

An initial pass at specifying the abbreviation tree. There's still more to define like the wire format, how these protocol strings can be exchanged, and how to know if your peer supports multistream select v2.

[haxxpop]

Looks good to me

[haxxpop]

Leaf nodes of the client's abbreviation tree, right?

[MarcoPolo]

I'm not sure I follow. Can you expand please?

[haxxpop]

I mean the client's abbreviation tree will never have non-leaf protocol strings, right?

[MarcoPolo]

yes. I've reworded it. Is it clearer now?

[MarcoPolo]

Do we need the hash function at all? The hash function acts as a mapping from a protocol string to a byte string with high likelihood of distinct short prefixes. But we could also imagine just using the index of the protocol string from the server's list instead.

If a server returned protocols ["/a", "/b", "/c"] this would be mapped to abbreviations [0x00, 0x01, 0x02]. The tombstones would solve the remove-reintroduce problem without increasing the list size. New protocols are added to the end. The implementation would also be simpler as it's just a list.

[marten-seemann]

Is it really necessary to support removing strings?

[MarcoPolo]

Whether to remove support for a protocol is an application concern. At this level, MSSv1 does support it, so it makes sense that the next version also supports.

That said, If supporting removing strings makes the solution much more complicated, that's a tradeoff we should discuss. It would be good to understand the complexity of supporting it first.

[dozyio]

Think I prefer an index based solution for simplicity of implementation. Presume the server would maintain the index.

Would it make sense to move some of this to an identify / identify push v2?

[MarcoPolo]

For the index based version, the server returns a list of protocol strings which implies the index. It also returns a list (or bitmap) of which protocol strings are tombstoned. The latter part is new to identify, and would have to be introduced.

The fewer changes to identify the better.

[haxxpop]

I think in the hash-based approach, you also don't need to maintain a per-peer state on the server side, right? You only need to maintain one tree.

In the index-based approach, you have to remember the order or indices of the protocol strings sent to each peer, unless you make sure you always send the same order of protocol strings to peers.

[sukunrt]

@MarcoPolo

For the index based version, the server returns a list of protocol strings which implies the index. It also returns a list (or bitmap) of which protocol strings are tombstoned. The latter part is new to identify, and would have to be introduced.

For a minimal identify change here:
The server can return another list along with the protocol list which is the int value of the corresponding protocol in the protocol list.

identify.Protocols[i] can be used with mss-v2 with identify.ProtocolIndex[i]

It's up to the server to do the tombstoning correctly.