diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000..6dd7e65 Binary files /dev/null and b/.DS_Store differ diff --git a/system/validator_revard/CHANGELOG.md b/system/validator_revard/CHANGELOG.md new file mode 100644 index 0000000..87efcae --- /dev/null +++ b/system/validator_revard/CHANGELOG.md @@ -0,0 +1,5 @@ +The purpose of the supplement. It was found that the guide is missing information about validator awards. +The information was taken from the joystream documents on this topic and from the polkadot handbook. +The document was checked by several authoritative joistream members and changes were made according to joistream blockchain. +It is possible that errors may be found in the future, which will need to be corrected. +The target audience is validators. diff --git a/system/validator_revard/README.md b/system/validator_revard/README.md new file mode 100644 index 0000000..5528ccb --- /dev/null +++ b/system/validator_revard/README.md @@ -0,0 +1,275 @@ +--- +description: Maintaining agreement over the growing history of the system. +--- + +# 🏭 Validation + +## Validator + +### Responsibilities + +* Run and maintain screening nodes that are always available and performant +* Help enforce the consensus rules of the network + +### Requirements + +* Experienced with how to setup and maintain high performance IT infrastructure +* Access to highly performant and reliable IT infrastructure, with high storage, (up & down) bandwidth and processing capacity +* Able to securely store keys +* Hold sufficient amount of the native platform token to put at stake + * currently **at least** JOY 41.667k in a single account, which is the minimum to sign up – actually getting a validator slot likely requires more + +#### Hardware Requirements + +The Joystream blockchain, and therefore the `joystream-node` is built on the [substrate](https://substrate.io/) framework, developed for the [Polkadot](https://polkadot.network/) ecosystem. As Joystream is still in infancy on mainnet, we refer to the their expertise for the technical specification and [recommendations](https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#reference-hardware): + +* **CPU** + * x86-64 compatible; + * Intel Ice Lake, or newer (Xeon or Core series); AMD Zen3, or newer (EPYC or Ryzen); + * 4 physical cores @ 3.4GHz; + * Simultaneous multithreading disabled (Hyper-Threading on Intel, SMT on AMD); + * Prefer single-threaded performance over higher cores count. A comparison of single-threaded performance can be found [here](https://www.cpubenchmark.net/singleThread.html). +* **Storage** + * An NVMe SSD of 1 TB (As it should be reasonably sized to deal with blockchain growth). An estimation of current chain snapshot sizes can be found [here](https://paranodes.io/DBSize). In general, the latency is more important than the throughput. +* **Memory** + * 16GB DDR4 ECC. +* **System** + * Linux Kernel 5.16 or newer. +* **Network** + * The minimum symmetric networking speed is set to 500 Mbit/s (= 62.5 MB/s). This is required to support a large number of parachains and allow for proper congestion control in busy network situations. + +It should be obvious that given the life span and size – thus state and transaction activity – of Polkadot relative to Joystream at this stage, it is certainly fine to scale down on things like storage... + +## Guides + +The instructions below cover Linux binaries only. If you want to build from source, clone the [repo](https://github.com/Joystream/joystream) and follow the build steps there. + +### Install and Deploy + +* Every time something is written in ``, this means you have to replace this with your input, without the `<>`. +* When something is written in `"double_quotes"`, it means the number/data will vary depending on your node or the current state of the blockchain. +* For terminal commands: + * `$` means you must type what comes afterwards + * `#` means it's just a comment/explanation for the readers convenience + +``` +# This is just a comment, don't type or paste it in your terminal! +$ cd ~/ +# Only type/paste the "cd ~/, not the preceding $ ! +``` + +For the purposes of simplicity, we will assume: + +1. You are user `joystream`, with sudo priveliges. +2. You want to save everything in `/home/joystream/bin/` + +#### Download Node Binary + +Find the latest release [here](https://github.com/Joystream/joystream/releases/latest) or get the tag from the command line with: + +``` +$ curl -sL https://api.github.com/repos/Joystream/joystream/releases/latest | jq -r ".tag_name" +``` + +At the time of writing, the latest release is `v12.1001.0`, whereas the last node binary is from version `v12.1000.0` (mainnet) + +``` +# Create the directory, and go there +$ mkdir ~/bin && cd ~/bin + +# Assuming the latest version is still v12.1000.0 +# Download the binary: +$ wget https://github.com/Joystream/joystream/releases/download/v12.1000.0/joystream-node-8.0.0-1a0d1f677df-x86_64-linux-gnu.tar.gz + +# unzip it: +$ tar -vxf joystream-node-8.0.0-1a0d1f677df-x86_64-linux-gnu.tar.gz + +# Download the chain spec: +$ wget https://github.com/Joystream/joystream/releases/download/v12.1000.0/joy-mainnet.json + +# test that your node works: +$ ./joystream-node --chain-spec joy-mainnet.json --pruning archive +``` + +Assuming it starts syncing, you can stop it right away with `ctrl+c` + +#### Configuration + +The node lets you set a variety of option flags. You can display them all with `./joystream-node --help` Some basic `options` you should enable or consider: + +> \--chain \ +> +> Specify the chain specification. It can be one of the predefined ones (dev, local, or staging) or it can be a path to a file with the chainspec (such as one exported by the `build-spec` subcommand). + +* **Required** + * Without this flag, you will not connect the chain. + +> \--pruning \ +> +> Specify the state pruning mode, a number of blocks to keep or 'archive'. Default is to keep all block states if the node is running as a validator (i.e. 'archive'), otherwise state is only kept for the last 256 blocks. + +* **Required for validators** + * If you want to be a validator, the node must run with `--pruning archive` + * If you start syncing without that flag enabled, you will have to wipe your node and sync again if you change your mind. + +> \--validator +> +> Enable validator mode. The node will be started with the authority role and actively participate in any consensus task that it can (e.g. depending on availability of local keys). + +* **Required for validators** + * Unlike with `--pruning`, it only has to be set when you are actually in the validator set to have an effect, so you don't have to re-sync if you forget while syncing. + +> \--name +> +> The human-readable name for this node. The node name will be reported to the telemetry server, if enabled. + +* **Optional** + * May serve some benefits if you want someone to nominate you, but may make it easier to identity you. + +*** + +As a validator, you should (as a bare minimum) be very restrictive in terms of RPC access to your node. Go through the options, and double check that the defaults are in line with your preferences and risk tolerance. + +#### Run as a Service + +Running as a service means that the node will continue running as a daemon, and you can enable it to restart in case of crashes and on reboot. + +It requires sudo privileges. If you are not user `root`, add `sudo` before commands. + +Example file below, with essentials only: + +``` +[Unit] +Description=Joystream Node +After=network.target + +[Service] +Type=simple +User=joystream +WorkingDirectory=/home/joystream/bin/ +ExecStart=/home/joystream/bin/joystream-node \ + --chain /home/joystream/bin/joy-mainnet.json \ + --pruning archive \ + --validator +Restart=on-failure +RestartSec=3 +LimitNOFILE=10000 + +[Install] +WantedBy=multi-user.target +``` + +``` +# Create/open a file with your favorite editor (I use nano below) +$ sudo nano /etc/systemd/system/joystream-node.service + +# Paste in the example file above, and do "ctrl+x", then "y" and "return" to save + +# start it up: +$ sudo systemctl start joystream-node + +# check that it's working: +$ sudo systemctl status joystream-node +# For a brief status, OR +$ sudo journalctl -f -n 100 -u joystream-node +# To monitor the log + +# If you are happy, enable it so it will start automatically on boot: +$ sudo systemctl enable joystream-node +``` + +### Setup Keys and Validate + +With your validator node up and running, you are now ready to set up keys and announce your intentions on chain. + +#### Generate Session Keys + +In the terminal on your node (will only work if you are on running the chain on the same machine!): + +``` +$ curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "author_rotateKeys", "params":[]}' http://localhost:9933 + +# Which should return: +{"jsonrpc":"2.0","result":"0xabc...123","id":1} +``` + +Where `0xabc...123` (a much longer string in reality) is a concatenation of four public keys hex-encoded. The private keys should have been injected in your `base-path`. Make sure you copy this string over somewhere, as you need it later. + +Assuming you only did this once (while running this _this_ chain), and you didn't set a different `--base-path` flag: + +``` +# On the current network, replace with joy_testnet_7 +$ ls -a ~/.local/share/joystream-node/chains//keystore + +# Which should return 4 files, each a long string starting with a 6. + +# You can confirm more precisely by: + +$ curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "author_hasSessionKeys", "params":["0xabc...123"]}' http://localhost:9933 + +# Which, if you have the corresponding ready to sign, should return: +{"jsonrpc":"2.0","result":true,"id":1} +``` + +**Warning:** + +* It's both bad practice, and a possible slashing risk, to keep multiple set of session keys on one node. If you wanted to try the command, made a mistake, or for whatever reason want to switch them, delete them. If you have had set them on chain (see next steps), you can change them before you get into the validator set. +* Keeping the same set of keys, on multiple nodes, all running with the `--validator` enabled, will cause a slash. A good backup system could include backup nodes, but be careful. It's better to get "booted" for an era than to double sign blocks. It will be treated as an attack even if just by accident. + +#### Configure Validator on Chain + +For the time being, we will only show how to do this with [Polkadot{.js} apps](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frpc.joystream.org#/explorer), a web based UI, and the [Polkadot{.js} extension](https://polkadot.js.org/extension/), a that works with both the aforementioned UI and Joystreams [own Pioneer](https://pioneerapp.xyz/#/profile). + +As the polkadot-js UI serves lots of projects in the substrate ecosystem, you have to make sure to set the correct endpoint, meaning which network (and node) you connect to. The link above does Joystream automatically, and sets it as default in local storage (until another is set). + +The network endpoint can also be set manually by clicking the top left corner, and selecting "Live Networks" -> "Joystream" (hosted by Jsgenesis) before clicking "Switch", from the meny that appears. + +This will display the logo, network name, node version and latest block height of the chain you are currently connected to as shown below. + +
+ +**With Polkadot-js** + +You need two keys for this, one to be the `controller` and one as the `stash`. The latter holds the stake and must sign at least once to "delegate" to the `controller` which is running the "day to day" operations. + +Assuming you are fully synched, your node is running, and you have + +**Steps:** + +1. Go to the "staking actions" tab - "Network" -> "Staking" -> "Accounts", and click the "+ Validator" button in the top right corner. +2. Select a `stash` and `controller` account from the dropdown, set the "value bonded" as the amount you want to stake, and choose a "payment destination", then hit "next". +3. Paste in the public session keys (`0xabc...123`), choose a "reward commission percentage" and whether you want to allow nominations or not, then click "Bond & Validate". + +If you are preparing this for later, click the "+ Stash" button instead. This allows you to wait for your session keys and/or synching your node. + +#### Being a Validator + +Assuming the transaction went through, you will now appear under the "waiting" tab [here](https://polkadot.js.org/apps/#/staking). That means you are in the queue for joining the validator set, but when (and whether) you actually join depends on the competition for getting a slot. + +At all times, there is a limit to how many can become validators. What that number is set by the council. The current value can be found in the [chain state](https://polkadot.js.org/apps/#/chainstate) -> "staking" -> "validatorCount". + +Suppose that number is `n`, and that there are `m` validators that, towards the end of each `era` were already validating or joined the queue: + +* If `n >= m`, all will be elected +* If `n "staking" -> "minValidatorBond". (In base value `HAPI`, meaning `1*10^-10 JOY`) + +Another reason the transaction can be rejected is if the maximum number of `bonded` accounts have declared as validators. That number can be found in the [chain state](https://polkadot.js.org/apps/#/chainstate) -> "staking" -> "maxValidatorsCount". How many there are currently can be found in the [chain state](https://polkadot.js.org/apps/#/chainstate) -> "staking" -> "counterForValidators". + +There are of course a limitless amount of other reasons the transaction could be rejected if you constructed it yourself in the cli. + +### Advanced Setup + +`TODO` + +## diff --git a/system/validator_revard/Reward-calculation.md b/system/validator_revard/Reward-calculation.md new file mode 100644 index 0000000..a3bfeba --- /dev/null +++ b/system/validator_revard/Reward-calculation.md @@ -0,0 +1,161 @@ +# Validator Payout Overview + +## Reward Calculation +Validators and nominators are rewarded at the end of each era. The total reward of an era is calculated using the era duration and the staking rate (the total amount of tokens staked by nominators and validators, divided by the total token supply). It aims to incentivize toward a defined staking rate. Total reward is split among validators and their nominators depending on the number of points they received during the era. Points are added to a validator using reward_by_ids or reward_by_indices. + +## The validator and its nominator split their reward as following: + +The validator can declare an amount, named commission, that does not get shared with the nominators at each reward payout through its ValidatorPrefs. This value gets deducted from the total reward that is paid to the validator and its nominators. The remaining portion is split among the validator and all of the nominators that nominated the validator, proportional to the value staked behind this validator (i.e. dividing the own or others by total in Exposure). + +All entities who receive a reward have the option to choose their reward destination through the Payee storage item (see set_payee), to be one of the following: + +Controller account, (obviously) not increasing the staked value. +Stash account, not increasing the staked value. +Stash account, also increasing the staked value. + +## How to calculate individual Nominator and Validator Rewards: + +Shared Validator Set Rewards - **THIS ONE DONE BY CHAIN IN THE END OF EACH ERA** In each era, a function of + +− totalIssuance.at(lastBlockOfEra) +− erasTotalStake(era) + + **erasRewardPoints(era).total** += validatorSetRewardInEra + +Individual Rewards for each Validator, before split between (vals + noms) +In each era, equal to: +(validatorSetRewardInEra × erasRewardPoints(era).stashOfVal / +erasRewardPoints(era).total) + + **slashedInEra(era,stashOfVal)** += totalValidatorRewardInEra + +Individual Rewards for each Validators (vals + noms) +In each era, equal to: + +valCommission = totalValidatorRewardInEra(era) × commission% / 100 +sharedReward = totalValidatorRewardInEra(era) − totalValidatorRewardInEra(era) × commission% / 100 += totalValidatorRewardInEra(era) × (1 00 − commission%) + +valReward(era) = valCommission + sharedReward × valOwnStake(era) / totalValStake(era) + +nomReward_i(era) = sharedReward × nomStake_i(era) / totalValStake(era) + +### Era Points +For every era (a period of time approximately 6 hours), validators are paid proportionally to the amount of era points they have collected. Era points are reward points earned for payable actions like: + +producing a non-uncle block in the Relay Chain. +producing a reference to a previously unreferenced uncle block. +producing a referenced uncle block. + +### NOTE +An uncle block is a Relay Chain block that is valid in every regard, but which failed to become canonical. This can happen when two or more validators are block producers in a single slot, and the block produced by one validator reaches the next block producer before the others. We call the lagging blocks uncle blocks. Era points create a probabilistic component for staking rewards. +If the mean of staking rewards is the average rewards per era, then the variance is the variability from the average staking rewards. The exact joys value of each era point is not known in advance since it depends on the total number of points earned by all validators in a given era. This is designed this way so that the total payout per era depends on joystream's inflation model, and not on the number of payable actions (f.e., authoring a new block) executed. + +## Payout Scheme +No matter how much total stake is behind a validator, all validators split the block authoring payout essentially equally. The payout of a specific validator, however, may differ based on era points, as described above. Although there is a probabilistic component to receiving era points, and they may be impacted slightly depending on factors such as network connectivity, well-behaving validators should generally average out to having similar era point totals over a large number of eras. + +Validators may also receive "tips" from senders as an incentive to include transactions in their produced blocks. Validators will receive 100% of these tips directly. +Validators will receive staking rewards in the form of the native token of that chain (joystream). +For simplicity, the examples below will assume all validators have the same amount of era points, and received no tips. + +**Validator Set Size (v): 4** +Validator 1 Stake (v1): 18 tokens +Validator 2 Stake (v2): 9 tokens +Validator 3 Stake (v3): 8 tokens +Validator 4 Stake (v4): 7 tokens +Payout (p): 8 joys +Payout for each validator (v1 - v4): +p / v = 8 / 4 = 2 tokens + +Note that this is different than most other Proof-of-Stake systems such as Cosmos. As long as a validator is in the validator set, it will receive the same block reward as every other validator. Validator v1, who had 18 tokens staked, received the same reward (2 tokens) in this era as v4 who had only 7 tokens staked. + +## Running Multiple Validators +It is possible for a single entity to run multiple validators. Running multiple validators may provide a better risk/reward ratio. Assuming you have enough joys, or enough stake nominates your validator, to ensure that your validators remain in the validator set, running multiple validators will result in a higher return than running a single validator. +For the following example, assume you have 18 joys to stake. For simplicity's sake, we will ignore nominators. Running a single validator, as in the example above, would net you 2 joys in this era. + +**Validator Set Size (v): 4** +Validator 1 Stake (v1): 18 joys <- Your validator +Validator 2 Stake (v2): 9 joys +Validator 3 Stake (v3): 8 joys +Validator 4 Stake (v4): 7 joys +Payout (p): 8 joys +Your payout = (p / v) * 1 = (8 / 4) * 1 = 2 + +**Running two validators, and splitting the stake equally, would result in the original validator v4 to be kicked out of the validator set, as only the top v validators (as measured by stake) are selected to be in the validator set. More important, it would also double the reward that you get from each era.** + +**Validator Set Size (v): 4** +Validator 1 Stake (v1): 9 joys <- Your first validator +Validator 2 Stake (v2): 9 joys <- Your second validator +Validator 3 Stake (v3): 9 joys +Validator 4 Stake (v4): 8 joys +Payout (p): 8 joys +Your payout = (p / v) * 2 = (8 / 4) * 2 = 4 + + ### With enough stake, you could run more than two validators. However, each validator must have enough stake behind it to be in the validator set. + +The incentives of the system favor equally-staked validators. This works out to be a dynamic, rather than static, equilibrium. Potential validators will run different numbers of validators and apply different amounts of stake to them as time goes on, and in response to the actions of other validators on the network. + +## Slashing +Although rewards are paid equally, slashes are relative to a validator's stake. Therefore, if you do have enough joys to run multiple validators, it is in your best interest to do so. A slash of 30% will, of course, be more joys for a validator with 18 joys staked than one with 9 joys staked. + +Running multiple validators does not absolve you of the consequences of misbehavior. Joystream punishes attacks that appear coordinated more severely than individual attacks. You should not, for example, run multiple validators hosted on the same infrastructure. A proper multi-validator configuration would ensure that they do not fail simultaneously. + +Nominators have the incentive to nominate the lowest-staked validator, as this will result in the lowest risk and highest reward. This is due to the fact that while their vulnerability to slashing remains the same (since it is percentage-based), their rewards are higher since they will be a higher proportion of the total stake allocated to that validator. + +To clarify this, let us imagine two validators, v1 and v2. Assume both are in the active set, have commission set to 0%, and are well-behaved. The only difference is that v1 has 90 joys nominating it and v2 only has 10. If you nominate v1, it now has 90 + 10 = 100 joys and you will get 10% of the staking rewards for the next era. If you nominate v2, it now has 10 + 10 = 20 joys nominating it, and you will get 50% of the staking rewards for the next era. In actuality, it would be quite rare to see such a large difference between the stake of validators, but the same principle holds even for smaller differences. If there is a 10% slash of either validator, then you will lose 1 joys in each case. + +### CAUTION +If a validator is oversubscribed in an era, staking rewards are distributed only to the the top 512 nominators and the rest of the nominators do not receive any rewards. This is not the case for slashing! Every active nominator of the validator committing slashable offence will be slashed. + +# Nominators and Validator Payments +Nominated stake allows you to "vote" for validators and share in the rewards (and slashing) without running a validator node yourself. Validators can choose to keep a percentage of the rewards due to their validator to "reimburse" themselves for the cost of running a validator node. Other than that, all rewards are shared based on the stake behind each validator. This includes the stake of the validator itself, plus any stake bonded by nominators. + +### INFO +Validators set their preference as a percentage of the block reward, not an absolute number of joys. joystream's block reward is based on the total amount at stake, with the reward peaking when the amount staked is at 50% of the total supply. The commission is set as the amount taken by the validator; that is, 0% commission means that the validator does not receive any proportion of the rewards besides that owed to it from self-stake, and 100% commission means that the validator operator gets all rewards and gives none to its nominators. + +In the following examples, we can see the results of several different validator payment schemes and split between nominator and validator stake. We will assume a single nominator for each validator. However, there can be numerous nominators for each validator. Rewards are still distributed proportionally - for example, if the total rewards to be given to nominators is 2 joys, and there are four nominators with equal stake bonded, each will receive 0.5 joys. Note also that a single nominator may stake different validators. + +Each validator in the example has selected a different validator payment (that is, a percentage of the reward set aside directly for the validator before sharing with all bonded stake). The validator's payment percentage (in joys) is listed in brackets ([]) next to each validator. Note that since the validator payment is public knowledge, having a low or non-existent validator payment may attract more stake from nominators, since they know they will receive a larger reward. + +**Validator Set Size (v): 4** +Validator 1 Stake (v1) [20% commission]: 18 joys (9 validator, 9 nominator) +Validator 2 Stake (v2) [40% commission]: 9 joys (3 validator, 6 nominator) +Validator 3 Stake (v3) [10% commission]: 8 joys (4 validator, 4 nominator) +Validator 4 Stake (v4) [ 0% commission]: 6 joys (1 validator, 5 nominator) +Payout (p): 8 joys +Payout for each validator (v1 - v4): +p / v = 8 / 4 = 2 joys + +**v1:** +(0.2 * 2) = 0.4 joys -> validator payment +(2 - 0.4) = 1.6 -> shared between all stake +(9 / 18) * 1.6 = 0.8 -> validator stake share +(9 / 18) * 1.6 = 0.8 -> nominator stake share +v1 validator total reward: 0.4 + 0.8 = 1.2 joys +v1 nominator reward: 0.8 joys + +**v2:** +(0.4 * 2) = 0.8 joys -> validator payment +(2 - 0.8) = 1.2 -> shared between all stake +(3 / 9) * 1.2 = 0.4 -> validator stake share +(6 / 9) * 1.2 = 0.8 -> nominator stake share +v2 validator total reward: 0.8 + 0.4 = 1.2 joys +v2 nominator reward: 0.8 joys + +**v3:** +(0.1 * 2) = 0.2 DOT -> validator payment +(2 - 0.2) = 1.8 -> shared between all stake +(4 / 8) * 1.8 = 0.9 -> validator stake share +(4 / 8) * 1.8 = 0.9 -> nominator stake share +v3 validator total reward: 0.2 + 0.9 joys = 1.1 joys +v3 nominator reward: 0.9 joys + +**v4:** +(0 * 2) = 0 joys -> validator payment +(2 - 0) = 2.0 -> shared between all stake +(1 / 6) * 2 = 0.33 -> validator stake share +(5 / 6) * 2 = 1.67 -> nominator stake share +v4 validator total reward: 0 + 0.33 joys = 0.33 joys +v4 nominator reward: 1.67 joys