diff --git a/src/index.md b/src/index.md index bedd5d8..db42410 100644 --- a/src/index.md +++ b/src/index.md @@ -234,7 +234,7 @@ features: 23.625H11.5835C12.2038 23.625 12.7085 24.1297 12.7085 24.75V27.0538C12.7084 27.0595 12.7085 27.0652 12.7085 27.0709V29.25C12.7085 29.8703 12.2038 30.375 11.5835 30.375Z" /> - title: Protocols + title: Protocol details: Deep dive into core protocol concepts. Understand how iExec enables privacy, governance, and monetization. diff --git a/src/protocol/proof-of-contribution.md b/src/protocol/proof-of-contribution.md index d0d0ba2..c11d77e 100644 --- a/src/protocol/proof-of-contribution.md +++ b/src/protocol/proof-of-contribution.md @@ -1,436 +1,192 @@ --- title: Proof of Contribution description: - PoCo protocol for trust, consensus, and secure payment in decentralized iExec - computing. + PoCo protocol for governance, trust, and secure payment in decentralized iExec + computing platform. --- -# Proof of Contribution +# Proof-of-Contribution (PoCo) -> PoCo is a protocol designed to provide trust in an open and decentralized -> environment of untrusted machines. +## What is PoCo? -The iExec platform provides a network where application providers, workers, and -users can gather and work together. The fully decentralized nature of iExec -implies that the system trusts no single agent by default, and that those agents -require incentives to contribute correctly. +PoCo (Proof-of-Contribution) is the trust and governance layer of the iExec +protocol. It ensures that off-chain computations running inside Trusted +Execution Environments (TEEs) are executed correctly, confidentially, and under +clear economic rules. -In this context, Proof-of-Contribution (PoCo) is the protocol that iExec uses -for consensus over off-chain computing. +PoCo provides three core guarantees: -## Protocol +- **Confidentiality**: data and secrets are only processed inside secure + enclaves. +- **Governance and access control**: users decide who can access their data. +- **Trusted payments and penalties**: contributors get paid automatically, and + misbehavior is economically discouraged. -### Objectives +## Why PoCo matters? -PoCo is a protocol that provides trust in an open and decentralized environment -of untrusted machines. +iExec is built around confidential computing, where computations run inside +Trusted Execution Environments. Users don’t need to trust the machine running +the task, TEE’s cryptographic attestation prove that execution happens in a +private way. -In addition to providing trust, PoCo also orchestrates the different -contributions to the iExec network and ensures payments are always fair and -timely. +PoCo uses this model to guarantee: -A major quality of PoCo is its modular design. It comes with features that are -context-specific. +- execution happens **inside a genuine enclave** +- secrets are handled **securely and privately** +- results come from **verified enclaves** +- payments and penalties are applied **automatically** on-chain. -### Result consolidation +In short: PoCo allows building production-grade confidential compute workflows, +backed by hardware security, without needing to design trust, access, or +monetization layers. -PoCo relies on replication to achieve result consolidation. This is purely a -software solution that enforces a confidence level on the result. -[The requester can customize this confidence level.](proof-of-contribution.md#replication-and-trust) +## How the TEE-centric workflow works? -This layer also supports the onchain consolidation of execution results that -Trusted Execution Environments (TEE) such as Intel SGX carry out. +This reflects the default workflow used today on iExec networks. -### Secure payment +1. ### The user triggers match orders on-chain operation -Once the iExec Marketplace seals a deal, the system locks requester funds to -ensure all resource providers receive payment for their contributions. Resources -can take the form of data, applications or computing power. +A requester matches the app, dataset, and workerpool orders. This creates a +**Deal** on-chain and locks the requester’s funds. -Workers must achieve consensus on the execution result to get the requester’s -funds. If consensus is not achieved, the system reimburses the requester. +PoCo now governs: -Worker and scheduler must stake RLC to participate as computing providers. Bad -behaviour from an actor results in a loss of stake. +- who has access +- what is paid +- under which conditions the task is considered valid -This is essential on the public blockchain, but you can set all values to 0 for -private blockchain networks. +2. ### The scheduler assigns the task to a TEE-enabled worker -### Permissioning +The workerpool selects an available worker with the required TEE capabilities. +No replication is needed, trust comes from hardware attestation, not from +multiple workers. -For an execution to happen, the different parties involved must sign a deal. A -permission mechanism controls access to applications, datasets and worker pools. +3. ### The worker executes the app inside a secure enclave -You can disable the secure payment layer for a private blockchain, or you can -also use it in the context of the public blockchain to increase security. An -example of permissioning is dataset restriction for a specific application. +The worker runs a confidential application inside its enclave: -### Overview +- the code is measured +- the environment is verified +- the enclave proves its authenticity through remote attestation +- PoCo verifies this attestation through the SMS -PoCo describes the succession of contributions that you need to achieve -consensus on a given result. Two blog articles detail its logic: +This guarantees: -- [PoCo series #1: Initial PoCo description](https://medium.com/iex-ec/about-trust-and-agents-incentives-4651c138974c) -- [PoCo series #3: Updated PoCo description](https://medium.com/iex-ec/poco-series-3-poco-protocole-update-a2c8f8f30126) +- no one can inspect the data +- the worker cannot tamper with the execution +- results come from a genuine, verified enclave -The -[nominal workflow](https://github.com/iExecBlockchainComputing/iexec-doc/raw/master/techreport/nominalworkflow-ODB.png) -is also available in the [technical report section](/references/glossary) +4. ### Secrets are transferred securely (SMS → Enclave) -Below are the details of the implementations: +If the task uses secrets (dataset decryption key, ...): -**1. Deal:** +- the Secret Management Service (SMS) enclave verifies the worker’s enclave +- secrets are provisioned for the specific enclave only +- secrets are only accessible and processed inside the TEE enclave. -[The Clerk seals a deal.](proof-of-contribution.md#brokering) This marks the -beginning of the execution. The system creates an event to notify the worker -pool’s scheduler. +This is fundamental for confidential and monetizable datasets. -The consensus timer starts when the parties sign the deal. The corresponding -task must complete before the end of this countdown. Otherwise, the scheduler -gets punished by a loss of stake and reputation, and the system reimburses the -user. +5. ### The enclave computes and produces the result -**2. Initialization:** +At the end of execution, the enclave: -The scheduler calls the `initialize` method. Given a deal id and a position in -the request order (within the deal window), this function initializes the -corresponding task and returns the -_taskid_.`bytes32 taskid = keccak256(abi.encodePacked(_dealid, idx));` +- makes the result available for the requester (on IPFS for example) +- signs a challenge to prove that the execution happened inside an enclave +- sends the proof to the PoCo via the worker -**3. Authorization signature:** +6. ### PoCo validates and finalizes the task on-chain -The scheduler designates workers that participate in this task. The scheduler’s -Ethereum wallet signs a message containing the worker’s Ethereum address, the -taskid, and (optionally) the Ethereum address of the worker's enclave. If the -worker doesn't use an enclave, you must fill this field with `address(0)`. +PoCo checks: -This Ethereum signature (authorization) goes to the worker through an off-chain -channel implemented by the middleware. +- worker permission to push a result for the task (through an off-chain + scheduler authorization) +- enclave authenticity by validating the the enclave challenge signature -**4. Task computation:** +If everything is valid the ask is finalized and funds are released according to +the on-chain rules: -Once the worker receives and verifies the authorization, the worker computes the -requested tasks. Results from this execution go in the `/iexec_out` folder. The -following values are then computed: +- the requester's locked money is finally seized +- the worker gets paid +- app & dataset owners get their revenue shares +- any misbehavior results in stake-based penalties for the scheduler -- _bytes32 digest_: a digest (sha256) of the result folder. -- _bytes32 hash_: the hash of the _digest_, that produces consensus -- _bytes32 seal_: the salted hash of the _digest_, that proves a worker’s knew - the _digest_ value before publishing it. - `resultHash == keccak256(abi.encodePacked( taskid, resultDigest))` - `resultSeal == keccak256(abi.encodePacked(worker, taskid, resultDigest))` - -In computer science, a deterministic algorithm is an algorithm which, given a -particular input, will always produce the same output. - -Both the `digest`, the `hash` and the `seal` compute automatically based on the -output of the application. If the output is not entirely deterministic, then the -application can specify a deterministic file that should use for building -consensus. In order to do so, the application just has to provide the path to -the deterministic file using a specific entry `deterministic-output-path` in -`${IEXEC_OUT}/computed.json`. - -Alternatively, if you use the application in an oracle context (the results are -designed to process on-chain by receiver smart-contracts), then the value of -this callback must specify the value of in `${IEXEC_OUT}/computed.json` under -the entry `callback-data`. - -If a TEE produces the result, the post-processing enclave will automatically -produce an `enclave-signature` entry that contains the enclave signature (of the -resultHash and resultSeal). TEE certification of results is transparent to the -application developer. - -**5. Contribution:** - -Once the worker performs the execution, the worker pushes its contribution using -the `contribute` method. The contribution contains: - -- _bytes32 taskid_ -- _bytes32 resultHash_ -- _bytes32 resultSeal_ -- _address enclaveChallenge_ - -The address of the enclave (specified in the scheduler’s authorization). If no -enclave specifies, you should set this parameter to `address(0)`. - -- _bytes enclaveSign_ - -The enclave signature. You need this if the `enclaveChallenge` is not -`address(0)`. Otherwise, you should set it to the empty byte string `0x`. - -- _bytes workerpoolSign_ - -The scheduler computes the signature at step 2. - -**6. Consensus:** - -During the contribution, the system updates and verifies the consensus. -Contributions are possible until the system reaches consensus, at which point -the contributions close. The system then enters a 2h reveal phase. - -**7. Reveal:** - -During the reveal phase, workers that have contributed to the consensus must -call the `reveal` method with the `resultDigest`. This verification confirms -that the `resultHash` and `resultSeal` they provided are valid. Failure to -reveal is equivalent to a bad contribution, and results in a loss of stake and -reputation. - -**8. Finalize:** - -Once the system reveals all contributions, or at the end of the reveal period if -some (but not all) reveals are missing, the scheduler must call the `finalize` -method. This task finalization, rewards good contribution and punishes bad ones. -You must call this before the end of the consensus timer. - -### Staking and Payment +## Brokering -Among the objectives of PoCo, the protocol ensures a worker that contributes -correctly receives a reward and, at the same time, that a requester won’t pay -unless the system achieves consensus. This locking achieves the requester’s -funds for the duration of the consensus, and unlocking them depending on the -outcome. +On iExec, **brokering** is the mechanism that matches all parties involved in a +computation: -Staking prevents bad behavior and encourage good contributions. +- the **application** (the logic to run) +- the **dataset** (optional, confidential input data) +- the **workerpool** (TEE-enabled workers) +- the **requester** (the user paying for the execution) -Your account, managed by the `Escrow` part of the `IexecClerk`, separates -between `balance.stake` (available, can be withdrawn) and `balance.locked` -(unavailable, frozen by a running task). The `Escrow` exposes the following -mechanism: +Each party publishes an **order** describing what they offer or request. When +compatible orders are combined, a **deal** is created and PoCo enforces all +economic, confidentiality, and governance rules. -`lock`: Moves value from the `balance.stake` to `balance.lock` +### Why off-chain brokering? -- Locks the requester stake for payment -- Locks the scheduler stake to protect against failed consensus -- Locks the worker stake when making a contribution +iExec uses an **off-chain order book and off-chain matching** system because it +provides major advantages: -`unlock`: Moves value from the `balance.lock` back to the `balance.stake` +- orders can be created, shared, and canceled **without gas costs** +- signatures make orders **trustless and verifiable** +- brokering is fast and flexible +- the blockchain is used only to **validate signatures** and **create the final + deal** -- Unlock the requester stake when consensus fails -- Unlock the scheduler stake when the system achieves consensus -- Unlock the worker stake when they contributed to a successful consensus +Although off-chain, the system is secure because each order is: -`seize`: Confiscate value from `balance.lock` +- encoded as a structured object +- hashed using **EIP-712** +- **signed** by the resource owner -- Seize the requester stake when the system achieves consensus (payment) -- Seize the scheduler stake when consensus fails (send to the reward kitty) -- Seize the worker stake when a contribution fails (redistributed to the other - workers in the task) - -`reward`: Award value to the `balance.stake` - -- Reward the scheduler when the system achieves consensus -- Reward the worker when they contributed to a successful consensus -- Reward the app and dataset owner - -The requester payment consists of 3 parts, one for the worker pool, one for the -application and one for the dataset. When a consensus finalizes, the payment -seizes from the requester and the application and dataset owners are rewarded -accordingly. The worker pool part puts inside the `totalReward`. Stake from the -losing workers also adds to the `totalReward`. The scheduler takes a fixed -portion of the `totalReward` as defined in the worker pool smart contract -(`schedulerRewardRatioPolicy`). - -The remaining reward is then divided between the successful workers -proportionally to the impact their contribution made on the consensus. If there -is anything left (division rounding, a few nRLC at most) the scheduler gets it. -The scheduler also gets part of the reward kitty. - -#### Parameters - -`FINAL_DEADLINE_RATIO = 10`, `CONTRIBUTION_DEADLINE_RATIO = 7`, -`REVEAL_DEADLINE_RATIO = 2` - -Parameters of the consensus timer. They express the number of reference timers -(category duration) that dedicate to each phase. These settings correspond to a -70%-20%-10% distribution between the contribution phase, the reveal phase and -the finalize phase. - -- `FINAL_DEADLINE_RATIO` This describes the total duration of the consensus. At - the end of this timer the consensus must finalize. If it is not, the user can - make a claim to get a refund. -- `CONTRIBUTION_DEADLINE_RATIO` This describes the duration of the contribution - period. The consensus can finalize before that, but no contribution will be - allowed after the timer to ensure enough time leaves for the reveal and - finalize steps. -- `REVEAL_DEADLINE_RATIO` This describes the duration of the reveal period. - Whenever a contribution triggers a consensus, a reveal period of this duration - reserves for the workers to reveal their contribution. Note that this period - will necessarily start before the end of the contribution phase. - -Let's consider a task of category GigaPlus, which reference duration is 1 hour. -If the task submitted at 9:27AM, the contributions must send before 4:27PM -(16:27). Whenever a contribution triggers a consensus, a 2 hours long reveal -period will start. Whatever happens, the consensus has to achieve by 7:27PM -(19:27). - -`WORKERPOOL_STAKE_RATIO = 30` - -Percentage of the worker pool price that has to stake by the scheduler. For -example, for a `20 RLC` task, with an additional `1 RLC` for the application and -`5 RLC` for the dataset, the worker will have to lock `26 RLC` in total and the -scheduler will have to lock (stake) `30% * 20 = 6 RLC`. - -This stake loses and transferred to the reward kitty if the consensus is not -finalized by the end of the consensus timer. - -`KITTY_RATIO = 10` - -Percentage of the reward kitty for the scheduler per successful execution. If -the reward kitty contains 42 RLC when a finalize calls, then the scheduler will -get 4.2 extra RLC and the reward kitty will leave with 37.8 RLC. - -`KITTY_MIN = 1 RLC` - -Minimum reward on successful execution (up to the reward kitty value). - -- If the reward kitty contains 42.0 RLC, the reward is 4.2 -- If the reward kitty contains 5.0 RLC, the reward should be 0.5 but gets raised - to 1.0 -- If the reward kitty contains 0.7 RLC, the reward should be 0.07 but gets - raised to 0.7 (the whole kitty) - -`reward = kitty.percentage(KITTY_RATIO).max(KITTY_MIN).min(kitty)` - -#### Example - -Let's consider a worker pool with the policies `workerStakeRatioPolicy = 35%` -and `workerStakeRatioPolicy = 5%`. - -- A requester offers `20 RLC` to run a task. The task is free, but it uses a - dataset that costs `1 RLC`. The requester locks `21 RLC` and the scheduler - `30% * 20 = 6 RLC`. The trust objective is `99%` (`trust = 100`) -- 3 workers contribute: - - The first one (`score = 12 → power = 3`) contributes `17`. It has to lock - `7 RLC` (35% of the `20 RLC` awarded to the worker pool). - - The second worker (`score = 100 → power = 32`) contributes `42`. It also - locks `7 RLC`. - - The third worker (`score = 300 → power = 99`) contributes `42`. It also - locks `7 RLC`. -- After the third contribution, the value `42` has reached a `99.87%` - likelihood. Consensus achieves and the two workers who contributed toward `42` - have to reveal. -- After both workers reveal, the scheduler finalizes the task: - - The requester locked value of `21 RLC` seizes. - - The dataset owner gets `1 RLC` for the use of its dataset. - - Stake from the scheduler unlocks. - - Stakes from workers 2 and 3 are also unlocked. - - The first worker stake seizes, and it loses a third of its score. The - corresponding `7 RLC` add to the `totalReward`. - - We now have `totalReward = 27 RLC`: - - We save 5% for the scheduler, `workersReward = 95% * 27 = 25.65 RLC` - - Worker 2 has weight `log2(32) = 5` and worker 3 has a weight - `log2(99) = 6`. Total weight is `5+6=11` - - Worker 2 takes `25.65 * 5/11 = 11.659090909 RLC` - - Worker 3 takes `25.65 * 6/11 = 13.990909090 RLC` - - Scheduler takes the remaining `1.350000001 RLC` - - If the reward kitty is not empty, the scheduler also takes a part of it. - -## Replication & Trust - -### How to achieve trust ? - -The PoCo offers a consensus mechanism that can certify the likelihood of a -result to be valid. This consensus relies on the scoring of workers and the -replication of a task’s execution to combine the score of the workers that come -up with the same result. This consensus is largely based on Sarmenta’s work -[[Sarmenta2002]](proof-of-contribution.md#references) with specific tuning of -the scoring function [[Trust2018]](proof-of-contribution.md#references). - -### Contribution credibility - -Each worker’s contribution has an associated credibility. This credibility -derives from the worker’s history score. As described in -[[Trust2018]](proof-of-contribution.md#references), a worker score is a positive -integer that increments for each valid and verified contribution. In case of bad -contribution, a worker loses one third of it’s score. This credibility can -express as a likelihood percentage but also as a weight value that can be used -to detect consensus without resorting to floating point arithmetics, more -details in [[Trust2018]](proof-of-contribution.md#references). - -### Requiring a trust level - -Based on [[Sarmenta2002]](proof-of-contribution.md#references) describing the -way to combine worker’s contribution and to evaluate a result’s likelihood, a -requester can ask for the level of trust as an input for the PoCo processing, to -impose a certain quality of service. The trust level corresponds to a minimum -correctness likelihood that a result must achieve to be valid. For example, a -trust level of 0 means any contribution would accept, regardless of the score of -the worker proposing it. On the other hand a trust level of 99.99% means a -result will only accept if the contribution towards it result shows a -correctness probability higher than 99.99%. - -The trust level expresses, on-chain, by an integer `trust` such that -`threshold = 1 - 1 / trust`. - -| **Trust** | **PoCo enforced confidence threshold** | -| --------- | -------------------------------------- | -| 0 | 0% | -| 1 | 0% | -| 2 | 50% | -| 100 | 99% | -| 10000 | 99.99% | -| 1000000 | 99.9999% | - -### Limitation - -This consensus mechanism requires replicable applications. Non-deterministic -applications, long-running jobs such as webservers, do not meet this requirement -out of the box. When it is possible, the application developer must provide a -deterministic result using the `deterministic-output-path` entry of the -`${IEXEC_OUT}/computed.json` file. Otherwise, the user can still run its -application on the iExec platform but would have to disable the PoCo’s consensus -layer. Hardware security as TEE is an option to overpass this limitation. - -#### References - -| | | -| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [Sarmenta2002] | (1, 2) Luis F.G.Sarmenta. Sabotage-tolerance mechanisms for volunteer computing systems. 2002. Future Generation Computer Systems, 18(4), 561–572 [Sarmenta2002 PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67.2962&rep=rep1&type=pdf) | -| [Trust2018] | (1, 2, 3) Trust management in the Proof of Contribution protocol. 2018. Technical report. [Trust2018 PDF](https://github.com/iExecBlockchainComputing/iexec-doc/raw/master/techreport/iExec_PoCo_and_trustmanagement_v1.pdf) | +This makes an order **as authoritative as if it were published on-chain**, +without paying gas. The **PoCo** smart contract validates every signature and +ensures correct matching. -## Brokering +### What brokering enables? -### Overview +- **Access control / permissioning** Different actors define who can use their + resources (specific apps, requesters, or workerpools). -We studied many possible evolutions of the brokering process. The first -requirement is to include bid orders to complete the already available ask -orders. It soon became clear that the evolution had to go beyond a simple -interaction. We considered on-chain and off-chain approach and finally went for -a solution where both the pairing and the order book are off-chain. It might -sound counterintuitive, but it has many advantages. +- **Dynamic pricing & monetization** Apps, datasets, and workerpools set prices; + requesters set maximum prices. -If the orders and the pairing handle off-chain, how can we build an on-chain -agreement knowing that all parties have agreed? Is there a threat to the -platform security? +- **Asynchronous, trust-minimized execution** After a successful `matchorders`, + the PoCo ensures that all parties can operate asynchronously and without + direct trust: + - the deal is created on-chain and acts as the single source of truth + - the requester does not need to stay online during execution + - TEE-enabled workers independently fetch tasks when ready + - results, proofs, and outputs are submitted later when execution completes -This option relies on the use of cryptographic signatures for order -authentication. We represent orders using structures containing all the required -details. The hash of this structure uniquely identifies the order. The structure -by itself is worthless as anyone could write and publish it. However, if we add -a valid cryptographic signature (of the identifying hash), then the origin of -the order can certify with the same level of security as if it was published -on-chain. This role fulfills a smart-contract called the iExecClerk. + PoCo guarantees that even though all actors act at different times, the + workflow remains secure, deterministic, and economically enforced. -An overview of the iExecODB (Open Decentralized Brokering) is available in this -blog article: +### Order structures -- [PoCo series #5: Open decentralized brokering on the iExec platform](https://medium.com/iex-ec/poco-series-5-open-decentralized-brokering-on-the-iexec-platform-67b266e330d8) +Each actor expresses intent through a signed **order**. There are four order +types, all using **EIP-712 signatures**: -### Orders structure +1. **AppOrder** — how the application can be used +2. **DatasetOrder** — how the dataset can be accessed +3. **WorkerpoolOrder** — what TEE workers are available +4. **RequestOrder** — what the requester wants to run -As discussed earlier, iExec introduces the offchain signature of orders as a new -core element of the iExec Open Decentralized Brokering, the iExec Clerk should -match these orders. There are 4 types of orders corresponding to the 4 actors -involved: the worker pool, the application, the dataset and the requester. Each -order types has to follow a specific structure and uses -[EIP-712](https://eips.ethereum.org/EIPS/eip-712) structure signature mechanism. +Every order includes: -### Orders description +- the resource address (app/dataset/workerpool/requester) +- price +- volume (number of times the order can be matched) +- optional matching restrictions +- a `tag` describing features (e.g., TEE requirement) +- a `salt` for uniqueness +- a cryptographic **signature** -#### **AppOrder** +#### AppOrder -```text +``` struct AppOrder { address app; @@ -445,23 +201,7 @@ struct AppOrder } ``` -- `app` Address of the smart contract describing the application. Must be - registered in the AppRegistry. -- `appprice` Price of a run of the application. -- `volume` Number of run authorized (by this order). -- `tag` Special requirements for the application (see - [tag](proof-of-contribution.md#tag)). -- `datasetrestrict` Matching restrictions. Dataset or group of datasets that can - match. Let null value to disable. -- `workerpoolrestrict` Matching restrictions. Workerpool or group of worker - pools that can match. Let null value to disable. -- `requesterrestrict` Matching restrictions. Requester or group of requesters - that can match. Let null value to disable. -- `salt` A random value to ensure order uniqueness. -- `sign` cryptographic signature of the order, the smart contract is securely - linked to application owner. - -### **DatasetOrder** +#### DatasetOrder ```text struct DatasetOrder @@ -478,23 +218,7 @@ struct DatasetOrder } ``` -- `dataset` Address of the smart contract describing the dataset. Must be - registered in the DatasetRegistry. -- `datasetprice` Price of a use of the dataset. -- `volume` Number of authorized uses (by this order). -- `tag` Special requirements of the dataset (see - [tag](proof-of-contribution.md#tag)). -- `apprestrict` Matching restrictions. App or group of apps that can match. Let - null value to disable. -- `workerpoolrestrict` Matching restrictions. Workerpool or group of workerpools - that can match. Let null value to disable. -- `requesterrestrict` Matching restrictions. Requester or group of requesters - that can match. Let null value to disable. -- `salt` A random value to ensure order uniqueness. -- `sign` cryptographic signature of the order, the smart contract is securely - linked to dataset owner. - -### **WorkerpoolOrder** +#### WorkerpoolOrder ```text struct WorkerpoolOrder @@ -513,25 +237,7 @@ struct WorkerpoolOrder } ``` -- `workerpool` Address of the smart contract describing the worker pool. Must be - registered in the WorkerpoolRegistry. -- `workerpoolprice` Price of an execution on the worker pool. -- `volume` Number of executions proposed (by this order). -- `tag` Special features proposed by the workerpool (see - [tag](proof-of-contribution.md#tag)). -- `category` Order category. -- `trust` Trust level used to consolidated results. -- `apprestrict` Matching restrictions. App or group of apps that can match. Let - null value to disable. -- `datasetrestrict` Matching restrictions. Dataset or group of datasets that can - match. Let null value to disable. -- `requesterrestrict` Matching restrictions. Requester or group of requesters - that can match. Let null value to disable. -- `salt` A random value to ensure order uniqueness. -- `sign` cryptographic signature of the order, the smart contract is securely - linked to worker pool manager. - -### **RequesterOrder** +#### RequesterOrder ```text struct RequestOrder @@ -555,54 +261,25 @@ struct RequestOrder } ``` -- `app` Address of the smart contract describing the application. Must be - registered in the AppRegistry. -- `appmaxprice` Maximum price allowed by the requester for the payment of the - application. -- `dataset` Address of the smart contract describing the dataset. Must be - registered in the DatasetRegistry. Null if no dataset requires. -- `datasetmaxprice` Maximum price allowed by the requester for the payment of - the dataset (if any). -- `workerpool` Matching restrictions. Worker pool or group of worker pools that - can run tasks from this order. Leave null value to disable check. -- `workerpoolmaxprice` Maximum price allowed by the requester for the payment of - the execution (scheduler + workers). -- `requester` Address of the requester (paying for the executions). -- `volume` Number of tasks that are part of this order (size of the Bag Of - Task). -- `tag` Special features required by the requester (see - [tag](proof-of-contribution.md#tag)). -- `category` tasks category required. -- `trust` Minimum trust level required by the requester. -- `beneficiary` Address of the beneficiary of the computation. Used to require - output data encryption. -- `callback` Address to callback with the results (following EIP1154 interface). - Let empty (null) if no callback requires. Learn more about the callback - mechanism. -- `params` Parameters of the application (application specific). -- `salt` A random value to ensure order uniqueness. -- `sign` Cryptographic signature of the order, the smart contract is securely - linked to requester. - -## Tag +## Additional protocol concepts + +### Tag The tag specifies the need or the availability of features that go beyond the specifications of the category. The tag is a requirement when it is part of an app order, a dataset order or a requester order. On the other hand, the tag in the workerpool order expresses the availability of the corresponding features. -In V3, tags are 32 bytes (256 bits) long array where each bit corresponds to a -feature. +Tags are 32 bytes (256 bits) long array where each bit corresponds to a feature. -| **Value** | **Description** | -| -------------------------------------------------------------------- | ---------------------- | -| `0x0000000000000000000000000000000000000000000000000000000000000001` | TEE (physical enclave) | -| `0x0000000000000000000000000000000000000000000000000000000000000002` | — | -| `0x0000000000000000000000000000000000000000000000000000000000000004` | — | -| `0x0000000000000000000000000000000000000000000000000000000000000008` | — | -| `0x0000000000000000000000000000000000000000000000000000000000000010` | — | -| … | … | -| `0x8000000000000000000000000000000000000000000000000000000000000000` | — | +**Pre-defined tags:** + +| **Value** | **Description** | +| ------------------------------------------------------------------------------- | --------------- | +| `0x0000000000000000000000000000000000000000000000000000000000000001` (`0b0001`) | TEE | +| `0x0000000000000000000000000000000000000000000000000000000000000003` (`0b0011`) | TEE Scone | +| `0x0000000000000000000000000000000000000000000000000000000000000005` (`0b0101`) | TEE Gramine | +| `0x0000000000000000000000000000000000000000000000000000000000000009` (`0b1001`) | TEE TDX | For orders matching, the worker pool order must enable all bits that enable in any of the app order, dataset order and requester order. Meaning that if the app @@ -610,267 +287,9 @@ order tag is `0x12 = 0x10 | 0x02`, the dataset order is `0x81 = 0x80 | 0x01` and the requester order is `0x03 = 0x02 | 0x01`, then the worker pool order must, at least, have a tag `0x93 = 0x80 | 0x10 | 0x02 | 0x01`. -### Matching Conditions - -In order to trigger an execution, a deal must register by the iExec Clerk. Deals -are formed when orders are successfully matched by the clerk. A match requires 3 -or 4 orders depending on the requester requirements, the dataset order is -optional. - -Orders compatibility required: - -**1. The worker pool’s category and the requester’s category must be equal.** - -```text -require(_requestorder.category == _workerpoolorder.category); -``` - -**2. The worker pool’s trust must be greater or equal to the requester’s -trust.** - -```text -require(_requestorder.trust == _workerpoolorder.trust); -``` - -**3. The app’s, dataset’s and worker pool’s prices must be less or equal to the -requester’s appmaxprice, datasetmaxprice and workerpoolmaxprice.** - -```text -require(_requestorder.appmaxprice >= _apporder.appprice); -require(_requestorder.datasetmaxprice >= _datasetorder.datasetprice); -require(_requestorder.workerpoolmaxprice >= _workerpoolorder.workerpoolprice); -``` - -**4. The worker pool’s tag must enable all the features required by the app’s -tag, the dataset’s tag and the worker pool’s tag.** - -```text -require(tag & ~_workerpoolorder.tag == 0x0); -require(tag & ~_workerpoolorder.tag == 0x0); -``` - -**5. If TEE tag requires, then application must be TEE compatible.** - -```text -require((tag ^ _apporder.tag)[31] & 0x01 == 0x0); -``` - -**6. The app provided by the apporder must match the one required by the -requester.** - -```text -require(_requestorder.app == _apporder.app); -``` - -**7. The dataset provided by the datasetorder must match the one required by the -requester.** - -```text -require(_requestorder.dataset == _datasetorder.dataset); -``` - -**8. If the requester specified a worker pool restriction, the worker pool must -match this value or be part of the corresponding group.** - -```text -require(_checkIdentity(_requestorder.workerpool, _workerpoolorder.workerpool, GROUPMEMBER_PURPOSE)); -``` - -**9. The application must fit the dataset’s and the worker pool’s application -restrictions (if any).** - -```text -require(_checkIdentity(_datasetorder.apprestrict, _apporder.app, GROUPMEMBER_PURPOSE)); -require(_checkIdentity(_workerpoolorder.apprestrict, _apporder.app, GROUPMEMBER_PURPOSE)); -``` - -**10. The dataset must fit the application’s and the worker pool’s restrictions -(if any).** - -```text -require(_checkIdentity(_apporder.datasetrestrict, _datasetorder.dataset, GROUPMEMBER_PURPOSE)); -require(_checkIdentity(_workerpoolorder.datasetrestrict, _datasetorder.dataset, GROUPMEMBER_PURPOSE)); -``` - -**11. The worker pool must fit the application’s and the dataset’s restrictions -(if any).** - -```text -require(_checkIdentity(_apporder.workerpoolrestrict, _workerpoolorder.workerpool, GROUPMEMBER_PURPOSE)); -require(_checkIdentity(_datasetorder.workerpoolrestrict, _workerpoolorder.workerpool, GROUPMEMBER_PURPOSE)); -``` - -**12. The requester must fit the application’s, the dataset’s and the worker -pool’s restrictions (if any).** - -```text -require(_checkIdentity(_apporder.requesterrestrict, _requestorder.requester, GROUPMEMBER_PURPOSE)); -require(_checkIdentity(_datasetorder.requesterrestrict, _requestorder.requester, GROUPMEMBER_PURPOSE)); -require(_checkIdentity(_workerpoolorder.requesterrestrict, _requestorder.requester, GROUPMEMBER_PURPOSE)); -``` - -**13. All resources must register in the corresponding registries.** - -```text -require(m_appregistry.isRegistered(_apporder.app)); -require(m_datasetregistry.isRegistered(_datasetorder.dataset)); -require(m_workerpoolregistry.isRegistered(_workerpoolorder.workerpool)); -``` - -**14. All orders must sign or presigned.** - -```text -require(_checkPresignatureOrSignature(App(_apporder.app).m_owner(), _apporder.hash(), _apporder.sign)); -require(_checkPresignatureOrSignature(Dataset(_datasetorder.dataset).m_owner(), _datasetorder.hash(), _datasetorder.sign)); -require(_checkPresignatureOrSignature(Workerpool(_workerpoolorder.workerpool).m_owner(), _workerpoolorder.hash(), _workerpoolorder.sign)); -require(_checkPresignatureOrSignature(_requestorder.requester, _requestorder.hash(), _requestorder.sign)); -``` - -**15. The deal produced must contain at least one task.** - -**16. Requester and worker pool must be able to stake.** - -### FAQ : How to write an order ? - -**[Requester] How do I enable PoCo’s consolidation of results?** - -A requester can enable the trust layer of the PoCo by setting the trust value in -the requestorder. As described here, the trust defines with the required -confidence level. If the requester wants à 99.99% confidence level on the -results, it must set the `trust` field to `10000`. - -**[Requester] How do I run a non deterministic application despite requiring -determinism?** - -The PoCo requires an application to be deterministic for the replication layer -to provide trust in the result. If an application is not deterministic, -consensus cannot achieve and replication is not necessary. - -In order to obtain a result, the requester must prevent replication by asking a -`trust` value of `0`. To protect your result, the requester can ask to run in an -enclave by setting the `tag` to `0x1`. - -**[Requester] How do I protect my result using encryption?** - -The result of an execution can be valuable to the end user, and the requester -might want to protect this result from leaking with encryption. - -Anyone can set up an encryption key in an SMS (Secret Management Service) of its -choice and set up the SMS address in the directory. - -Once a user set an encryption key (see TODO), any computation result can be -encrypted with, you have to set up the beneficiary address in the -RequesterOrder. - -An application can only perform result encryption inside an enclave. No -encryption key provides the SMS to an application that doesn't run outside an -enclave. - -**[Dataset owner] How do I limit the usage of my dataset to a specific -application?** - -iExec’s Data wallet and Data store is a complete solution to monetize valuable -datasets preserving privacy. Before uploading a dataset you should encrypt it -using the iExec SDK. Through this process, the encryption key becomes the -valuable data that needs protection. - -The encryption key must store in an SMS and the address of the corresponding SMS -recorded in the directory. The SMS stores this encryption key and will only -communicate it to an application running in an enclave. - -Before a worker runs this application, the worker must first prove that its -access is legitimate by providing the scheduler authorization. The SMS will -verify that this authorization’s signature is valid and that the corresponding -task registers onchain. This means that any deal signed in the iExec Clerk will -grant access to the dataset’s encryption key. - -Therefore, in order to restrict the dataset’s usage, the dataset owner should -set up restriction before signing a brokering order. This happens through the -`apprestrict` field of the datasetorder. The dataset owner can deploy a -`SimpleGroup` smart contract, have the `apprestrict` field point to it, and then -whitelisting the applications that will have access to the dataset’s encryption -key. - -**[Scheduler] How do I protect myself from non-deterministic applications?** - -When a scheduler publishes an order, it makes a commitment to achieve consensus -on any task that is part of a deal made. While everything happens to ensure an -application cannot hurt a worker pool’s machines, not reaching the consensus -would cause a loss of stake for the scheduler. The scheduler must therefore take -action to prevent this. - -Whenever the scheduler proposes to certify a result using the PoCo’s trust -layer, it should make sure the application’s developer took the actions required -to make it compatible with the PoCo. On the other hand, any application could be -executed with the PoCo’s trust layer disabled. - -A scheduler could therefore emit two kinds of workerpoolorder: - -- A workerpoolorder offering execution with the PoCo’s trust layer disabled - (`trust = 0`) and accepting all applications (`apprestrict = 0`) -- A workerpoolorder offering secure execution of whitelisted tasks. The - application whitelist would use the `GroupInterface` to verify by the iExec - Clerk. This group could either manage the scheduler or by a certification - authority that would check application's determinism. - -## Other technical choices - -### Callback - -Some requesters might want an onchain callback with the result of the execution. -The callback mechanism uses [[EIP-1154]](proof-of-contribution.md#references-1). -The result is a `bytes` value that is set during the `finalize`. The -`IexecProxy` implements both side of the -[[EIP-1154]](proof-of-contribution.md#references-1). - -**Pull:** - -Results identify by their `taskid` and can pull through the `resultFor` method. - -**Push:** - -In order to use the push approach, the requester can use the `callback` field to -specify the address of a smart contract that implements the `receiveResult` -method specified in [[EIP-1154]](proof-of-contribution.md#references-1). This -method calls during the finalization with a minimum of 200000 nanoRLC gas to -proceed [[*]](proof-of-contribution.md#references-1). - -In order to protect the scheduler and the workers, any error raised during this -callback ignores and will not prevent the finalization from happening. The same -mechanism goes for the callback running out of gas. - -### Consensus & Reveal duration - -When orders match, IexecClerk records the deal which details the parameters of -the task. If a consensus on a result achieves before the countdown, the task is -successful. After the countdown, as no consensus reaches, the execution is -failed. The duration of the consensus timer is a balance between the quality of -service offered to the requester (short timer) and the margin available for the -scheduler and the worker to achieve consensus (and go through the reveal -process). - -The maximum duration of a task governs the category the task fits in. While the -consensus duration can obviously not be shorter than the task runtime, a -significant margin requires for the scheduler to do its job correctly. Multiple -workers are likely to contribute and extra time must plan for the revealing and -finalization steps. - -The consensus timer starts when the deal records the IexecClerk. - -In case of failure, the requester can claim the refund. - -In the v3-alpha version, this timer lasts for 10 category runtime, for all -category. The reveal timer starts whenever a consensus reaches and determines -the time frame the workers have to reveal their contributions. This should be -long enough for worker to have time to reveal while not being too long so that -the requester waits too long for its result or the consensus fails because the -scheduler cannot finalize in time. In the v3-alpha version, this timer lasts for -2 runtime whatever the category runtime. This leaves a gap of at least 1 time -the category runtime for the scheduler to finalize the task. - ### Reward kitty -When a consensus fails, the requester gets a refund and the scheduler loses its +When a task fails, the requester gets a refund and the scheduler loses its stake. In order to remove an attack vector, the requester does not get any of the seized stake. If this was a feature, anyone could build a flawed application that would not reach consensus to drain money from the scheduler. This would @@ -883,10 +302,3 @@ also gets a small part of the reward kitty. As described in the protocol parameters section, this reward is `reward = kitty.percentage(KITTY_RATIO).max(KITTY_MIN).min(kitty)`. - -### References - -| | | -| -------------------------------------------------------------- | --------------------------------------------------------------------- | -| [[EIP-1154]](proof-of-contribution.md#other-technical-choices) | [EIP-1154: Oracle Interface](https://eips.ethereum.org/EIPS/eip-1154) | -| [[*]](proof-of-contribution.md#other-technical-choices) | value susceptible to change. |