Definitions
This page contains definitions of terms used in the Spark protocol
Clients
Storage Clients (not Stations) who received datacap and made deals using that datacap.
Checker / Checker Node
Nodes that do retrievability checks. In practice, these are primarily Station nodes running the Spark module.
Committee
Checker / Checker Nodes that have performed the same Retrieval Task in a round
- Committees aren’t formed during tasking. Nodes pick tasks based on the hash of their self-generated id. The scheduler doesn’t assign tasks to nodes, because at the beginning of the round it doesn’t yet know who will participate. See Task Sampling for more info.
Eligible Deal
(providerId, payloadCID, clients)- Note: in the context of this doc, these are eligible deals for retrieval testing
- Created during Deal Ingestion
- Created by fil-deal-ingester
- Consumed by Task Sampling
- Callout: we only have some of the payloadCIDs. While there is at least one for every deal, deals with multiple payload CIDs only have one payload CID in the Eligible Deals DB.
Retrieval Task
(roundId, providerId, payloadCID)- For each round, randomly sourced from Eligible deals
- Created during Task Sampling
- Created by Spark Tasking Service
- Consumed by
- Checker / Checker Node for performing Retrieval Checking
- Spark-Evaluate as part of Measurement Submission
Round Retrieval Task List
[Retrieval Task]- The set of retrieval tasks that were selected for a given round.
- Downloaded by Checkers as part of Task Sampling
- Note: creation/consumption is the same as Retrieval Task
Retrieval Result
(providerId, indexerResult, protocol, providerAddress, startAt, statusCode, firstByteAt, byteLength, carTooLarge, carChecksum, endAt, timeout)- These are the outcome of Retrieval Checking and is the key data structure in a Retrieval Task Measurement.
- Not every Retrieval Result leads to a Retrieval Task Measurement (see components)
- Components:
- RPC API call to convert MinerID to PeerID. If this fails, then no Retrieval Task Measurement is produced.
- IPNI query
- CAR transfer (includes DNS and TCP errors)
- CAR validation
- Created by Checker / Checker Node
- Consumed by Spark-Evaluate to determine the
retrievalResultCode.
Retrieval Task Measurement
(roundId, providerId, payloadCID, checkerId, retrievalResult)- This is what is submitted by a Checker / Checker Node for Measurement Submission.
- Created during Retrieval Checking and Measurement Submission
- Created by Checker / Checker Node
- Spark-API receives measurements and store them in the publishing queue.
- Spark-Publish uploads them to Storacha.
- Spark-Evaluate downloads the measurements and process them as part of the round evaluation.
- Consumed by Spark-API
Committee Retrieval Task Measurements
(roundId, providerId, payloadCID, retrievalTaskMeasurements)- This is the set of ALL Retrieval Task Measurements submitted by the network for a Committee, including invalid/fraudulent measurements.
- Created at the end of Measurement Submission
- Created by multiple by Checker / Checker Nodes
- Consumed by Spark-Evaluate
Accepted Retrieval Task Measurement
(roundId, providerId, payloadCID, checkerId, retrievalResult, retrtievalResultCode, clients)- A Retrieval Task Measurement that passes the fraud detection checks during Measurement Evaluation
- Created during Measurement Evaluation
- Created by Spark-Evaluate
- Consumed by Spark-Evaluate
Committee Accepted Retrieval Task Measurements
(roundId, providerId, payloadCID, checkerId, retrievalResult, retrtievalResultCode, clients)- The Accepted Retrieval Task Measurement for a Committee.
- Created during Measurement Evaluation. This is what is stored off chain, with a pointer on chain.
- Created by Spark-Evaluate
- Consumed by Spark-Evaluate