Indexing and querying data at the protocol level

[criadoperez]

The protocol will have thousands or hundreds of thousands of json files representing specific data (so predefined formats).
We need an efficient way to index and query all these files, once we have them stored on IPFS.

Preferably in a decentralized manner.

I can think of the following possible solutions. Currently, I'm only considering The Graph, Ceramic or IPLD, but wanted to lay out here more options that are also debatable.

Option	Level of Decentralization	Pros	Cons	Impact of MFS/Revocation	Consistency Model
The Graph	High	- Decentralized and scalable indexing and querying. - Supports complex and flexible queries via GraphQL. - Well-suited for blockchain and IPFS data. - Easy to define schemas and mappings.	- Requires familiarity with GraphQL and subgraph development. - Hosted service or running a Graph node required.	- Not natively designed for MFS. - Data revocation needs additional mechanisms.	Eventual Consistency
ElasticSearch	Low	- Powerful full-text search capabilities. - Advanced querying and analytics. - Scalable and mature ecosystem. - Rich developer tools and community support.	- Requires setup and management of ElasticSearch clusters. - Centralized unless combined with decentralized storage solutions.	- Can integrate with MFS for mutable data management. - Revocation handled via database updates.	Strong Consistency (with cluster setup)
Database with IPFS Integration	Low to Medium	- Familiar relational or document-based database querying. - Mature ecosystems with extensive tooling and support. - Easy to manage data relationships and indexing.	- Centralized unless using a distributed database. - Requires additional infrastructure for database management.	- Can integrate with MFS for mutable data management. - Revocation handled via database updates.	Strong Consistency (traditional databases)
ComposeDB (Ceramic Network)	High	- Decentralized, mutable data network. - Strong integration with IPFS. - Allows for dynamic and mutable data updates.	- Relatively new and evolving ecosystem. - May have a learning curve and require adaptation to new tools.	- Native support for mutable data. - Provides better mechanisms for data revocation.	Eventual Consistency
BigchainDB	High	- Combines blockchain technology with a database. - Decentralized and tamper-resistant. - IPFS integration for distributed file storage.	- Overhead of blockchain operations. - Complexity in setup and management. - Less mature compared to traditional databases.	- Not specifically designed for MFS. - Data revocation requires additional logic.	Eventual Consistency
IPLD (InterPlanetary Linked Data)	High	- Directly integrated with IPFS. - Enables creation of complex data structures. - Decentralized and content-addressable.	- Requires more custom development for indexing and querying. - May need additional infrastructure for efficient querying.	- Can work with MFS but requires custom implementation. - Revocation needs custom handling.	Eventual Consistency

[criadoperez]

@0xbarath and @denciu we have a change now... We no longer have json files, but DDEX messages which are xml files. And these files need to follow the standard.
Does the decision of using The Graph still stand?