InterPlanetary Linked Data (IPLD)

IPLD is the foundational data model for the InterPlanetary File System (IPFS). Every file and directory is represented as an IPLD Merkle DAG, where each block is identified by its Content Identifier (CID). This enables:

• Deduplication: Identical content generates the same CID, stored only once.
• Verifiability: Data integrity is guaranteed by the cryptographic hash in the CID.
• Persistence: Content remains accessible as long as one node stores the linked blocks.

IPLD is used to model and traverse blockchain data. For example, Ethereum blocks, transactions, and state tries can be represented as IPLD objects. This allows tools to:

• Query across chains using a single, consistent data model.
• Navigate linked data (e.g., from a transaction to its block header) via CID links.
• Enable interoperability by providing a common 'language' for disparate blockchain data structures like Bitcoin UTXOs or Filecoin deals.

OrbitDB is a peer-to-peer database built on IPFS and IPLD. It uses IPLD to create CRDTs (Conflict-Free Replicated Data Types) for distributed data synchronization. Key features include:

• Automatic conflict resolution for offline-first applications.
• Immutable operation log where each entry is an IPLD node linked by CIDs.
• Decentralized indexing where database state is a verifiable, traversable Merkle DAG.

IPLD Schemas provide a type system for IPLD data, similar to JSON Schema or Protocol Buffers. They enable:

• Data validation and structure definition for IPLD nodes.
• Code generation for type-safe data handling in various programming languages.
• Logical data views by allowing the same underlying data to be represented and validated through different schema 'lenses', enhancing data interoperability.

IPLD enables mutable pointers to immutable data. InterPlanetary Name System (IPNS) creates a CID that can be updated to point to the latest version of a dataset, using IPLD links. DNSLink uses DNS TXT records to point a domain name to an IPFS/IPLD address. This solves the 'linking to moving content' problem in decentralized systems.

IPLD Selectors are a query language for traversing and selecting nodes from a Merkle DAG. They allow applications to:

• Fetch specific subsets of a large graph without downloading everything (e.g., just the metadata of a file).
• Enable efficient syncing and partial replication between peers.
• Form the basis for more complex graph queries over decentralized data, providing a powerful alternative to traditional database queries.

Projects like OrbitDB use IPLD as their underlying data store to build serverless, distributed, peer-to-peer databases. Key features include:

• CRDTs for Conflict-Free Data: Uses Conflict-Free Replicated Data Types (CRDTs) stored as IPLD objects to ensure consistency across peers.
• IPFS-Backed Storage: All database operations (insert, query, sync) are performed directly on the IPFS network, leveraging its content-addressed storage.

Developers use IPLD libraries to build applications that interact with decentralized data.

• Data Querying: Tools like IPLD Explorer and dag-jose allow for introspection and manipulation of IPLD graphs.
• Library Support: Official implementations exist for JavaScript (js-ipld) and Go (go-ipld-prime), providing the core logic for creating, parsing, and traversing IPLD data.

A Content Identifier (CID) is a self-describing content-addressed identifier used in IPLD and IPFS. It is a cryptographic hash of the data it points to, ensuring integrity and verifiability.

• Key Property: A CID contains a multihash, which specifies the hash function and length used.
• Self-Describing: It includes a codec identifier (e.g., dag-cbor, dag-json) describing how to interpret the data.
• Example: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi

The InterPlanetary File System (IPFS) is a peer-to-peer hypermedia protocol for storing and sharing data in a distributed file system. IPLD is its underlying data model.

• Relationship: IPFS uses IPLD to structure and link all content (files, directories, blocks).
• Mechanism: Data is broken into blocks, each addressed by its CID, and linked via Merkle DAGs.
• Use Case: Enables decentralized websites (dweb), resilient data storage, and content-addressed distribution.

A Merkle DAG (Directed Acyclic Graph) is a data structure where each node is identified by the cryptographic hash of its contents, and nodes are linked to form a graph without cycles.

• Core of IPLD: All IPLD data forms a Merkle DAG, enabling efficient verification and deduplication.
• Properties: Immutable (hash changes if data changes) and verifiable (anyone can recompute hashes).
• Application: Used for versioned file systems (like Git), blockchain structures, and complex datasets.

Multiformats is a collection of self-describing protocol suites that IPLD relies on for interoperability. They provide future-proof, extensible identifiers.

• Multihash: Specifies hash function and digest length (e.g., sha2-256). Used within CIDs.
• Multicodec: Identifies data serialization formats (e.g., dag-cbor, raw).
• Multibase: Encodes binary data into various string encodings (e.g., base58btc).
• Purpose: Ensures systems can evolve without breaking existing data.

Filecoin is a decentralized storage network built on top of IPFS and IPLD. It creates a marketplace for storage and retrieval using blockchain and cryptographic proofs.

• IPLD Foundation: All data stored on Filecoin is structured and referenced using IPLD's Merkle DAGs and CIDs.
• Incentive Layer: Adds economic incentives for provable, long-term storage of IPLD data.
• Proof Systems: Uses Proof-of-Replication and Proof-of-Spacetime to verify storage commitments.

DAG-JSON and DAG-CBOR are the primary serialization formats for IPLD data, extending JSON and CBOR with formal links.

• DAG-CBOR: The canonical format for IPLD. It's a Concise Binary Object Representation with a defined schema for encoding CIDs as links.
• DAG-JSON: A JSON-based format, easier for debugging, that also supports CID links.
• Key Feature: Both formats allow embedding a CID as a special link object ({ "/": "<cid>" }), enabling the graph structure.

A Content Identifier (CID) is the foundational addressing system in IPLD. It's a self-describing content-addressed identifier that uniquely points to a piece of data. A CID contains:

• A cryptographic hash of the data (e.g., SHA2-256).
• A codec identifier specifying how to interpret the data (e.g., dag-cbor).
• A multihash format for future-proofing the hash function. This structure allows any system to verify the integrity and format of the data it retrieves.

The IPLD Data Model defines a universal set of data types that can be represented across different serialization formats. It includes:

• Nodes: The basic unit, which can be a map, list, integer, string, boolean, byte array, or link (CID).
• Links: Special nodes that point to other nodes via CIDs, forming the graph structure. This abstraction allows tools to work with data from IPFS, Filecoin, Git, and other protocols using a common interface.

DAG-JSON and DAG-CBOR are the primary serialization formats for IPLD data. They encode the IPLD Data Model into bytes for storage and transmission.

• DAG-CBOR (Concise Binary Object Representation) is the recommended default. It is compact, fast to parse, and supports all IPLD data types.
• DAG-JSON is a JSON-based format, useful for debugging and web compatibility, but less efficient. Both formats have strict rules for encoding CIDs as a special link object ({ "/": "<cid>" }).

An IPLD Selector is a declarative query language for traversing and selecting specific nodes within a Merkle DAG. Instead of fetching an entire graph, you can specify a path or pattern to retrieve only the needed data. Key concepts include:

• Path Expressions: Navigate through links (e.g., a/b/0/c).
• Recursive Selectors: Match patterns deep within the graph.
• Conditional Matches: Select nodes based on their content. This enables efficient partial data retrieval for applications like blockchain state queries.

IPFS (InterPlanetary File System) is the most prominent application of IPLD. IPLD provides the data layer for IPFS, enabling:

• Content Addressing: Every file and directory is represented as an IPLD DAG, addressed by its CID.
• Deduplication: Identical data blocks are stored only once.
• Flexible Structures: Data can be organized as balanced trees, trickle DAGs, or custom graphs. IPLD's role is to define what the data is, while IPFS handles how to store, retrieve, and discover it across a peer-to-peer network.

IPLD Schemas provide a type system and structural validation for IPLD data, similar to JSON Schema or Protobuf. They allow developers to:

• Define expected node types (structs, enums, unions).
• Specify field names and their value types.
• Create type aliases for complex structures.
• Generate code bindings in multiple programming languages. Schemas bring consistency and safety to applications building on IPLD, ensuring data conforms to a predefined structure across different implementations.