Compilation of possible uses for PubSub. These are (should) be accompained by a description, requirements and expected API.
- chat / communication systems
- collaborative editors
- live streaming
- PubSub type: topic based
- Javascript library preferred
- message could be a too
- latency: delivery in < 1ms ;)
- needs to be push (on subscriber side)
- topology: ideally all-to-all (distributed), would be ok with many-to-all (decentralized)
- links: redis has a solid interface (https://www.npmjs.com/package/redis#publish--subscribe)
- ipfs.sub('topic', (sender, message) => ...)
topiccan include wildcards:topic-from-haad-*
- ipfs.publish('topic', message)
- topology: some-to-many
- type: topic, content, type, concept (can be any) // type and content - subscription represented as a query, reuse IPLD with predicates, inspiration from XPATH pub-sub ??
- reliability: does not need to be a requirement //reliability/correctness related properties:- delivery guarantees, enforce ordering, allow gaps.
- realtime: not 100% required // efficiency-related properties:- timeliness, limitation of rate/period/length/size of infromation received
- encryption: important
- interface:
- subscriber:
- Be able to subscribe on one or more topics and validate that each message was generated by a trusted source // regular expressions over topic space, "content-based" condition/predicates on each level of namespace -- generic content-based queries/subscriptions over the real inner content require some kind of prior agreement on format (simpe text, XML ?, JSON, etc., lots of inspiration for the interfaces can be derived from XML/XPATH - though actually very inefficient.
- Subscribe and validate all the updates through the PubKey found in the IPRS record (https://github.com/ipfs/specs/tree/master/iprs-interplanetary-record-system)
pubsub subscribe --{topic, content, type, concept}="[...]" --auth=/iprs/QmHash- publisher:
- Be able to join a 'ring of publishers'
- Publish under a key or several keys
- Simple example:
- Wordpress webpage // subscribe to all updates in setion / page / set of pages; subscribe to updates by person / set of people, to updates with given content (string) or specific update metadata iinformation
- Have several roles (Author/Editor/Subscriber), organize the network based on these roles (e.g Author link to Editor and subscribers, subscribers to subscribers)
- Converge the news feeds through CRDT (offload that responsability from the pubsub protocol) // stock ticker based on topic (market / stock domain / companu names), based on content (e.g., variation over X% ) // betting system warning system based on domain / competition / team, etc. and based on content (odds over X, odds raising over X%, etc.)
- subscriber:
I want to be able to get updates on the new hash that an IPNS key is signing,
When there is a new update (that invalidate the previous one) and the owner signs it, The owner should be able to propagate the update to the rest of the network Even if the owner does not propagate to the rest, if users fetch it and they have subscribers, they should be able to propagate it
- Properties:
- Can skip messages (unreliable)
- No need of ordering (only the latest one really counts!)
- Q: which one is the latest though? (causal ordering!? to be defined)
- Little flooding through the network
- Does not need to persist beyond IPNS expiration time
- On new messages, we can just skip gossiping/emitting the previous ones
- In essence: replaces the DHT via a lazy pub/sub, you only get updates when peers connected to you receive an IPNS update
- API:
- Subscriber
- .sub(HASH) (we could actually make this hierarchical with IPLD pathing HASH/friends/0/blog)
- .unsub(HASH)
- Publisher
- .pub(HASH, content) (no need to re-publish for all the hierarchical changes since .pub should be clever enough)
- Subscriber
Creat shared multiplayer environments, potentially thousands of users. Similar requirements to chat, but events are more frequent and smaller
- Interface: Library
- Ideally something that works in JavaScript/browser via WebRTC
- Pushx
- Persistent (maybe?)
- Low latency
- Ability to prioritize events (some inputs can be discarded, user knows priority)
- Complex filtering (users only need to track events based on their location)
- Ordering of events can be causal
- Subsystem that listens to messages sent from a cluster of nodes pertaining to particular types of events or metrics. Has some ability to alert based on particular thresholds for a metric. Displays graphs/visualizations.
-Ideal interface:
- func publish(topic, message) error
- func read(topic) (message, error) Properties:
- Persistant queues of messages per topic, but of bounded length if no reads occur, futher older messages may be cleared.
- Effortless topic creation (there's no create topic or other topic management on the publisher side)
Support a global network of software sensors. A 'software sensor' is a node in the proposed network that takes zero or more input streams and shares messages in a single output stream. Software sensors have a unique identity and produce signed messages that can be verifiably traced back to the emitting sensor. We imagine two types of software sensors: Atomic and Composite.
Atomic Sensors broadcast messages that contain a single, arbitrary data structure (as defined by some protocol). The data contained in Atomic sensor messages has pointers to previous broadcasts, and are used in arbitrary compositions by network participants and consumers. Atomic Sensors may be a mix of hardware and software, or software only. We envision software sensors often running on embedded hardware and publishing messages based on sensor readings.
Composite Sensors broadcast messages that contain a single, arbitrary data structure whose end value(s) are composed from messages from various, upstream Atomic and Composite sensors in the network. In this sense, Composite sensors can be seen as generating higher-order data and messages. These higher-order messages can be consumed by other network participants. Composite sensors also contain knowledge of the streams they tap to create their published messages. This allows subscribers to access the tree of intermediate messages that led to the publication of a given message all the way down to the originating Atomic sensors.
Atomic Example
-
Hardware + software: A single sensor that consists of a physical water temperature sensor that is able to automatically broadcast a message with the current water temperature, to the network, every minute.
-
Software-only example: A research team that, based on their latest findings, updates a value representing the volume of E. coli required to synthesize biofuel in a given aqueous environment. Whenever the value changes, a message containing the new value is automatically broadcast to the network.
Composite Example
-
Hardware + software example: A single sensor that reads data directly from several physical water temperature sensors, and also subscribes to several water temperature
Atomicsensors in the network. It then composes the various temperatures into a single average value and automatically broadcasts it to the network in its single, composited message. -
Software-only example: A single sensor that only subscribes to several water temperature
Atomicsensors in the network and composes the various temperatures into a single average value and automatically broadcasts to the network it in a message.
- Topic based
- JS library preferred at the outset
- Low latency (for actuation on perceived 'real time' data)
- Push (for subscriber)
Subscribe: ipfs.sub('topic', (sender, message) => ...)
topic:- can include wildcards (see above example:
topic-from-haad-*) - might be a stream / software sensor id
- this implies signed messages
- can include wildcards (see above example:
Publish: ipfs.publish('topic', message)
message:- Preferrable for it to be actual data (could be a to look up; obvious tradeoffs)