Cursor sequence for the main firehose increments by 10

[ggwc82]

It's described in the documentation that event sequences may have arbitrary gaps however from observing the main firehose, there appears to be a consistency in that each event increases In sequence by 10.

Is the main firehose implemented with consistent steps, and therefore can I rely on this, despite what the document says? (i.e. arbitrary gaps is optional and based on design, but can be removed altogether).

[bnewbold]

By "main firehose" do you mean the relay firehose at bsky.network? That is a popular relay, but technically there isn't a single "main" firehose or relay for the network.

We just double-checked and the sequence numbers are mostly incrementing by one coming from that relay. They should also be incrementing by one coming from the bluesky-developed PDS implementation event stream. In all of the infra that Bluesky runs, we would probably only include gaps in a migration or recovery situation, not normal operation.

[ggwc82]

Yes, sorry still getting familiar with the taxonomy for BlueSky. Is there a directory where I can find all the available firehose relays?

Regarding the incrementing by 10, yes you are right - I do see it incrementing by 1 not 10 (this turns out to be an implementation detail of the cursor in the Bluesky generator application I am using).

I'll design according to normal sequential increments and perhaps log or raise warnings if gaps are detected.

One additional question - when can we expect some form of authentication on the bsky.network firehose, or will it stay open to all? Thank you.

[DavidBuchanan314]

While it's not a relay, my PDS implementation used timestamp-based seqs (which I believe is spec-compliant), meaning every event had an arbitrarily large gap - something you may want to consider. (although realistically you probably won't ever see this from a relay)

[bnewbold]

Authentication: we think it is pretty important for there to be accessible and permissionless/antagonistic-interoperability firehose providers. we have no plans to require auth for the relay we run.

That being said, no-cost and no-auth means no SLA or guarantees. We can certainly imagine a "supported" infrastructure tier with guarantees. And we don't think it is against the values of the protocol generally to have infrastructure access be negotiated. What would be against the values of the protocol would be restricting access to data/content, or making public content differently or only partially visible to some actors in the network.

One analogy is time info and NTP, or DNS: for some high-throughput/high-reliability situations, you could imagine gating access to an NTP server, or connections to a DNS server. But it is critical that time info and access to the DNS system generally are abundant and available; nobody is excluded from knowing the current time.

There is a second public relay at relay.pop2.bsky.network, and i'm just now getting an additional demo relay and up running at relay-ovh.demo.bsky.network (it will be unreliable and may get wiped).