Add Nats JetStream as a sink#2148
Conversation
@koocbor could expand upon the situation? As far as I can tell, from the publishing perspective, Jetstream vs. Core is effectively the same. Are you sure it is not a configuration of your subject to be a Jetstream storage? |
|
@yordis - Feeling quite dumb here, you are correct. My assumptions and order of operations were wrong, and I was not creating the stream before publishing, so I never had the JetStream persistence - hence was losing messages if there were no consumers subscribed. Once I manually created the stream first I now have a persistent stream. I'm going to close this and am happy to use the existing NATS sink as-is. Sorry to waste your time. |
|
@koocbor no problem, maybe you could send a documentation clarification instead |
I like Sequin as a CDC solution, however in my current project we use NATS JetStream for our events. We tried using the existing NATS sink, but our consumers are occasionally missing events if they happen to be down when a CDC message is published to the NATS subject.
This PR adds support for NATS JetStream as a sink for sequin. The existing NATS sink doesn't provide persistence making it non-ideal in scenarios where the event consumers might not always be running.
The biggest change in this PR is to use
Gnat.requestinstead of the existingGnat.pubto send messages to a JetStream subject. There are also a few additional configuration values for the NATS JetStream sink such as stream name and domain.I am not an elixir developer and have tried to follow the existing patterns as closely as possible so please forgive any egregious errors.
I also updated the documentation for the existing NATS sink that had a repeated line in the how-to document.
Thanks for considering.