Haraka dies when connection to redis is lost

[DoobleD]

This is probably more of an issue with node-redis than Haraka's redis plugin.

If the connection to redis is lost (e.g. redis failover, transient network issue, ...), an exception is thrown by node-redis that isn't catched, and Haraka dies. Ideally, Haraka should be resilient to such events, and instead attempt to reconnect for a reasonable amount of time.

My understanding is that node-redis is actually supposed to try reconnecting, via the default socket.reconnectStrategy.

It can be reproduced quite easily by simply starting redis + Haraka, then stopping redis and wait for a few seconds.

[CRIT] [-] [core] Error: Socket closed unexpectedly
[CRIT] [-] [core]     at Socket.<anonymous> (<path_to_haraka>/node_modules/@redis/client/dist/lib/client/socket.js:194:118)
[CRIT] [-] [core]     at Object.onceWrapper (node:events:634:26)
[CRIT] [-] [core]     at Socket.emit (node:events:519:28)
[CRIT] [-] [core]     at TCP.<anonymous> (node:net:339:12)
[NOTICE] [-] [core] Shutting down

I'm not sure if there's something that can be done about it in haraka-plugin-redis. I'm open to ideas, and at the very least I thought I should mention the issue.

[DoobleD]

After a deeper look, I realize this actually happens because of other haraka plugins creating redis connections without setting an error handler on the client, which translates in redis-node's error events generating exceptions (as per node's docs).

At least these plugins are concerned:

• haraka-plugin-watch, here.
• haraka-plugin-karma, here.

The package haraka-plugin-redis also has 2 client creations that don't set an error handler, here and here, though I'm unsure how they are used.

Should we create PRs to add error handlers to all of these?

With error handlers everywhere I cannot reproduce this exact issue. I can still get Haraka to die by attempting to e.g. send an email while redis is loading the RDB dump, which also generates an exception. That may be a separate issue.

[DoobleD]

Thanks for merging #67 and #61 @msimerson. No merge for #46?