Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Litestream Azure Blob support #445

Open
Rob787 opened this issue Jun 14, 2023 · 3 comments
Open

Enable Litestream Azure Blob support #445

Rob787 opened this issue Jun 14, 2023 · 3 comments

Comments

@Rob787
Copy link

Rob787 commented Jun 14, 2023

Hi,

First of all thanks for the amazing project :)

It seems that the Litestream implementation is hardcoded for S3 storage only? Is it possible to enable all the types that Litestream support (maybe with a new ENV variable) or at lease Azure Blobs?

https://litestream.io/guides/azure/

@mtlynch
Copy link
Owner

mtlynch commented Jun 15, 2023

Thanks for the kind words!

Yeah, the way we pass environment variables to Litestream is pretty narrow, so we limit the functionality that Litestream offers.

It should be possible to adjust it for more flexibility, but I'm not likely to implement it since I don't use Litestream features beyond what's currently exposed. But if someone wanted to work on this, I'd accept it as long as it didn't make the existing scenarios really hard to preserve.

@Rob787
Copy link
Author

Rob787 commented Jun 15, 2023

Ah ok, got it! But for my understanding, is Litestream itself implemented in the package fully? So it's just a question of manipulating the litestream.yml replicas settings?

In that case it would be providing an env variable that toggles a certain replica type in the yml and then use the existing variables to fill the rest? Like, if type x use this part of the yaml.

Looking at the one for Azure, it's not too different from the S3 one:

dbs:
  - path: /path/to/local/db
    replicas:
      - type: abs
        account-name: STORAGEACCOUNT
        account-key:  ACCOUNTKEY
        bucket:       CONTAINERNAME
        path:         PATH

LITESTREAM_ACCESS_KEY_ID variable could be used for the account-name
LITESTREAM_SECRET_ACCESS_KEY could be account-key
LITESTREAM_BUCKET is bucket obviously

Or am I think too simple about it? :) I'm not a hard-core developer, so not fully in power to judge and implement unfortunately.

@mtlynch
Copy link
Owner

mtlynch commented Jun 16, 2023

Yeah, it would be a matter of piping in the logic so that you can specify a different format of litestream.yml file. Because right now the litestream.yml file PicoShare uses assumes S3:

https://github.com/mtlynch/picoshare/blob/c67d347217c6c091f1b80f99f93fbc84c6f16446/litestream.yml

To get Litestream to replicate to Azure, we'd need to write the file like this:

https://litestream.io/guides/azure/#configuration-file-usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants