Feature Request: Import trails from external providers

[SehrGuterCode]

First of all, thanks for creating this project, I've been searching for something like this for a long time.

I record all my tours via Komoot and would love to be able to directly import them from there as it would take me a lot of time to manually download and import hundreds of GPX files. Support for other providers like Strava would also probably be pretty useful to people using them.

[Flomp]

Thanks for checking it out!
Integrating with Komoot and Strava would likely entail enabling authorization with their APIs. I will need to look into how much work that is.
Currently, I can offer wanderer's /trail/upload API endpoint to automatically upload GPX files and create trails. Furthermore, I'm working to implement this feature for the next patch.

[Fmstrat]

Adding request for Garmin if this ever happens.

[Fmstrat]

Might also make sense for Fittrackee since this is a similar audience.

[Solear]

A vote for Komoot! It would awesome!

[Flomp]

Hey there,

I can see that this is a highly requested feature (understandably so). However, looking at the mentioned providers, it's evident that most do not offer a sufficient API.

Komoot does not offer an API at all:
https://support.komoot.com/hc/articles/7464746034458-komoot-API

Alltrails also

Garmin offers an API however only for companies:
https://www.garmin.com/en-US/forms/GarminConnectDeveloperAccess/

Only Strava offers an easy way to pull data from their systems:
https://developers.strava.com/docs/reference/

I will start working on the strava integration soon, but for all other providers, it seems that you will need to export/import manually.

[PhilemonDev]

Komoot does not offer an API at all: https://support.komoot.com/hc/articles/7464746034458-komoot-API

There are undocumented API endpoints that allow pulling tour/GPX data from Komoot (python implementation, experimental rust implementation). If you do not want to include unofficial APIs in this project I understand that, then I'll probably write my own small service some day that syncs Komoot with wanderer (using the upload endpoint you mentioned above).

[Flomp]

I have implemented integrations for strava and komoot. I kindly ask you to test the integrations before releasing v0.15.0. You can update the Docker image for wanderer-web and wanderer-db to v0.15.0-beta to receive early access.

The new intergrations settings are located under Settings -> Integrations.

Strava

You need to create an API-Application in strava first:
https://developers.strava.com/docs/getting-started/#account

Provide wanderer with your Client ID and Client Secret, turn the integration on and authorize wanderer. Afterwards, your routes and activities should be synced.

Komoot

Simply provide wanderer with your username and password of your komoot account and turn the integration on.

Important

Please note that the sync runs as a cronjob every 5 minutes for testing purposes. Don't leave the beta version running for too long to avoid unnecessary traffic to the APIs.

Please let me know here if you find any bugs, problems etc.

[CyberSparkCloud]

Nice work thanks! This is the feature I’ve been waiting for! I even tried to script a sync between Strava / Wanderer myself and got it partly working but gave up.

One question.. is it possible to change the sync time so I can leave it running without hitting rate limits? 5 minutes is nice but I’d be quite happy with an hourly or even longer (up to a day) sync interval.

[Flomp]

The sync interval will be customisable via an environment variable (with the default probably being once per day). For beta testing I just didn't want people to have to wait for hours until the first sync.

[CyberSparkCloud]

Perfect thanks. I'll try the beta version tonight.

[SehrGuterCode]

@Flomp thank you so much for implementing this feature! I just tried out the Komoot integration, and it looks great! However, it only loaded 57 tours, which are all marked as "planned" on my Komoot profile (I have more planned tours but for some reason it only loaded 57). It didn't load any of my completed tours on Komoot (I have hundreds of them on my profile). Is there any way for me to debug this, such as log output I could check? I'm using Docker, but attaching to the db container via docker logs -f wanderer-db did not yield any results apart from the start-up message where it prints the routes.

[Flomp]

Thanks for testing! I assume that the 58th tour threw an exception. The komoot API is unfortunately very finicky and completely undocumented so I guess you are getting a response that I have not considered yet.
Regarding your completed tours. I just found out that my completed tours do not have a difficulty. This means: they are saved and I can see them in my database. However, they get filtered out by meilisearch because they do not have a difficulty set. Is that maybe also the case with yours?

You will find logs in the pocketbase admin dashboard. Here is how you can access it. The logs are located under /_/#/logs. They will have a log level of "warn" (you can filter for it).
Let me know if I can help you with anything.

Thanks!

EDIT:
I updated the beta image: tours without a difficulty now get a default one. So maybe this fixes the issue? You would need to clear your old import though as I don't overwrite tours that have already been imported.

[SehrGuterCode]

Thanks for the response. I just checked the logs and there was an error due to a tour being too large:

I had two planned tours with tens of thousands of kilometers on my Komoot profile which I made as a joke. This would definitely explain why some planned tours weren't loaded. I deleted them and since have not been getting any errors anymore.

I also checked the trails collection and there were actually all of my tours in there, but as you suspected, all of my completed tours didn't have a difficulty set. I deleted them all and let the integration redownload them and now they are displayed properly!

Something else I noticed: The integration settings load the password I set for Komoot when opening them. Is it possible to only allow overwriting, but not retrieving the password (or does that not work since the permissions set on Pocketbase would allow retrieving it anyways)? If somebody was to hack the account of a user who has this integration enabled, they would also gain access to the Komoot account which is not optimal.

[Flomp]

Great! Happy to hear that it's working.
That is a valid point. I shouldn't leave credentials in clear text on the server. It's a bit tricky because I cannot one-way hash them like passwords as I will need to send the clear text password to komoot/strava.
Probably the easiest way would be to create an encryption key env variable that is only known to site admins and encrypt the credentials with that. The server can decrypt the credentials with this key from the env variable and send them to the providers.
Will implement that tomorrow 👍

[Flomp]

Integrations with encrypted secrets are part of v0.15.0.