Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PID/Hostname File Locking for Cluster Environments. #46

Open
willow-ahrens opened this issue Jan 29, 2025 · 0 comments
Open

PID/Hostname File Locking for Cluster Environments. #46

willow-ahrens opened this issue Jan 29, 2025 · 0 comments
Milestone

Comments

@willow-ahrens
Copy link
Contributor

The PID file locking ecosystem in Python may not be robust to the networked filesystems common to HPC clusters, which may have multiple hosts on multiple operating systems.

Quoting @amilsted:

Julia core uses https://github.com/vtjnash/Pidfile.jl to do hostname/PID-based locking to JuliaLang/julia#42405 and JuliaLang/julia#42405, because this type of locking is JuliaLang/julia#31953.

A successful fix for this issue would identify and integrate a python pidfile implementation which offers the same (tested) guarantees as https://github.com/vtjnash/Pidfile.jl. Further discussion in #45, where FileLock may be used as an interim solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants