README
¶
lockfs - Thread-Safe Wrapper for absfs
The lockfs package provides thread-safe wrappers for absfs filesystem interfaces. It uses hierarchical sync.RWMutex locking to enable safe concurrent access from multiple goroutines while preventing races between file operations and filesystem mutations.
Features
- Full absfs interface compliance: Implements
Filer,FileSystem, andSymlinkFileSysteminterfaces - Hierarchical locking: File operations hold a read lock on the parent filesystem, preventing races with filesystem mutations
- RWMutex-based: Read operations allow concurrent access; write operations use exclusive locks
- Wrapped file handles: Files returned from Open/Create/OpenFile are automatically wrapped for thread-safe access
- Zero dependencies beyond absfs itself
Installation
go get github.com/absfs/lockfs
Usage
Wrapping a FileSystem
package main
import (
"github.com/absfs/lockfs"
"github.com/absfs/memfs"
)
func main() {
// Create an underlying filesystem
mfs, _ := memfs.NewFS()
// Wrap it for thread-safe access
fs, _ := lockfs.NewFS(mfs)
// Now safe to use from multiple goroutines
go func() {
fs.Create("/file1.txt")
}()
go func() {
fs.Create("/file2.txt")
}()
}
Wrapping a Filer
filer, _ := lockfs.NewFiler(myFiler)
Wrapping a SymlinkFileSystem
sfs, _ := lockfs.NewSymlinkFS(mySymlinkFS)
Hierarchical Locking
The key feature of lockfs is its hierarchical locking strategy. When you perform a file operation (like Read or Write), the operation acquires:
- Filesystem read lock - Prevents filesystem mutations (Create, Remove, Rename, etc.) from executing concurrently
- File-level lock - Serializes operations on the specific file handle
This design prevents races between scenarios like:
- Reading from an open file while another goroutine removes or truncates it
- Writing to a file while another goroutine renames it
- Multiple file handles to the same path operating concurrently
Goroutine A: f.Read() Goroutine B: fs.Remove("/file")
| |
+-- fs.RLock() +-- fs.Lock() [BLOCKED]
+-- file.Lock() |
+-- ... read ... |
+-- file.Unlock() |
+-- fs.RUnlock() |
+-- [NOW PROCEEDS]
Thread Safety Semantics
Filesystem Operations
All filesystem operations are protected by a RWMutex:
| Operation | Lock Type | Notes |
|---|---|---|
| Stat, Lstat | RLock | Concurrent reads allowed |
| Open | RLock | Concurrent opens allowed |
| Getwd | RLock | Concurrent reads allowed |
| Readlink | RLock | Concurrent reads allowed |
| Create, OpenFile | Lock | Exclusive access |
| Mkdir, MkdirAll | Lock | Exclusive access |
| Remove, RemoveAll | Lock | Exclusive access |
| Rename | Lock | Exclusive access |
| Chmod, Chown, Chtimes | Lock | Exclusive access |
| Chdir | Lock | Exclusive access |
| Truncate | Lock | Exclusive access |
| Symlink | Lock | Exclusive access |
File Operations
Each File has its own RWMutex plus holds the parent filesystem's read lock during operations:
| Operation | FS Lock | File Lock | Notes |
|---|---|---|---|
| Name | None | None | Immutable after creation |
| Stat | RLock | RLock | Concurrent reads allowed |
| ReadAt | RLock | RLock | Position-independent read |
| Read | RLock | Lock | Modifies file position |
| Write, WriteAt, WriteString | RLock | Lock | Exclusive file access |
| Seek | RLock | Lock | Modifies file position |
| Truncate | RLock | Lock | Exclusive file access |
| Readdir, Readdirnames | RLock | Lock | Modifies directory cursor |
| Sync | RLock | Lock | Exclusive file access |
| Close | None | Lock | No filesystem lock needed |
Lock Ordering
To prevent deadlocks, locks are always acquired in this order:
- Filesystem lock (if needed)
- File lock (if needed)
And released in reverse order via defer.
Limitations
Underlying Filesystem Thread Safety
The lockfs wrapper serializes access at the lockfs level, but cannot fix thread-safety issues within the underlying filesystem's implementation. If the underlying filesystem has internal races, those may still occur.
Best practice: Use lockfs to add thread safety to single-threaded filesystem implementations.
Performance Considerations
- File operations hold the filesystem read lock, which blocks filesystem mutations
- Multiple concurrent reads are efficient (RLock allows multiple readers)
- Write-heavy workloads may see contention on the filesystem lock
- Consider using separate filesystem instances for isolated workloads
absfs
Check out the absfs repo for more information about the abstract FileSystem interface and features like FileSystem composition.
License
This project is governed by the MIT License. See LICENSE
Documentation
¶
Index ¶
- type File
- func (f *File) Close() error
- func (f *File) Name() string
- func (f *File) Read(p []byte) (int, error)
- func (f *File) ReadAt(b []byte, off int64) (n int, err error)
- func (f *File) ReadDir(n int) ([]fs.DirEntry, error)
- func (f *File) Readdir(n int) ([]os.FileInfo, error)
- func (f *File) Readdirnames(n int) ([]string, error)
- func (f *File) Seek(offset int64, whence int) (ret int64, err error)
- func (f *File) Stat() (os.FileInfo, error)
- func (f *File) Sync() error
- func (f *File) Truncate(size int64) error
- func (f *File) Write(p []byte) (int, error)
- func (f *File) WriteAt(b []byte, off int64) (n int, err error)
- func (f *File) WriteString(s string) (n int, err error)
- type FileSystem
- func (f *FileSystem) Chdir(dir string) error
- func (f *FileSystem) Chmod(name string, mode os.FileMode) error
- func (f *FileSystem) Chown(name string, uid, gid int) error
- func (f *FileSystem) Chtimes(name string, atime time.Time, mtime time.Time) error
- func (f *FileSystem) Create(name string) (absfs.File, error)
- func (f *FileSystem) Getwd() (dir string, err error)
- func (f *FileSystem) Mkdir(name string, perm os.FileMode) error
- func (f *FileSystem) MkdirAll(name string, perm os.FileMode) error
- func (f *FileSystem) Open(name string) (absfs.File, error)
- func (f *FileSystem) OpenFile(name string, flag int, perm os.FileMode) (absfs.File, error)
- func (f *FileSystem) ReadDir(name string) ([]fs.DirEntry, error)
- func (f *FileSystem) ReadFile(name string) ([]byte, error)
- func (f *FileSystem) Remove(name string) error
- func (f *FileSystem) RemoveAll(path string) (err error)
- func (f *FileSystem) Rename(oldpath, newpath string) error
- func (f *FileSystem) Stat(name string) (os.FileInfo, error)
- func (f *FileSystem) Sub(dir string) (fs.FS, error)
- func (f *FileSystem) TempDir() string
- func (f *FileSystem) Truncate(name string, size int64) error
- type Filer
- func (f *Filer) Chmod(name string, mode os.FileMode) error
- func (f *Filer) Chown(name string, uid, gid int) error
- func (f *Filer) Chtimes(name string, atime time.Time, mtime time.Time) error
- func (f *Filer) Mkdir(name string, perm os.FileMode) error
- func (f *Filer) OpenFile(name string, flag int, perm os.FileMode) (absfs.File, error)
- func (f *Filer) ReadDir(name string) ([]fs.DirEntry, error)
- func (f *Filer) ReadFile(name string) ([]byte, error)
- func (f *Filer) Remove(name string) error
- func (f *Filer) Rename(oldpath, newpath string) error
- func (f *Filer) Stat(name string) (os.FileInfo, error)
- func (f *Filer) Sub(dir string) (fs.FS, error)
- type SymlinkFileSystem
- func (f *SymlinkFileSystem) Chdir(dir string) error
- func (f *SymlinkFileSystem) Chmod(name string, mode os.FileMode) error
- func (f *SymlinkFileSystem) Chown(name string, uid, gid int) error
- func (f *SymlinkFileSystem) Chtimes(name string, atime time.Time, mtime time.Time) error
- func (f *SymlinkFileSystem) Create(name string) (absfs.File, error)
- func (f *SymlinkFileSystem) Getwd() (dir string, err error)
- func (f *SymlinkFileSystem) Lchown(name string, uid, gid int) error
- func (f *SymlinkFileSystem) Lstat(name string) (os.FileInfo, error)
- func (f *SymlinkFileSystem) Mkdir(name string, perm os.FileMode) error
- func (f *SymlinkFileSystem) MkdirAll(name string, perm os.FileMode) error
- func (f *SymlinkFileSystem) Open(name string) (absfs.File, error)
- func (f *SymlinkFileSystem) OpenFile(name string, flag int, perm os.FileMode) (absfs.File, error)
- func (f *SymlinkFileSystem) ReadDir(name string) ([]fs.DirEntry, error)
- func (f *SymlinkFileSystem) ReadFile(name string) ([]byte, error)
- func (f *SymlinkFileSystem) Readlink(name string) (string, error)
- func (f *SymlinkFileSystem) Remove(name string) error
- func (f *SymlinkFileSystem) RemoveAll(path string) (err error)
- func (f *SymlinkFileSystem) Rename(oldpath, newpath string) error
- func (f *SymlinkFileSystem) Stat(name string) (os.FileInfo, error)
- func (f *SymlinkFileSystem) Sub(dir string) (fs.FS, error)
- func (f *SymlinkFileSystem) Symlink(oldname, newname string) error
- func (f *SymlinkFileSystem) TempDir() string
- func (f *SymlinkFileSystem) Truncate(name string, size int64) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
File wraps an absfs.File with hierarchical locking for thread-safe access.
File operations acquire both: 1. A read lock on the parent filesystem (prevents filesystem mutations during I/O) 2. An appropriate lock on the file itself (serializes operations on this handle)
This ensures that operations like fs.Create("/file") cannot race with f.Read() on an existing handle to the same file.
func (*File) Close ¶
Close closes the file. Uses exclusive file lock only (closing doesn't need filesystem lock).
func (*File) Name ¶
Name returns the name of the file. This is safe without locking since the name is immutable after file creation.
func (*File) Read ¶
Read reads up to len(p) bytes into p. Uses exclusive file lock (modifies position) with filesystem read lock.
func (*File) ReadAt ¶
ReadAt reads len(b) bytes from the file starting at byte offset off. Uses read locks on both filesystem and file (position-independent).
func (*File) ReadDir ¶
ReadDir reads the contents of the directory associated with file and returns a slice of up to n DirEntry values, as would be returned by ReadDir. If n <= 0, ReadDir returns all the DirEntry values from the directory in a single slice. Uses exclusive file lock (modifies directory cursor) with filesystem read lock.
func (*File) Readdir ¶
Readdir reads the contents of the directory. Uses exclusive file lock (modifies directory cursor) with filesystem read lock.
func (*File) Readdirnames ¶
Readdirnames reads the names of directory entries. Uses exclusive file lock (modifies directory cursor) with filesystem read lock.
func (*File) Seek ¶
Seek sets the offset for the next Read or Write. Uses exclusive file lock (modifies position) with filesystem read lock.
func (*File) Stat ¶
Stat returns the FileInfo for the file. Uses read locks on both filesystem and file.
func (*File) Sync ¶
Sync commits the file's contents to stable storage. Uses filesystem read lock and exclusive file lock.
func (*File) Truncate ¶
Truncate changes the size of the file. Uses filesystem read lock and exclusive file lock.
func (*File) Write ¶
Write writes len(p) bytes to the file. Uses exclusive locks on both filesystem and file.
type FileSystem ¶
type FileSystem struct {
// contains filtered or unexported fields
}
FileSystem wraps an absfs.FileSystem with a RWMutex for thread-safe access. Read operations use RLock for concurrent access, write operations use Lock. Files returned from Open/Create/OpenFile use hierarchical locking to coordinate with the FileSystem, preventing races between file operations and filesystem mutations.
func NewFS ¶
func NewFS(fs absfs.FileSystem) (*FileSystem, error)
NewFS creates a new thread-safe FileSystem wrapper.
func (*FileSystem) Chdir ¶
func (f *FileSystem) Chdir(dir string) error
Chdir changes the current working directory.
func (*FileSystem) Chmod ¶
func (f *FileSystem) Chmod(name string, mode os.FileMode) error
Chmod changes the mode of the named file to mode.
func (*FileSystem) Chown ¶
func (f *FileSystem) Chown(name string, uid, gid int) error
Chown changes the owner and group ids of the named file.
func (*FileSystem) Create ¶
func (f *FileSystem) Create(name string) (absfs.File, error)
Create creates the named file, truncating it if it already exists. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*FileSystem) Getwd ¶
func (f *FileSystem) Getwd() (dir string, err error)
Getwd returns the current working directory.
func (*FileSystem) Mkdir ¶
func (f *FileSystem) Mkdir(name string, perm os.FileMode) error
Mkdir creates a directory in the filesystem, return an error if any happens.
func (*FileSystem) MkdirAll ¶
func (f *FileSystem) MkdirAll(name string, perm os.FileMode) error
MkdirAll creates a directory named path, along with any necessary parents.
func (*FileSystem) Open ¶
func (f *FileSystem) Open(name string) (absfs.File, error)
Open opens the named file for reading. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*FileSystem) OpenFile ¶
OpenFile opens a file using the given flags and the given mode. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*FileSystem) ReadDir ¶
func (f *FileSystem) ReadDir(name string) ([]fs.DirEntry, error)
ReadDir reads the named directory and returns all its directory entries.
func (*FileSystem) ReadFile ¶
func (f *FileSystem) ReadFile(name string) ([]byte, error)
ReadFile reads the named file and returns its contents.
func (*FileSystem) Remove ¶
func (f *FileSystem) Remove(name string) error
Remove removes a file identified by name, returning an error, if any happens.
func (*FileSystem) RemoveAll ¶
func (f *FileSystem) RemoveAll(path string) (err error)
RemoveAll removes path and any children it contains.
func (*FileSystem) Rename ¶
func (f *FileSystem) Rename(oldpath, newpath string) error
Rename renames (moves) oldpath to newpath.
func (*FileSystem) Stat ¶
func (f *FileSystem) Stat(name string) (os.FileInfo, error)
Stat returns the FileInfo structure describing file. If there is an error, it will be of type *PathError.
func (*FileSystem) Sub ¶
func (f *FileSystem) Sub(dir string) (fs.FS, error)
Sub returns a filesystem corresponding to the subtree rooted at dir.
func (*FileSystem) TempDir ¶
func (f *FileSystem) TempDir() string
TempDir returns the default directory for temporary files.
type Filer ¶
type Filer struct {
// contains filtered or unexported fields
}
Filer wraps an absfs.Filer with a RWMutex for thread-safe access. Read operations (Stat) use RLock for concurrent access, write operations use Lock. Files returned from OpenFile use hierarchical locking to coordinate with the Filer.
func (*Filer) OpenFile ¶
OpenFile opens a file using the given flags and the given mode. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*Filer) Remove ¶
Remove removes a file identified by name, returning an error, if any happens.
type SymlinkFileSystem ¶
type SymlinkFileSystem struct {
// contains filtered or unexported fields
}
SymlinkFileSystem wraps an absfs.SymlinkFileSystem with a RWMutex for thread-safe access. Read operations use RLock for concurrent access, write operations use Lock. Files returned from Open/Create/OpenFile use hierarchical locking to coordinate with the SymlinkFileSystem, preventing races between file operations and filesystem mutations.
func NewSymlinkFS ¶
func NewSymlinkFS(fs absfs.SymlinkFileSystem) (*SymlinkFileSystem, error)
NewSymlinkFS creates a new thread-safe SymlinkFileSystem wrapper.
func (*SymlinkFileSystem) Chdir ¶
func (f *SymlinkFileSystem) Chdir(dir string) error
Chdir changes the current working directory.
func (*SymlinkFileSystem) Chmod ¶
func (f *SymlinkFileSystem) Chmod(name string, mode os.FileMode) error
Chmod changes the mode of the named file to mode.
func (*SymlinkFileSystem) Chown ¶
func (f *SymlinkFileSystem) Chown(name string, uid, gid int) error
Chown changes the owner and group ids of the named file.
func (*SymlinkFileSystem) Chtimes ¶
Chtimes changes the access and modification times of the named file.
func (*SymlinkFileSystem) Create ¶
func (f *SymlinkFileSystem) Create(name string) (absfs.File, error)
Create creates the named file, truncating it if it already exists. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*SymlinkFileSystem) Getwd ¶
func (f *SymlinkFileSystem) Getwd() (dir string, err error)
Getwd returns the current working directory.
func (*SymlinkFileSystem) Lchown ¶
func (f *SymlinkFileSystem) Lchown(name string, uid, gid int) error
Lchown changes the numeric uid and gid of the named file. If the file is a symbolic link, it changes the uid and gid of the link itself. If there is an error, it will be of type *PathError.
On Windows, it always returns the syscall.EWINDOWS error, wrapped in *PathError.
func (*SymlinkFileSystem) Lstat ¶
func (f *SymlinkFileSystem) Lstat(name string) (os.FileInfo, error)
Lstat returns a FileInfo describing the named file. If the file is a symbolic link, the returned FileInfo describes the symbolic link. Lstat makes no attempt to follow the link. If there is an error, it will be of type *PathError.
func (*SymlinkFileSystem) Mkdir ¶
func (f *SymlinkFileSystem) Mkdir(name string, perm os.FileMode) error
Mkdir creates a directory in the filesystem, return an error if any happens.
func (*SymlinkFileSystem) MkdirAll ¶
func (f *SymlinkFileSystem) MkdirAll(name string, perm os.FileMode) error
MkdirAll creates a directory named path, along with any necessary parents.
func (*SymlinkFileSystem) Open ¶
func (f *SymlinkFileSystem) Open(name string) (absfs.File, error)
Open opens the named file for reading. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*SymlinkFileSystem) OpenFile ¶
OpenFile opens a file using the given flags and the given mode. The returned File is wrapped for thread-safe access with hierarchical locking.
func (*SymlinkFileSystem) ReadDir ¶
func (f *SymlinkFileSystem) ReadDir(name string) ([]fs.DirEntry, error)
ReadDir reads the named directory and returns all its directory entries.
func (*SymlinkFileSystem) ReadFile ¶
func (f *SymlinkFileSystem) ReadFile(name string) ([]byte, error)
ReadFile reads the named file and returns its contents.
func (*SymlinkFileSystem) Readlink ¶
func (f *SymlinkFileSystem) Readlink(name string) (string, error)
Readlink returns the destination of the named symbolic link. If there is an error, it will be of type *PathError.
func (*SymlinkFileSystem) Remove ¶
func (f *SymlinkFileSystem) Remove(name string) error
Remove removes a file identified by name, returning an error, if any happens.
func (*SymlinkFileSystem) RemoveAll ¶
func (f *SymlinkFileSystem) RemoveAll(path string) (err error)
RemoveAll removes path and any children it contains.
func (*SymlinkFileSystem) Rename ¶
func (f *SymlinkFileSystem) Rename(oldpath, newpath string) error
Rename renames (moves) oldpath to newpath.
func (*SymlinkFileSystem) Stat ¶
func (f *SymlinkFileSystem) Stat(name string) (os.FileInfo, error)
Stat returns the FileInfo structure describing file. If there is an error, it will be of type *PathError.
func (*SymlinkFileSystem) Sub ¶
func (f *SymlinkFileSystem) Sub(dir string) (fs.FS, error)
Sub returns a filesystem corresponding to the subtree rooted at dir.
func (*SymlinkFileSystem) Symlink ¶
func (f *SymlinkFileSystem) Symlink(oldname, newname string) error
Symlink creates newname as a symbolic link to oldname. If there is an error, it will be of type *LinkError.
func (*SymlinkFileSystem) TempDir ¶
func (f *SymlinkFileSystem) TempDir() string
TempDir returns the default directory for temporary files.
Source Files