snd

Afero Sync & Delete fs

The S&D file system relies on:

• A temporary file system to store files as fast as possible
• A permanent file system for the destination files

It allows to quickly accept new files and synchronize them on a slower file system.

The rationale is that when using some cloud-based file systems like S3, Google Drive or Dropbox on an FTP server, some devices consider the slowness as an error.

Requirements

• Go 1.24 or later

Features

• Asynchronous file synchronization from temporary to destination filesystem
• Automatic garbage collection of old files
• Built-in logging using Go's standard log/slog package
• Configurable cleanup behavior (file age, minimum file count, cleanup period)

Installation

go get github.com/fclairamb/afero-snd

Usage

package main

import (
    "log/slog"
    "os"

    "github.com/fclairamb/afero-snd"
    "github.com/spf13/afero"
)

func main() {
    // Create a new sync & delete filesystem
    fs, err := snd.NewFs(&snd.Config{
        Destination: afero.NewOsFs(),
        Temporary:   afero.NewMemMapFs(),
        Logger:      slog.New(slog.NewTextHandler(os.Stdout, nil)),
    })
    if err != nil {
        panic(err)
    }
    defer fs.Close()

    // Use the filesystem
    file, err := fs.Create("test.txt")
    if err != nil {
        panic(err)
    }
    defer file.Close()

    // Write to file - changes are queued for sync
    file.WriteString("Hello, World!")
}