Cache configuration documentation

CACHE_CONFIGURATION.md

@@ -0,0 +1,279 @@
+Wasmtime Cache Configuration
+============================
+
+The cache configuration file uses the [toml] format.
+You can create a configuration file at the default location with:
+```
+$ wasmtime --create-cache-config
+```
+It will print the location regardless of the success.
+Please refer to the  `--help` message for using a custom location.
+
+All settings, except `enabled`, are **optional**.
+If the setting is not specified, the **default** value is used.
+***Thus, if you don't know what values to use, don't specify them.***
+The default values might be tuned in the future.
+
+Wasmtime assumes all the options are in the `cache` section.
+
+Example config:
+```toml
+[cache]
+enabled = true
+directory = "/nfs-share/wasmtime-cache/"
+cleanup-interval = "30m"
+files-total-size-soft-limit = "1Gi"
+```
+
+Please refer to the [cache system] section to learn how it works.
+
+If you think some default value should be tuned, some new settings
+should be introduced or some behavior should be changed, you are
+welcome to discuss it and contribute to [the Wasmtime repository].
+
+[the Wasmtime repository]: https://github.com/CraneStation/wasmtime
+
+Setting `enabled`
+-----------------
+- **type**: boolean
+- **format**: `true | false`
+- **default**: `true`
+
+Specifies whether the cache system is used or not.
+
+This field is *mandatory*.
+The default value is used when configuration file is not specified
+and none exists at the default location.
+
+[`enabled`]: #setting-enabled
+
+Setting `directory`
+-----------------
+- **type**: string (path)
+- **default**: look up `cache_dir` in [directories] crate
+
+Specifies where the cache directory is. Must be an absolute path.
+
+[`directory`]: #setting-directory
+
+Setting `worker-event-queue-size`
+-----------------
+- **type**: string (SI prefix)
+- **format**: `"{integer}(K | M | G | T | P)?"`
+- **default**: `"16"`
+
+Size of [cache worker] event queue.
+If the queue is full, incoming cache usage events will be dropped.
+
+[`worker-event-queue-size`]: #setting-worker-event-queue-size
+
+Setting `baseline-compression-level`
+------------------
+- **type**: integer
+- **default**: `3`, the default zstd compression level
+
+Compression level used when a new cache file is being written by the [cache system].
+Wasmtime uses [zstd] compression.
+
+[`baseline-compression-level`]: #setting-baseline-compression-level
+
+Setting `optimized-compression-level`
+------------------
+- **type**: integer
+- **default**: `20`
+
+Compression level used when the [cache worker] decides to recompress a cache file.
+Wasmtime uses [zstd] compression.
+
+[`optimized-compression-level`]: #setting-optimized-compression-level
+
+Setting `optimized-compression-usage-counter-threshold`
+------------------
+- **type**: string (SI prefix)
+- **format**: `"{integer}(K | M | G | T | P)?"`
+- **default**: `"256"`
+
+One of the conditions for the [cache worker] to recompress a cache file
+is to have usage count of the file exceeding this threshold.
+
+[`optimized-compression-usage-counter-threshold`]: #setting-optimized-compression-usage-counter-threshold
+
+Setting `cleanup-interval`
+------------------
+- **type**: string (duration)
+- **format**: `"{integer}(s | m | h | d)"`
+- **default**: `"1h"`
+
+When the [cache worker] is notified about a cache file being updated by the [cache system]
+and this interval has already passed since last cleaning up,
+the worker will attempt a new cleanup.
+
+Please also refer to [`allowed-clock-drift-for-files-from-future`].
+
+[`cleanup-interval`]: #setting-cleanup-interval
+
+Setting `optimizing-compression-task-timeout`
+------------------
+- **type**: string (duration)
+- **format**: `"{integer}(s | m | h | d)"`
+- **default**: `"30m"`
+
+When the [cache worker] decides to recompress a cache file, it makes sure that
+no other worker has started the task for this file within the last
+[`optimizing-compression-task-timeout`] interval.
+If some worker has started working on it, other workers are skipping this task.
+
+Please also refer to the [`allowed-clock-drift-for-files-from-future`] section.
+
+[`optimizing-compression-task-timeout`]: #setting-optimizing-compression-task-timeout
+
+Setting `allowed-clock-drift-for-files-from-future`
+------------------
+- **type**: string (duration)
+- **format**: `"{integer}(s | m | h | d)"`
+- **default**: `"1d"`
+
+### Locks
+When the [cache worker] attempts acquiring a lock for some task,
+it checks if some other worker has already acquired such a lock.
+To be fault tolerant and eventually execute every task,
+the locks expire after some interval.
+However, because of clock drifts and different timezones,
+it would happen that some lock was created in the future.
+This setting defines a tolerance limit for these locks.
+If the time has been changed in the system (i.e. two years backwards),
+the [cache system] should still work properly.
+Thus, these locks will be treated as expired
+(assuming the tolerance is not too big).
+
+### Cache files
+Similarly to the locks, the cache files or their metadata might
+have modification time in distant future.
+The cache system tries to keep these files as long as possible.
+If the limits are not reached, the cache files will not be deleted.
+Otherwise, they will be treated as the oldest files, so they might survive.
+If the user actually uses the cache file, the modification time will be updated.
+
+[`allowed-clock-drift-for-files-from-future`]: #setting-allowed-clock-drift-for-files-from-future
+
+Setting `file-count-soft-limit`
+------------------
+- **type**: string (SI prefix)
+- **format**: `"{integer}(K | M | G | T | P)?"`
+- **default**: `"65536"`
+
+Soft limit for the file count in the cache directory.
+
+This doesn't include files with metadata.
+To learn more, please refer to the [cache system] section.
+
+[`file-count-soft-limit`]: #setting-file-count-soft-limit
+
+Setting `files-total-size-soft-limit`
+------------------
+- **type**: string (disk space)
+- **format**: `"{integer}(K | Ki | M | Mi | G | Gi | T | Ti | P | Pi)?"`
+- **default**: `"512Mi"`
+
+Soft limit for the total size* of files in the cache directory.
+
+This doesn't include files with metadata.
+To learn more, please refer to the [cache system] section.
+
+*this is the file size, not the space physically occupied on the disk.
+
+[`files-total-size-soft-limit`]: #setting-files-total-size-soft-limit
+
+Setting `file-count-limit-percent-if-deleting`
+------------------
+- **type**: string (percent)
+- **format**: `"{integer}%"`
+- **default**: `"70%"`
+
+If [`file-count-soft-limit`] is exceeded and the [cache worker] performs the cleanup task,
+then the worker will delete some cache files, so after the task,
+the file count should not exceed
+[`file-count-soft-limit`] * [`file-count-limit-percent-if-deleting`].
+
+This doesn't include files with metadata.
+To learn more, please refer to the [cache system] section.
+
+[`file-count-limit-percent-if-deleting`]: #setting-file-count-limit-percent-if-deleting
+
+Setting `files-total-size-limit-percent-if-deleting`
+------------------
+- **type**: string (percent)
+- **format**: `"{integer}%"`
+- **default**: `"70%"`
+
+If [`files-total-size-soft-limit`] is exceeded and [cache worker] performs the cleanup task,
+then the worker will delete some cache files, so after the task,
+the files total size should not exceed
+[`files-total-size-soft-limit`] * [`files-total-size-limit-percent-if-deleting`].
+
+This doesn't include files with metadata.
+To learn more, please refer to the [cache system] section.
+
+[`files-total-size-limit-percent-if-deleting`]: #setting-files-total-size-limit-percent-if-deleting
+
+[toml]: https://github.com/toml-lang/toml
+[directories]: https://crates.io/crates/directories
+[cache system]: #how-does-the-cache-work
+[cache worker]: #how-does-the-cache-work
+[zstd]: https://facebook.github.io/zstd/
+[Least Recently Used (LRU)]: https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU)
+
+How does the cache work?
+========================
+
+**This is an implementation detail and might change in the future.**
+Information provided here is meant to help understanding the big picture
+and configuring the cache.
+
+There are two main components - the *cache system* and the *cache worker*.
+
+Cache system
+------------
+
+Handles GET and UPDATE cache requests.
+- **GET request** - simply loads the cache from disk if it is there.
+- **UPDATE request** - compresses received data with [zstd] and [`baseline-compression-level`], then writes the data to the disk.
+
+In case of successful handling of a request, it notifies the *cache worker* about this
+event using the queue.
+The queue has a limited size of [`worker-event-queue-size`]. If it is full, it will drop
+new events until the *cache worker* pops some event from the queue.
+
+Cache worker
+------------
+
+The cache worker runs in a single thread with lower priority and pops events from the queue
+in a loop handling them one by one.
+
+### On GET request
+1. Read the statistics file for the cache file,
+   increase the usage counter and write it back to the disk.
+2. Attempt recompressing the cache file if all of the following conditions are met:
+   - usage counter exceeds [`optimized-compression-usage-counter-threshold`],
+   - the file is compressed with compression level lower than [`optimized-compression-level`],
+   - no other worker has started working on this particular task within the last
+     [`optimizing-compression-task-timeout`] interval.
+
+   When recompressing, [`optimized-compression-level`] is used as a compression level.
+
+### On UPDATE request
+1. Write a fresh statistics file for the cache file.
+2. Clean up the cache if no worker has attempted to do this within the last [`cleanup-interval`].
+   During this task:
+   - all unrecognized files and expired task locks in cache directory will be deleted
+   - if [`file-count-soft-limit`] or [`files-total-size-soft-limit`] is exceeded,
+     then recognized files will be deleted according to
+     [`file-count-limit-percent-if-deleting`] and [`files-total-size-limit-percent-if-deleting`].
+     Wasmtime uses [Least Recently Used (LRU)] cache replacement policy and requires that
+     the filesystem maintains proper mtime (modification time) of the files.
+     Files with future mtimes are treated specially - more details
+     in [`allowed-clock-drift-for-files-from-future`].
+
+### Metadata files
+- every cached WebAssembly module has its own statistics file
+- every lock is a file

ci/azure-build-release.yml

@@ -71,7 +71,7 @@ steps:
     if [ "$AGENT_OS" = "Windows_NT" ]; then
       ext=.exe
     fi
-    cp LICENSE README.md target/release/{wasmtime,wasm2obj}$ext $BUILD_BINARIESDIRECTORY/$BASENAME
+    cp LICENSE README.md CACHE_CONFIGURATION.md target/release/{wasmtime,wasm2obj}$ext $BUILD_BINARIESDIRECTORY/$BASENAME
   displayName: Copy binaries
 
 - bash: |

installer/msi/wasmtime.wxs

@@ -58,6 +58,9 @@
             <Component Id="README" Guid="*">
                 <File Id="README.md" Source="README.md" KeyPath="yes" Checksum="yes"/>
             </Component>
+            <Component Id="CACHE_CONFIGURATION" Guid="*">
+                <File Id="CACHE_CONFIGURATION.md" Source="CACHE_CONFIGURATION.md" KeyPath="yes" Checksum="yes"/>
+            </Component>
         </DirectoryRef>
 
         <DirectoryRef Id="BINDIR">
@@ -74,6 +77,7 @@
             <ComponentRef Id="wasm2obj.exe" />
             <ComponentRef Id="LICENSE" />
             <ComponentRef Id="README" />
+            <ComponentRef Id="CACHE_CONFIGURATION" />
             <ComponentRef Id="InstallDir" />
         </Feature>
         <Feature Id="AddToPath"

src/bin/wasm2obj.rs

@@ -50,7 +50,7 @@ use std::str;
 use std::str::FromStr;
 use target_lexicon::Triple;
 use wasmtime_debug::{emit_debugsections, read_debuginfo};
-use wasmtime_environ::cache_init;
+use wasmtime_environ::{cache_create_new_config, cache_init};
 use wasmtime_environ::{
     Compiler, Cranelift, ModuleEnvironment, ModuleVmctxInfo, Tunables, VMOffsets,
 };
@@ -63,7 +63,8 @@ The translation is dependent on the environment chosen.
 The default is a dummy environment that produces placeholder values.
 
 Usage:
-    wasm2obj [--target TARGET] [-Odg] [--cache | --cache-config=<cache_config_file>] [--create-cache-config] [--enable-simd] <file> -o <output>
+    wasm2obj [--target TARGET] [-Odg] [--cache | --cache-config=<cache_config_file>] [--enable-simd] <file> -o <output>
+    wasm2obj --create-cache-config [--cache-config=<cache_config_file>]
     wasm2obj --help | --version
 
 Options:
@@ -73,10 +74,12 @@ Options:
     -g                  generate debug information
     -c, --cache         enable caching system, use default configuration
     --cache-config=<cache_config_file>
-                        enable caching system, use specified cache configuration
+                        enable caching system, use specified cache configuration;
+                        can be used with --create-cache-config to specify custom file
     --create-cache-config
-                        used with --cache or --cache-config, creates default configuration and writes it to the disk,
-                        will fail if specified file already exists (or default file if used with --cache)
+                        creates default configuration and writes it to the disk,
+                        use with --cache-config to specify custom config file
+                        instead of default one
     --enable-simd       enable proposed SIMD instructions
     -O, --optimize      runs optimization passes on the translated functions
     --version           print the Cranelift version
@@ -91,7 +94,7 @@ struct Args {
     flag_g: bool,
     flag_debug: bool,
     flag_cache: bool, // TODO change to disable cache after implementing cache eviction
-    flag_cache_config_file: Option<String>,
+    flag_cache_config: Option<String>,
     flag_create_cache_config: bool,
     flag_enable_simd: bool,
     flag_optimize: bool,
@@ -123,10 +126,25 @@ fn main() {
         Some(prefix)
     };
 
+    if args.flag_create_cache_config {
+        match cache_create_new_config(args.flag_cache_config) {
+            Ok(path) => {
+                println!(
+                    "Successfully created new configuation file at {}",
+                    path.display()
+                );
+                return;
+            }
+            Err(err) => {
+                eprintln!("Error: {}", err);
+                process::exit(1);
+            }
+        }
+    }
+
     let errors = cache_init(
-        args.flag_cache || args.flag_cache_config_file.is_some(),
-        args.flag_cache_config_file.as_ref(),
-        args.flag_create_cache_config,
+        args.flag_cache || args.flag_cache_config.is_some(),
+        args.flag_cache_config.as_ref(),
         log_config,
     );