Discussion: CLI cache clean commands

[kevinkassimo]

(Migrated from gitter)
We could potentially introduce some basic cache cleaning commands for Deno.
Currently 2 super-duper simple and crude ideas:

1. Clean by domain, something like deno clean https://github.com deletes all cached files under that domain (this one is somewhat easy to implement)
   • This could be slightly more granular: deno clean -d https://unpkg.com/[email protected]/
   • Rationale: assume related files to be hosted under the same domain/common path segment
2. Clean by dependency exclusion. Since when we are building dependency tree each time we run a file, we can use such info to dictate deno to clean every file EXCEPT those that serves as dependency of a file. This also potentially allows third-party cli to create "dep lockers" to prevent certain dependency from being cleaned.
   • e.g. deno clean -p ./mydeps.ts. If mydeps.ts imports from all https://deno.land/std/, running this command would delete every cached remote files except for deno_std ones

[ry]

I'm for this. I think the most common use case would be to delete the dependencies of a script, so maybe that shouldn't need a flag:

deno clean - Deletes cached downloads and compilation artifacts

Given a path or URL to a module, deno clean will traverse the dependency tree of the module
removing all cached dependencies. Example: deno clean https://deno.land/std/http/file_server.ts

I think the dep exclusion sounds good, maybe with the --exclude flag?

[kevinkassimo]

@ry I'm actually slightly worried about the "delete the dependencies of a script" use case. Since we do not really have the concept of a "project" in fixed locations and thus cannot do reference count of certain dep usage for all Deno scripts, if a script happened to import from common modules like deno_std, this would impact many other scripts spreading across the file system when we run deno clean.

[hayd]

clear > clean.

[kitsonk]

I think people would prefer an aggressive clear/clean. With the ability to exclude stuff, I think it would be overkill to automatically exclude dependencies for one script that are used by another script. While there are chances that it might invalidate the cache of something else and then replace it with something that makes the other script not work, just as much of the reverse is true as well and good reason why people shouldn't let their dependencies float.

[kevinkassimo]

@kitsonk I think there is a potentially better use scenario for the exclusion feature: assume that user defines their own DENO_DIR location and carries all the dep files along with the project, dep exclusion could help remove unnecessary files while keeping only the cached files that are actually used. (I think this might be a valid scenario: imagine a user initially tried to use e.g. servest and later switches to oakserver to build a web server, s/he might want to get rid of all the old files that are no longer imported to reduce repo size)

[kitsonk]

yes, I can see "clean everything but what X needs" is a more common use case to reduce size...

One side thought in general about generating a dependency graph, how do we deal with dynamic import()?

[kevinkassimo]

I believe I have raised a similar question for deno fetch in the past... Our resolution at that time was to "just ignore them"... I believe v8 would inform us with dynamic imports requests only during runtime, so unless we uses some tools to generate AST (TS compiler maybe?) we won't be able to know whether and where these dynamic imports occur

[rhideg]

Hi, we've made a simple script that deletes the https cache (from the directory that deno info returns).
https://raw.githubusercontent.com/rhideg/deno-scripts/master/deno-cache-delete.sh

[wobsoriano]

Hi, we've made a simple script that deletes the https cache (from the directory that deno info returns).
https://raw.githubusercontent.com/rhideg/deno-scripts/master/deno-cache-delete.sh

Thanks for this. Works!

[hayd]

Related, it would be good to have some other cache interactively. In deno lambda I do a weird remapping of artifacts because the script path (on a users machine) is not the same as the script path on lambda. It'd be nice if that were a deno cache command... Perhaps it's too niche a use case?

[jsejcksn]

I think it's logical for this functionality to be an additional argument to deno cache, using something like --delete or --invalidate, and it would change the behavior of the command to delete the cached module and its dependencies.

An example:

Caching:

% deno cache --reload http://localhost:8080/local-modules/[email protected]/mod.ts 
Download http://localhost:8080/local-modules/[email protected]/mod.ts
Download http://localhost:8080/local-modules/[email protected]/deps.ts
Download https://deno.land/x/[email protected]/mod.ts
Download https://deno.land/[email protected]/testing/asserts.ts
Download http://localhost:8080/local-modules/[email protected]/mod.ts
Download https://deno.land/[email protected]/fmt/colors.ts
Download https://deno.land/[email protected]/testing/diff.ts
Compile http://localhost:8080/local-modules/[email protected]/mod.ts

Deleting:

% deno cache --delete http://localhost:8080/local-modules/[email protected]/mod.ts 
Delete http://localhost:8080/local-modules/[email protected]/mod.ts
Delete http://localhost:8080/local-modules/[email protected]/deps.ts
Delete https://deno.land/x/[email protected]/mod.ts
Delete https://deno.land/[email protected]/testing/asserts.ts
Delete http://localhost:8080/local-modules/[email protected]/mod.ts
Delete https://deno.land/[email protected]/fmt/colors.ts
Delete https://deno.land/[email protected]/testing/diff.ts

[lucacasonato]

Closing as duplicate of #3437, as it is more detailed.

[johandalabacka]

Hi, we've made a simple script that deletes the https cache (from the directory that deno info returns).
https://raw.githubusercontent.com/rhideg/deno-scripts/master/deno-cache-delete.sh

Thanks I made a variant in deno inspired by your script

#!/usr/bin/env deno run --allow-run --allow-write

const p = Deno.run({
  cmd: ['deno', 'info'],
  stdout: 'piped',
  env: {NO_COLOR: "1"} // No color in output
})
const status = await p.status()
if (! status.success) {
  Deno.exit(status.code)
}
const t = new TextDecoder()
const output = t.decode(await p.output())
p.close()

const m = output.match(/Remote modules cache: "([^"]+)"/)
if (! m) {
  console.log('Remote modules cache not found')
  Deno.exit(1)
}
const folder = m[1]
if (confirm(`Are you sure you want to delete "${folder}"?`)) {
  console.log('deleting...')
  Deno.remove(folder, {recursive: true})
}