-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How does this help existing code? #1
Comments
The problem we're trying to solve is exactly with the legacy codebases that won't realistically change (e.g. they are outside of direct author control, are 3rd party scripts, are too costly to refactor etc. - see example of such dependency). Such code can use Trusted Types implicitly without code changes via default policy. Evalable is targetted to legacy code like that that forms part of the application whose authors wish to harden, such that no new The intention is not to encourage |
I don't understand how this proposal helps with that case: if you have legacy code using the default policy, it is presumably passing a string to |
Oh, I see. I believe this is more clear when viewed with the second proposal from @mikesamuel. The type check (via evalable symbol here) happens in |
Why would |
Fyi, per @annevk's request, I combined the host callout and evalable proposals, and transferred this issue to the merged proposal. |
Correct. The args list and its return value are both internal lists created by other spec abstractions or host callouts, not JS arrays.
I tend to use the term "eval" loosely to mean code from string loaders including Are you objecting specifically to I don't know of any feature that is straightforward to adopt and that might reduce the attack surface for pre-existing uses of eval that would not also aid writing If not, it seems that this standard would prevent all features that might reduce the attack surface for pre-existing uses of eval. So I have to question whether this is an appropriate standard to apply. That said, I also would rather developers not use any of those flavours of "eval" in new code. But if I had to choose between
It took quite a while to come to this preference, because dynamic code loading seems like using dynamite to catch fish. My change in view was based on several realizations:
I think it's important to distinguish between 2 different kinds of legacy:
Both new and legacy applications developed alongside TT may have legacy dependencies. Legacy applications may use trusted types. Just not for all trust decisions. There are a few use cases where I expect legacy dependencies would persist in libraries even in Here are some common use cases: Constant patterns<meta http-equiv="Content-Security-Policy" content="trusted-types default">
<script>
// Application code defines a default policy
Trusted.createPolicy(
'default',
{
createScript(x) {
if (/^return this;?/.test(x)) {
// Make sure there's a semicolon at the end so that
// the result can be safely concatenated with other TrustedScripts.
return 'return this;';
}
throw new Error();
},
});
</script>
<script>
// Legacy library code uses `Function` as a strict mode escape to access the global scope
// appears in lodash, and other libraries that have large installed bases, code size constraints,
// and need to support old interpreters.
(new Function('return this')())
</script> Formulaic metaprogrammingI'm mostly familiar with this in Node.js code. Note that fairly recent versions of depd used Since this is formulaic, it's easy to recognize and handle using a default policy. Code that hasn't been migrated yetOne common use case for large, legacy applications is ad-hoc reporting: letting the user specify arithmetic equations which are used to generate a column in an HTML table that shows data.
Starting the migration process and being able to turn on some protections even if the attack surface is still large is a necessary first step. There's a problem akin to penguins clustering at the water's edge reluctant to enter. There's little incentive for libraries like depd or Mathjs to increase code size to not use eval until they have a significant number of clients that would be able to reduce their attack surface by not using eval. Smaller libs like Mathjs (josdejong was unusually proactive) have no incentive as long as they can say "depd blocks almost all my clients anyway." Large libs like lodash & depd likewise have little incentive as long as they believe there's a long tail of small libs preventing. I see eval TT guards as a way to break that deadlock. Anyway, apologies for the length, and thanks for agreeing to take this offline. |
Thanks for the reply!
All of the above.
I would strongly prefer that these migrators be forced to get rid of
I agree that it's a huge improvement to have a default policy which can restrict which strings are allowed to be passed through to Can you give a concrete example of how the addition of |
I would also prefer this. There's no reason it could not have happened over the last 3 years, so it seems to me there would have been pressure on widely used libraries like lodash, jquery, and others to get rid of We would have seen pressure on webpack and other widely used tools to not produce output that uses eval, but if there is such pressure, I've not seen it. @arturjanc Do you know of any stats on CSP policies migrating from 'unsafe-eval' to no 'unsafe-eval' on the web at large that might shed light on whether this is a viable goal?
This is an excellent question. The HostBeforeCompileValue changes are sufficient for That leaves uses of I can't point you at code, but it actually took us an inordinately long time to migrate a large Google application to turn off It used*
These inputs were large, and because of the performance requirements, trying to validate this using the default policy would not work, so we did something like (but not exactly because our client-side trusted-types relied on static analysis instead of policy objects) const unpackResponse = (() => {
const uncheckedConversionPolicy = new TrustedTypesPolicy(
'...',
// Approved for use in a limited scope. See below.
{ createScript(s) { return s; } });
return ({ status, responseText }) => {
if (status === 200 && ...) {
// backend.example.com is a known producer of TrustedScript responses
// since it uses TrustedTypes for server side discipline.
return (0, eval)(uncheckedConversionPolicy.createScript(responseText));
}
throw new Error(...);
};
})(); You can imagine that this was benchmarked nine ways from Sunday so the actual code probably differs from this in important ways, but these are the kind of legacy-ish requirements that we hope to make it possible to disentangle. Marking a value trusted manually has overhead that is independent of the size of the result, and can be made based on indicators apparent to a scoped policy like:
which are not apparent to the unscoped default policy. * there's some chance it still does |
I don't have current data, other than the observation that ~82% of sites with CSP used Based on a more anecdotal data set, it seems like many popular sites are still actively relying on My intuition matches @mikesamuel's here: it seems difficult to require developers to completely remove |
Thanks. Better not to have to amend GreenSpun's 10th to include JS:
|
Thanks for the example. My understanding then is that
It is not intended to help with any other cases. Is that right? If that's so, I would like to see data indicating that this case in particular is common enough to warrant a change to JS before we went ahead with it. I know Sidebar: with the caveat that I have not fully read the trusted types proposal, why is it not possible to implement an equivalent of <meta http-equiv="Content-Security-Policy" content="trusted-types default">
<script>
// Application code defines a default policy
{
let trusted = new Set;
trusted.has = trusted.has.bind(trusted);
trusted.add = trusted.add.bind(trusted);
trusted.delete = trusted.delete.bind(trusted);
window.trusted = trusted;
Trusted.createPolicy(
'default',
{
createScript(x) {
if (!trusted.has(x)) {
throw new Error();
}
trusted.delete(x);
return x;
},
});
}
</script>
<script>
{
let trusted = window.trusted;
delete window.trusted; // there are other ways of doing this, like import maps; it's just an example.
const unpackResponse = (() => {
return ({ status, responseText }) => {
if (status === 200 && ...) {
// backend.example.com is a known producer of TrustedScript responses
// since it uses TrustedTypes for server side discipline.
trusted.add(responseText);
return (0, eval)(responseText);
}
throw new Error(...);
};
})();
}
</script> |
I don't think this is the case w.r.t. point 1. Why is evalable not useful for 3rd party code? The default policy is an escape hatch that we can use if the code continues to use
One indication I can provide is we did a review of popular JS libraries a while ago - https://ai.google/research/pubs/pub46450. 10 out of 16 used eval() in a way that's essential to them, mostly for implementing expression language support for the templating system.
Default policy is an escape hatch, and we'd rather promote a design that uses regular, named policies, that allow you to decouple trusted value production from consumption. Using types to represent a "trusted" value allows us to write static analysis rules that guide developers and warn them on errors that would break at runtime (like e.g. eval(string) if no default policy exists, or it doesn't convert any value). Additionally, types allow us to group values that might be used in various similar context. For example, |
I was following @mikesamuel's taxonomy above:
Is there reason to expect these dependencies to migrate to TrustedScripts, and not to migrate off of
Interesting. I read the paper, but don't see this specific claim in it; can you point me to where I should be looking?
Are you planning to build this static analysis into Chrome's devtools? If not, is there reason to expect adoption of such a static analysis outside of Google? Would it at least be accurate to say that |
Small correction here. It would be less than 10, I believe we also ticked the box for frameworks that have non-eval based parsers. I see however that this is still common: Vue.js uses eval in full version , so does Ractive, AngularJS in non-CSP mode, Underscore, all the jQuery-family libraries etc. |
No. Evalable is also targeted at library code.
Correct.
It may provide value where it is not practical right now.
Correct.
In strict mode, assuming no stack introspection, and assuming you can enumerate the local symbols you want the dynamically loaded code to access, I agree with this mostly, but there are probably some weird meta-programming cases where For example, webpack has apparently benchmarked various ways to provide source mapping with delayed loading, and concluded that there are efficient ways that involve
That's a good question, but I just don't know. |
@bakkot, What's your level of comfort with this now? |
Here's a concrete use case for direct eval that cannot be ported to // Sometimes we want node's require, not webpack's.
const nativeRequire = eval('require'); This is some node.js code, so not browser relevant, so I understand if you say that Trusted Types should not be concerned with this. The notable features are:
I came across this in the wild (a Twitter thread actually) which I can dig up if anyone is interested. |
In that case, the package should use the "browser" field to specify a separate browser-only entry point (or |
It is practically impossible to migrate existing codebases completely off eval in the web platform at scale, and the data confirms this - e.g. Table 3 from CSP Is Dead, Long Live CSP shows that >81% of CSPs have to use Most of the eval usage does not cause Cross Site Scripting though, as externally-controlled data is not passed to these calls. Annotating these calls with evalable allows to lock down the configuration. A migration path is simple, and much, much easier than migrating off eval - and the authors naturally prefer that. We have given multiple examples of why this is preferable, I'm not sure reiterating that brings the discussion forward. I just want to point out that the feature is driven by author's feedback. We want to make our websites more secure and are willing to spend some time rewriting the code, but our analysis shows we can't move off eval. CCing @devd who mentioned that usecase as well. |
@bakkot had expressed interest in alternative approaches that we considered. I collected some of the history of the major design iterations we went through within Google. It's got my byline since I talk about my experiences in app development, but it touches on attempts by a lot of different people. |
Thanks for putting that together, @mikesamuel, and for talking this over with me offline; it is very helpful in understanding where you're coming from with this proposal. So, to summarize this thread:
Obviously this would only happen on a fairly long timescale. So this relies heavily on a belief that the ecosystem will still be such that Is all of that accurate? As a small quibble with your design history document, you say
but it seems to me that it is untenable only under the assumption that the common practices for loading JS on the web will remain essentially static even on the timescales which would be needed for this proposal to see wide adoption. I am not convinced that this will be the case; I had understood that it was the intent of browsers to obviate the need for bundling by use of HTTP/2 push or web packages or other means. |
@bakkot
Agreed. We are also interested in "will not migrate off in the short term."
In our experience, showing a feasible migration path puts pressure on removing eval.
It's a long road, but I think "decade" may be an overstatement. Some will adopt faster than others. Gmail took less than a year to get significant benefit from server-side trusted types, and about 2 years for the bulk of the migration to complete, even though it took several more years before it was eval free.
An unmaintained library that delegates its security-sensitive operations to a maintained library should be fine. In our experience, showing a feasible migration path for many clients makes it cost-effective to find new maintainers. |
OK. I am satisfied that the tradeoff of "some people will use this who would instead have just avoided Thanks for talking through this with me, and for more fully laying out the target audience of this proposal. I'll go ahead and close this issue. |
I raised this at TC39 today; copying it out here.
I don't think we should be adding features to aid writing
eval
in any new code. I understand that isn't the goal: that the broader goal of this proposal is to allow more fine-grained control overeval
so that people stuck using legacy codebases aren't forced to just useunsafe-eval
. But since legacy codebases are, by definition, not using trusted types or whatever other new feature allowed producingevalable
things, how does this help meet that goal?The text was updated successfully, but these errors were encountered: