Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If AST #5160

Merged
merged 5 commits into from
Feb 15, 2019
Merged

If AST #5160

merged 5 commits into from
Feb 15, 2019

Conversation

helixbass
Copy link
Collaborator

@GeoffreyBooth PR for If AST

@@ -817,8 +817,8 @@ grammar =
If: [
o 'IfBlock'
o 'IfBlock ELSE Block', -> $1.addElse $3
o 'Statement POST_IF Expression', -> new If $3, LOC(1)(Block.wrap [$1]), type: $2, statement: true
o 'Expression POST_IF Expression', -> new If $3, LOC(1)(Block.wrap [$1]), type: $2, statement: true
o 'Statement POST_IF Expression', -> new If $3, LOC(1)(Block.wrap [$1]), type: $2, postfix: true
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The statement option to the If constructor was ignored and corresponded to it being a postfix if (which we now need to track), so renamed it to postfix

@@ -270,7 +270,9 @@ exports.Base = class Base
# as JSON. This is what the `ast` option in the Node API returns.
# We try to follow the [Babel AST spec](https://github.com/babel/babel/blob/master/packages/babel-parser/ast/spec.md)
# as closely as possible, for improved interoperability with other tools.
ast: (o) ->
ast: (o, level) ->
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly to how we just added support for o.scope, we now need to support passing level on o

So allow passing it as a second argument to ast() (like the signature of compileToFragments())

@@ -1121,7 +1124,7 @@ exports.Return = class Return extends Base
astType: -> 'ReturnStatement'

astProperties: (o) ->
argument: @expression?.ast(o) ? null
argument: @expression?.ast(o, LEVEL_PAREN) ? null
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So went back through existing calls to .ast() for child node AST generation and populated the appropriate level arguments

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someday I'd like to understand how you know which level to use. That would be good to add in a big comment somewhere if it's not written out already (yeah, I'm too lazy to look 😀 )

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The levels used for child nodes in AST generation here should basically match the corresponding levels used for those children when compiling to JS

But as far as how those work, I haven't seen a thorough explanation - there's a little general comment and examples where the LEVEL_*s are defined towards the bottom of nodes.coffee

Copy link
Collaborator

@vendethiel vendethiel Feb 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Try compiling and see if it requires more parentheses or not" in general is how it's used. That's why we've had wrong parsing precedence with little to no issue for a long while:
#2199 (comment)

super()
@condition = if options.type is 'unless' then condition.invert() else condition
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to defer inverting @condition so that the original condition is still intact at AST generation time

@processedConditionCache ?= if @type is 'unless' then @condition.invert() else @condition

isStatementAst: (o) ->
o.level is LEVEL_TOP
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what needed o.level to be set - in order to more closely mimic Babel AST, If ASTs should be of type IfStatement (ie a JS if statement) if we're at LEVEL_TOP and a ConditionalExpression (ie a JS ternary) otherwise

else
@elseBody?.ast(o, LEVEL_TOP) ? null
postfix: !!@postfix
inverted: @type is 'unless'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

postfix and inverted are extensions to the Babel AST

Copy link
Collaborator

@GeoffreyBooth GeoffreyBooth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Just some nits.

src/nodes.coffee Outdated
@@ -270,7 +270,9 @@ exports.Base = class Base
# as JSON. This is what the `ast` option in the Node API returns.
# We try to follow the [Babel AST spec](https://github.com/babel/babel/blob/master/packages/babel-parser/ast/spec.md)
# as closely as possible, for improved interoperability with other tools.
ast: (o) ->
ast: (o, level) ->
o = extend {}, o
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been trying to phase out these helpers when built-ins can do, like Object.assign in this case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, updated to use Object.assign()

@@ -1121,7 +1124,7 @@ exports.Return = class Return extends Base
astType: -> 'ReturnStatement'

astProperties: (o) ->
argument: @expression?.ast(o) ? null
argument: @expression?.ast(o, LEVEL_PAREN) ? null
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someday I'd like to understand how you know which level to use. That would be good to add in a big comment somewhere if it's not written out already (yeah, I'm too lazy to look 😀 )

@GeoffreyBooth
Copy link
Collaborator

You've knocked out some big ones lately, and quickly. How much more do you think there is?

Copy link
Collaborator Author

@helixbass helixbass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GeoffreyBooth updated per your comments

You've knocked out some big ones lately, and quickly. How much more do you think there is?

As far as the remaining node classes, these are some of the big categories:

  • classes
  • interpolated strings/regexes and JSX text
  • for/while/loop

And there are probably a couple stragglers, eg I think you mentioned that PassthroughLiteral wasn't covered yet

src/nodes.coffee Outdated
@@ -270,7 +270,9 @@ exports.Base = class Base
# as JSON. This is what the `ast` option in the Node API returns.
# We try to follow the [Babel AST spec](https://github.com/babel/babel/blob/master/packages/babel-parser/ast/spec.md)
# as closely as possible, for improved interoperability with other tools.
ast: (o) ->
ast: (o, level) ->
o = extend {}, o
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, updated to use Object.assign()

@@ -1121,7 +1124,7 @@ exports.Return = class Return extends Base
astType: -> 'ReturnStatement'

astProperties: (o) ->
argument: @expression?.ast(o) ? null
argument: @expression?.ast(o, LEVEL_PAREN) ? null
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The levels used for child nodes in AST generation here should basically match the corresponding levels used for those children when compiling to JS

But as far as how those work, I haven't seen a thorough explanation - there's a little general comment and examples where the LEVEL_*s are defined towards the bottom of nodes.coffee

@GeoffreyBooth GeoffreyBooth merged commit 806a442 into jashkenas:ast Feb 15, 2019
@helixbass helixbass deleted the if-ast branch February 15, 2019 17:38
@Asc2011
Copy link

Asc2011 commented Feb 16, 2019

big thanks @helixbass & @GeoffreyBooth for this huge step ahead - ASTexplorer shows much more detail now. I'm working on low hanging fruit alike CS.TaggedTemplateCall a.k.a. Babel.TaggedTemplateExpressions and friends. JSX-related-structures after that.
How about testing ? I've seen some ast-related test-cases. What do you think about using a JSON-schema to validate ASTs ? Sooner or later a documented schema would be a nice-to-have - i wonder why I can't find one for the babel-AST ?
greets Andreas

@helixbass
Copy link
Collaborator Author

@Asc2011 sure, just for some context about these AST PRs:

I've already got a complete version of the target AST implemented on my prettier branch, so these PRs are basically me pulling chunks of code off of that branch and then @GeoffreyBooth and I have been cleaning them up and reviewing them for eg consistency

So working on AST implementations for remaining classes like TaggedTemplateCall is probably unnecessary/conflicting since there is already an existing implementation

But if you're looking to participate, I'd suggest this:

  • You're certainly more than welcome to follow along with and comment on the ongoing PRs as far as I'm concerned
  • There are possibly a couple tasks on our TODO that aren't already covered on my branch, eg looking to update to use OptionalCallExpression/OptionalMemberExpression to align with Babel 7 AST. These could be good candidates for you to work on if you're interested
  • The initial target consumers of this Coffeescript AST that I've been working on are a Prettier plugin and ESLint custom parser plugin. I haven't documented either yet but if you're interested in contributing there are definitely tasks on both projects and I could help get you set up

How about testing ? I've seen some ast-related test-cases.

Yes, so far we've been adding tests validating AST structure and location data to test/abstract_syntax_tree.coffee and test/abstract_syntax_tree_location_data.coffee

Sooner or later a documented schema would be a nice-to-have

Yes, documentation is something we haven't discussed yet. I'd think that documenting the diff against Babel AST (and linking to its documentation) would be a minimal version

I think it might be nice to have a guide to the codebase too, so that someone exploring/hacking on the Coffeescript codebase could understand at least the AST-related stuff

@GeoffreyBooth
Copy link
Collaborator

Sooner or later a documented schema would be a nice-to-have - i wonder why I can’t find one for the babel-AST ?

There’s this: https://github.com/babel/babel/blob/master/packages/babel-parser/ast/spec.md. I’ve generally been using astexplorer.net with babylon7 selected.

I think it might be nice to have a guide to the codebase too, so that someone exploring/hacking on the Coffeescript codebase could understand at least the AST-related stuff

FYI we have this: https://github.com/jashkenas/coffeescript/wiki/%5BHowTo%5D-How-parsing-works though obviously improvements are welcome 😄 and/or a new wiki page about the AST stuff.

@Asc2011
Copy link

Asc2011 commented Feb 16, 2019

I've already got a complete version of the target AST implemented on my [prettier] branch,

wups, that went quick. How are the coffee-resque AST-Types e.g. Existence and Range. handled ? I'll take a look..

But if you're looking to participate, I'd suggest this:

  • You're certainly more than welcome to follow along with and comment on the ongoing PRs
    sure, I'll do that.
  • The initial target consumers of this Coffeescript AST that I've been working on are a Prettier plugin and ESLint custom parser plugin. I haven't documented either yet but if you're interested in contributing there are definitely tasks on both projects and I could help get you set up

great, as Prettier is already part of ASTExplorer, that would be my starting-point/ -setup.

Yes, so far we've been adding tests validating AST structure and location data to

Yes, documentation is something we haven't discussed yet. I'd think that documenting the diff against Babel AST (and linking to its documentation) would be a minimal version

@GeoffreyBooth agreed to write smth. for the wiki. A JSON-schema that describes the overlap and differences between (Babel)-AST and a CS-AST will help interested contributors in the future. As a bonus, one can use it to validate stuff during dev. Looking at the CS-codebase I wondered if the current approach - staying with the cs-naming-conventions and then transform/rename nodes on export will hold. But let me first see and hopefully understand how things were done...
Regarding ASTExplorer, I did a local install and added a ast2js-section. It takes a JSON-AST (made by CS) and then (re-)generates JS-code using babel-generate. Thats really impressive :-). So in the (near-) future, CS will deliver a partially/entirely Babel-compatible AST to the pipeline. From here on transformation & code-generation follows. Sounds good to me - or did i get it wrong ?
@GeoffreyBooth on my system, working on the codebase, esp. the code-generating parts, is slow. I remember a leaner codebase was/is on the wish-list. I shrinked my nodes.coffee down to 3500 loc - still enough to produce a AST. Is there any consensus/plan on how things could get a bit re-arranged ? This should be put in the wiki, too.

@GeoffreyBooth
Copy link
Collaborator

I wondered if the current approach - staying with the cs-naming-conventions and then transform/rename nodes on export will hold

We’ve definitely been thinking about streamlining the compiler once AST generation is done. nodes.coffee wouldn’t need two separate “output” methods, one for JS source and another for AST JSON. It could generate only the AST JSON, and then another step could convert that into JS source (either using our own code if we want to keep at zero dependencies, or using something like babel-generate or Prettier itself). But all that refactoring is a later phase, we wanted to focus on AST generation alone first.

Not sure why it’s slow for you. The slow part of recompiling the compiler is the parser, which doesn’t need to happen unless you change grammar.coffee which is infrequent most of the time. I use https://github.com/GeoffreyBooth/coffeescript-gulp to watch the CoffeeScript repo for changes and recompile; it skips the parser unless it sees a change in grammar.coffee.

@Asc2011
Copy link

Asc2011 commented Feb 17, 2019

But all that refactoring is a later phase, we wanted to focus on AST generation alone first.

ic, so let's do this :-) i'm glad to fill a new wiki-page about the AST/Babel-tooling stuff.

Not sure why it’s slow for you. The slow part of recompiling the compiler is the parser, which doesn’t need to happen unless you change grammar.coffee which is infrequent most of the time.

I did not touch the grammar. I guess it's Atom or smth. misconfigured on my end or mojave, I dunno.

anyway, the AST-output from Julians CS-branch looks good, but seems to break the local ASTExplorer :-(
@helixbass i could need some assistance here. I failed on integrating your prettier-branch with ast-explorer. I see a bunch of deps in coffeescript/node_modules. graceful-fs and read-pkg appear in the stack trace. I found read-pkg, read-pkg-upand load-json-file in different versions side-by-side ?
I could not yet figure out which pkg requires those convenience-modules. Maybe you have a idea ?
Any help appreciated.

@GeoffreyBooth
Copy link
Collaborator

Personally I haven’t been piping the AST output into anything. I’ve just been having the compiler print it to the console and maybe format it there, e.g. ./bin/coffee --ast test.coffee | jq '.program.body', and then I visually compare that output against the output in astexplorer.net for the equivalent input JavaScript.

@helixbass
Copy link
Collaborator Author

@Asc2011 ya I'm not clear on how you're using ASTExplorer?

That prettier branch does contain various other stuff (including some dependencies) since that branch started off as an attempt to pipe transformed AST through Prettier to produce transformed JS. If you npm install with that branch checked out (I'd recommend checking out a separate copy in a different directory to avoid having to switch back and forth with eg the ast branch) are you still seeing dependency issues? If that's blocking you, I could try and take a quick pass at that branch to strip out the additional dependencies (which aren't needed for the AST generation code path)

@Asc2011
Copy link

Asc2011 commented Feb 18, 2019

@helixbass i did a local install. It's a react-web-app, served by webpack.
I bet its easier to keep your prettier-branch separated from ast-explorer and use the coffee-command via shell. I need the CS-master anyway to be able to compare generated AST-results.
I could not find anything on OptionalCallExpression/OptionalMemberExpression ? It this already part of the Babel-AST. It is not yet in the spec-doc the Geoffrey mentioned ?

@helixbass
Copy link
Collaborator Author

@Asc2011 ya last I checked it didn't look like Babel had updated their docs to reflect OptionalCallExpression/OptionalMemberExpression yet. Basically it's how they represent "soaking" (eg a?.b) when using the corresponding "optional chaining" JS proposed syntax

Here's a Babel issue that seemed to have a fair amount of information: babel/babel#7256

I haven't fully wrapped my head around it but one interesting aspect is that the usage of OptionalMemberExpression/OptionalCallExpression "cascades"

For example, look at the AST (with babylon7 parser) for a?.b.c in ASTExplorer - notice how the outer member expression is also an OptionalMemberExpression, but with optional: false

@helixbass
Copy link
Collaborator Author

Also here's where we discussed it briefly in a previous AST PR: #5117 (comment)

@Asc2011
Copy link

Asc2011 commented Feb 25, 2019

@helixbass hi again. I scanned the TCO39-proposals that will use soaking and OptionalMemberExpression/OptionalCallExpression-nodes. These proposals are at stage-1 and in recent discussions from january they somehow realized that anything .? or ?? will collide with the ternary-operator. That will gonna take some extra-time, I guess.
I thought about how to handle the PassthroughLiteral. It seems, that babel.traverse expects a already valid Babel-AST. The babel-types are considered AFAIK private. So one cannot temporarily create aPassThru-Node, just to transform such a node via the babel-plugin into smth. that babel will finally accept.
So I remembered the WithStatement which Babel supports, but CoffeeScript never uses or creates. So why not (re-) use the WithStatement as a transport into babel and then write a transform-plugin that :

  • extracts the backtracked-string from the WithStatement-node
  • uses babel to compile that string and finally
  • insert the babel-generated node(s) back into the AST

The whole process is a bit a 'catch-22'-thing. As it's only required if somebody uses the coffee --ast option.
In case CS produces all the JS-code than the problem does not arise at all.
As an alternative, one could compile the backticked-strings right away and insert them into the CS-AST. That way, any syntax-errors inside the JS-parts would surface during CS ast-production. Somewhat asymmetric, since such errors would typically show up during the JS-transpilation or during run-time.
Both approaches would introduce a dependency towards babel..
Lastly, maybe one could simply put the backticked-js into a StringLiteral. A transform-plugin would then grab all StringLiteral-nodes, that have no context/are not assigned to anything. And replace those with what babel compiles.
What do you think ? Asking the babel-guys if they can provide a node-type for that purpose seems pointless, since such would not add any purpose for them ?

@Asc2011
Copy link

Asc2011 commented Feb 26, 2019

Another unused node-type is theFunctionDeclaration came to my mind. This one works in strict-mode. Whereas the WithStatement does not.
As a babel-dependency would only be required when coffee --ast is used and backticked-expressions are found. So all common tasks work as before. So if one is interested in working with the AST, then he might be willing to add babel as a dev-dependency, anyway. In that case, CS would not need to create PassthroughLiterals. Instead any backticked-js-code could be immeadiatly compiled during the first-run and out pops a valid Babel-AST ?

@GeoffreyBooth
Copy link
Collaborator

We were planning on creating new node types for nodes that don’t exist in the Babel AST. This corresponds with how on some existing Babel AST node types, we’re adding custom properties (like whether an if is postfix (end of line) or not). The idea is for the CoffeeScript AST to be a superset of Babel’s, not to fit within it.

@Asc2011
Copy link

Asc2011 commented Feb 26, 2019

@GeoffreyBooth well, fair enough - so then lets forget all my comments from above :-)
Am I right in believing, that it' s preferable if we could get away without having to rely on babel-tooling during cs-compilation ? As you said, having a babel-node that carries not-yet-transpiled code would solve that, too.

@GeoffreyBooth
Copy link
Collaborator

Am I right in believing, that it’ s preferable if we could get away without having to rely on babel-tooling during cs-compilation ?

I’m not sure I understand. Currently the CoffeeScript compiler doesn’t rely on Babel unless you use the transpile option.

The CoffeeScript AST is just an output like the JavaScript source output. The obligation is on downstream tools receiving this AST to understand what to do with it. @helixbass has been working on plugins for Prettier and eslint to enable those tools to understand how to process the CoffeeScript AST. Since those tools already accept the Babel AST as input, the closer we are to that the simpler the plugin can be.

@helixbass
Copy link
Collaborator Author

Ya I haven't tried to process a CS AST with "Babel itself", just (as @GeoffreyBooth mentioned) with tools that know how to process a "Babel-style" AST

So that's interesting if in fact Babel can't handle ASTs with node types that are unknown to it, but I don't think it's within scope of our AST generation (or something that would have a big impact on our approach) for us to try and create a "compatibility layer" there. I wonder if eg Babel syntax plugins are able to introduce new node types (it seems like they'd have to be able to)?

@Asc2011 as far as the OptionalMemberExpression stuff, do you mind linking to that TC39 discussion? Sounds interesting. But I don't think that should have any effect on our AST "spec", we'd just be aiming to generate equivalent AST as Babel 7 does when it sees "soak" operators

@GeoffreyBooth
Copy link
Collaborator

Also it's been on our to do list for a long time to create a Babel plugin for CoffeeScript like how there's one for TypeScript. That might be a bit tricky since TypeScript is a superset of JavaScript and we're not, but I hope it's still possible. If the plugin can pass onto Babel JS source rather than an AST, creating such a plugin can happen now and not need to wait on this AST work.

@edemaine
Copy link
Contributor

@GeoffreyBooth I think CS is technically a superset of JS via the backtick operator. 😄 But I'm no expert on what's needed for a Babel plugin, and whether this perspective helps.

@Asc2011
Copy link

Asc2011 commented Feb 26, 2019

I’m not sure I understand. Currently the CoffeeScript compiler doesn’t rely on Babel unless you use the transpile option.

Thank you, you got me right. And now I've learned that it should stay that way. I was asking because @helixbass copheescript-branch for prettier makes use of babylon et al. And I thought maybe there are plans/ideas to integrate such tools directly. Might that be during cs->ast-generation or elsewhere. As a side-note babel-plugin-macros looks quite interesting.
So what's next ? Writing a petition to create CoffeeScript-specific node-types ? I've read that on the babel-side that people might have concerns about that - but maybe that's just hearsay.

@GeoffreyBooth
Copy link
Collaborator

Before we petition Babel, I think we should see how far we can get without any changes on their side. If a plugin can be written that outputs JavaScript source, and Babel can accept that as input, that’s all we need to do.

@Asc2011
Copy link

Asc2011 commented Feb 27, 2019

@helixbass regarding the mentioned tc39-proposal(s):
stage-1 optional-chaining. Mind the comprehensive links in the references- & prior-art sections. The dawning of the impossible in this issue-discussion. This one started in 2017 and had a babel-transform-plugin, which I can't find. Maybe here in babels experimental-section. Or ask @rattralex he had been somehow involved ?
stage-1 nullish-coalescing is related to the above proposal. Both are driven by G. Isenberg - who has ~10 proposals in-the-works.

@GeoffreyBooth making a babel-syntax-plugin as a starting point. My current believe is that syntax-plugins are not open for public-usage. AFAIK same for the babel-types(ast-types). Such types are documented but I can't find information describing the process. Once you have defined a needed additional ast-type - thinking PassthroughLiteral. Maybe a simple PR to them ? I'll put more research into this...
Mind that in the babel-handbook exists a section 'Converters' marked as work-in-progress.

If a plugin can be written that outputs JavaScript source, and Babel can accept that as input

Surely babel accepts any JS-code that CoffeeScript has generated. From here on - after babel has produced a ast - one can apply one..many transformation-plugins, maybe grouped into presets. Many prototyped tc39-proposals are among those plugins == transformation-plugins.
So what you are saying is, lets just feed the JS-code into babel and go on ? Yeah, that requires no plugin at all ? Unless you want to transform the PassthruLiteral via a plugin. That is easily doable, but i failed upon babel, not accepting the cs-ast, complaining about that particular unknown node-type. Thats why I suggested to masquerade backticked-js-code inside a ast-node-type that babel already knows, likes and appreciates, maybe FunctionDeclaration cos' it has a body.
But being so close to feeding a ast into babel - lets forget about backticks for now - wouldn't that be more efficient and straight-forward ?

@GeoffreyBooth
Copy link
Collaborator

But being so close to feeding a ast into babel - lets forget about backticks for now - wouldn’t that be more efficient and straight-forward ?

No, because Babel won’t know how to handle our more exotic AST node types. It won’t know what to convert foo? into. That’s the job of the CoffeeScript compiler, or for a CoffeeScript plugin within Babel.

There’s a chance we could get Babel to include a CoffeeScript converter along the lines of their TypeScript one. According to this the TypeScript converter simply removes all the type annotations before piping the revised source into Babel; a parallel CoffeeScript one could simply run the source through the CoffeeScript compiler before piping the output JS source into Babel. I would start there. If you could get that to work, we could submit it as a PR to Babel (assuming that converters aren’t supported via configuration the way plugins are).

@Asc2011
Copy link

Asc2011 commented Mar 1, 2019

@GeoffreyBooth thx for the good read. I remain a bit sceptic about redoing what typescript/MS achieved. As you suggested, i started trying stuff we can do without help. So i defined a PassthroughLiteral like so (a) :

types = require 'babel-types'
{ default: defineType, assertValueType } = require '.././node_modules/babel-types/lib/definitions'

defineType 'PassthroughLiteral', {
  builder: ['value']
  fields:
    value:
      validate: assertValueType 'string'
  visitor: ['value']
  aliases: ['JS']
}

hopeing babel would allow to add a new ast-type temporarily to the definitions = the set of all legal ast-types. If that had worked, one could apply babels elegant transform-Api on the coffeescript-ast. But i could not make this work.
I used copheescript's ast-output. Sry for not being clear on this. CoffeeScripts more exotic-node-types Existence, Range are already renamed/reshaped into babel-types. Leaving the PassThru-node as the last to take care for :

(b) one could straight-compile the JS contents of the PassThru-node in the PassthroughLiteral::constructor

babelAst = babel.transformSync @originalValue, {
  ast: on
  sourceType: 'script'
  plugins: babel.loadOptions().plugins
}
@astFragment = babelAst.ast.program.body

In case it does not throw, then babel returns a ast-fragment to be merged later into the CoffeeScript-ast. That works nicely. Surely the location-info needed to be adjusted. I tried backticked-JS with transformations from 'pipeline-operator' and 'class getter'-stuff. Works fine, but the location-info lost some accuracy due to the transforms and thus may require a bit more attention.@babelAst.code can provide the javascript-code.

(c) temporarily masquerade the PassthroughLiteral inside a unused but legal babel-ast-type e.g.FunctionDeclaration prior to the transformation :
Doing so one can present a spec-compliant ast-node to babel, which should be transformed via a self-craftedbabel-plugin. I tested that positive. Thanks to the transformation-Api such can be done within 3-lines-of-code. I did so, but did not check the accuracy of the location-info. I assume that the babels' transformation-Api can take care of that, too. Actually one could 'tunnel' the remaining handful of Exotics via a FunctionDeclaration. E.g. putting CoffeeScripts ast-name into the Function-parameter or maybe the Fn-name. That Function-node-containment will be short-lived and transformed very soon, anyway.
A handcrafted-transformation-plugin could be included and properly setup inside the CS-distribution. By (ab)-using the good-old FunctionDeclaration-node as a temporal-containment, one can be sure, that it won't be name-changed or dropped. That naturally happens with new node-types that are part of proposals < stage-4.
I'd prefer the b + c -variants. Because one could do that right away. No tiny in-between-ast-layer that smbd. needs to take care of. It does not require another ast-/json-transformation-tool. Keeping that as-simple-as-possible.
If the bride's name is babel, then lets try to use that to-the-max. And powerful it is..
Looking at the tc39-proposals, my impression is that all the 'missing'-node-types for Existence, Ranges etc. are all being worked on. So the song goes like tiiiii~me is on our side - yes it is. :-) Just sit & wait and adjust type-names once they reach stage-4 to stay compliant.
And using time to figure out if maybe babel can be used to make things easier for the CoffeeScript-codebase ? Or looking at more interesting proposals like pattern-matching. That said, the Elvis-operator is already here - proven & tested - so let's just wait for a proper ast-name to arrive, maybe someday in 2023. Or maybe finding a coffeefied-syntax for this little stage-3 ugly ?
You might have noticed already, i'm looking at developer-experience. Shipping a babel-plugin that is crucial to the proper working of CoffeeScript, opens up the chance to misconfigure it. Once you use an array of plugins than the processing order becomes very important - saying - the CS-transform MUST be processed first. If one can handle that transformation internal, then the consumer/user/developer cannot screw it up. babel-plugin-macro adresses that problem, but still one can get it wrong.

@GeoffreyBooth
Copy link
Collaborator

@Asc2011 I would start a lot less ambitious if I were you. As I understand things, there’s a Babel TypeScript plugin/converter that intercepts the source code before anything else in Babel sees it, and it strips out all the type annotations. Then this revised source gets passed to Babel for Babel to do its thing with like any other JS source. That’s it. There’s no TypeScript AST getting generated, there’s no AST getting passed as input to Babel. Babel gets JS source, the same as if you had run the TypeScript file through the TypeScript compiler to generate a .js file and then sent that .js file as input to Babel.

So see if you can achieve the same but for CoffeeScript. It intercepts the source before any part of Babel sees it, runs it through the CoffeeScript compiler (loaded as a Node dependency) and then the resulting JavaScript source code (not AST) is sent onward into Babel. That’s it.

Or looked at another way, see if you can achieve this “do something to the source before Babel ever sees it” with Illiterate. That’s even more generic. If you can make that work, Babel would be able to process Literate JavaScript and Literate TypeScript and Literate JSX 😉 And you’d only have to replace the call to Illiterate with a call to CoffeeScript and then you have the CoffeeScript equivalent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants