Add templating mechanism to automatically generate `.c`/`.h` files #2098

amomchilov · 2024-11-25T21:47:22Z

This PR automates the creation of these pre-existing files:

constants.h
constants.c
ruby_objs.h
ruby_objs.c

In this initial PR, we aim to generate these files to match as closely to their original hand-authored versions. To facilitate this, we added some trickery to the ERB files, to get it as close as practical. These will be removed in a follow-up PR.

amomchilov · 2024-11-25T21:47:54Z

config.yml

+  - name: RBS::Types::Alias
+    fields:
+      - name: name
+        c_name: typename # Temporary name to match existing C code. Will be removed in next PR.


The identifiers used in C don't always match those used in the Ruby API. Where necessary, we used this c_name: field to preserve the C code as it was.

amomchilov · 2024-11-25T21:48:20Z

ext/rbs_extension/constants.c

 VALUE RBS_AST_Declarations;
+VALUE RBS_AST_Directives;
+VALUE RBS_AST_Members;
+VALUE RBS_Parser;
+VALUE RBS_Types;
+VALUE RBS_Types_Bases;


These are special-cased in the template. The rest of the file is code-genned, but in a different order than the original. All the same constants are still here, though.

amomchilov · 2024-11-25T21:50:07Z

ext/rbs_extension/parser.c

@@ -963,25 +963,25 @@ static VALUE parse_simple(parserstate *state) {
    return type;
  }
  case kBOOL:
-    return rbs_base_type(RBS_Types_Bases_Bool, rbs_location_current_token(state));
+    return rbs_bases_bool(rbs_location_current_token(state));


These are just working names for now, we'll reconsider these once we decide what the new public C API looks like.

amomchilov · 2024-11-25T21:51:53Z

ext/rbs_extension/parser.c

@@ -963,25 +963,25 @@ static VALUE parse_simple(parserstate *state) {
    return type;
  }
  case kBOOL:
-    return rbs_base_type(RBS_Types_Bases_Bool, rbs_location_current_token(state));


All 9 of these types used to have their own Ruby classes, but shared this same "constructor" (rbs_base_type()).

We will eventually want 9 different C structs to model these (to keep a simple 1:1 correspondence to the Ruby classes), with 9 different constructor functions. We did that here, since it simplifies automation.

amomchilov · 2024-11-25T21:54:17Z

ext/rbs_extension/parser.c

@@ -1836,14 +1839,18 @@ VALUE parse_mixin_member(parserstate *state, bool from_interface, position comme
  rbs_loc_add_required_child(loc, rb_intern("keyword"), keyword_range);
  rbs_loc_add_optional_child(loc, rb_intern("args"), args_range);

-  return rbs_ast_members_mixin(


Likewise, we split this one constructor function (rbs_ast_members_mixin()) into 3 new functions:

rbs_ast_members_extend()

rbs_ast_members_include()

rbs_ast_members_prepend()

These will be modelled by 3 different C structs, to match the 3 different Ruby classes that exist for them.

amomchilov · 2024-11-25T21:56:16Z

ext/rbs_extension/parser.c

-  rbs_loc_add_required_child(loc, rb_intern("colon"), colon_range);
-  rbs_loc_add_optional_child(loc, rb_intern("kind"), kind_range);
-
-  return rbs_ast_members_variable(klass, name, type, location, comment);


Same with rbs_ast_members_variable(), which gets split into:

rbs_ast_members_instance_variable()

rbs_ast_members_class_variable()

rbs_ast_members_class_instance_variable()

amomchilov · 2024-11-25T21:56:41Z

ext/rbs_extension/parser.c

  default:
    rbs_abort();
  }
-
-  return rbs_ast_members_visibility(


Split into two:

rbs_ast_members_public()

rbs_ast_members_private()

amomchilov · 2024-11-25T21:57:14Z

ext/rbs_extension/parser.c

@@ -2153,17 +2146,17 @@ VALUE parse_attribute_member(parserstate *state, position comment_pos, VALUE ann
  rbs_loc_add_optional_child(loc, rb_intern("ivar_name"), ivar_name_range);
  rbs_loc_add_optional_child(loc, rb_intern("visibility"), visibility_range);

-  return rbs_ast_members_attribute(


Split into 3 types and functions:

rbs_ast_members_attr_reader

rbs_ast_members_attr_writer

rbs_ast_members_attr_accessor

amomchilov · 2024-11-25T21:58:28Z

ext/rbs_extension/parser.c

+      if (unchecked) {
+        rb_funcall(param, rb_intern("unchecked!"), 0);
+      }


This was moved to here from rbs_ast_type_param(), since it doesn't fit the pattern used by the rest of the constructor functions. It's cleaner to just add it here, than to special-case it in generated code.

soutaro · 2024-11-27T14:28:29Z

Thanks @amomchilov!

I hope #2101 fixes the windows compile issue.

soutaro

🎉 Waiting for the build issue is fixed.

Generate the c/h code for constant definitions and object initialization. We make sure the Rake task builds the required templates so we can compile the extension. Co-authored-by: Alexandre Terrasa <[email protected]>

amomchilov commented Nov 25, 2024

View reviewed changes

amomchilov force-pushed the templated-code-gen branch from b8d3eb3 to a888013 Compare November 25, 2024 22:21

amomchilov mentioned this pull request Nov 25, 2024

Add templating mechanism to automatically generate .c/.h files Shopify/rbs#2

Closed

amomchilov force-pushed the templated-code-gen branch from 3597db7 to 3831ac4 Compare November 25, 2024 23:22

amomchilov marked this pull request as ready for review November 26, 2024 01:28

soutaro approved these changes Nov 27, 2024

View reviewed changes

soutaro added this to the RBS 3.7 milestone Nov 27, 2024

amomchilov and others added 2 commits November 27, 2024 17:36

Add templating mechanism to generate .c/.h files

574ccd1

Generate the c/h code for constant definitions and object initialization. We make sure the Rake task builds the required templates so we can compile the extension. Co-authored-by: Alexandre Terrasa <[email protected]>

Add activesupport as explicit depenency

6ff4d34

amomchilov force-pushed the templated-code-gen branch from 3831ac4 to 6ff4d34 Compare November 27, 2024 22:36

amomchilov mentioned this pull request Nov 27, 2024

Clean up temporary workarounds from templated code #2102

Merged

soutaro added this pull request to the merge queue Nov 28, 2024

Merged via the queue into ruby:master with commit 7c561d0 Nov 28, 2024
18 checks passed

amomchilov mentioned this pull request Nov 28, 2024

WIP: node templates Shopify/rbs#1

Closed

soutaro added the Released PRs already included in the released version label Dec 5, 2024

amomchilov mentioned this pull request Dec 6, 2024

[Epic] Pure C API #2133

Open

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add templating mechanism to automatically generate `.c`/`.h` files #2098

Add templating mechanism to automatically generate `.c`/`.h` files #2098

amomchilov commented Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

amomchilov Nov 25, 2024

soutaro commented Nov 27, 2024

soutaro left a comment

Add templating mechanism to automatically generate .c/.h files #2098

Add templating mechanism to automatically generate .c/.h files #2098

Conversation

amomchilov commented Nov 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soutaro commented Nov 27, 2024

soutaro left a comment

Choose a reason for hiding this comment

Add templating mechanism to automatically generate `.c`/`.h` files #2098

Add templating mechanism to automatically generate `.c`/`.h` files #2098