Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript runtime resolver bundling #577

Open
Hideman85 opened this issue Mar 12, 2023 · 5 comments
Open

Javascript runtime resolver bundling #577

Hideman85 opened this issue Mar 12, 2023 · 5 comments

Comments

@Hideman85
Copy link

Hi there,

First of all this is a quite new feature and I'm not quite aware of all the support, I checked AWS doc that explains a lot but I'm not sure about the limitations yet.

It feels plenty of lambda could be eliminated with the JS resolvers but then for this we need to bundle those resolvers and make sure we are not using node built-in or anything unsupported.

I am wondering if we can combine with serverless-esbuild to bundle the resolvers.code in addition can it be a s3file? I am wondering what is the size limit?

Thanks in advance for your help.

I was also checking the RFC on appsync community and I seen some interesting proposals like if we are able to call the datasource on our own instead of statically define pipeline functions this look really promising. Pretty sure we can kill lot of lambda and avoid extra cost/response time 🚀

@bboure
Copy link
Collaborator

bboure commented Mar 12, 2023

H @Hideman85

You might be interested in PR #576 🙂
It adds support for automatic bundling and TS transplation (if needed).

Also, recently wrote an article about bundling JS code for AppSync, you might be interested too.

The size limit for JS resolvers is 32,000 characters (source)

@Hideman85
Copy link
Author

Oh men this is great, thanks a lot for your reply 💯

@Hideman85
Copy link
Author

Hideman85 commented Mar 20, 2023

Hi @bboure
I was checking deeper the limitations on the js runtime and some are quite annoying like those:
image

I was thinking of an high level API to build queries like mongodb, today we are using dynamoose in lambdas and that is pretty useful. dynamoose wont be compatible at all since the whole internal is promise based but I guess something alike non async to build the queries is achievable.

I was wondering if you would know a way to statically evaluate code while minifying? I havent found anything yet in esbuild but this is something available in terser tho 🤔 (with some limit still)

I just tried a very small example just to know the viability of a solution like this.

If you have some time and more knowledge with bundlers/minifiers I would love some help.

Example using factories/builder pattern that wont be allowed in js runtime

I know js runtime is limited but lot of high level code like those example can be statically evaluated and reduced down to some primitives. Here we can imagine building the filter conditions of a query for dynamodb using a builder syntax like .where('column').eq(42)

Code sample

const factory = (str, num) => ({ str, num })
export const a = { foo: 'bar', ...factory('Hello', 1) }

//  Test a builder pattern
const builder = /* @__PURE__ */() => {
  const obj = {};
  const methods = {
    str: str => {
      obj.str = str
      return methods
    },
    num: num => {
      obj.num = num
      return methods
    },
    foo: foo => {
      obj.foo = foo
      return methods
    },
    build: () => {
      return obj
    }
  }
  return methods
}
export const b = builder().foo('bar').str('Hello').num(1).build()

//  Class compilled as functions?
class Builder {
  obj = {}
  str(str) {
    this.obj.str = str
    return this
  }
  num(num) {
    this.obj.num = num
    return this
  }
  foo(foo) {
    this.obj.foo = foo
    return this
  }
  build() {
    return this.obj
  }
}
export const c = new Builder().foo('bar').str('Hello').num(1).build()
Terser config
{
  module: true,
  compress: {
    ecma: '2020',
    expression: true,
    evaluate: true,
    hoist_funs: true,
    hoist_vars: true,
    module: true,
    passes: 10,
    toplevel: true,
    unsafe: true,
  },
  mangle: {
    module: true,
    toplevel: true,
  },
  output: {
    comments:false,
  },
  parse: {},
  rename: {},
  ecma: '2020',
}

I used the following:

Right now I stuck with the builder pattenr because it does not seem to want to be statically interpreted, I tried to check and mark pure functions with no luck.

So I would love some help with this and I guess then properly TS checked building api could be done for standardise dynamodb resolver writting using factories/builder.

Example of output (formatted)
export const a = {
    foo: "bar",
    str: "Hello",
    num: 1
};

// Look at the first expression fully evaluated

export const b = (() => {
    const o = {}, t = {
        str: r => (o.str = r, t),
        num: r => (o.num = r, t),
        foo: r => (o.foo = r, t),
        build: () => o
    };
    return t;
})().foo("bar").str("Hello").num(1).build();

// But here no luck 🤔

export const c = (new class {
    obj={};
    str(o) {
        return this.obj.str = o, this;
    }
    num(o) {
        return this.obj.num = o, this;
    }
    foo(o) {
        return this.obj.foo = o, this;
    }
    build() {
        return this.obj;
    }
}).foo("bar").str("Hello").num(1).build();

@bboure
Copy link
Collaborator

bboure commented Mar 20, 2023

Disclaimer: I am not too familiar with Mongo

One important thing to notice is that JS resolvers can't call any external library by themselves.
There is no network access. And as you mentioned, no async (promises) either. I think classes are not supported either.

However, there might be some things we can do. AFAIK, MongoDB is an HTTP-based API.
So we could technically rely on an HTTP resolver in AppSync to send the requests.
e.g. have a helper function/API that generates a valid HTTP request that is later sent to Mongo.

@Hideman85
Copy link
Author

Sorry for the confusion I did not meant to call mongodb but having a schema and query builder that is looking like it. Look at dynamoosejs

It looks like this:

const dynamoose = require("dynamoose");

const personSchema = new dynamoose.Schema({
    "id": String,
    "name": String,
    "age": Number,
    "team": String,
}, {
    "timestamps": {
        "createdAt": ["createDate", "creation"],
        "updatedAt": ["updateDate", "updated"],
    },
});

const personModel = new dynamoose.Model('Persons', personSchema);

Then it can execute directly against dynamodb or return the request (this would be then the return object of the JS resolver)

export function request({ args }) {
  return personModel.query().where('team').eq('admin').and().where('age').ge(args.minAge ?? 18).build();
}

// This could be statically evaluated during building time/minification time to
export function request({ args }) {
  return {
    operation: 'Query',
    query: {
      expression: '#team = :team',
      expressionNames: {
        '#team': 'team',
      },
      expressionValues: {
        ':team': 'admin',
      },
    },
    filter: {
      expression: '#age >= :age',
      expressionNames: {
        '#age': 'age',
      },
      expressionValues: {
        ':age': args.minAge ?? 18,
      },
    },
  };
}

More a kind of meta programming, I'm not so sure if there is a better way to implement an abstraction like this. Of course dynamoose would not suite the JS Runtime requirements at all. But this is what I try to acheive at compilation time so then it fits the runtime and the source remais highly maintanable and well typechecked.

Let me know your thoughts on this and if you already seen something similar existing, thanks 🙏.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants