Can't pass function to evaluationContext (Improvement?)

[Vandivier]

Steps to reproduce

Tell us about your environment:

• Puppeteer version: "^0.12.0"
• Platform / OS version: Windows
• URLs (if applicable): https://github.com/Vandivier/data-science-practice/tree/function-to-evaluation-context-repro

What steps will reproduce the problem?

Please include code that reproduces the issue.

see https://github.com/Vandivier/data-science-practice/tree/function-to-evaluation-context-repro

clone and execute npm run-script get-riders

What is the expected result?

Objects will be printed to console which contain the key-value pair 'hi', 'sup', as well as a key 'fpEvaluateInPage' with a function as the value. Additionally, this function should be executed, printing 'hi' to the console on it's own line.

What happens instead?
Objects are printed to console only containing the key-value pair 'hi', 'sup'

[aslushnikov]

@Vandivier i looked into your repository and quickly got lost. Please, feel free to re-file with a repro that would be easier to follow.

[Vandivier]

lol. Since when is "I can't understand" a reason to close an issue?

Here's the issue in it's simplest form. Consider the following object:

let options = {
    'fpEvaluateInPage': function (_options) {
        console.log('hi');
        return Promise.resolve({
            'sTableHtml': 'hi'
        });
    },
    'hi': 'sup'
}

If this object is passed as options to evaluationContext, options.hi exists and options.fpEvaluateInPage does not exist. Or at very least the console.log('hi') fails to print in combination with _page.on('console', _fCleanLog);, where _fCleanLog is defined as follows:

function _fCleanLog(ConsoleMessage) {
    console.log(ConsoleMessage.text + EOL);
}

[aslushnikov]

@Vandivier if I understand correctly, you're passing function from node.js context over to the page context as one of the arguments in the page.evaluate:

const puppeteer = require('puppeteer');

(async() => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  const myObject = {fn: e => 7 + e};
  const result = await page.evaluate(myObject => myObject.fn.call(1), myObject);
  console.log(result);

  await browser.close();
})();

This won't work since myObject, and specifically function inside it, is not Serializable. To pass function over to the page context, you'll have to stringify it first and then re-create in browser:

const puppeteer = require('puppeteer');

(async() => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  const myFunction = e => e + 1;
  const result = await page.evaluate(myFunctionText => {
    // re-creating function in browser context
    const myFunction = new Function(' return (' + myFunctionText + ').apply(null, arguments)');
    return myFunction.call(null, 7);
  }, myFunction.toString());
  console.log(result);

  await browser.close();
})();

[Vandivier]

Great! Yes, that is what I was talking about and your solution works.

It would be even more convenient, tho, if Puppeteer handled this internally.

How do you feel about a PR?

[aslushnikov]

How do you feel about a PR?

@Vandivier I'd rather not. We currently rely on a generic notion of Serializable objects, and it's quite easy to predict what will safely cross the boundary and whatnot.

Adding a custom codepath for function objects, we'll set a precedent and start an uphill battle with the endless stream of custom serializers/deserializers. We'll never support all the usecases, and it will be hard to predict what's supported.

[yaakovyitzchak]

Hi is there any way to get progress of a page.evaluate? if I'm doing something like this:

await page.evaluate(cb => {
        
        var x = new XMLHttpRequest();
        var url = location.href;
        x.onreadystatechange = function(){
            console.log((x.response));
        }
        x.responseType="arraybuffer";
        x.open("GET",url,true);
        x.send("");
    });

How do I call some kind of progress callback whenever I get a chunk of data?

[0xtaf]

anything new on this subject?