Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Got an error reading communication packets #2539

Closed
metanovii opened this issue Mar 28, 2024 · 9 comments
Closed

Got an error reading communication packets #2539

metanovii opened this issue Mar 28, 2024 · 9 comments

Comments

@metanovii
Copy link

version packages:
percona mysql 8.0.34-26
nodejs version v18.13.0
mysql2 3.9.3

config

  database: {
    type: 'mysql',
    host: process.env.THE_BEST_APP_INT_THE_WORLD_TYPE_ORM_MYSQL_HOST,
    port: parseInt(process.env.THE_BEST_APP_INT_THE_WORLD_TYPE_ORM_MYSQL_PORT),
    username: process.env.THE_BEST_APP_INT_THE_WORLD_TYPE_ORM_MYSQL_USERNAME,
    password: process.env.THE_BEST_APP_INT_THE_WORLD_TYPE_ORM_MYSQL_PASSWORD,
    database: process.env.THE_BEST_APP_INT_THE_WORLD_TYPE_ORM_MYSQL_DATABASE,
    autoLoadEntities: true,
    namingStrategy: new SnakeNamingStrategy(),
    logging: true,
    debug: true,
  }

Debug log

$ nest start prod
Add command: ClientHandshake
 raw: 0a382e302e33342d3236002e62c8000e441c256f2e272300ffffff0200ffdf1500000000000000000000670547297f53261858272a680063616368696e675f736861325f70617373776f726400
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 undefined ==> ClientHandshake#unknown name(0,,81)
Server hello packet: capability flags:3758096383=(long password, found rows, long flag, connect with db, no schema, compress, odbc, local files, ignore space, protocol 41, interactive, ssl, ignore sigpipe, transactions, reserved, secure connection, multi statements, multi results, ps multi results, plugin auth, connect attrs, plugin auth lenenc client data, can handle expired passwords, session track, deprecate eof, ssl verify server cert, remember options, multi factor authentication)
Sending handshake packet: flags:280687567=(long password, found rows, long flag, connect with db, odbc, local files, ignore space, protocol 41, ignore sigpipe, transactions, reserved, secure connection, multi results, plugin auth, connect attrs, plugin auth lenenc client data, session track, multi factor authentication)
0 13132334 <== ClientHandshake#unknown name(1,,161)
0 13132334 <== 9d000001cff3ba1000000000e000000000000000000000000000000000000000000000006c6561645f636f72655f6461746170686f6e655f7161001467b272d5f8dd1d87f05f62c1d42701f63e0fdc7e6c6561645f636f7265006d7973716c5f6e61746976655f70617373776f726400300c5f636c69656e745f6e616d650c4e6f64652d4d7953514c2d320f5f636c69656e745f76657273696f6e05332e392e33
 raw: fe6d7973716c5f6e61746976655f70617373776f7264000e441c256f2e2723670547297f53261858272a6800
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> ClientHandshake#unknown name(2,,48)
0 13132334 <== ClientHandshake#unknown name(3,,24)
0 13132334 <== 1400000367b272d5f8dd1d87f05f62c1d42701f63e0fdc7e
 raw: 00000002400000000c010a096c6561645f636f7265
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> ClientHandshake#unknown name(4,maybeOK,25)
query: SELECT VERSION() AS `version`
Add command: Query
        Sending query command: SELECT VERSION() AS `version`
0 13132334 <== Query#unknown name(0,,34)
0 13132334 <== 1e0000000353454c4543542056455253494f4e2829204153206076657273696f6e60
 raw: 011d000002
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> Query#unknown name(1,,5)
        Resultset header received, expecting 1 column definition packets
 raw: 036465660000000776657273696f6e000ce00024000000fd01001f000005000003
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> Query#unknown name(2,,33)
        Column definition:
          name: version
          type: 253
         flags: 1


Compiled text protocol row parser:

For nicer debug output consider install cardinal@^2.0.0
(function () {
  return class TextRow {
    constructor(fields) {
    }
    next(packet, fields, options) {
      this.packet = packet;
      const result = {};
      // "version": VAR_STRING
      result["version"] = packet.readLengthCodedString(fields[0].encoding);
      return result;
    }
  };
})()

 raw: fe000002000a000004
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> Query#unknown name(3,EOF,9)
 raw: 09382e302e33342d323605000005
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> Query#unknown name(4,,14)
 raw: fe00000208
Trace: 
    at PoolConnection.handlePacket (/srv/app/node_modules/mysql2/lib/connection.js:433:17)
    at PacketParser.onPacket (/srv/app/node_modules/mysql2/lib/connection.js:97:12)
    at PacketParser.executeStart (/srv/app/node_modules/mysql2/lib/packet_parser.js:75:16)
    at Socket.<anonymous> (/srv/app/node_modules/mysql2/lib/connection.js:104:25)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
0 13132334 ==> Query#unknown name(5,EOF,9)
error Command failed with exit code 1.

messages from wireshark

M...
8.0.34-26./b..x.*K6NR....................X
.2|.S.ALLo.caching_sha2_password.....................................best_app_qa...f(......;d.P
.....Xbest_app.mysql_native_password.0._client_name.Node-MySQL-2._client_version.3.9.3,....mysql_native_password.x.*K6NR.X
.2|.S.ALLo......f(......;d.P
[email protected]_app.....SELECT VERSION() AS `version`..........def....version....$..................
...	8.0.34-26.........3......#08S01Got an error reading communication packets

What am I doing wrong?
@BMO-tech
Copy link

Try reverting to 3.9.2. I was getting handshake errors today too (completely different setup than you, so no promises), and reverting from 3.9.3 back to 3.9.2 fixed it for me.

@metanovii
Copy link
Author

metanovii commented Mar 29, 2024 •
edited
Loading

Try reverting to 3.9.2. I was getting handshake errors today too (completely different setup than you, so no promises), and reverting from 3.9.3 back to 3.9.2 fixed it for me.

I rolled back the version to ^2.8.0 and it solved my problem. %)) And plugin auth mysql_native_password of course, sha2 does not work.

@pedrovanzella
Copy link

There's something weird with 3.9.3. I was getting SSL cert errors on RDS with the newest version too. I upgraded the certs in the cluster and version 3.9.2 works fine with the newest certs provided by AWS, but 3.9.3 doesn't.

@wellwelwel
Copy link
Collaborator

@pedrovanzella, can you show an example of your changes using the version 3.9.2?

@pedrovanzella
Copy link

@pedrovanzella, can you show an example of your changes using the version 3.9.2?

@wellwelwel there were no changes. I was running < 3.9.2 with the old RDS certs. I upgraded the cluster certs, and my code still worked. I upgraded to 3.9.3, and I started getting errors related to the certificates (they were bubbled up to sequelize, as a SequelizeConnectionError: unable to get local issuer certificate. I downgraded to 3.9.2 and I was able to connect again.

I mentioned this here in this issue because @BMO-tech had mentioned handshake errors, which is what this boiled down to - following the stack down I would see HANDSHAKE_SSL_ERROR.

@wellwelwel
Copy link
Collaborator

@pedrovanzella, @BMO-tech

I think the PR #2542 fixes this behavior, could you check it?

Closing this issue to continue this topic on #2541

@pedrovanzella
Copy link

@wellwelwel Just tested 3.9.4 and I still get the same behavior.

@wellwelwel
Copy link
Collaborator

Thanks, @pedrovanzella 🙋🏻‍♂️

We are handing over this responsibility to a dependency dedicated to it: mysqljs/aws-ssl-profiles#8

I'm thinking of "rebasing" the certificates again, just to make sure.

I trust this can be fixed soon.

@pedrovanzella
Copy link

@wellwelwel I commented on another issue already, but after some more testing, it does look like the issue is with RDS Proxy only. So it looks like you could just include the global CA certs in the file and we would be all set for all use cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pedrovanzella @BMO-tech @metanovii @wellwelwel