bug: failed to find any SSL certificate by SNI:

[MarkCupitt]

Current Behavior

This Error occurs:

2023/01/12 06:09:20 [error] 64#64: *426241 [lua] radixtree_sni.lua:170: match_and_set(): failed to find any SSL certificate by SNI: e3.engineering.billrush.work, context: ssl_certificate_by_lua*, client: 192.168.50.80, server: 0.0.0.0:9443

Certificate craeted with this certbot command

sudo certbot -d e3.engineering.billrush.work -v --rsa-key-size 4096 --manual --preferred-challenges dns certonly

Certificate Uploaded to apisix via dashboard. Certificate accepted and sni and expiry is displayed

Routes on that SNI host cause the error to issue

Tried with Latest Chrome and Firefiox, same issue

Ssl Config

      ssl:
        enable: true
        listen:
          - port: 9443
            enable_http2: true
        ssl_protocols: "TLSv1.2 TLSv1.3"
        ssl_ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA"

System is Ubuntu 22.04LTS which should have the DST Root CA X3 available

Expected Behavior

Routes shoudl work, no error should be logged

Error Logs

2023/01/12 06:09:20 [error] 64#64: 426241 [lua] radixtree_sni.lua:170: match_and_set(): failed to find any SSL certificate by SNI: e3.engineering.billrush.work, context: ssl_certificate_by_lua, client: 192.168.50.80, server: 0.0.0.0:9443

Steps to Reproduce

Install via helm
enable ssl in config
load cert via dashboard
create route using same host as SNI in cert

Environment

• APISIX version (run apisix version):
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[MarkCupitt]

I noticed in the Helm config, these options, Have assume they are not required ot be set, as nothing documented

  tls:
    enabled: true
    servicePort: 443
    containerPort: 9443
    existingCASecret: ""
    certCAFilename: ""

[happyoka]

I also faced the same issue. failed to find any SSL certificate.

[tokers]

@MarkCupitt How did you send a request?

The certificate matching depends on the TLS SNI sent from the Client Hello packet. So please check out if your clients send the correct TLS SNI.

[MarkCupitt]

the client is latest chrome and firefox, both tested, the route is for a html page

…

On Thu, 12 Jan 2023, 5:05 pm Alex Zhang, ***@***.***> wrote: @MarkCupitt <https://github.com/MarkCupitt> How did you send a request? The certificate matching depends on the TLS SNI sent from the Client Hello packet. So please check out if your clients send the correct TLS SNI. — Reply to this email directly, view it on GitHub <#8663 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJ7RUCEJRKHPYMPF7P6VL3WR7COVANCNFSM6AAAAAATY3TZKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[tokers]

the client is latest chrome and firefox, both tested, the route is for a html page
…
On Thu, 12 Jan 2023, 5:05 pm Alex Zhang, @.> wrote: @MarkCupitt https://github.com/MarkCupitt How did you send a request? The certificate matching depends on the TLS SNI sent from the Client Hello packet. So please check out if your clients send the correct TLS SNI. — Reply to this email directly, view it on GitHub <#8663 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJ7RUCEJRKHPYMPF7P6VL3WR7COVANCNFSM6AAAAAATY3TZKM . You are receiving this because you were mentioned.Message ID: @.>

Whether a route is for a HTML page doesn't matter. How did you send a request? What does the URL look like?

[MarkCupitt]

sorry on brevity on phone right now

https://e3.engineering.billrush.work/toptog in the browser both chrome and Firefox .. latest updates applied

[MarkCupitt]

the sni and cert are issued to e3.engineering.billrush.work

the dashboard gets the correct SNI from the cert

[tokers]

@MarkCupitt Are you using APISIX 3.0? If so, this may due to the incompatibility between APISIX 3.x and APISIX Dashboard 2.x.

[MarkCupitt]

@tokers yes, we are using v3.x (latest helm version actually) .. but we have an issue logged using the api upload as well .. #8665 which we could not get to work, it would accept the cert, and return 200, but dashboard failed to get the expiry date out of the cert, the dashboard DID extract the SNI correctly.

Problem is that the cert displayed in the dashboard but was not able to be deleted

Again, apisix logged the Same error as per above on both upload methods. (api and dashboard UI)

I also tried downgrading the cert from 4096 to 2048, just in case, no difference

[tokers]

@tokers yes, we are using v3.x (latest helm version actually) .. but we have an issue logged using the api upload as well .. #8665 which we could not get to work, it would accept the cert, and return 200, but dashboard failed to get the expiry date out of the cert, the dashboard DID extract the SNI correctly.

Problem is that the cert displayed in the dashboard but was not able to be deleted

Again, apisix logged the Same error as per above on both upload methods. (api and dashboard UI)

I also tried downgrading the cert from 4096 to 2048, just in case, no difference

@MarkCupitt I mean, the certificate path prefix for APISIX fetching from the ETCD doesn't match the one Dashboard uses to write the certificate. i.e.g, APISIX fetches from the key space /apisix/ssls/ while dashboard writes to /apisix/ssl/.

[happyoka]

how to solve the issue ? To get the correct SSL cert

[MarkCupitt]

@tokers ok, understand, but given issue with #8665 (we seem unable to get it to work via api either) how do we achieve this so we can get the SNI matching to work, Im unsure on what we need to do

[MarkCupitt]

Issue for upload via V3 api still remains

====================================================
Solution for Dashboard Concern

Seems the Dashboard ONLY supports the V2 api which uses /ssl, whereas the V3 version of apisix requires certs to be loaded accessed form api /ssls

#8173
#8599
#8183
ccc43ea

[tokers]

Issue for upload via V3 api still remains

==================================================== Solution for Dashboard Concern

Seems the Dashboard ONLY supports the V2 api which uses /ssl, whereas the V3 version of apisix requires certs to be loaded accessed form api /ssls

#8173 #8599 #8183 ccc43ea

Yes.

[MarkCupitt]

Certs loaded by the dashboard V2 will NOT be seen in the routing, until the dashboard is made compatible, you MUST use the API's to load and manage certs

Additionally, if you use a Wildcard Cert ["*.example.com"] the route MUST use hosts: ["*.example.com"]. if you need granular route matching, you will need to add a filter, or match on vars and/or priority

if you use host: "my.example.com" it will not match, is it uses the literal SNI value *.example.com

@tokers Might save a lot of grief for people if this could be added to the documentation in the following locations:

https://apisix.apache.org/docs/apisix/admin-api/#request-body-parameters
https://apisix.apache.org/docs/apisix/admin-api/#ssl-api

Documenting in case anyone else has this issue

[adangadang]

apisix 3.9 [error] 50#50: *181148 [lua] init.lua:212: http_ssl_client_hello_phase(): failed to match any SSL certificate by SNI:

https://apisix.apache.org/docs/apisix/certificate/
curl http://127.0.0.1:9180/apisix/admin/ssls/1
-H "X-API-KEY: $admin_key" -X PUT -d '
{
"cert" : "'"$(cat server2.crt)"'",
"key": "'"$(cat server2.key)"'",
"snis": ["test2.com"]
}'