Falco Unable to insmod module on Amazon Linux 2 EKS

[rama-akbar]

Describe the bug

We want to install Falco into our EKS cluster, but got an Unable to insmod module error.

Expected behaviour

Falco running sucess

Error Message

2024-02-19 12:26:52 INFO  Running falcoctl driver install
                      ├ driver version: 7.0.0+driver
                      ├ driver type: kmod
                      ├ driver name: falco
                      ├ compile: true
                      ├ download: true
                      ├ arch: x86_64
                      ├ kernel release: 5.10.198-187.748.amzn2.x86_64
                      └ kernel version: #1 SMP Tue Oct 24 19:49:54 UTC 2023
2024-02-19 12:26:52 INFO  Found distro target: amazonlinux2
2024-02-19 12:26:52 INFO  Check if kernel module is still loaded.
2024-02-19 12:26:52 INFO  OK! There is no module loaded.
2024-02-19 12:26:52 INFO  Check all versions of kernel module in dkms.
2024-02-19 12:26:52 INFO  OK! There are no module versions in dkms.
2024-02-19 12:26:52 INFO  Trying to download a driver.
                      └ url: https://download.falco.org/driver/7.0.0%2Bdriver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
2024-02-19 12:26:54 INFO  Driver downloaded.
                      └ path: /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
2024-02-19 12:26:54 WARN  Unable to insmod module.
                      ├ driver: /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
                      └ err: exit status 1

Environment

• Falco version:
  falcosecurity/falco-no-driver:0.37.1
  falcosecurity/falco-driver-loader:0.37.1
  falcosecurity/falcoctl:0.7.2

• System info:

• Cloud provider or hardware configuration: AWS (EKS)

• OS:
  Amazon Linux 2

• Kernel:
  kernel release: 5.10.198-187.748.amzn2.x86_64
  kernel version: Digwatch compiler #1 SMP Tue Oct 24 19:49:54 UTC 2023

• Installation method:
  Helm Chart

[FedeDP]

Hi! Thanks for opening this issue! Can you manually try insmod /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko and paste here the output?

[FedeDP]

/assign

[rama-akbar]

@FedeDP

Output of insmod, we deploy it using helm chart from official falco - https://github.com/falcosecurity/charts/blob/master/charts/falco/values.yaml

insmod: ERROR: could not insert module root/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko: Operation not permitted

[FedeDP]

Mmh it seems like you are missing some permissions; how did you deploy the Falco helm chart? Did you modify anything from default values?

[rama-akbar]

Hi @FedeDP
here is diff

left = ours
right = current chart https://github.com/falcosecurity/charts/blob/master/charts/falco/values.yaml

https://www.diffchecker.com/HPWzTzsU/

---

Is there posibility latest version of falco didn't support on Amazon Linux 2 ?

[FedeDP]

Mmmh from your diff, it seems like you are using ebpf driver?

kind: ebpf
ebpf:
path:

But why did you clear up the path? Also, your OP showed that falcoctl tried to install kmod instead (and indeed the error is about insmod).

Is there posibility latest version of falco didn't support on Amazon Linux 2 ?

Nope, we support amazonlinux2.

[rama-akbar]

Mmmh from your diff, it seems like you are using ebpf driver?

kind: ebpf
ebpf:
path:

But why did you clear up the path? Also, your OP showed that falcoctl tried to install kmod instead (and indeed the error is about insmod).

I'm not sure actually, because chart we used is chart when we deploy falco version 0.34.1, if we deploy falco 0.34.1 it works fine.

but when we deploy latest version 0.37.1, we getting the kernel module issue

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[poiana]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

[hitsub2]

Same issue here with Bottlerocket

EKS 1.29
OS: Bottlerocket

helm install command

helm install falco falcosecurity/falco
--create-namespace \
--namespace falco \
--set tty=true \
--set collectors.kubernetes.enabled=true \ 
--set driver.kind=kmod

error logs by kubectl logs falco-4dtwz -n falco -c falco-driver-loader

* Setting up /usr/src links from host
2024-07-11 02:02:06 INFO  Running falcoctl driver config
                      ├ name: falco
                      ├ version: 7.2.0+driver
                      ├ type: kmod
                      ├ host-root: /host
                      └ repos: https://download.falco.org/driver
2024-07-11 02:02:06 INFO  Storing falcoctl driver config
2024-07-11 02:02:06 INFO  Running falcoctl driver install
                      ├ driver version: 7.2.0+driver
                      ├ driver type: kmod
                      ├ driver name: falco
                      ├ compile: true
                      ├ download: true
                      ├ target: bottlerocket
                      ├ arch: x86_64
                      ├ kernel release: 6.1.92
                      └ kernel version: #1 SMP PREEMPT_DYNAMIC Mon Jun 24 17:51:17 UTC 2024
2024-07-11 02:02:06 INFO  Check if kernel module is still loaded.
2024-07-11 02:02:06 INFO  OK! There is no module loaded.
2024-07-11 02:02:06 INFO  Check all versions of kernel module in dkms.
2024-07-11 02:02:06 INFO  OK! There are no module versions in dkms.
2024-07-11 02:02:06 INFO  Trying to download a driver.
                      └ url: https://download.falco.org/driver/7.2.0%2Bdriver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
2024-07-11 02:02:08 INFO  Driver downloaded.
                      └ path: /root/.falco/7.2.0+driver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
2024-07-11 02:02:08 WARN  Unable to insmod module.
                      ├ driver: /root/.falco/7.2.0+driver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
                      └ err: exit status 1

[FedeDP]

Hi! Since this is a recent kernel release, the preferred method is to use the modern_ebpf driver that does not need any external artifact.
In the meantime i'll investigate about why we are failing to inject kmod there; thanks for reporting!

[poiana]

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

[poiana]

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.