Browser profile gets corrupted with eCryptFS

[slaecker]

I'm on a fresh install of Arch Linux with KDE and encrypted home directory.
What I did after initial setup (https://wiki.archlinux.org/index.php/Profile-sync-daemon#Setup_and_installation):

1. Set USE_OVERLAYFS="yes" in ~/.config/psd/psd.config
2. Launched qupzilla to get the initial directory ~/.config/qupzilla/, changed some settings and quit the browser.
3. Launched PSD via systemctl --user start psd
4. Profile was synced to /run/user/1000 and linked back to .config correctly
5. Launched qupzilla again

An error message was displayed saying:
Incopatible profile version ... and the browser profile was reset to default settings. I then tried the same steps with Google Chrome and experienced the same, after enabling PSD an error message is shown and the profile is reset to default.

I only start and stop PSD manually (service not enabled) because I thought maybe home encryption was the source of the issue (though I read you implemented compatibility).

% uname -a
Linux slaecklap 4.3.3-3-ARCH #1 SMP PREEMPT Wed Jan 20 08:12:23 CET 2016 x86_64 GNU/Linux

% cat ~/.config/psd/psd.conf | grep -Ev '(^#|^$)' 
USE_OVERLAYFS="yes"

% systemctl --user -l status psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon

Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync: send_files failed to open "/home/slaecker/.config/qupzilla/profiles/default/networkcache/538.1-Qt5.5.1/data8/c/2ztfkkml.d": No such device or address (6)
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync: send_files failed to open "/home/slaecker/.config/qupzilla/profiles/default/networkcache/538.1-Qt5.5.1/data8/c/35m3zc5l.d": No such device or address (6)
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync: send_files failed to open "/home/slaecker/.config/qupzilla/profiles/default/networkcache/538.1-Qt5.5.1/data8/d/13iujbum.d": No such device or address (6)
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync: send_files failed to open "/home/slaecker/.config/qupzilla/profiles/default/networkcache/538.1-Qt5.5.1/data8/d/3gganc0m.d": No such device or address (6)
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync: send_files failed to open "/home/slaecker/.config/qupzilla/profiles/default/networkcache/538.1-Qt5.5.1/prepared/Lh4757.d": No such device or address (6)
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: IO error encountered -- skipping file deletion
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: qupzilla resync successful
Jan 23 12:06:40 slaecklap profile-sync-daemon[4806]: qupzilla unsync successful
Jan 23 12:06:40 slaecklap systemd[595]: Stopped Profile-sync-daemon.

% psd p
Profile-sync-daemon v6.21

 Systemd service is currently unknown.
 Systemd resync-timer is currently unknown.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/slaecker/.config/psd/psd.conf:

 browser/psname:  qupzilla/qupzilla
 owner/group id:  slaecker/1001
 sync target:     /home/slaecker/.config/qupzilla
 tmpfs dir:       /run/user/1000/slaecker-qupzilla
 profile size:    16M
 overlayfs size:  
 recovery dirs:   none

[graysky2]

Interesting... I am not expert with encryption. How are you doing it? LUKS? Can you try it without overlayfs to rule out something there?

1. Stop psd
2. Edit the config to not use overlayfs
3. Make a fresh profile and setup
4. Start psd
5. Try the browser, verify the sync works
6. Quit the browser
7. Restart psd
8. Launch again... still good?

owner/group id: slaecker/1001

Also, is your group ID really 1001? Can you post the output of id

[slaecker]

Indeed after disabling overlayfs everything looks good. Someone should file a bug report for overlayfs ;)

% id
uid=1000(slaecker) gid=1001(slaecker) [...]

[graysky2]

Now, can you repeat the 8 steps above except re-enable overlayfs? Show that this is in fact reproducible. If it is, I'm not sure how or why the encryption you're using would mess up overlayfs. Can you provide more details about your encryption?

[slaecker]

Yes, it is reproducible.

1. Stop psd
2. Edit the config to re-enable overlayfs
3. Start psd
4. Profile is "destroyed", qupzilla moaning again about incompatible profile version
5. Quit the browser
6. Edit the config to disable overlayfs
7. Restart psd
8. Launch browser again, change some settings
9. Restart psd
10. All good (in fact I'm using PSD without overlayfs since my last comment stable and reliable with different browsers)

For encrypting my home folder I followed the steps in the Arch Wiki:
https://wiki.archlinux.org/index.php/ECryptfs#Encrypting_a_home_directory
# ecryptfs-migrate-home -u username
Afterwards https://wiki.archlinux.org/index.php/ECryptfs#Auto-mounting

[graysky2]

Thank you for repeating to verify. I will have to re-create your encrypted setup to troubleshoot further. Can you verify that the problem is true with another browser as well? This would help to rule out something specific to qupzilla and suggest something in the psd process is breaking with this particular type of encryption.

[slaecker]

As written in the initial bug report I also confirmed The issue with Google Chrome so it's not qupzilla specific.

[graysky2]

As written in the initial bug report I also confirmed The issue with Google Chrome so it's not qupzilla specific.

Sorry, missed that. That implies the latter from my previous comment. I will need to spin up a VM with your setup. I cannot currently commit to this given lack of free time but I will keep this open.

[Smurv]

I don't know if it will help, but I have same experience. I'm on Debian Testing and it started yesterday, after update to 4.5.0-1 kernel. I also have USE_OVERLAYFS="yes" and Firefox profile broke up. Also Chromium is complaining about profile (something like "can't read your setting") when PSD is running. When I set USE_OVERLAYFS="no" everything is working fine.

[kepi]

Isn't it possible that you have btrfs as you filesystem? I had huge problems with profile-sync-daemon and overlayfs on btrfs.

[graysky2]

@kepi - If that is the case, can you detail these problems? If general, a warning should be featured in the manpage and wiki.

@slaecker - Do you have btrfs for your /home?

[kepi]

I have whole system on btrfs and tried to use psd with overlayfs. It took me about 30 computer freezes :D and corrupted disk before I find out that only thing that I changed (beside of new laptop) was overlayfs in psd :)

You can find some info in https://bugzilla.redhat.com/show_bug.cgi?id=1255512

I'm not sure if this was already resolved in upstream, but it seams not. And if yes, it will take long time before everybody will have new kernel.

I think warning would be great and maybe check in psd even better, because it is probably easy to check if btrfs is present on filesystem where you are using overlayfs.

Sorry for not reporting this earlier, It happend to me about 2 months ago but I had so many problems with skylake and freezes that I completely forgot about this issue.

[graysky2]

I tried btrfs a while ago and lost data withit on a non-graceful reboot. Never again.

[graysky2]

Is this a Fedora-only bug?

[slaecker]

From my side at the time when I experienced this issue I didn't habe btrfs,
I was on ext4/encfs on Arch Linux. Meanwhile I'm on openSUSE with xfs/luks
and have no issues.

Am 20.08.2016 12:47 schrieb "graysky" [email protected]:

Is this a Fedora-only bug?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#158 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIL051h_JCxDmcaTATx7CZPYOcVwYJdVks5qhts3gaJpZM4HK66R
.

[kepi]

@graysky2 no, I have Debian. Its probably problem in brfs itself. But It apparently doesn't have to be connected to original problem reported here as at least slaecker doesn't have btrfs. Probably better to open new issue for it.

[vizzah]

Got my profiles wiped today upon first psd run on Ubuntu Xenial with eCryptfs home.

psd failed to create any backups (shouldn't it do them before applying overlayFs, which seems to be culprit here).

System info:

Linux xps 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:01:55 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux

Profile-sync-daemon v6.23 on Ubuntu 16.04.1 LTS

Sep 23 19:58:55 xps systemd[1561]: Reloading.
Sep 23 19:58:59 xps systemd[1561]: Started Profile-sync-daemon.
Sep 23 19:58:59 xps systemd[1561]: Starting Timed resync...
Sep 23 19:58:59 xps profile-sync-daemon[6529]: No status data could be sent: $NOTIFY_SOCKET was not set
Sep 23 19:58:59 xps profile-sync-daemon[6529]: #33[01mgoogle-chrome sync successful#033[00m
Sep 23 19:58:59 xps profile-sync-daemon[6529]: #33[01mfirefox sync successful#033[00m
Sep 23 19:58:59 xps profile-sync-daemon[6529]: #33[01mfirefox sync successful#033[00m
Sep 23 19:58:59 xps systemd[1561]: Started Timed resync.

Once Firefox (or Chrome) was started, it crashed immediately:

Sep 23 20:00:02 xps kernel: [211576.693936] BUG: unable to handle kernel NULL pointer dereference at 0000000000000030
Sep 23 20:00:02 xps kernel: [211576.693964] IP: [] ecryptfs_privileged_open+0x51/0x1b0
Sep 23 20:00:02 xps kernel: [211576.693986] PGD 0
Sep 23 20:00:02 xps kernel: [211576.693994] Oops: 0000 [#1] SMP
Sep 23 20:00:02 xps kernel: [211576.694005] Modules linked in: overlay sunrpc binfmt_misc veth xt_CHECKSUM iptable_mangle xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack bridge stp llc iptable_filter ip_tables x_tables ipheth bnep pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) bbswitch(OE) vboxdrv(OE) hid_multitouch nls_iso8859_1 snd_hda_codec_hdmi dell_led snd_hda_codec_realtek i2c_designware_platform i2c_designware_core snd_hda_codec_generic dell_wmi mxm_wmi dell_laptop dcdbas intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_intel kvm_intel snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_seq_midi snd_seq_midi_event irqbypass snd_rawmidi snd_seq snd_seq_device snd_timer joydev cdc_ether serio_raw usbnet snd r8152 mii soundcore brcmfmac mei_me input_leds brcmutil cfg80211 rtsx_pci_ms mei memstick idma64 virt_dma intel_lpss_pci processor_thermal_device shpchp intel_soc_dts_iosf hci_uart btbcm btqca btintel bluetooth int3403_thermal wmi dell_smo8800 int3402_thermal int340x_thermal_zone intel_lpss_acpi intel_lpss int3400_thermal acpi_thermal_rel intel_hid acpi_pad acpi_als mac_hid sparse_keymap kfifo_buf industrialio parport_pc ppdev lp parport autofs4 drbg ansi_cprng algif_skcipher af_alg dm_crypt hid_generic usbhid rtsx_pci_sdmmc crct10dif_pclmul crc32_pclmul i915_bpo aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd intel_ips i2c_algo_bit psmouse drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci nvme drm rtsx_pci libahci i2c_hid hid pinctrl_sunrisepoint video pinctrl_intel fjes [last unloaded: nvidia]
Sep 23 20:00:02 xps kernel: [211576.694507] CPU: 7 PID: 6821 Comm: firefox Tainted: P W OE 4.4.0-36-generic #55-Ubuntu
Sep 23 20:00:02 xps kernel: [211576.694530] Hardware name: Dell Inc. XPS 15 9550/0N7TVV, BIOS 01.02.10 06/30/2016
Sep 23 20:00:02 xps kernel: [211576.694550] task: ffff88021c265880 ti: ffff8800212b0000 task.ti: ffff8800212b0000
Sep 23 20:00:02 xps kernel: [211576.694583] RIP: 0010:[] [] ecryptfs_privileged_open+0x51/0x1b0
Sep 23 20:00:02 xps kernel: [211576.694608] RSP: 0018:ffff8800212b3bb0 EFLAGS: 00010246
Sep 23 20:00:02 xps kernel: [211576.694622] RAX: ffff8800212b3bd8 RBX: ffff8802681a9a68 RCX: ffff88021b224e40
Sep 23 20:00:02 xps kernel: [211576.694639] RDX: ffffffff821d03e0 RSI: ffffffff81cb1a9e RDI: ffff8800212b3bd0
Sep 23 20:00:02 xps kernel: [211576.694658] RBP: ffff8800212b3c20 R08: ffff8802bddd9f40 R09: ffff8802ad30ab90
Sep 23 20:00:02 xps kernel: [211576.694676] R10: ffff88027b818b40 R11: 0000000000000000 R12: 0000000000000000
Sep 23 20:00:02 xps kernel: [211576.694694] R13: ffff88021b224e40 R14: 0000000000000000 R15: ffff8801c89d5010
Sep 23 20:00:02 xps kernel: [211576.694712] FS: 00007f249013d740(0000) GS:ffff8802bddc0000(0000) knlGS:0000000000000000
Sep 23 20:00:02 xps kernel: [211576.694732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 23 20:00:02 xps kernel: [211576.694747] CR2: 0000000000000030 CR3: 0000000021117000 CR4: 00000000003406e0
Sep 23 20:00:02 xps kernel: [211576.694765] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 23 20:00:02 xps kernel: [211576.694783] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Sep 23 20:00:02 xps kernel: [211576.694801] Stack:
Sep 23 20:00:02 xps kernel: [211576.694810] ffff8800212b3bf8 ffffffff812276bb ffffffff811ebaa7 ffffffff00000000
Sep 23 20:00:02 xps kernel: [211576.694832] ffff880200000000 ffff8800212b3bd8 ffff8800212b3bd8 0000000000000000
Sep 23 20:00:02 xps kernel: [211576.694854] 0000000000000024 0000000085f23c80 ffff8802681a9800 ffff8802681a9a38
Sep 23 20:00:02 xps kernel: [211576.694876] Call Trace:
Sep 23 20:00:02 xps kernel: [211576.694885] [] ? inode_init_always+0x10b/0x1b0
Sep 23 20:00:02 xps kernel: [211576.694902] [] ? kmem_cache_alloc+0x187/0x1f0
Sep 23 20:00:02 xps kernel: [211576.694918] [] ecryptfs_get_lower_file+0x7d/0xe0
Sep 23 20:00:02 xps kernel: [211576.694936] [] ecryptfs_open+0x71/0x200
Sep 23 20:00:02 xps kernel: [211576.694951] [] do_dentry_open+0x1ff/0x310
Sep 23 20:00:02 xps kernel: [211576.694966] [] ? ecryptfs_release+0x30/0x30
Sep 23 20:00:02 xps kernel: [211576.694982] [] vfs_open+0x54/0x80
Sep 23 20:00:02 xps kernel: [211576.694996] [] path_openat+0x1b7/0x1330
Sep 23 20:00:02 xps kernel: [211576.695011] [] ? ecryptfs_d_revalidate+0x41/0x70
Sep 23 20:00:02 xps kernel: [211576.695028] [] do_filp_open+0x91/0x100
Sep 23 20:00:02 xps kernel: [211576.695043] [] ? __alloc_fd+0x46/0x190
Sep 23 20:00:02 xps kernel: [211576.695058] [] do_sys_open+0x138/0x2a0
Sep 23 20:00:02 xps kernel: [211576.695073] [] SyS_open+0x1e/0x20
Sep 23 20:00:02 xps kernel: [211576.695087] [] entry_SYSCALL_64_fastpath+0x16/0x71
Sep 23 20:00:02 xps kernel: [211576.695103] Code: c2 e0 03 1d 82 49 89 cd 65 48 8b 04 25 28 00 00 00 48 89 45 d8 31 c0 48 8d 45 90 c7 45 a8 00 00 00 00 48 8d 78 20 e8 bf 0c db ff <49> 8b 44 24 30 48 89 5d 90 4c 89 65 a0 4c 89 75 98 48 8b 40 28
Sep 23 20:00:02 xps kernel: [211576.695206] RIP [] ecryptfs_privileged_open+0x51/0x1b0
Sep 23 20:00:02 xps kernel: [211576.695225] RSP
Sep 23 20:00:02 xps kernel: [211576.695234] CR2: 0000000000000030
Sep 23 20:00:02 xps kernel: [211576.701506] ---[ end trace 46031f3f464a2558 ]---

Rsync has also crashed (and that apparently wiped profile files), but psd received successful status:

Sep 23 20:01:24 xps kernel: [211659.426569] Code: c2 e0 03 1d 82 49 89 cd 65 48 8b 04 25 28 00 00 00 48 89 45 d8 31 c0 48 8d 45 90 c7 45 a8 00 00 00 00 48 8d 78 20 e8 bf 0c db ff <49> 8b 44 24 30 48 89 5d 90 4c 89 65 a0 4c 89 75 98 48 8b 40 28
Sep 23 20:01:24 xps kernel: [211659.427502] RIP [] ecryptfs_privileged_open+0x51/0x1b0
Sep 23 20:01:24 xps kernel: [211659.428447] RSP
Sep 23 20:01:24 xps kernel: [211659.429324] CR2: 0000000000000030
Sep 23 20:01:24 xps kernel: [211659.430198] ---[ end trace 46031f3f464a255c ]---
Sep 23 20:01:24 xps profile-sync-daemon[7828]: /usr/bin/profile-sync-daemon: line 387: 7945 Killed rsync -aX --delete-after --inplace --no-whole-file --exclude .flagged "$DIR/" "$BACK_OVFS/"
Sep 23 20:01:24 xps profile-sync-daemon[7828]: #33[01mfirefox resync successful#033[00m

I believe there must be a warning that the tool is not wholly tested with eCryptfs and there is high chance of data loss.

[graysky2]

I believe there must be a warning that the tool is not wholly tested with eCryptfs and there is high chance of data loss.

Agreed. a434cb6 and 3df065a

Glad to hear you had some backups. Would like to chase down this bug.

[vizzah]

Perfect - thanks!

(I also posted on github so you have a track of the issue)

I believe there must be a warning that the tool is not wholly
tested with eCryptfs and there is high chance of data loss.

Agreed.
a434cb6
and
3df065a

Glad to hear you had some backups. Would like to chase down this bug.