File ownership/group problem

[nkarthickannan]

Hi,

I have installed Magento 2 using composer. My environment is as follows:

Ubuntu 14.04
Apache 2.2
PHP 5.5.9
MySQL 5.6

"mage2" is the owner of the Magento 2 files and folders. mage2 user is included in the www-data group. The file permissions are set using this command sudo find . -type d -exec chmod 770 {} ; && sudo find . -type f -exec chmod 660 {} ; && sudo chmod u+x bin/magento

The installation goes well and I can view the website. Then I have created a sample module. After creating the module.xml file, composer.json file & registration.php file, I have enabled the module using bin/magento module:enable command.

Then I was prompted to run the bin/magento setup:upgrade command and so I did it. After this I tried to go to the Advanced configuration page in the admin panel to check whether my module is enabled or not. But I ended up with an error.

The error says "Magento is not able to create files under var/generation/Magento" directory. Then I issued the "ls -l" command inside var/generation directory. It seems the Magento directory is owned by mage2 and the group is also mage2. Then I have changed the group to www-data by this command sudo chown -R :www-data *. After running this command that error has gone.

So, my question is why Magento does not follow the file ownership & group from it's parent directory while creating new files and directories?

Thanks

[daim2k5]

From http://www.maximehuran.fr/en/install-magento-2/:

Ownership and permissions creation
To find the user you need, type this commande :

Ubuntu / Apache: ps -ef | grep apache2 Ubuntu / Nginx : ps -ef | grep nginx

In my case, the user is “www-data”. I will use this value during the next steps, change it if you have a different value.
Get the user’s group, it can be the same value as the user :
groups www-data

Go on the magento 2 root folder(/usr/share/nginx/html/magento2 or /var/www/magento2), and type this command :
cd /var/www/magento2 sudo chown -R :www-data .

On the same folder, set permissions 770 on folders, 660 on files :
sudo find . -type d -exec chmod 770 {} \; && sudo find . -type f -exec chmod 660 {} \; && sudo chmod u+x bin/magento

Add your current linux user on the www-data group :
sudo adduser <username> www-data

[nkarthickannan]

@daim2k5

Yes I followed the exact same sequence of steps. In my case also, www-data is the Apache user. I have added my current linux user to the primary group of www-data which is again www-data.

My question is different. Everytime I run the bin/magento setup:upgrade command, the var/generation/Magento directory and its contents are getting the wrong group.

I am running this command as my current linux user (karthic). So, the owner is karthic (which is correct), but the group should be www-data. Instead Magento is generating the files with karthic as group. This creates the problem.

If you are not able to understand, I'll record my screen and share the same with you.

Thanks

[daim2k5]

@karthickannan ok, the problem is that you run the command "bin/magento setup:upgrade" with the user "karthic" so the group is also "karthic". Please try to run the command as www-data user.

You find here more information how to run a command as different user
http://askubuntu.com/questions/294736/run-a-shell-script-as-another-user-that-has-no-password

[nkarthickannan]

@daim2k5

But the Magento documentation recommends to run the Command line utilities as the owner (karthic) and not as the web server user (www-data). Please refer this screen shot https://nimbus.everhelper.me/client/notes/share/337111/kCAAvOIOAZ5EQkb2qvOVUt7nODN26m8f/

The entire page can be viewed here http://devdocs.magento.com/guides/v2.0/install-gde/prereq/apache-user.html

Thanks

[Vinai]

Setting the sticky bit on pub/static and pub/media folders while they have the correct group should also work: chmod -R g+s pub/static pub/media.
As long as your system user shares the group with the web server user, new files and sub-directories will be created using the web servers group, even if that group is not the default group of your system user.

[nkarthickannan]

@Vinai

It doesn't solve this problem.

Please see the attached screen shot. This is when the var/generation/Magento directory is owned by ameex/www-data (ameex is my current username). The website is loading properly.

Now I have run the bin/magento setup:upgrade command. The ownership has now changed to ameex/ameex instead of ameex/www-data. The website is throwing error.

I have tried the chmod -R g+s pub/static pub/media command as per your reply. But it doesn't resolve this problem. Then I manually changed the ownership back to ameex/www-data by this command sudo chown -R :www-data * (issued inside the var/generation directory). After this the website is again working properly.

Thanks

[Vinai]

You have to set the sticky bit and ensure the group ownership is www-data.
If the issue is the var/generation directory, then you have to set the sticky bit there of course:
find var/generation -type d -exec chmod g+s {} \;

[nkarthickannan]

@Vinai

It seems setting the sticky bit for var/generation solves the issue. Could you please explain why this is not included in the documentation?

Thanks

[Vinai]

There always are many ways to do things depending on many factors. It is hard creating documentation that suits all situations and all knowledge levels. Using the available tools without understanding the consequences can be dangerous, too. I'm glad I'm not tasked to create the docs.

[nkarthickannan]

@Vinai

:) Anyway thank you for sorting this out. I'm closing this issue.

Thanks

[tanberry]

@Vinai thanks for helping those of us who are tasked with creating the docs! ;-)
And thanks, @karthickannan for persisting through this; we will add info to the docs about the need to set the sticky bit for var/generation, as well. And, of course, please feel free to help the DevDocs team by doing PRs and opening issues on https://github.com/magento/devdocs!

[nkarthickannan]

@tanberry

Thank you for your acknowledgement. I'm not sure where to modify this information in the https://github.com/magento/devdocs. It is having plenty of files.

If possible please let me know where should I add so that I can create a PR to add the sticky bit.

Thanks

[tanberry]

:-) For that, we will need to consult @xcomSteveJohnson, the author of the configuration docs. We will get back to you with suggested page(s) for this specific info, but in meantime take a look at http://devdocs.magento.com/guides/v2.0/extension-dev-guide/Contribute_edg.html and at http://devdocs.magento.com/guides/v2.0/howdoi/howdoi_contribute.html. Thanks!

[ghost]

Troubleshooting topic on this

[Vinai]

I have to correct my earlier comment. The chmod g+s switch sets the set-group-id bit, not the sticky bit. My bad, I confused the name.
Reference PR magento/devdocs#432

[nkarthickannan]

@xcomSteveJohnson

Thank you for adding this info in the documentation. Please let me know why this is included as a Trouble shooting option but not as a mandatory one.

I think this should be added in this page (http://devdocs.magento.com/guides/v2.0/install-gde/prereq/integrator_install.html#instgde-prereq-compose-access) along with other directory & file permissions.

If you have any valid reasons, I'm glad to hear.

Thanks

[ghost]

@karthickannan Main reason is that I have never seen this issue. I have installed Magento 2 many times on Ubuntu and CentOS. Other than this GitHub issue, I am not aware it's been reported.

That's why I put it in troubleshooting.

[davidalger]

@karthickannan IMO this wouldn't belong in the standard "do this" procedures because setting the setgid flag can also cause issues. Ex: the CLI user may no longer be able to remove the files created by the web user, even if it belongs to the group. So while this does accomplish "forcing" the group to be set correctly, it does have other side-effects. This wouldn't be an issue if, for example, php-fpm is used and is running as the CLI user managing the site, but would be an issue were mod_php used and thus the CLI user is different than the one creating the files.

[nkarthickannan]

@xcomSteveJohnson That makes sense !! Thank you

@davidalger chown -R magento:www-data * In this command, magento is the owner of the files and www-data is the group. CLI commands will be run as magento user. By setting the setgid flag, we are forcing the owner:group relationship to be the same. Hence the CLI commands will create the files with magento as owner and www-data as group.

Why owner will not be able to delete files? I don't understand your point.

Thanks

[davidalger]

@karthickannan The only issue is in possible differences in file ownership and how the sticky bit applies, and I realize I was a bit unclear…the issue is not with the setgid flag, but with the sticky bit. And I realize now (after re-reading, that is), that the sticky bit wasn't included in the recommended change, only referenced by accident. :)

With the permission setup you outlined above and php-fpm running as the magento user, all would be dandy. Files created by CLI user will be magento:www-data and files created by the PHP process will also by magento:www-data, and apache/nginx would be able to read the static files because they are grouped under www-data. File ownership for new files will be magento because the CLI user and the PHP process are running as the magento user. The group will be www-data because of the setgid behavior.

Consider an installation (same permissions setup) where the CLI user is magento and apache + mod_php are used to serve the site and they are running as www-data user and www-data group. Files created by the CLI user will be owned by magneto and grouped www-data. However, and this is where things start to go sideways, files created by the php process will be owned by www-data and grouped www-data. Note that the owner is not magento here, but www-data. The setgid flag only applies to group, not owner. There is a setuid flag to do the same for users, but I believe that only works on BSD, not on *nix systems (where it just does nothing for directories).

Given the above (i.e. second) scenario, and assuming the sticky bit was also applied, the CLI user would run into trouble renaming or deleting files created by mod_php since they are owned by www-data and the effect of the sticky-bit is to prevent those operations for anyone except the owner even in cases where group membership applies.

But, as I noted that I misunderstood that the sticky bit wasn't actually intended to be referred to above… ;)

[nkarthickannan]

@davidalger

First of all, thank you for your time in drafting such a big response :)

1. I completely understand that sticky bit was not intended. It was a minor mistake. Instead of setgid he mistakenly told sticky bit. So, we need not to worry about this point.

2. In this issue we are talking about the files under {MAGENTO_INSTALLATION_DIRECTORY}var/generation/Magento. I hope that only the CLI commands will populate the contents in this directory and not the web server. Hence the owner will always be magento (as per my example). So owner can always delete/edit them as needed.

Hope I'm clear and not confusing you.

Thanks

[sbwcws]

I m running into same issue

[ideadx]

Here is the permission sets I ran on Nginx and still getting dashboard warning error, please help:

"our web server is set up incorrectly and allows unauthorized access to sensitive files. Please contact your hosting provider."

#Update permissions:

sudo find var vendor pub/static pub/media app/etc var -type f -exec chmod u+w {} ;
sudo find var vendor pub/static pub/media app/etc var -type d -exec chmod u+w {} ;
sudo chmod u+x bin/magento

sudo find . -type d -exec chmod 770 {} ; && sudo find . -type f -exec chmod 660 {} ; && sudo chmod u+x bin/magento; sudo chmod -R g+w /var/www//{pub,var}

#update the permission to write:
sudo chown -R :www-data .

#Generate the content
sudo php bin/magento setup:static-content:deploy

[veloraven]

I'm closing this issue as GitHub issue tracker is intended for technical issues only. Please refer to the Community Forums or the Magento Stack Exchange site for technical questions.
If there are any issues which should be fixed please create new GitHub issue following the Issue reporting guidelines: with steps to reproduce, actual result and expected result. Please, also identify which version of Magento you are running.

[vtransy]

You can follow this post to set right permission for user run with nginx php
https://mrvts.wordpress.com/2018/08/11/how-to-develop-magento-2-with-nginx-without-sudo-chmod-777-for-var-pub-and-generation-directory/