[Feature] Add command to setup legacy settings/designs symlinks #107

andrerom · 2017-09-07T15:27:44Z

This is done for doc effort in: ibexa/documentation-developer#28
Which is from issue: https://jira.ez.no/browse/EZP-27142
Part of Epic: https://jira.ez.no/browse/EZEE-1455

Without this feature, each and every user would have to find out the hard way that
Composer sometimes wipes out the ezpublish_legacy folder, and also that versioning
this folder is not really recommended so you somehow need to deal with settings and designs.

So this command setups and symlinks the following into legacy on compser install and update:

src/legacy_files:
- designs/*
- settings/override
- settings/siteaccess/*

The feature in this PR is primary aiming at upgrade/migration use cases, where
people are coming from 5.4 or 2014.x and similar and need to move over settings and designs.

Review wanted from partners that have their own way of doing this, as you will know
better what you need. /cc @emodric @joekepley @peterkeung ..

Why no extension folder handling?

Extensions are handled by composer, and also by legacy bundles command by this bundle,
so if you need an extension in src folder you can make app bundle a legacy bundle.
Hence it felt easily confusing to give another option for this.

Why no storage and storage/packages folder handling?

As site storage folder is not typically versioned in git, I guess this should be handled by something else then src/. Input on how people tend to deal with this is more then welcome!

NOTE: But merged version of this PR contains hints to user during setup on manually handling storage folder, and with similar pointers in documentation this is somewhat covered for now, even if it is not automated (yet).

Why place them directly in `src/legacy_files`?

Opted for this mainly because placing it in bundle is more of a legacy bundles feature, and also makes sure we are forward compatible with Symfony 4/Flex's single bundle src layout.

…esigns symlinks Without this feature, each and every user would have to find out the hard way that Composer sometimes wipes out the ezpublish_legacy folder, and also that versioning this folder is not really recommended. The feature in this PR is primary aiming at upgrade/migration use cases, where people are coming from 5.4 or 2014.x and similar and need to move over settings and designs. Review wanted from partners that have their own way of doing this, as you will know better what you need. *Why no extension folder handling?* Extensions are handled by composer and also by legacy bundles, so if you need an extension in src folder you can make app bundle an legacy bundle. Hence it felt wrong to give another option for this which could be confusing. *Why no storage and storage/packages folder handling?* As site storage folder is not typically versioned, I guess this should be handled by something else. Input on how people tend to deal with this more then welcome.

emodric · 2017-09-07T16:13:41Z

I will go through the code in couple of days since I'm on a short break right now, but this looks okay to me.

At Netgen, we do something similar, with the difference that legacy_files is stored in a bundles' Resources folder, making it possible for every bundle to add it's own legacy stuff (i.e. legacy kernel patches). We also use different folders for different environments (e.g. legacy_files (used in every env), legacy_files_dev, legacy_files_prod) so we can have different configuration for different servers.

andrerom · 2017-09-07T16:58:15Z

Cool, looking forward to it 😃 Btw reason why I opted for not putting legacy files in bundle was among other things because symfony 4 has single bundle layout with no bundle dir afaik, and legacy bundles feature already allow you to place extensions and legacy stuff inside it even if it takes a bit more config + extension structure.

brookinsconsulting · 2017-09-08T14:43:26Z

+1

andrerom · 2017-09-13T12:11:40Z

@emodric had a chance to have a look yet? :)

emodric · 2017-09-13T12:25:02Z

@andrerom Nope, but will do it ASAP and let you know!

emodric · 2017-09-13T14:16:15Z

bundle/Command/LegacySrcSymlinkCommand.php

+         */
+        $filesystem = $this->getContainer()->get('filesystem');
+
+        if (!$filesystem->exists($srcArg)) {


What's the use creating the source folder structure?

Migration/Upgrade use case:

install legacy bridge (will execute this step)

manually move files over from old to new structure

So no manual steps needed for creating the folders, and it can be explicit referred to in doc where you move them as a default convention

How hard is it to type mkdir -p src/legacy_files/settings ? :) Doesn't seem that much useful to me if it's documented right, but okay, if you think it can help, sure, but I still don't like that script is running with -c automatically every time.

I'm with @emodric here on having -c as default. Leaving off -c will throw the exception and help the users who have not read the docs be aware they can and should migrate their existing config to the new location.

PR updated, also added much more output to try to explain the concepts better for user.

better? (changes done in new commit)

emodric · 2017-09-13T14:16:45Z

INSTALL.md

        "eZ\\Bundle\\EzPublishLegacyBundle\\Composer\\ScriptHandler::installAssets",
        "eZ\\Bundle\\EzPublishLegacyBundle\\Composer\\ScriptHandler::installLegacyBundlesExtensions",
-        "eZ\\Bundle\\EzPublishLegacyBundle\\Composer\\ScriptHandler::generateAutoloads"
+        "eZ\\Bundle\\EzPublishLegacyBundle\\Composer\\ScriptHandler::generateAutoloads",
+        "eZ\\Bundle\\EzPublishLegacyBundle\\Composer\\ScriptHandler::symlinkLegacySrcFiles"


Maybe just symlinkLegacyFiles ?

If we rename folder name, yes

emodric · 2017-09-13T14:17:17Z

bundle/Command/LegacySrcSymlinkCommand.php

+            ->setName('ezpublish:legacy:symlink_src')
+            ->setDefinition(
+                array(
+                    new InputArgument('src', InputArgument::OPTIONAL, 'The src directory for legacy files', 'src/legacy_files'),


How about src/legacy as the default?

I was trying to not imply it should contain legacy itself, but if this is preferred or something else I'm open for it.

emodric · 2017-09-13T14:17:47Z

bundle/Command/LegacySrcSymlinkCommand.php

+    protected function configure()
+    {
+        $this
+            ->setName('ezpublish:legacy:symlink_src')


ezpublish:legacy:symlink?

(I'm not a fan of acronyms :P)

emodric · 2017-09-13T14:18:06Z

bundle/Command/LegacySrcSymlinkCommand.php

+                    new InputArgument('src', InputArgument::OPTIONAL, 'The src directory for legacy files', 'src/legacy_files'),
+                )
+            )
+            ->addOption('create', 'c', InputOption::VALUE_NONE, 'Create src folder structure if it does not exists')


does not exist

emodric · 2017-09-13T14:19:31Z

bundle/Command/LegacySrcSymlinkCommand.php

+        $legacyRootDir = rtrim($this->getContainer()->getParameter('ezpublish_legacy.root_dir'), '/');
+
+        if ($force || !$filesystem->exists("$legacyRootDir/settings/override")) {
+            $filesystem->symlink(


Should probably check if "$srcArg/settings/override" exists before symlinking.

emodric · 2017-09-13T14:20:03Z

bundle/Command/LegacySrcSymlinkCommand.php

+            $symlinks[] = "$legacyRootDir/settings/override";
+        }
+
+        $directories = ['design', 'settings/override'];


settings/siteaccess ?

yes, wrong string here, override is handled above :)

emodric · 2017-09-13T14:24:07Z

@andrerom Left a couple of remarks, but in general, this looks okay.

What I would like to definitely see is symlinking based on Symfony environment used when running the script (src/legacy_files_dev, src/legacy_files_prod and src/legacy_files (for all environments)).

peterkeung · 2017-09-13T18:12:24Z

"Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)
We don't depend on Composer for deployments and don't use it on production environments. We use it only on dev to pull in code. So we do end up versioning ezpublish_legacy.
Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

emodric · 2017-09-14T07:13:07Z

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

Symlinking design folder is simpler than creating and activating the extension together with design.ini.append.php. For simpler usecases, symlinking designs to ezpublish_legacy/design is a welcome feature.

andrerom · 2017-09-14T07:17:48Z

"Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)

From slack #ezplatfomr-use on september 5th leading up to this PR:

One issue I know of is if the composer.json setting ezpublish-legacy-dir was missing, which can happen when you switch between branches to one not setting it. However as of legacy-installer v2.0.4 that is set by default now, so unless you set it to other setting it should always be set: https://github.com/ezsystems/ezpublish-legacy-installer/releases/tag/v2.0.4 (edited)

[16:48]
Oh, there is one other case. When composer removes the folder and does a new checkeout, for instance moving from dist to source or opposite. Also might happen on dist updates somtimes

Long story short: use symlinks or similar for var, settings, extensions and design folders. And add a step in your composer post-update / post-install step which makes sure they are setup.

Not experienced it? I guess if you don't use composer in prod (which is a good thing, I guess you then have deployment pipeline for instance involving composer when rolling out new code, anyway) it won't be an issue for you.

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

I assumed some people use it given it was never deprecated in favour of extensions, as for extensions they are as said already handled by other features of this bundle.

Maybe I should get the folder creation step to place a readme in legacy_folder directory to explain what it is for and not.

andrerom · 2017-09-20T12:48:35Z

updated. Regarding folder creation, maybe create on install? the point is to have a meaningful output to users and also to avoid additional steps. Also for the sake of trying to keep the documentation on this straight forward without to much if's and buts :)

emodric · 2017-09-20T13:52:01Z

@andrerom If you think it can help, you can leave it as is. Since you don't have to use the command in your stack if empty folder structure bothers you, this should be okay :)

andrerom · 2017-09-20T17:35:33Z

Ok, then who else should we get to review this then?

emodric · 2017-09-22T13:34:55Z

Ok, then who else should we get to review this then?

@wizhippo maybe ?

wizhippo

Besides my thought on the default creation of of the src directory structure above I'm fine with this as is.

I too am curious how others are handling storage and should it be addressed, perhaps in another PR. I'm glad I have had backups a couple of times.

andrerom · 2017-09-28T11:37:00Z

I too am curious how others are handling storage and should it be addressed, perhaps in another PR. I'm glad I have had backups a couple of times.

yes, at the very least would need to alert the user when we detect that folder being removed (overwritten by symlink) is not empty, like now reported in #110.

…at is going on

glye · 2017-10-03T09:59:18Z

bundle/Command/LegacySrcSymlinkCommand.php

+any design/settings project files, and symlinks these into <info>ezpublish_legacy/</info> which is installed by composer.
+
+The benefit of this is:
+1. Being able to version your design/config files in git withouth versing legacy itself


versing -> versioning

glye

(more nitpicks)

glye · 2017-10-03T11:52:32Z

bundle/Command/LegacySrcSymlinkCommand.php

+Aborting: The src directory "$srcArg" does not exist.
+
+You can create the directory by running <info>ezpublish:legacy:symlink -c</info>, OR by creating the folders you need
+manually among the once supported by this command:


once -> ones

glye · 2017-10-03T11:54:19Z

bundle/Command/LegacySrcSymlinkCommand.php

+        if ($symlinkFolderStr) {
+            $output->writeln("The following folders where symlinked: '$symlinkFolderStr'.");
+        } else {
+            $output->writeln('No folders where symlinked, use force option if they need to be re-created.');


May be overkill, but: --force, to clarify?

andrerom · 2017-10-03T12:41:45Z

Thanks all for reviews!

Merging as I assume the only remaining concerns by @emodric and @wizhippo is now handled given structure is no longer automatically created, but instead instructions for how to do it is outputted (and it is covered in migration doc).

andrerom force-pushed the legacy_src_symlink branch from 96faf7a to ef7a3c2 Compare September 7, 2017 15:38

ezsystems deleted a comment from ezrobot Sep 7, 2017

andrerom mentioned this pull request Sep 11, 2017

EZP-27142: Update migration guide for 1.11 and legacy bridge ibexa/documentation-developer#28

Merged

4 tasks

emodric reviewed Sep 13, 2017

View reviewed changes

andrerom added 2 commits September 20, 2017 14:33

Fixup for review comments

77ddde6

Fix #85: assets should be symlinked before being dumped

19bc4d2

wizhippo reviewed Sep 22, 2017

View reviewed changes

Change to not create folders by defalt, and add much more ouput on wh…

ffa53c9

…at is going on

ezsystems deleted a comment from ezrobot Oct 2, 2017

andrerom force-pushed the legacy_src_symlink branch from 95e2b72 to ffa53c9 Compare October 2, 2017 11:36

andrerom requested a review from glye October 3, 2017 09:04

glye reviewed Oct 3, 2017

View reviewed changes

typos

5005027

glye approved these changes Oct 3, 2017

View reviewed changes

Improving output

157da20

andrerom merged commit 5dd5ecf into master Oct 3, 2017

andrerom deleted the legacy_src_symlink branch October 3, 2017 12:41

[Feature] Add command to setup legacy settings/designs symlinks #107

[Feature] Add command to setup legacy settings/designs symlinks #107

Conversation

andrerom commented Sep 7, 2017 • edited Loading

Why no extension folder handling?

Why no storage and storage/packages folder handling?

Why place them directly in src/legacy_files?

emodric commented Sep 7, 2017

andrerom commented Sep 7, 2017 • edited Loading

brookinsconsulting commented Sep 8, 2017

andrerom commented Sep 13, 2017

emodric commented Sep 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrerom Sep 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrerom Sep 13, 2017 • edited Loading

Choose a reason for hiding this comment

emodric commented Sep 13, 2017

peterkeung commented Sep 13, 2017

emodric commented Sep 14, 2017

andrerom commented Sep 14, 2017 • edited Loading

andrerom commented Sep 20, 2017 • edited Loading

emodric commented Sep 20, 2017 • edited Loading

andrerom commented Sep 20, 2017

emodric commented Sep 22, 2017

wizhippo left a comment

Choose a reason for hiding this comment

andrerom commented Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

glye left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrerom commented Oct 3, 2017 • edited Loading

andrerom commented Sep 7, 2017 •

edited

Loading

Why place them directly in `src/legacy_files`?

andrerom commented Sep 7, 2017 •

edited

Loading

andrerom Sep 13, 2017 •

edited

Loading

andrerom Sep 13, 2017 •

edited

Loading

andrerom commented Sep 14, 2017 •

edited

Loading

andrerom commented Sep 20, 2017 •

edited

Loading

emodric commented Sep 20, 2017 •

edited

Loading

andrerom commented Sep 28, 2017 •

edited

Loading

andrerom commented Oct 3, 2017 •

edited

Loading