Feat: content in sub folders #2897

erezrokah · 2019-11-19T17:29:47Z

Fixes: #513

~~Still work in progress, though I tested with github/gitlab/bitbucket.~~

Enable it by adding content_in_sub_folders: true to your folder collection (still need to write the documentation).
By enabling the flag slashes will not be sanitized and will be used for directory structure.
e.g doing:

The slug option remains the same.
You can add a path template for a collection similar to the slug generation template:

folder: 'posts'
slug: '{{month}}-{{day}}-{{slug}}'
path: '{{year}}/{{slug}}'

Will result in the file saved in posts/2019/11-19-title.md for example.

The slug key in the path template is the post format value of the slug.

TODO:

Think about filtering items when reading them from the repository
Think if I really need all the encoding/decoding
Fix GraphQL query to get nested trees
Use path config instead of content_in_sub_folders and keep slug behaviour
UI improvements
Test with open authoring
Only sanitize template variables. BREAKING CHANGE - fixes Filename sanitation should only apply to variable parts #1254

Things to consider:

POSSIBLE BREAKING CHANGE - folder collections entries are now retrieved up to a depth of 10 for all the backends. If someone relied on the CMS retrieving only up to a depth of 1, they might experience unwanted behavior.
Filtering entities on extraction - feels like a separate issue that can be done once we have more use cases.

Further improvements (I think they should be done in another PR):

internally we should not use slug in our redux store (and everywhere else for that matter). We should use the entry full path and just URI encode it to make sure it doesn't break the routing.
We should also not reference the term slug in the backends (specifically the GitHub one).

tomrutgers · 2019-11-19T17:36:41Z

Question: do we need the content_in_sub_folders property? The slashes in the slug already imply that right?

erezrokah · 2019-11-19T18:02:25Z

Question: do we need the content_in_sub_folders property? The slashes in the slug already imply that right?

That would make it a breaking change I think

chrfritsch · 2019-11-20T08:32:06Z

Would a site structure like the following also be handled by this?

index.md
contact.md
teams
- team1.md
- team2.md
aboutus.md

erezrokah · 2019-11-20T08:42:13Z

Would a site structure like the following also be handled by this?

index.md

contact.md

teams

team1.md

team2.md

aboutus.md

I think you'll need to set up two collections, one for the teams nested directory and one for the parent, then define a filter for each collection (We're still thinking on how to do the filtering as the current mechanism is very limited).

chrfritsch · 2019-11-20T08:50:27Z

Uhh, that's so not nice TBH. Creating a site hierarchy is part of the editors' work (not developers) and should not depend on changing the cms config.

erezrokah · 2019-11-20T09:00:27Z

Uhh, that's so not nice TBH. Creating a site hierarchy is part of the editors' work (not developers) and should not depend on changing the cms config.

Actually this might be possible with a single collection if you use a field with the slug template.
e.g. {{fields.slug}} then have the editor update that field in the UI.

Another improvement will be to preview the destination path in the UI (like suggested in the issue thread).

chrfritsch · 2019-11-20T11:38:22Z

Uhh, that's so not nice TBH. Creating a site hierarchy is part of the editors' work (not developers) and should not depend on changing the cms config.

Actually this might be possible with a single collection if you use a field with the slug template.
e.g. {{fields.slug}} then have the editor update that field in the UI.

Another improvement will be to preview the destination path in the UI (like suggested in the issue thread).

Ok, I will give it a try. Thank you. :)

barthc · 2019-11-24T23:29:42Z

@erezrokah path preview:

And of course would need some UI improvements.

barthc · 2019-11-26T11:22:55Z

packages/netlify-cms-core/src/components/Editor/EditorControlPane/EditorControlPane.js

@@ -55,6 +56,7 @@ export default class ControlPane extends React.Component {

    return (
      <ControlPaneContainer>
+        {collection.has('path') && <PathPreview collection={collection} entry={entry} />}


Would be nice if the path preview can come immediately before the identifier field.

Good idea I'll update it

barthc · 2019-11-26T11:27:25Z

packages/netlify-cms-core/src/selectors/entryDraft.js

+  try {
+    // slugFormatter throws in case an identifier is missing from the entry
+    // we can safely ignore this error as this is just a preview path value
+    return slugFormatter(collection, entry.get('data'), config.get('slug'));


Would be nice to add the file extension.

barthc · 2019-11-26T22:15:16Z

packages/netlify-cms-backend-github/src/GraphQLAPI.js

@@ -186,6 +186,22 @@ export default class GraphQLAPI extends API {
    }
  }

+  addFiles(allFiles, entries, path) {


array reduce would be more cleaner.

I was thinking something like so:

function addFiles(entries, path) { return entries.reduce((acc, item) => { if (item.type === 'tree') { return [...acc, ...addFiles(item.object.entries, `${path}/${item.name}`)]; } return [ ...acc, { ...item, path: `${path}/${item.name}`, size: item.blob && item.blob.size } ]; }, []); }

barthc · 2019-11-27T09:24:22Z

packages/netlify-cms-backend-github/src/queries.js

@@ -92,17 +93,47 @@ export const statues = gql`
  ${fragments.object}
 `;

+const buildFilesQuery = (depth = 10) => {
+  const PLACE_HOLDER = 'PLACE_HOLDER';


Good thinking with the PLACE_HOLDER 👍

erezrokah · 2019-11-27T10:17:38Z

BREAKING CHANGE: non template parts of the slug will not be sanitized

erquhart · 2019-11-27T14:09:56Z

Two issues with displaying the path:

most nontechnical users won’t need the info and may be confused by it
multiple fields could be involved in path formation

What’s more likely to be sought by an editor is the resulting URL path. We could handle this in a separate issue/PR and make some product decisions there.

What do you all think, agree/disagree?

_{Sent with GitHawk}

erezrokah · 2019-11-27T14:19:07Z

Two issues with displaying the path:

most nontechnical users won’t need the info and may be confused by it

multiple fields could be involved in path formation

What’s more likely to be sought by an editor is the resulting URL path. We could handle this in a separate issue/PR and make some product decisions there.

What do you all think, agree/disagree?

Sent with GitHawk

I agree the preview is very raw atm. Having the resulting URL path sounds much better

erezrokah · 2019-11-27T16:21:52Z

Removed the path preview completely for now

erquhart

Killer work, Erez 💪 - let's test for the possible breaking change you mentioned, but I'm good with merging and publishing to beta as is.

erquhart · 2019-11-28T04:13:19Z

Actually, second thought, I'm going to hold off on releasing since we're going into holiday, and many have come to depend on our long lived beta channel not breaking. If folks here really want to play with it over the next couple of days, make some noise.

chrfritsch · 2019-11-28T04:54:37Z

I think many people, including me, are waiting for this feature. So please publish it 😊💪🏻🔥🚀

flyfoxuk · 2019-11-28T09:05:44Z

Yes please, would like to test it

earthboundkid · 2019-12-19T16:24:24Z

Can this be turned off? It sort of broke my content organization.

hulloanson · 2020-12-21T12:01:18Z

Edit: rephrased

I have a repo structured like this:

/
- blog 1
  - index.md
  - audio.mp3
  - image.jpg
- blog 2
  - index.md
  - another_audio.mp3
  - more_audio.mp3
  - even_more_audio.mp3

etc...

and the following collection configuration (irrelevant fields not listed):

- folder: "/"
  slug: "{{slug}}"
  path: "{{slug}}/index"
  media_folder: ""
  public_folder: ""

Is it the intended way to use this feature? If not, how should I author on netlify cms with this structure?

I tried using the feature without success of course. Description of what happened below if you're interested.

------- Here is the experiment results ----------

Expected behavior:

Lists entries like "blog 1", "blog 2", etc.

Actual behavior:

Produced something like this:

Before going any further though, I do understand the empty entries are media files. I probably need to filter them.

The problem is: every time I scrolled to the bottom, all entries with text updated to the same name it happened to load from the last request. Such entries all have the link `https://example.com/#/collections/bites/entries/index

My uneducated guess was the "/index" I wrote in path / the "index.md" in each sub-folder messed with redux store keys but I have not studied the implementation... yep. that's all I've tried. just sharing. my concern is still on how I should author with the file structure I proposed.

Thank you!

erezrokah · 2020-12-21T19:36:42Z

Hi @huguestennier, I would suggest moving your content into a dedicated directory, e.g. content and then use:

- folder: "content"
  slug: "{{slug}}"
  path: "{{slug}}/index"
  media_folder: ""
  public_folder: ""

If that doesn't work I suggest opening a new issue for it.

hulloanson · 2020-12-22T06:43:37Z

Thank you @erezrokah ! It works if I put contents in a subfolder :) I don't even have to filter media files!

Amazing work!

erezrokah changed the title ~~Feat: content in sub folders~~ [WIP] Feat: content in sub folders Nov 19, 2019

erezrokah force-pushed the feat/content_in_sub_folders branch 2 times, most recently from 984a745 to d4e71ee Compare November 26, 2019 10:20

barthc reviewed Nov 26, 2019

View reviewed changes

erezrokah changed the title ~~[WIP] Feat: content in sub folders~~ Feat: content in sub folders Nov 26, 2019

barthc reviewed Nov 26, 2019

View reviewed changes

erezrokah force-pushed the feat/content_in_sub_folders branch from 36eceaf to 9f0dd4e Compare November 27, 2019 09:14

barthc reviewed Nov 27, 2019

View reviewed changes

erezrokah added 12 commits November 27, 2019 12:20

refactor: extract slug formatting into backendHelper

043f7a0

feat: add content_in_sub_folders option and support in test backend

875feaa

fix: decode slug when formatting commit message

92cbf6b

feat(backend-github): get files recursively

80a7689

fix: properly decode/encode path/slug

053bedc

feat(backend-gitlab-bitbucket): get files recursively

6d110e8

fix: don't re-sanitize slug when entry exists

e2cdabc

fix(backend-github): fix recursive flag(v4 API)

4f250cc

fix: encode/decode slug only during navigation

8eaaf69

feat: use path config instead of content_in_sub_folders

498d5bb

fix(backend-github): fix parsing of slug when loading entries

d97be99

feat: add path preview to editor

6688cde

erezrokah added 10 commits November 27, 2019 12:21

fix: show extension in path preview

44b70cb

fix: consider collection name when parsing content key

70effa4

feat(github-graphql): use nested query to retrieve files

740c4c2

test(cypress): update GitHub GraphQL mock data

cf6faaa

feat(editor-ui): put path preview next to identifier field

17db8b9

fix(open-authoring): update open authoring slug parsing

ecf05fd

refactor: use array reduce instead of recursion

3d180ec

feat: only sanitize template variables

f883558

BREAKING CHANGE: non template parts of the slug will not be sanitized

refactor: update getAllFiles method

84c89d9

chore: code cleanup

10ad13d

erezrokah force-pushed the feat/content_in_sub_folders branch from 451661f to 10ad13d Compare November 27, 2019 10:30

fix: remove path preview from UI

1368473

erquhart approved these changes Nov 28, 2019

View reviewed changes

erquhart merged commit afcfe5b into master Nov 28, 2019

erquhart deleted the feat/content_in_sub_folders branch November 28, 2019 03:39

barthc mentioned this pull request Nov 30, 2019

fix: keep editor slug path #2934

Merged

IanKBovard mentioned this pull request Dec 3, 2019

Verify Netlify-CMS works with current MVP build hpe-dev-incubator/hpe-dev-portal#39

Closed

2 tasks

nathankitchen pushed a commit to nathankitchen/netlify-cms-backend-azure that referenced this pull request Feb 24, 2020

feat: content in sub folders (decaporg#2897)

daeea6b

vladdu pushed a commit to vladdu/netlify-cms that referenced this pull request Jan 26, 2021

feat: content in sub folders (decaporg#2897)

1054210

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: content in sub folders #2897

Feat: content in sub folders #2897

erezrokah commented Nov 19, 2019 •

edited

Loading

tomrutgers commented Nov 19, 2019

erezrokah commented Nov 19, 2019

chrfritsch commented Nov 20, 2019

erezrokah commented Nov 20, 2019

chrfritsch commented Nov 20, 2019

erezrokah commented Nov 20, 2019 •

edited

Loading

chrfritsch commented Nov 20, 2019

barthc commented Nov 24, 2019

barthc Nov 26, 2019

erezrokah Nov 26, 2019

barthc Nov 26, 2019

barthc Nov 26, 2019

barthc Nov 27, 2019

erezrokah Nov 27, 2019

barthc Nov 27, 2019

erezrokah commented Nov 27, 2019

erquhart commented Nov 27, 2019

erezrokah commented Nov 27, 2019 •

edited

Loading

erezrokah commented Nov 27, 2019

erquhart left a comment

erquhart commented Nov 28, 2019 •

edited

Loading

chrfritsch commented Nov 28, 2019

flyfoxuk commented Nov 28, 2019

earthboundkid commented Dec 19, 2019

hulloanson commented Dec 21, 2020 •

edited

Loading

erezrokah commented Dec 21, 2020

hulloanson commented Dec 22, 2020 •

edited

Loading

Feat: content in sub folders #2897

Feat: content in sub folders #2897

Conversation

erezrokah commented Nov 19, 2019 • edited Loading

TODO:

Things to consider:

Further improvements (I think they should be done in another PR):

tomrutgers commented Nov 19, 2019

erezrokah commented Nov 19, 2019

chrfritsch commented Nov 20, 2019

erezrokah commented Nov 20, 2019

chrfritsch commented Nov 20, 2019

erezrokah commented Nov 20, 2019 • edited Loading

chrfritsch commented Nov 20, 2019

barthc commented Nov 24, 2019

barthc Nov 26, 2019

Choose a reason for hiding this comment

erezrokah Nov 26, 2019

Choose a reason for hiding this comment

barthc Nov 26, 2019

Choose a reason for hiding this comment

barthc Nov 26, 2019

Choose a reason for hiding this comment

barthc Nov 27, 2019

Choose a reason for hiding this comment

erezrokah Nov 27, 2019

Choose a reason for hiding this comment

barthc Nov 27, 2019

Choose a reason for hiding this comment

erezrokah commented Nov 27, 2019

erquhart commented Nov 27, 2019

erezrokah commented Nov 27, 2019 • edited Loading

erezrokah commented Nov 27, 2019

erquhart left a comment

Choose a reason for hiding this comment

erquhart commented Nov 28, 2019 • edited Loading

chrfritsch commented Nov 28, 2019

flyfoxuk commented Nov 28, 2019

earthboundkid commented Dec 19, 2019

hulloanson commented Dec 21, 2020 • edited Loading

erezrokah commented Dec 21, 2020

hulloanson commented Dec 22, 2020 • edited Loading

erezrokah commented Nov 19, 2019 •

edited

Loading

erezrokah commented Nov 20, 2019 •

edited

Loading

erezrokah commented Nov 27, 2019 •

edited

Loading

erquhart commented Nov 28, 2019 •

edited

Loading

hulloanson commented Dec 21, 2020 •

edited

Loading

hulloanson commented Dec 22, 2020 •

edited

Loading