EZP-28176: Support for parallel and selective indexing command

[andrerom]

issue: https://jira.ez.no/browse/EZP-28176
Depends on ezsystems/ezpublish-kernel#2139

As a Developer I want a indexing command which is:

• faster at indexing (parallel indexing)
• able to refresh parts of index (by date or content id)
• able to omit purging so index is gradually refreshed on running prod setup
• able to remove index items if no longer existing (when I provide content id's to index)

Todo:

• Split out SolrCreateIndexCommand.php changes into own PR and improvement for 1.0 and up
• Re start travis once kernel PR has been merged

[MarioBlazek]

@andrerom did you maybe thought about adding option for refreshing index for subtree ?

[andrerom]

@MarioBlazek No, but that's a good idea. Things like that is very simple to add here now (on kernel side in the command there, the command here is deprecated, so I don't intend to add anything more on it here, PR's welcome tough if someone wants to port things over), but I'll definitely add that one asap ;)

[MarioBlazek]

@andrerom we've implemented that on current project. I can do PR on kernel.

[andrerom]

@MarioBlazek If so then do the PR on top of ezsystems/ezpublish-kernel#2139, you'll only need a new getStatement* method and handle the console option same way as since.

EDIT: subtree already added there now, thanks for the suggestion.

[adamwojs]

if we eliminate continue statement code will be more readable IMHO.

try {
    $info = $contentHandler->loadContentInfo($contentId);
    if ($info->isPublished) {
        $documents[] = $this->searchHandler->generateDocument(
            $contentHandler->load($contentId, $info->currentVersionNo)
        );
    }
} catch(NotFoundException $e) {
    // Catch this so we delete the index for this content below 
    $this->searchHandler->deleteContent($contentId);
}

[andrerom]

you are forgetting the non published code, it would mean:

try {
    $info = $contentHandler->loadContentInfo($contentId);
    if ($info->isPublished) {
        $documents[] = $this->searchHandler->generateDocument(
            $contentHandler->load($contentId, $info->currentVersionNo)
        );
    } else {
        $this->searchHandler->deleteContent($contentId);
    }
} catch(NotFoundException $e) {
    $this->searchHandler->deleteContent($contentId);
}

prefer this? (I'm agnostic to it tbh, but I can see this is probably slightly easier to read)

[adamwojs]

Yes, I prefer this version because it explicitly says when content is added/removed from the search index. But taking into account the forgotten case, we can stay as is 😉

[andrerom]

@alongosz Any preferences here?

[andrerom]

This and constructor suggestion fixed in 8a27d12

Also fixed both in kernel PR

[alongosz]

current version looks ok

[adamwojs]

Do we need to check if searchHandler is supported in each call of purge and updateSearchIndex?

[andrerom]

alternative is that I overload ctor

EDIT:: Or that we change IncrementalIndexer to be interface so constructor is not shared, but then loosing the BC method in the process, unless we combine it with a temporary abstract for that.

[adamwojs]

Overloading the constructor is OK for me, let's keep it simple. WDYT @alongosz?

[alongosz]

Overloading the constructor is OK for me, let's keep it simple.

+1, makes sense