Tpetra: transferAndFillComplete fails

[cgcgcg]

Bug Report

@trilinos/tpetra

Description

EMPIRE has reported an issue where transferAndFillComplete fails from within MueLu's rebalancing:

Trilinos/packages/muelu/src/Rebalancing/MueLu_RebalanceAcFactory_def.hpp

Lines 104 to 105 in 07a6632

	XpetraList.set("Timer Label","MueLu::RebalanceAc-" + Teuchos::toString(coarseLevel.GetLevelID()));
	rebalancedAc = MatrixFactory::Build(originalAc, *rebalanceImporter, *rebalanceImporter, targetMap, targetMap, rcp(&XpetraList,false));

It seems that the problem is that originalAc is block diagonal and hence has no importer.

Trilinos/packages/tpetra/core/src/Tpetra_CrsMatrix_def.hpp

Lines 8891 to 8953 in 07a6632

	bool bSameDomainMap = BaseDomainMap->isSameAs (*getDomainMap ());
	if (! restrictComm && ! MyImporter.is_null () && bSameDomainMap ) {
	// Same domain map as source matrix
	//
	// NOTE: This won't work for restrictComm (because the Import
	// doesn't know the restricted PIDs), though writing an
	// optimized version for that case would be easy (Import an
	// IntVector of the new PIDs). Might want to add this later.
	Import_Util::getPids (*MyImporter, SourcePids, false);
	}
	else if (restrictComm && ! MyImporter.is_null () && bSameDomainMap) {
	// Same domain map as source matrix (restricted communicator)
	// We need one import from the domain to the column map
	IntVectorType SourceDomain_pids(getDomainMap (),true);
	IntVectorType SourceCol_pids(getColMap());
	// SourceDomain_pids contains the restricted pids
	SourceDomain_pids.putScalar(MyPID);
	SourceCol_pids.doImport (SourceDomain_pids, *MyImporter, INSERT);
	SourcePids.resize (getColMap ()->getNodeNumElements ());
	SourceCol_pids.get1dCopy (SourcePids ());
	}
	else if (MyImporter.is_null () && bSameDomainMap) {
	// Matrix has no off-process entries
	SourcePids.resize (getColMap ()->getNodeNumElements ());
	SourcePids.assign (getColMap ()->getNodeNumElements (), MyPID);
	}
	else if ( ! MyImporter.is_null () &&
	! domainTransfer.is_null () ) {
	// general implementation for rectangular matrices with
	// domain map different than SourceMatrix domain map.
	// User has to provide a DomainTransfer object. We need
	// to communications (import/export)
	// TargetDomain_pids lives on the rebalanced new domain map
	IntVectorType TargetDomain_pids (domainMap);
	TargetDomain_pids.putScalar (MyPID);
	// SourceDomain_pids lives on the non-rebalanced old domain map
	IntVectorType SourceDomain_pids (getDomainMap ());
	// SourceCol_pids lives on the non-rebalanced old column map
	IntVectorType SourceCol_pids (getColMap ());
	if (! reverseMode && ! xferDomainAsImport.is_null() ) {
	SourceDomain_pids.doExport (TargetDomain_pids, *xferDomainAsImport, INSERT);
	}
	else if (reverseMode && ! xferDomainAsExport.is_null() ) {
	SourceDomain_pids.doExport (TargetDomain_pids, *xferDomainAsExport, INSERT);
	}
	else if (! reverseMode && ! xferDomainAsExport.is_null() ) {
	SourceDomain_pids.doImport (TargetDomain_pids, *xferDomainAsExport, INSERT);
	}
	else if (reverseMode && ! xferDomainAsImport.is_null() ) {
	SourceDomain_pids.doImport (TargetDomain_pids, *xferDomainAsImport, INSERT);
	}
	else {
	TEUCHOS_TEST_FOR_EXCEPTION(
	true, std::logic_error, "Tpetra::CrsMatrix::"
	"transferAndFillComplete: Should never get here! "
	"Please report this bug to a Tpetra developer.");
	}

I removed the check for bSameDomainMap on

Trilinos/packages/tpetra/core/src/Tpetra_CrsMatrix_def.hpp

Line 8914 in 07a6632

else if (MyImporter.is_null () && bSameDomainMap) {

since that is the only case with null importer, and this fixed the EMPIRE issue.
Could someone from the Tpetra team confirm that this is a good fix?

(All Tpetra tests pass on my system.)

[cgcgcg]

@jhux2 @bathmatt

[cgcgcg]

The added test sems to pass with UVM off.