Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Teuchos: throw exception at duplicate YAML names #2308

Merged
merged 2 commits into from
Feb 28, 2018

Conversation

ibaned
Copy link
Contributor

@ibaned ibaned commented Feb 27, 2018

[#2306]

@trilinos/teuchos

Description

Please see issue #2306 . I also did some unrelated cleanup.

Motivation and Context

@bathmatt requested that there should be an error (exception) thrown when the same name shows up twice in the same ParameterList. This is a useful debugging tool to catch user error.

Related Issues

How Has This Been Tested?

A custom build of Teuchos passed all tests.

@ibaned ibaned added the type: enhancement Issue is an enhancement, not a bug label Feb 27, 2018
@ibaned ibaned self-assigned this Feb 27, 2018
@ibaned ibaned added the stage: in progress Work on the issue has started label Feb 27, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_autotester_test

  • Build Num: 439
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 304
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 15
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Using Repos:

Repo: TRILINOS (ibaned/Trilinos)
  • Branch: issue-2306
  • SHA: 29def6b
  • Mode: TEST_REPO

Pull Request Author: ibaned

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_autotester_test

  • Build Num: 439
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 304
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 15
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA 29def6b
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a technical reason that Teuchos::Array is being removed for std::vector?

Otherwise, the changes look fine. Very straightforward.

@@ -277,8 +278,8 @@ class Reader : public Teuchos::Reader {
break;
}
case Teuchos::YAML::PROD_BMAP_FSEQ: {
TEUCHOS_ASSERT(rhs.at(4).type() == typeid(Array<Scalar>) ||
rhs.at(4).type() == typeid(Array<Array<Scalar> >));
TEUCHOS_ASSERT(rhs.at(4).type() == typeid(std::vector<Scalar>) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ibaned, why are all of these being switched over from Teuchos::Array to std::vector? Teuchos::Array just uses std::vector in non-debug checking mode but has very strong runtime checking in a debug-mode build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw the comment "I also did some unrelated cleanup" in the description field.

Unless there is some technical reason to use std::vector instead of Teuchos::Array, the usage of Teuchos::Array should be kept for safety. Even in this context, std::vector::operator is completely unchecked. And we can't turn on checked STL because we need to link against C++ TPLs that are built without checked STL turned on. For the motivation for using Teuchos::Array over raw std::vector, see section "5.5.3 Teuchos::Array" in:

If you have questions about this, then let's talk.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The motivation was to avoid having to define std::ostream& operator<<(std::ostream&, Scalar const&) and std::ostream& operator<<(std::ostream&, std::vector<Scalar>&). It is possible to access std::vector in a checked way with std::vector::at(), I can make sure to use that if thats the main issue. Otherwise if Teuchos::Array still sounds better then I can put all that back.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with std::vector::at() is that it always checks. There is no portable way to turn on and off array bounds checking with std::vector due to C++ TPLs that need to be linked in.

What is the issue with operator<<() exactly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just seems like unnecessary boilerplate (recall the same issue with Teuchos::any). I used to have empty implementations for both. I can put them back just to get Teuchos::Array working just to have toggle-able bounds checking. Most of TeuchosParser already unconditionally calls std::vector::at since I don't consider it performance-critical.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::vector::at() does not completely make std::vector safe. YOu can also easily over-step and increment std::vector iterator returned from std::vector::begin(). The motivation for Teuchos::Array as an off-by-one error I made in a code several years ago that segfaulted in a very strange place. When I ran valgrind on it, it segfaulted inside of valgrind. It took me several days to find the off-by-one error with std::vector and I only found it by staring at the code for days. Teuchos::Array would have caught that defect in 1 sec.

Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the usage of Teuchos::Array for superior debug-mode run-time checking?

@@ -277,8 +278,8 @@ class Reader : public Teuchos::Reader {
break;
}
case Teuchos::YAML::PROD_BMAP_FSEQ: {
TEUCHOS_ASSERT(rhs.at(4).type() == typeid(Array<Scalar>) ||
rhs.at(4).type() == typeid(Array<Array<Scalar> >));
TEUCHOS_ASSERT(rhs.at(4).type() == typeid(std::vector<Scalar>) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw the comment "I also did some unrelated cleanup" in the description field.

Unless there is some technical reason to use std::vector instead of Teuchos::Array, the usage of Teuchos::Array should be kept for safety. Even in this context, std::vector::operator is completely unchecked. And we can't turn on checked STL because we need to link against C++ TPLs that are built without checked STL turned on. For the motivation for using Teuchos::Array over raw std::vector, see section "5.5.3 Teuchos::Array" in:

If you have questions about this, then let's talk.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS BEEN REVIEWED, BUT NOT ACCEPTED OR REQUIRES CHANGES

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@ibaned
Copy link
Contributor Author

ibaned commented Feb 28, 2018

Teuchos::Array is back in.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_autotester_test

  • Build Num: 441
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 306
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 17
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Using Repos:

Repo: TRILINOS (ibaned/Trilinos)
  • Branch: issue-2306
  • SHA: dded496
  • Mode: TEST_REPO

Pull Request Author: ibaned

Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ibaned, thanks for making the change back to Teuchos::Array. I just have lost too many days (and weeks) of my life tracking down undefined memory errors.

I wonder if there is some way that we can use C++11 features to avoid having to create dummy operators when they don't already exist for a type to be able to use these classes?

@ibaned
Copy link
Contributor Author

ibaned commented Feb 28, 2018

@bartlettroscoe we could use the same trick I used in Teuchos::any, in Teuchos::Array. Today I came across a drawback of that trick, however. operator== is technically defined for std::vector<T>, but doesn't get instantiated unless one calls it. So the compile-time test logic I put in Teuchos::any said that std::vector<Scalar> was comparable, therefore it instantiated that function. However, Scalar itself was not comparable, and it still failed to compile. So there are subtleties there for containers of containers that are not quite resolved. But we can make things a bit more usable.

@bartlettroscoe
Copy link
Member

Can we put in recursive checks for "Is comparable"? Can the check for "is comparable" for Teuchos::Array check to see if the type T is comparable and if not return false?

@ibaned
Copy link
Contributor Author

ibaned commented Feb 28, 2018

Maybe... you may have to add a recursive definition for every existing container type, or come up with some common concept of a container they all satisfy. I'd say thats future work, and I intend to hit the merge button on this particular pull request unless someone objects.

@ibaned ibaned merged commit 2797a9a into trilinos:develop Feb 28, 2018
@ibaned ibaned deleted the issue-2306 branch February 28, 2018 17:37
@ibaned ibaned removed the stage: in progress Work on the issue has started label Feb 28, 2018
@ibaned
Copy link
Contributor Author

ibaned commented Feb 28, 2018

Hmm the autotester hasn't quite passed on the latest commit, so I did that a bit too early. Hopefully it does pass, otherwise we'll revert the merge.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_autotester_test

  • Build Num: 441
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 306
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 17
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 2308
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH issue-2306
TRILINOS_SOURCE_REPO https://github.com/ibaned/Trilinos
TRILINOS_SOURCE_SHA dded496
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA bc22b0d

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ bartlettroscoe ]!

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

@bartlettroscoe
Copy link
Member

Maybe... you may have to add a recursive definition for every existing container type, or come up with some common concept of a container they all satisfy. I'd say thats future work, and I intend to hit the merge button on this particular pull request unless someone objects.

Yes, I was not intending for this to hold up this PR. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement Issue is an enhancement, not a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants