diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9cf3609333d..ef6ce00bfbd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Fixed missing device mapping in the `allennlp.modules.conditional_random_field.py` file.
 - Fixed Broken link in `allennlp.fairness.fairness_metrics.Separation` docs
 - Ensured all `allennlp` submodules are imported with `allennlp.common.plugins.import_plugins()`.
 - Fixed `IndexOutOfBoundsException` in `MultiOptimizer` when checking if optimizer received any parameters.
diff --git a/allennlp/modules/conditional_random_field.py b/allennlp/modules/conditional_random_field.py
index c40e19359b7..7b2cb43f683 100644
--- a/allennlp/modules/conditional_random_field.py
+++ b/allennlp/modules/conditional_random_field.py
@@ -323,7 +323,7 @@ def forward(
         """
 
         if mask is None:
-            mask = torch.ones(*tags.size(), dtype=torch.bool)
+            mask = torch.ones(*tags.size(), dtype=torch.bool, device=inputs.device)
         else:
             # The code below fails in weird ways if this isn't a bool tensor, so we make sure.
             mask = mask.to(torch.bool)