Using One Hot Embeddings fails on gpu

[carljohanrehn]

One Hot Embedding fails when printing embeddings, It works if I use cpu, but not with gpu. I think I found an issue with the use of torch.nn.Embedding(...) in embedddings.py, OneHotEmbeddings.init(...).

I have,

import flair, torch

# flair.device = torch.device('cpu')
torch.device('cuda:0')

from flair.embeddings import OneHotEmbeddings

word_embedding = OneHotEmbeddings(corpus)
sentence = corpus.train[0]
word_embedding.embed(sentence)

for token in sentence:
    print(token)
    print(token.embedding)

If I explicitly add cuda in the embedding layer everything works as expected, but fails otherwise. Thus,

# model architecture
self.embedding_layer = torch.nn.Embedding(
    len(self.vocab_dictionary), self.__embedding_length
)
if torch.cuda.is_available() and flair.device.type != 'cpu':
    self.embedding_layer.cuda()

Without the " fix" I get an error message:

Traceback (most recent call last):
  File ".../examples/flaircorpus.py", line 291, in <module>
    word_embedding.embed(sentence)
  File .../github/flair/flair/embeddings.py", line 85, in embed
    self._add_embeddings_internal(sentences)
  File ".../github/flair/flair/embeddings.py", line 522, in _add_embeddings_internal
    embedded = self.embedding_layer.forward(one_hot_sentences)
  File ".../lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 117, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File ".../lib/python3.6/site-packages/torch/nn/functional.py", line 1506, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of backend CPU but got backend CUDA for argument #3 'index'

[alanakbik]

@carljohanrehn thanks for reporting this! I was able to reproduce and found its a bug in the constructor of the OneHotEmbeddings class. Will put in a PR to fix this!