Accessing [CLS] Token Embeddings

[reinaldncku]

Hello,

I would like to access the embedding that corresponds to the [CLS] token. This is my code, by the way.

from flair.embeddings import BertEmbeddings
from flair.data import Sentence, Token
from pytorch_pretrained_bert import BertTokenizer

bert = BertEmbeddings(layers="-1", pooling_operation="first")

sample_txt = "Hello there i am here"
sentence = Sentence(sample_txt)
embedded_sample_txt = bert.embed(sentence)

print (embedded_sample_txt)

However, when I try to access the BERT-embedded text, [CLS] is not included there. This is the output:

[Sentence: "Hello there i am here" - 5 Tokens]

May I know how to I access/show the [CLS] token? Thanks!

[LorrinWWW]

the same question!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alanakbik]

You can now get the embedding from the CLS token using the new TransformerDocumentEmbeddings class (see #1494). It's part of master branch now and will be in the next Flair release.