How to compute smatch directly from triplets

[Wangpeiyi9979]

Hi, thanks for your nice work.
For some reason, I need to compute the smatch directly from the triples such as

[('b', ':instance', 'believe-01'),
 ('b', ':ARG1', 'c8'),
 ('c8', ':instance', 'capable-01'),
 ('c8', ':ARG2', 'i'),
 ('i', ':instance', 'innovate-01'),
 ('i', ':ARG0', 'p'),
 ('p', ':instance', 'person'),
 ('p', ':mod', 'e2'),
 ('e2', ':instance', 'each'),
 ('c8', ':ARG1', 'p'),
 ('b', ':ARG0', 'p2'),
 ('p2', ':instance', 'person')]

Could you tell me how to do this using smatch.
Thanks a lot!

[snowblink14]

You might need some customization. Depending on your input format, you may want to add some wrappers around get_best_match() function (which get the highest triple match number between two sets of triples), then compute the metrics similar to the score_amr_pairs() function in smatch.py.

[Wangpeiyi9979]

Hi, when I use the function get_best_match(), I meet a strange problem as follows:

As we can see, the best_match_num(21) > the pred_triplet_num(16).

[Wangpeiyi9979]

Also, I find that even though I fix the random seed in smatch.py, I also get different smatch scores when different runs.

[snowblink14]

Can you post the code you use to call the function? I cannot reproduce your results.
See below:

[Wangpeiyi9979]

[Wangpeiyi9979]

When I run smatch.get_best_match for the single case, it's OK

[Wangpeiyi9979]

So strange, the same case output different results

[snowblink14]

match_triple_dict is a global variable. It is cleared in score_amr_pairs() function. Since you are calling get_best_match() directly without using score_amr_pairs() as the wrapper, you need to clear this variable before computing every pair of triples.

[Wangpeiyi9979]

Thanks for your help!!!