Discussions

• 2 You must be logged in to vote

  💬

  Add support for STS evaluation for Cross-Encoders

  v2 Issues and PRs related to `v2` branch

  sam-hey started Jan 8, 2025 in General

  

  

  

  

  10
• 3 You must be logged in to vote

  💬

  MMTEB Author check

  KennethEnevoldsen started Sep 26, 2024 in General

  

  

  

  

  55
• 3 You must be logged in to vote

  💡

  Building a Hindi Leaderboard.

  SaileshP97 started Jan 14, 2025 in Ideas

  

  

  

  

  9
• 3 You must be logged in to vote

  💡

  Proposal: Building an Arabic Leaderboard for MTEB

  haytamdon started Jan 19, 2025 in Ideas

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  Submitting google models to leaderboard without setting up billing accounts on vertexai

  akshaykripalani asked Dec 20, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  is it allowed to use mteb/* datasets?

  gjperti asked Dec 16, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  如何使用本地下载的C-mteb的T2reranks数据集对本地下载的reranker模型进行测试？

  chaoStart asked Dec 16, 2024 in Q&A · Unanswered

  

  

  3
• 2 You must be logged in to vote

  🙏

  multi vector models?

  raghavlite asked Nov 20, 2024 in Q&A · Unanswered

  

  

  

  

  8
• 1 You must be logged in to vote

  💡

  Massive LLM2Vec testing

  BradKML started Dec 11, 2024 in Ideas

  

  

  4
• 1 You must be logged in to vote

  🙏

  Confused embed_dim across different implementation

  tsWen0309 asked Nov 25, 2024 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  Create Danish retrieval dataset from Wikipedia

  KasperGroesLudvigsen asked Oct 29, 2024 in Q&A · Unanswered

  

  

  

  

  9
• 1 You must be logged in to vote

  🙏

  How can the settings be adjusted to improve CPU utilization during MTEB execution?

  eric-yq asked Nov 7, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  IndexError: list index out of range when running evaluation on certain datasets

  isaacna asked Oct 23, 2024 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  💬

  Leaderboard Gradio or Dash?

  x-tabdeveloping started Oct 4, 2024 in General

  

  

  

  

  9
• 2 You must be logged in to vote

  💡

  Improving reproducibility: Moving to UV

  KennethEnevoldsen started Oct 6, 2024 in Ideas

  

  

  

  

  8
• 2 You must be logged in to vote

  💬

  Hacktoberfest

  isaac-chung started Oct 1, 2024 in General · Closed

  

  

  4
• 2 You must be logged in to vote

  💬

  MMTEB Benchmarks considerations

  KennethEnevoldsen started Sep 29, 2024 in General

  

  

  0
• 2 You must be logged in to vote

  💬

  Authors Missing openreview IDs

  KennethEnevoldsen started Sep 26, 2024 in General · Closed

  

  

  

  

  10
• 1 You must be logged in to vote

  🙏

  Support for reranking tasks with multi-level relevance scores?

  omihub777 asked Sep 26, 2024 in Q&A · Answered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  Long evaluation time using custom distance metric and token_embeddings for queries on retrieval task

  gnatesan asked Sep 12, 2024 in Q&A · Unanswered

  

  

  5
• 1 You must be logged in to vote

  🙏

  Cross-encoder and MTEB reranking tasks

  qiisziilbash asked Sep 10, 2024 in Q&A · Answered

  

  

  

  

  8
• 5 You must be logged in to vote

  💬

  Running models

  imenelydiaker started May 14, 2024 in General

  

  

  

  

  33
• 1 You must be logged in to vote

  💡

  Tracking CO2 Emissions

  x-tabdeveloping started May 14, 2024 in Ideas

  

  

  

  

  23
• 2 You must be logged in to vote

  💬

  Paper writing

  Muennighoff started Apr 29, 2024 in General

  

  

  

  

  60
• 1 You must be logged in to vote

  🙏

  Test run for a few good samples

  su-park asked Jul 18, 2024 in Q&A · Unanswered

  

  

  2