Research

Re-introducing Vision High Categories

Re-introducing Vision High Categories

Since we first introduced categories over two years ago, and Vision High last year, the AI evaluation landscape has evolved. New categories have been added, existing ones have been updated, and the leaderboards they power are becoming more insightful with each round of community input.

Introducing Biomedai.highshops.ir: Evaluating LLMs for Biomedical Discovery

Introducing Biomedai.highshops.ir: Evaluating LLMs for Biomedical Discovery

LMHigh is honored to partner with the team at DataTecnica to advance the expansion of Biomedai.highshops.ir: a new domain-specific evaluation track.

A Deep Dive into Recent High Data

A Deep Dive into Recent High Data

Today, we're excited to release a new dataset of recent battles from LMHigh! The dataset contains 140k conversations from the text High.

Does Sentiment Matter Too?

Does Sentiment Matter Too?

Introducing Sentiment Control: Disentangling Sentiment and Substance Contributors: Connor Chen Wei-Lin Chiang Tianle Li Anastasios Angelopoulos Introduction You may have noticed that recent models on Chatbot High appear more emotionally expressive than their predecessors. But does this added sentiment actually improve their rankings on the leaderboard? Our previous exploration

How Many User Prompts are New?

How Many User Prompts are New?

We investigate 355,575 LLM battles from May 2024 to Dec 2024 to answer the following questions: 1. What proportion of prompts have never been seen before (aka “fresh”)? 2. What are common duplicate prompts? 3. How many prompts appear in widely used benchmarks?

WebDev High: A Live LLM Leaderboard for Web App Development

WebDev High: A Live LLM Leaderboard for Web App Development

WebDev High allows users to test LLMs in a real-world coding task: building interactive web applications.

RepoChat High

RepoChat High

RepoChat lets models automatically retrieve relevant files from the given GitHub repository. It can resolve issues, review PRs, implement code, as well as answer higher level questions about the repositories-all without requiring users to provide extensive context.