A Discussion of the Brilliant Level System

After posting $13$ problems to Brilliant.org, I've finally created my first $400$ point problem! The reason that I'm posting this is that it definitely should not have been the first.

I don't mean to promote my problem, but I suggest that anyone reading this should try out this problem that I posted about a week ago. It may seem unassuming at first, but I assure you that once you try it you'll shortly realize that this problem isn't for the faint of heart. If I may quote fellow Brilliant user Mr. Pi Han Goh, when he read my solution to the problem he stated "Holy Crap! This is soooooooooooooo long."

After writing this problem, I expected it to easily be a Level $5$ , worth something in the high $300$ s, but after a few days and $82$ views, I was surprised to find that it's only been rated a Level $4$ , worth a measly $160$ points.

I completely understand why it's currently rated this way. Brilliant.org rates their users and their questions based on a variation of the ELO rating system. For those of you who are unfamiliar with this system, it's a self-adjusting system of rating people (and in this case problems) based upon their interaction with other people with ratings. The way it works is that everyone is given a score to begin with. Those with higher scores are deemed "better" and those with lower scores are deemed "worse", comparatively. The system assumes that a person with a higher score is going to win if matched up against someone with a lower score. Each match affects the score of the two people involved. If two people are evenly matched, the scores of each person won't change that much after the match. If the person with the higher score wins, then that person's score goes up a little, and the person who lost goes down a little. However, if the person with the lower score wins, this is considered an "upset" and the weaker person's score will go way up, while the supposed "better" person's score will go way down.

Brilliant's users and problems follow a similar system. If a person who has a low ranking solves a really hard problem, then that person's ranking will skyrocket, while that problem's ranking will plummet. Likewise, if a person of a high rank can't solve a problem, that problem's score is likely to increase. The system really is ingenious, and seems to work beautifully most of the time. As you can probably guess, I would like to talk about the times where it doesn't work.

When I wanted to post the above problem, I happened to be ranked Level $4$ in Combinatorics, not because I couldn't solve tougher problems but because I hadn't solved enough yet to get my rank there. Brilliant prevents you from posting problems above your level of aptitude, therefore the highest level I could post the problem at was Level $4$ , which I thought was fine because eventually the ELO system would bring my problem up to the level that I thought it should be at. I thought that enough people would try the problem (and fail) that the problem's level would eventually raise to the level that I expected the problem to be at. However, after $82$ views, the problem only has had $3$ attempts, and this got me curious.

Why were people only viewing my problem and not trying to solve it? Did people try to work out the problem and then realize it was too hard and never tried to submit an answer? I personally think that that's probably unlikely. While not everyone may feel this way, I know that if I start working on a problem, I can't sleep knowing that I don't know how to solve a problem. If I don't get the problem the first time through I'll make good use of those $2$ nd and $3$ rd tries, and if by then I haven't gotten it right I'll scour the solution page looking for ways to solve the problem that I hadn't thought of, in case a problem like it comes across my feed again. So if that's not the reason, what could it be? My best guess? The problem isn't worth their time.

When you read solve a $400$ point problem, there's a sense of pride that comes with it. It feels good to know that you just solved one of the hardest problems on Brilliant.org. You're part of an elite few, the upper echelon of a certain branch of mathematics. What would you rather solve, a tough problem worth $400$ points or one only worth $160$ ? Someone who reads my problem is probably thinking "I could spend all day on this problem and get $160$ points, or I could knock out $5$ $200$ point questions in half an hour." If my problem isn't being attempted, then the ELO system can't be put into action, and so my problem sits, waiting to be attempted, ranked lower than it should be.

My question to you is, what are some changes that can be made to Brilliant's leveling system that can account for these kind of erroneous appropriations? Should the so-called "level cap" on posting problems be eliminated? What would be the implications of doing so? Is there a way of asking users who have attempted the problem whether they thought the level of the problem was appropriate? How can we get a better assessment of a problem's difficulty right from the get-go? Please feel free to leave a comment so we can discuss this issue together.

EDIT: My problem has since been simplified so that it's no longer as tedious to solve as it once was. It's now at Level $5$ hovering around $350$ points. Thanks Brilliant.org users for your helping me fix this issue!

Markdown	Appears as
`italics` or `_italics_`	italics
`bold` or `__bold__`	bold
- bulleted - list	bulleted list
1. numbered 2. list	numbered list
Note: you must add a full line of space before and after lists for them to show up correctly
paragraph 1 paragraph 2	paragraph 1 paragraph 2
`[example link](https://brilliant.org)`	example link
`> This is a quote`	This is a quote
# I indented these lines # 4 spaces, and now they show # up as a code block. print "hello world"	# I indented these lines # 4 spaces, and now they show # up as a code block. print "hello world"

Math	Appears as
Remember to wrap math in `\(` ... `\)` or `\[` ... `\]` to ensure proper formatting.
`2 \times 3`	$2 \times 3$
`2^{34}`	$2^{34}$
`a_{i-1}`	$a_{i-1}$
`\frac{2}{3}`	$\frac{2}{3}$
`\sqrt{2}`	$\sqrt{2}$
`\sum_{i=1}^3`	$\sum_{i=1}^3$
`\sin \theta`	$\sin \theta$
`\boxed{123}`	$\boxed{123}$

A Discussion of the Brilliant Level System

Comments