Problems with incorrect answers

We’ve seen a number of issues raised by members regarding problems with incorrect answers. We have been doing a lot of work behind the scenes to make flagging and moderation much better on Brilliant. I wanted to share our philosophy behind the work, and let you know what to expect in the coming weeks.

Philosophy

We started Brilliant to create a community for ambitious minds, where members could create and share interesting problems, notes, and sets. We kickstarted that community by developing a small arsenal of problems that we liked and thought were interesting, and publishing them to Brilliant for anyone who wanted to solve and discuss those problems. However, we knew that the best problems that our small team could create would be no match for the best problems created by a community of hundreds of thousands of active, very smart members.

Part of our task is to surface the best items for you. Like Tumblr, anyone can post anything on Brilliant, and we can’t guarantee that it will all be good. But we can put measures in place to quickly hide problems and notes that are irrelevant, spam, or wrong, and highlight the ones that are excellent and deserve to be shared.

How moderation will work

Near-term

When multiple users flag a problem, the problem will automatically display a banner that says “This problem may not be trustworthy”.
- If the author believes that the problem is correct and it should not have been flagged, the author will be able to ask a Moderator to arbitrate the dispute.
- The author may acknowledge that the problem is incorrect, and either delete the problem, or edit the problem and ask a Moderator to remove the banner. All users whose ratings changed on the problem will have their ratings automatically restored.
- If the author does not take any action within a few days, all users whose ratings changed on the problem will have their ratings automatically restored. The author may still get the banner removed, edit the problem, or delete the problem, but further actions will not affect the ratings of the people who tried the problem.

Mid-term

We will offer qualified users god-like powers to moderate various domains of Brilliant.
Moderators will have 1-click ability to ban or promote items, and the ability to ban users. You will be rewarded by a fresh feeling each morning when you wake up to make the world a more orderly and informed place, early access to new features, and some other awesome secret perks.
We will notify users in a notifications menu about ratings restorations from incorrect problems.

Markdown

Appears as

*italics* or _italics_

italics

**bold** or __bold__

bold


- bulleted
- list

bulleted
list


1. numbered
2. list

numbered
list

Note: you must add a full line of space before and after lists for them to show up correctly

paragraph 1

paragraph 2

paragraph 1

paragraph 2

[example link](https://brilliant.org)

example link

> This is a quote

This is a quote

    # I indented these lines
    # 4 spaces, and now they show
    # up as a code block.

    print "hello world"

# I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"

Math

Appears as

Remember to wrap math in $ ... $ or \[ ... \] to ensure proper formatting.

2 \times 3

2 \times 3

2^{34}

2^{34}

a_{i-1}

a_{i-1}

\frac{2}{3}

\frac{2}{3}

\sqrt{2}

\sqrt{2}

\sum_{i=1}^3

\sum_{i=1}^3

\sin \theta

\sin \theta

\boxed{123}

\boxed{123}

Easy Math Editor

This discussion board is a place to discuss our Daily Challenges and the math and science related to those challenges. Explanations are more than just a solution — they should explain the steps and thinking strategies that you used to obtain the solution. Comments should further the discussion of math and science.

When posting on Brilliant:

Use the emojis to react to an explanation, whether you're congratulating a job well done , or just really confused .
Ask specific questions about the challenge or the steps in somebody's explanation. Well-posed questions can add a lot to the discussion, but posting "I don't understand!" doesn't help anyone.
Try to contribute something new to the discussion, whether it is an extension, generalization or other idea related to the challenge.
Stay on topic — we're all here to learn more about math and science, not to hear about your favorite get-rich-quick scheme or current world events.

Markdown	Appears as
`italics` or `_italics_`	italics
`bold` or `__bold__`	bold
- bulleted - list	bulleted list
1. numbered 2. list	numbered list
Note: you must add a full line of space before and after lists for them to show up correctly
paragraph 1 paragraph 2	paragraph 1 paragraph 2
`[example link](https://brilliant.org)`	example link
`> This is a quote`	This is a quote
# I indented these lines # 4 spaces, and now they show # up as a code block. print "hello world"	# I indented these lines # 4 spaces, and now they show # up as a code block. print "hello world"

Math	Appears as
Remember to wrap math in `$` ... `$` or `\[` ... `\]` to ensure proper formatting.
`2 \times 3`	$2 \times 3$
`2^{34}`	$2^{34}$
`a_{i-1}`	$a_{i-1}$
`\frac{2}{3}`	$\frac{2}{3}$
`\sqrt{2}`	$\sqrt{2}$
`\sum_{i=1}^3`	$\sum_{i=1}^3$
`\sin \theta`	$\sin \theta$
`\boxed{123}`	$\boxed{123}$

Comments

Instead of removing problems, it will be better if the moderator can edit the problem's answer or wording. This is because some problem aren't really bad, just maybe the author have a typo or gives a wrong answer.

Christopher Boo - 7 years, 2 months ago

Oh that's absolutely a fact. There've been some otherwise really great problems posted here, but were ruined by ambiguous wording of the problem stated. A lot of the times, I find myself thinking of how I'd have posed the problem better, improving on it.

Michael Mendrin - 7 years, 2 months ago

I totally agree

Eddie The Head - 7 years, 2 months ago

Cool. It would be nice if brilliant picks us to be brilliant moderators,

Beakal Tiliksew - 7 years, 2 months ago

Agreed.

Priyansh Sangule - 7 years, 2 months ago

This is great. I've usually been able to tell if a problem is wrong if it's not pretty looking, so I'm cool with that. Who will be chosen to be moderators? Will it be people who are trustworthy and very active on Brilliant and have good LaTeX and not very high levels and who are very friendly and are comfortable with technology and always have smiley faces at the end of their sentences and who follow LOTS of people and who have the initials F. H.? Because if so... :D

Finn Hulse - 7 years, 2 months ago

Yea, it would be soooo exciting if we could be picked as moderators. For me, it would be like a dream come true! But the idea of me being able to ban users at the click of the button still feels like too much responsibility; I would be content if I could just help with spreading good problems and removing bad problems. $\ddot\smile$

Daniel Liu - 7 years, 2 months ago

If I get picked as a moderator, it would a dream come true for me too.

You shouldn't worry about the ban button. If you're not comfortable with using it, you're not obligated to. In fact, it should only be used in extreme cases (remember _?). I do have a suggestion about this. I think that all bans should be reviewed by the Brilliant team just as a safety net. I agree that the moderators are trustworthy, but it's always good to make sure if, for example, someone else got access to your account.

Trevor B. - 7 years, 2 months ago

@Trevor B. – I agree with your suggestion.

However, I have not heard of __. Mind telling me a story?

@Daniel Liu – Sure!

When this user solved problems (and if he even solved them legitimately I wasn't sure), he would almost always post a solution containing only the answer to the question. For example, on Chung Gene Keun's problem to find the digit sum of the digit sum of the first 50,000,000 digits of $e,$ (something like that, I don't remember the exact problem), the answer was 413 (not really, but no spoilers!), so the user wrote a solution: "413." No analysis, no methodology, no (in the case of the problem) lines of code, no anything, which was infuriating because it was a really hard problem. I never figured out how to do it myself, but I would have been really interested in figuring it out. Repeat this 200+ times and you have what _ did. He was subsequently banned.

@Trevor B. – I see the name ___ in the recent solvers list sometimes but no profile picture shows up.. After reading your comment, I tried to click on the name, but Brilliant says this page does not exist. But if we hover our mouse on his name, we see his profile summary. We can still follow him. Quite curious, don't you think?

Ameya Salankar - 7 years, 2 months ago

@Ameya Salankar – Hmmm....

@Finn Hulse – I shudder to think how it would be to be banned from Brilliant!

@Trevor B. – Prof Premraj reminds me a little bit of Dr Sonnhard Graubner in AoPS. I don't know if you've heard of Art of Problem solving before?

@Daniel Liu – I've heard of AOPS, but I've never used it. It must have been really annoying there as well.

It's kind of strange that I never created an account. At Mu Alpha Theta Nationals last summer, the guy who created it gave a speech during the opening ceremonies and all of the people at my school that used AOPS were staring at him in admiration. But I'm not going to blame them. If Calvin gave a speech at a math competition I was in, I'd stare in admiration myself.

@Trevor B. – They have a really cool feature called "For The Win" which is modeled off of the MATHCOUNTS Countdown round. It's really great. My account name is FINNN.

@Trevor B. – Have you seen Calvin's picture on the "About" page? And Aaron's?

I was wondering that how do you guys come across these "bad problems"?

Pranav Arora - 7 years, 2 months ago

@Pranav Arora – Usually you find them in the "Explore" section. People go there to look for good problems to share (and I need to do it more myself). But in addition to the good problems that you'll find there, you are going to find the problems saying that Jack had 8 marbles after Jill took away 7 marbles from his collection of 16.

...You will be rewarded by a fresh feeling each morning when you wake up to make the world a more orderly and informed place...

I can just imagine Brilliant.org hacking into your brain, and automatically releasing a "fresh feeling" every time you wake up. :P

Ha ha :-D

Ashtik Mahapatra - 7 years, 2 months ago

Maam,, I have noticed a bug in brilliant, Hope u will solve it. Every time, I open any problem, I see more number of ""Reveal __ Solutions"" Than the actually solution. For instance, while solving a problem, Before solving it, I saw ,, Reveal 5 solutions, But when I solved it, It showed only 2 solutions, Hope, this bug might be solved

Dinesh Chavan - 7 years, 2 months ago

I've seen this too.

Excuse me, Sue, would you mind checking out this post? It's a feature request that I think is really cool. :D

This is just great. Can't wait to see how Brilliant evolves.

Yeah it is.

I have a feeling that only real popular members would be chosen as moderators. Oh well... now I would recommend Brilliant more than ANY other website.

Where is this notifications menu???

Being picked as a moderator is like a dream come true!!! :D But there are plenty of people in this community who are wayyyyyyyyyy more talented than me.....

the correct answer to this problem is 4331 -but the sum in the question is 11 and not 10

Logical Arithmetic Warmup

1 2 3 4 5

Try again

I am thinking of a 4-digit number. The sum of the digits in my number is 10 and the product of the digits is 36. Also, the first digit is three larger than the last digit.

What number am I thinking of?

Oops. The answer is 4331.

17% of people got this right.

Median time to solve: 1 minute

This problem is moderately challenging for a Level 1 in Logic (you).

P Post - 5 years, 1 month ago

I have sent many notes concerning errors in scoring, answers etc but these issues have not been addressed. What is happening?

E Koh - 1 year, 3 months ago

Math	Appears as
Remember to wrap math in `\(` ... `\)` or `\[` ... `\]` to ensure proper formatting.
`2 \times 3`	$2 \times 3$
`2^{34}`	$2^{34}$
`a_{i-1}`	$a_{i-1}$
`\frac{2}{3}`	$\frac{2}{3}$
`\sqrt{2}`	$\sqrt{2}$
`\sum_{i=1}^3`	$\sum_{i=1}^3$
`\sin \theta`	$\sin \theta$
`\boxed{123}`	$\boxed{123}$