Are there a lot of questions, even rated ones, that have incorrect "correct" answers?
I've noticed that there's a high percentage of questions posted here, even the ones that are rated, that even for correct answers submitted, reject them as incorrect? When a request for clarification has been made, none is hardly ever forthcoming?
Note by
Michael Mendrin
7 years, 3 months ago
No vote yet
1 vote
Easy Math Editor
This discussion board is a place to discuss our Daily Challenges and the math and science related to those challenges. Explanations are more than just a solution — they should explain the steps and thinking strategies that you used to obtain the solution. Comments should further the discussion of math and science.
When posting on Brilliant:
*italics*or_italics_**bold**or__bold__paragraph 1
paragraph 2
[example link](https://brilliant.org)> This is a quote# I indented these lines # 4 spaces, and now they show # up as a code block. print "hello world"\(...\)or\[...\]to ensure proper formatting.2 \times 32^{34}a_{i-1}\frac{2}{3}\sqrt{2}\sum_{i=1}^3\sin \theta\boxed{123}Comments
For example, just now a new question asks about the value of 0^0, and the supposed "correct" answer is 1. In fact, it is indeterminate. If this is going to be a website devoted to mathematics, shouldn't we be concerned about inaccuracies such as this?
Log in to reply
Most of them are trolls...
Hey Michael – thanks for posting this concern. We are working on a number of these issues. There are a few different things going on:
Brilliant recently provided an interface for members to create and post problems of their own, to share to their friends or people they know on Brilliant.
When a user posts a problem, it appears in the Explore section of Brilliant. We do not vet the problems that appear Explore section – the section is there for members to discover what's happening on the site and find new people to follow.
If the user's problem becomes very popular, it gets reshared by a Topic Feed (e.g., Best of Algebra). We do make an effort to check that problems in the Topic Feeds are correct, but we are a small team and don't catch everything. Our error rate is in the single digits, but we agree that is still too high. We are working on a better user-moderation system, so that users are able to flag incorrect problems and we can respond quickly to remove them. For now, if members report issues with any of the problems in the Topic Feeds, we typically fix the problems immediately.
We are also working on a troll-hunting feature, that will ban users who consistently post wrong problems, as well as improvements to the "Request for clarification" system.
Hope this helps.
Log in to reply
I know this will be a hard problem for the staffs but I think most of the users here will like to give a helping hand. I have some suggestion below:
Maybe we can give some users, probably at the "people to follow" or others who are advanced, the power to approve/disapprove the problem before they go to the explore section. So, if the problem gets some approval(I suggest three), then they will go to the explore section. Else, they will not appear in other users explore section or wait till they fixed their error and gets the approval. The team can be as many advanced user as possible so that the problems can be approve/disapprove as fast as possible. The reason I have this suggestion is because brilliant.org have worldwide users and have users online any time so that the problems can be approve/disapprove any time. Moreover, the users who are advance are less likely to make mistakes so their consideration can be taken immediately. Then, we can just see correct problems and not problems with flaws all around. I may have thought less but if you think it is a good solution or you have some improvements, you can comment! If this works well, we can vote some users to be the member of the team!
Log in to reply
Christopher, I think your idea can be thought upon by the staff.
However, I have a concern regarding the idea. If the advanced users end up solving "Explore" trolls and the problems which turn out to be far less than their level, it would not be so encouraging for them besides the fact they would have to take out some time for such trolls.
What I feel is that a significant fraction of such users would not end up solving it such questions, which is pretty obvious, I believe, which would dampen its probability of being approved by three.
Also if I am at an intermediate level I would end up gaining the most as I would not only have the reshares of 'Best Of' but also filtered explore questions.
On the other hand, the advanced users, the pioneers of Brilliant, would end up losing which I believe is not beneficial to the Brilliant community.
What I could think as a solution to this problem is firstly the underlying self realization that it is wrong on your part to post a troll. Moreover if you are uncertain about the answer, it should be rather posted as a note and not a problem.
If someone is found to be lacking such self motivation they should be penalized by probably decreasing their rating points or something like that.
Still the question remains how to identify the correctness of an answer. One of the possible solutions is that to make it compulsory to add a solution to the question when it is submitted. Frankly, I have no idea about the feasibility of such a move and I would be really glad if anyone replies with better and more viable solutions as such ideas are for the betterment of the Brilliant community.
Log in to reply
Yes I know your concerns. Before Brilliant.org changes it format, users who want to submit a problem must provide a solution too. Then, the problems will be viewed by the Challenge Master. If they think it is suitable, they will send you an email to notice you and the problem will be out in the following week.
However, it has changed its format. More and more great problems can be viewed by many users now. However, only little amount of users post problems with wrong answer, maybe because of some careless typing error or what. If these users effected this group and make all of us must provide a solution, in my opinion it will discourage someone to post problems as some solution may be long. They will need more time to only post a problem.
So from your concern, I think of another solution. The solution before still can be taken, but all the users may have a rating for their "reliance point". This means that if the user have good ratings on the "reliance point", the problem don't need to go through the approve/disapprove section. On the other hand, if one is a new user, the "reliance point" of his/her will be 0 and must go through the approve/disapprove section. If the user keeps on posting problems with correct answer, his or her "reliance point" will go up until they reach the level that don't need to go through the approve/disapprove section. If they post problems with wrong answer, their "reliance point" will go down. The flexibility of the "reliance point" can be adjusted by the staffs.
If you can add on some suggestion, please comment and make brilliant.org a better site!
Log in to reply
Yeah this would be a good idea.
Hey Christopher, thanks for these suggestions. Some form of what you're suggesting – a special Moderator class of users, and a posting reputation score – are in the works! Look forward to your feedback once we make them available.
Log in to reply
Hi @Suyeon Khim ma'am , what is this posting reputation score, has it been implemented as of yet ?
Thanks for the same :)
Those refinements or suggetions seem very reasonable. I understand that Brilliant is a still evolving website, and I can see that the central difficulty is that it wants to encourage users to have initiative in creating and posting problems, and yet when unchecked, it can discourage users from wasting a lot of time on bogus problems with bogus "correct" ansnwers. The format is to keep the solution hidden until users first try solving them, which makes it impractical for user to protest bogus "correct" answers publicly because then other users can see where the solution is going. And no small staff can possibly keep up with all problems being posted, or all the "request for clarification" coming in from disgruntled users.
Hmmm.... I don't know the best way to deal with this problem. Work in progress, I guess. Please keep up the good work.
Log in to reply
Thanks Michael!
Thanks for replying ...
I'm glad someone raised this issue; I have had the same experience. It's very frustrating when you submit an answer that is correct, but it gets rejected as incorrect. I imagine it is the same for other Brilliant users. Even worse, there are probably many users who don't even realize they actually have the right answer, and continue to struggle with the problem in vain.
By permitting any user to submit a post, the website has been opened up to many interesting and diverse mathematical problems. Unfortunately, without any vetting process, it also means that some posts will have little mathematical value, and other posts will contain errors.
The main difficulty for me is that when I come across a problem with an incorrect answer (or at least what I think is an incorrect answer), there is no good way to communicate this. The only way is when a solution just happens to be posted, and I can respond with my own thoughts. It would be nice if there was some kind of mechanism that allowed for such a discussion, without spoiling the problem for other users.
Sudeep proposes that when users post a problem, they must submit an accompanying solution as well. I think this idea is worth thinking about. It is not as onerous as it may seem; if you feel sure enough in submitting a problem, then you should know how to solve the problem in the first place. A solution would also help ensure that the answer was correct (or provide a basis for a challenge, as mentioned in the previous paragraph), and other users would benefit from seeing how to solve the problem.
The current interface is quite strict: Three strikes, and you're out. I don't think this is necessarily a bad thing. But it does mean that users who post problems should be held to the same standard as the users who try to solve problems.
I think this issue is paramount, and I hope the staff is taking it seriously. If Brilliant is going to thrive, then users must have confidence that their hard work and persistence will not be in vain.
Log in to reply
I'm happy to see that a number of users (and staff) are taking this matter seriously, because how this problem will be handled will impact the future of Brilliant. Even though I regularly tackle a slew of problems at one sitting, I actually decide on how much time I'll spend on a problem. If it intrigues me, I'll devote quite a bit of my time until I understand it thoroughly. Unfortunately, the wording of the problem can be extremely critical, where I have to sit and stare the question over and over to try to make sure I understand the intent of of the person who posted it. And then I still get it "wrong", and I find myself spending endless time trying to figure out how could I have missed it---only to finally realize that the question and the supposed answer is bogus. What that does is to make me more cynical about Brilliant and less willing to spend that much time analyzing the problem. I end up devoting my time only on problems posted by "trusted" contributors, such as staff members, and meanwhile if I don't get "you're solved it" response from an iffy contributor, I'm likely to just skip it and not bother to spend any more time on it and move on to more interesting problems elsewhere. My free time is limited, I'd like to make the best use of it and not waste it on bogus questions with bogus answers.
The most enjoyable problems are the ones that I'll "take it up the hills", where I'll think it over as I hike the trails by my house. And then I come down with a solution. But I have to have trust that I'm not just wasting my time on the problem.
Log in to reply
Dear Mr. Haussmann and Mr. Mendrin, I've made about 31 problems and both of you almost can solve all my problems that I made. I have to admit that you're both really awesome, maybe it takes years for me to reach your level. I'm just wondering about my problems that I made, are there any of them that you think incorrect?
Log in to reply
I'm still new to Brilliant. If one posts a problem, can he/she tell who's giving the correct answers? Maybe I ought to try my hand creating some problems.
I've already learned that "Tunk-Fey Ariawan" is a trusted contributor of problems, so I'm willing to spend more time finding the correct answers to your problems. I'll let you know if I disagree with any of your solutions. None so far.
Log in to reply
Even if you're still new here Sir, but I personally admit your capability in problems solving. Ya, I think it's a good idea if you post some problems. I'd love to see it and I hope I can solve it. :)
I'm really glad to know that so far I didn't make any incorrect problem. I hope I am never disagree with you Sir. :D
Mr. Ariawan, I have attempted most of your problems, so if I'm not in the list of solvers, it is probably because our answers do not match. If you wish to check, I would encourage you to post your solution.
Log in to reply
I'll post the solutions but to be more specific could you please mention which problems Sir?
I've already posted several solutions. You may see The Game of Death (Part I) solution, Romeo and Juliet solution, and The Dice Experiment solution. If there is anything you need or ask, you may send me an email to: ariawan_11@yahoo.com.
Log in to reply
Great, I will take a look.
By "high" percentage, it's in single digits, but that's still too high.
I have an idea. A popular practice in the internet market is for sellers to have a "trustworthiness" rating, based on consumer satisfaction with completed orders. Perhaps each Brilliant user could have a separate and public rating based on accuracy of his/her questions and answers, based on peer review. Isn't mathematics about peer review anyway? Many of the suggestions here sort of point towards a peer-review system anyway, as it should be.