Recent changes in the solution submission system

First of all, I'd like to say that I think it is great that you are trying to improve the submission system to be the best as can be. However, you've made some decisions for which I'd like a motivation:

  • Two weeks ago, voting started after three days. Last week, voting was possible right from the beginning, but the votes were hidden. I had my doubts about this because it partly brought back the timezone problem, as early solutions were more likely to get the most votes. It seemed to work pretty well though. But this week, the vote count is again visible right from the start. The only thing different from a few weeks ago is the fact that they're shown in random order. This brings back the timezone issues people have been complaining about, so I'd like to know why you chose to use this, as opposed to what you had.
  • From this week on, we can only submit one problem per problem set. Why? I'd like to submit multiple solutions per problem set to practice my proof-writing, and if I can't make my solutions visible, I'm not very motivated to write them in the first place. Also, I don't think you've put it very clear, as it says "You may publish one solution per week.", which wasn't clear to me. Per problem? Per set? In total? I just assumed it was per problem. After that, I put a lot of effort in another solution, and only after I'd written it, it said that I can't publish it, because I had already written one this week.

Also, I've seen quite some questions about this system from other users, without an answer by the staff:

  • Why can't we edit our solutions? I see it a lot that the solution writer finds a mistake in his own work and just puts it in a comment, while it would be much neater if he could edit his original solution. Of course, copying other solutions shouldn't be encouraged, but what you could do about this, is only allowing a certain amount of characters that's edited, or the possibility to submit an edit request rather than edit it right away, so a staff member can decide whether the edit may be put into effect or not.
  • Do we get points for correct solutions? I remember that somewhere it said you can get up to 1000 points for a correct solution, and you can be deducted 50 points for a wrong solution, but that's the last time I heard of it. When I go back in the past problems tab to the first week of the renewed system, I see that you haven't chosen featured solutions (which is good I suppose, as people get to see multiple ways of solving a problem) and I have no idea whether I got points for my solutions or not.
  • A few weeks ago, after you submit the right answer, it showed the key techniques for that problem, but that's gone now. Will it come back? If it will, then it would be cool if you can spend points in order to see those techniques for a problem that you haven't solved yet, in order to figure out how you can approach it best.
  • As I pointed out a while back, there seemed to be a limit on the number of solutions that could be submitted per problem, and last week, it was still there. Now that we can only submit one solution per problem set, it's not very likely that a problem will get over 50 solutions, but the limit shouldn't be there.

Here are some other issues I encountered with the new system:

  • Sometimes I try to upvote a comment, but it doesn't do anything. Other times, it does work. It's pretty strange.
  • The solution box is way too small to begin with. I know I can enlargen it, but after I preview my solution and go back to editing, the box is small again. This is pretty annoying. This also applies to the box I'm typing in right now, it doesn't remember its size.

To conclude, some feature requests:

  • The most requested feature probably is the ability to discuss a problem rather than submit and discuss a solution. Although being able to submit solutions and discuss those with others is fantastic, discussing the problem itself might be even cooler.
  • Before I submit an answer to a problem, it says "Show xx solutions", but after I submit the right answer, it doesn't say how many solutions have been submitted, while I'd like to use this number as an indicator of whether it's worth it to publish mine. I'd like to suggest a box on the right similar to the Solved by xx users box, but then for who submitted solutions, and more importantly perhaps, how many solutions have been submitted.
  • It would be handy if the checkmark in the green circle would be filled with some color if I wrote a solution for that particular problem, to make it easy to see for which I haven't done so yet, and to easily check how my solutions are doing, whether they have comments/notes, etc.

That's it for now. I'm sure you have good reasons for the choices you've made, but please, communicate more often with your userbase.

#FeatureRequests

Note by Tim Vermeulen
7 years, 10 months ago

No vote yet
37 votes

  Easy Math Editor

This discussion board is a place to discuss our Daily Challenges and the math and science related to those challenges. Explanations are more than just a solution — they should explain the steps and thinking strategies that you used to obtain the solution. Comments should further the discussion of math and science.

When posting on Brilliant:

  • Use the emojis to react to an explanation, whether you're congratulating a job well done , or just really confused .
  • Ask specific questions about the challenge or the steps in somebody's explanation. Well-posed questions can add a lot to the discussion, but posting "I don't understand!" doesn't help anyone.
  • Try to contribute something new to the discussion, whether it is an extension, generalization or other idea related to the challenge.
  • Stay on topic — we're all here to learn more about math and science, not to hear about your favorite get-rich-quick scheme or current world events.

MarkdownAppears as
*italics* or _italics_ italics
**bold** or __bold__ bold

- bulleted
- list

  • bulleted
  • list

1. numbered
2. list

  1. numbered
  2. list
Note: you must add a full line of space before and after lists for them to show up correctly
paragraph 1

paragraph 2

paragraph 1

paragraph 2

[example link](https://brilliant.org)example link
> This is a quote
This is a quote
    # I indented these lines
    # 4 spaces, and now they show
    # up as a code block.

    print "hello world"
# I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"
MathAppears as
Remember to wrap math in \( ... \) or \[ ... \] to ensure proper formatting.
2 \times 3 2×3 2 \times 3
2^{34} 234 2^{34}
a_{i-1} ai1 a_{i-1}
\frac{2}{3} 23 \frac{2}{3}
\sqrt{2} 2 \sqrt{2}
\sum_{i=1}^3 i=13 \sum_{i=1}^3
\sin \theta sinθ \sin \theta
\boxed{123} 123 \boxed{123}

Comments

Yes, I completely agree. I wrote my first solution, and submitted it without seeing the "one solution per week" part. I think my solution for another problem would be very interesting to share and get feedback on; but I cannot submit it. The reasoning for this decision might be that too many solutions are submitted, leaving novel and well-written solutions unseen, but right now it might be causing more problems than it solved.

One additional thing that might be helpful are individual problem forums, where the problem can be discussed, including possible extensions, advice on solving similar problems, etc.

Daniel Chiu - 7 years, 10 months ago

Log in to reply

While there may be some (or many) flawed solutions, remember that brilliant as a community are not all (prospective) math majors/professors. Allowing only one solution per problem allows the people who are less experienced in the language of math to have a chance to write a solution and see how they do, while the people who are more experienced write solutions for the more challenging problems.

In the state it was before, there would be a bunch of "inferior" solutions being put at the bottom and not discussed while the same few people who had superior solutions shined at the top, leaving everyone else in the dust.

It's all a learning experience, we are not in a competition to see who does the best. That is also probably why the challenge masters (or just Calvin L., who knows) are replying much more actively this week. This way it is much more of a "discussion" rather than "tell me the answer please."

Michael Tong - 7 years, 10 months ago

Log in to reply

Wait was this in response to Ivan? I don't really see the connection with my comment.

Daniel Chiu - 7 years, 10 months ago

Log in to reply

@Daniel Chiu Yes, I think this one was meant for me. And that was exactly my entire point, which is why I agree with the whole one-solution-per-week/set (or whatever) concept.

I just think that it should be increased to something like 2 solutions per problem set, due to the fact that there is often more than one problem that I'd personally like to talk to other people about (this actually stands for a lot of other people too).

That way, the more experienced people can take care of the more advanced problems, whereas the people who aren't so sure about themselves can get their fair amount of practice and eventually progress through content.

Ivan Sekovanić - 7 years, 10 months ago

Log in to reply

@Ivan Sekovanić I don't really agree with what you're saying. Perhaps the best way for the less experienced solution writers to get better is to write a solution, then read the best ones, and compare them. If you restrict the users from submitting more than one solution, you are most probably missing out on very well-written solutions on some problems.

It is true that the solutions which aren't formatted are barely given any attention. I know that math is not about formatting but about, well, math… but those solutions just do not look attractive to read, while it is so easy to format your math equations. I think that Brilliant should try to make sure that everyone knows how to format their solutions before writing them, and those solutions will get more attention.

Another way of bringing more attention to the solutions by less experienced writers, would be to have such a solution pop up on the screen of an experienced solution writer, so he can give feedback on formatting as well as mathematical content, possibly for points.

Tim Vermeulen - 7 years, 10 months ago

Log in to reply

@Tim Vermeulen Well honestly, I'm not quite sure if people can improve by just watching other people's solutions. My point was, if they try for themselves and someone else points out the mistakes they made (if any), they'd learn a lot better. In terms of formatting and in terms of math.

Just because some solutions aren't "eye candy" it doesn't mean that they are any less valid or even better than some other, fancy ones. That's why I think they should be given a chance to get theirs reviewed and not be overshadowed by other ones.

On another note, formatting should definitely be promoted more. As Tim said, it is both easy and efficient. Plus, people your solutions will catch other people's eyes more easily, increasing the chances of them being read and discussed.

Another thing I'd like to mention is that I agree that there should be more than one solution submission available for each problem set. I just don't think there should be eight.

These are my two cents, at least.

Ivan Sekovanić - 7 years, 10 months ago

Although I actually fell for the one-solution-per-week thing myself and definitely share your pain, I must say that allowing only one solution per week is somewhat a good thing to do at this point. It's simply way too much work for the Challenge Masters, especially since some people simply do not bother using LaTeX at all, whose solutions can be particularly hard to read and understand sometimes. Regardless of that though, their solutions might be surprisingly better than some other, better versed ones, therefore they deserve to be pointed out despite the style of expression.

Your idea of a separate forum for all those willing to participate in problem discussions is really great, however I'm not sure if that would be at all possible to accomplish. This is due to the fact that the solutions people post will have to be reviewed in any case by a Challenge Master in order to point out if there is something wrong with a given solution. Therefore, the situation will not differ by a lot if this happens, sadly.

I guess an optimal way of satisfying everyone would be to give us something along the lines of 2 or 3 solutions per week or such. That way, if people come up with a unique way of solving a certain problem or several ones, they may express themselves freely and share it with the community.

Ivan Sekovanić - 7 years, 10 months ago

Log in to reply

Well, solutions don't really have to be reviewed by a Challenge Master since other members are reviewing. Hopefully, discussion threads would attract enough attention such that mistakes would be caught, but I do see that it could cause some problems.

A possible solution would be to appoint moderators, who have shown to be helpful and knowledgeable members to reduce the load. I think this is a feasible option, and I think it could help with your first point as well.

Daniel Chiu - 7 years, 10 months ago

Log in to reply

@Daniel Chiu Yeah you have a pretty good point there. That might definitely be the best solution.

Ivan Sekovanić - 7 years, 10 months ago

I have the same issue. I would like to make visible my solution for "Shourya's non-real roots", actually I think it would be the best solution so far ; however there is no option to make my not-so-good solution for "Degree 99 Polynomial" invisible again.

I think maybe allowing 4 out of 8 visible solutions would cut down on the noise?

As things stand now what is the point of even writing out a solution once you've made one visible? We may end up with some ingenious ideas being used to solve a problem, but nobody else ever finds out about it because the author didn't see the value in writing a solution he couldn't publish.

Matt McNabb - 7 years, 10 months ago

Log in to reply

Agreed

Shourya Pandey - 7 years, 10 months ago

I think maybe allowing 4 out of 8 visible solutions would cut down on the noise?

My point exactly. Couldn't agree more.

Ivan Sekovanić - 7 years, 10 months ago

Another point; I normally take all week to finish all the problems, so I have to wait till the last day to know which solution I want to publish. By then most people will never see it or be able to vote on it.

Having thought about this some more I actually would prefer no restriction on writing solutions. Everybody still has the chance to write a solution, and everybody can vote on all solutions. (Voters don't just have to pick one). All solutions "have a chance".

If a solution is poorly formatted, or pretty much the same as 10 other solutions, so what? The submitter will know that he's submitted an effective solution. Removing examples of other solutions for him to look at doesn't really help him.

If people are motivated by wanting to get the highest-voted solution then the thing to do is for them to study other solutions and get better at writing theirs. Not to remove the good solutions so that a poorly-written solution "wins".

(Perhaps the site could have a guide on how to write solutions -- as most of us know, the process of finding the answer is very different from the process of writing a proof once you know the answer).

Finally, when I finish a problem I am eager to see what other people have written and look at the various different approaches, and learn new techniques to use next week. Anything that reduces the number of interesting solutions appearing is a bad thing, IMHO.

Matt McNabb - 7 years, 10 months ago

Another point I would like to make is that encouraging people to write solutions may help to verify whether people cheated or not (of course this is very hard to do in general, but I guess every bit helps).

Sometimes the Olympiad problems are solvable by brute force computing , and you still get the points and the leaderboard position if you did it that way, or googled the wording of the problem, or whatever.

Matt McNabb - 7 years, 10 months ago

Log in to reply

I'm not sure if that'll work. If someone used WolframAlpha to solve a problem, he isn't very likely to write that in his solution. He'll probably not write a solution at all.

Tim Vermeulen - 7 years, 10 months ago

Oh yeah, I actually wanted to add that but forgot, I'll add it now. Thanks.

Tim Vermeulen - 7 years, 10 months ago

Wow, so... a lot of ideas and topics of discussion in this thread, so I am going to respond to different points in different comments to try to preserve a conversation flow.

Why can you only submit one solution per problem set this week?

The challenge masters, who have seen lots of solutions from users on Brilliant over the months, have noticed a sharp decline in the quality of solutions that people have been submitting since implementing solution discussions. Many and in some cases most of the solutions submitted for a problem have been poorly written and wrong.

From commenting behavior so far in those discussions, this would not be obvious to most of you, but it is important to keep in mind that about 1/4 to half the people who are reading the solution discussions are people who got the problem wrong. Seeing 30-50 solutions, 80% of which are poorly formatted and expressed is demotivating to people who entered the discussion in frustration to begin with.

We thought that limiting the number of solutions you could submit per week would force people to be more deliberate about what they post publicly. Remember, you can write as many solutions as you want in a week, we have restricted how many you can publish publicly. It is our hope that this encourages people who want to practice their proof-writing to write their solutions up and choose to post their favorite one, their best solution, or their solution to a problem that needs it the most-- only after reviewing what others have written. If you keep it private, you can still see and compare your solution to others, and hopefully learn from having written it. We are not sure if 1 solution per week is the right number, or whether it is more. We will likely experiment.

In addition, challenge masters will take a somewhat more active role on solutions which have numerous errors, if the community has not already stepped in. This is necessary to make those solutions helpful/useful to people who enter the solutions section completely stumped by the problem. However, it is our belief and hope that the combined participation of everyone can grow to the point that all of you can collectively clarify and elaborate upon posted solutions better than a few of us.

Peter Taylor Staff - 7 years, 10 months ago

Log in to reply

It's indeed a shame that the level of sometimes the majority of the solutions is pretty poor. But what about the best solutions? Did that level also drop? After all, whenever someone did not solve the problem and comes back to see a solution, he will most likely only look at the best one. I'd say that in theory the level of the best solution should even rise, because if you know the best solution, you can share it (as opposed to in the old system where you had to be chosen).

The current limit of 1 solution per problem set per week surely cuts down on submitted solutions per problem, and there are probably users that now do submit only their best solution, but is that really for the better? After all, bad/moderate solutions can't really harm the solution discussion, as long as it is dealt with (with a note by a challenge master, a comment, or just 0 upvotes). And whenever a user has more awesome solutions than he's allowed to submit, then the community is missing out on some of them, which doesn't improve the discussion at all.

I'm currently not at all encouraged to write a solution when I'm not allowed to submit it, other users might do it, but I just feel like it is lost. Maybe this will change if you would create some sort of personal archive, with all the solutions a user wrote (with the corresponding problem), regardless of whether it was public or not.

Maybe it is an idea to only allow formatted solutions. This might sound unfair to people who don't know how to format them, but frankly, their unformatted solutions will probably not get much attention anyways, and it is so easy to learn how to do it. You could filter those unformatted solutions by i.e. letting the staff mark a solution as unformatted which makes it private to its writer.

Another thing that I'd really like is the ability for the staff to mark each solution either right or wrong, i.e. whether that solution would have gotten the full points in a contest. You could also distinguish between wrong and partly correct. Ideally, the partly correct and wrong solutions would be filtered out (maybe put in a separate section underneath).

To summarize, while the intentions are definitely good, I don't think that limiting the number of solutions you can submit will improve the overall quality of the solutions, or better yet, the quality of the best solutions. There are indeed quite some poor solutions that cover up the real gems, but those can probably be filtered out by a combination of the community and the staff. While limiting the number of allowed submission does cut in the number of poor solutions, it also cuts in the number of great ones.

Tim Vermeulen - 7 years, 10 months ago

Log in to reply

Quality drop in solutions

What was actually most concerning to us was the drop in quality of the best solutions. Even those, have generally been less well done than they were when we selected you randomly once in a blue moon to write them.

EDIT: see Calvin's comment

non-constructive volume

Bad/moderate solutions can drag down the whole conversation if their volume is too large. To have a fair voting system there needs to be a period when you view them randomly. Large volumes of solutions reduce the likelihood that a viewer will easily find a good one.

The need to make private solutions matter more

It is very important feedback that private solutions have little practical value in the current design. The personal archive, with your solutions, bookmarked problems, comments you have made is something that is very high on our development priority list. Give it a month or more

Staff marking things right or wrong

Ultimately on Brilliant, we believe every user should be entitled to having an opinion on what is right and wrong, and a responsibility to back it up with reasoning(or at the very least reasons), evidence/demonstration, and/or a good story. We would like to avoid going down the road of having arbiters of "Truth" and creating that kind of status distinction among users. Currently the staff are marking solutions because not enough people are questioning and discussing the solutions that are posted. We anticipate and hope overtime, that as the site design improves incrementally, everyone on here will play a greater part in helping the discussions arbitrate what solutions are good and correct, what ones are intelligent and interesting but wrong, and which ones are just wrong/incomplete/poorly written etc... Eventually challenge masters will recede into the mist and only comment as the whimsy strikes them, just like everyone else.

Your last sentence:

While limiting the number of allowed submission does cut in the number of poor solutions, it also cuts in the number of great ones.

Is true. As you have pointed out, there are instances where there is a problem with no good solutions, and someone has a good one but already spent their solution on an earlier problem. It is our prediction that the community will get better at discussing and voting on solutions if they have a smaller chunk of them to chew on every week. We believe a better discussion surrounding solutions will start inspiring people to write better solutions. When both the solutions and the conversations they generate get better, we will likely increase the number you can submit per week. We will consider raising the number you can submit even just to experimentally see if it is better, before necessarily seeing improvement. We are not sure 1 per topic is the right number.

Peter Taylor Staff - 7 years, 10 months ago

The quality of the best solutions did drop significantly last week. This became apparant mid-week, when we realized that many of the voted solutions would previously have been marked incomplete, and very few would have received the distinction of being worthy of featured (which corresponds to receiving 3 times the points of the problem). On the whole, I have seen a huge improvement in the quality of solutions that were submitted this week, compared to the previous week.

I would disagree with only allowing formatted solutions, especially at the lower levels. This is an unnecessary barrier to entry, which would be much more stringent than limiting the number of solutions. I do not think that it should be a requirement for you to know LaTex, in order to communicate mathematical ideas with others.
However, I'd make the distinction that ideas should be clearly and logically presented in a friendly way. It is fine to write 12x + 3x = 15x, but not to write 12x3xx2x=72x2. You should not make the reader have to guess what you mean.

When we decided to open up the number of solutions, it quickly became impossible for me to review each and every solution. (Bear in mind that you are only seeing a small subset of problems every week.) Instead, we will be relying on the community to vote and comment on solutions, to help each other improve their presentation skills. We will be providing slight guidance, especially in pointing out subtle intricacies. For example, in A sum of roots of roots, almost every solution make the false claim that x22x+1=x1 \sqrt{x^2-2x+1} = x-1.

Following up your request for discussions to each problem, as echoed by others, this will be occurring in the solution discussion itself. I do not see any reason why discussions should be held in a separate area, which would make toggling between the two pretty troublesome. Of course, the community needs to participate in this too, and not just keep silent while expecting a conversation which they are interested in to magically happen (e.g. if you did not start this post, none of the 30+ comments would be written up). I do try and seed some conversation starters, like in this Complicated System of Equations.

Calvin Lin Staff - 7 years, 10 months ago

Log in to reply

@Calvin Lin With regard to the problem discussions, I see enough reason to separate the solutions and other discussions. For example, when the problem reminds me of a problem I'd seen before, I don't think it makes sense if I put that in my solution, or in a comment to another solution, while it might be completely irrelevant to that solution. Or when I generalize the problem (or simply want to ask if someone can generalize it), it would make a mess if it would have to be a reply on a random solution, just because there's no other place to put it.

Tim Vermeulen - 7 years, 10 months ago

@Calvin Lin In my opinion some users may not be submitting solutions anymore because there are no points for submitting a good solution.

Sambit Senapati - 7 years, 10 months ago

FYI, we are monitoring this discussion and will likely chime in shortly with some more information about the new solution discussion format and why some of the features are the way they are. This note is just so you know you're not being ignored, cause that's frustrating :)

David Mattingly Staff - 7 years, 10 months ago

Why is editing not allowed?

Editing is also not allowed at the moment as we evaluate reward systems. Writing a terrible solution, getting to see others, and quickly editing your own solution is an issue. Please do not hesitate to use the comments section to add clarifications to your solution, or pointing out typos/mistakes that would allow your meaning to be misconstrued. We might consider implementing wither an auto formatted edit box, or a character limit on edits but our concerned with improving other things more.

In general, keep in mind that the small typos and botched symbols don't actually annoy others nearly as much as they annoy you :) Your grammar mistakes are usually much less severe impediments to having your ideas understood by others, than flawed reasoning and cluttered but grammatically correct writing. So be brave, do your best, and don't sweat it too much if it doesn't end up perfect.

Peter Taylor Staff - 7 years, 10 months ago

Log in to reply

I appreciate all of your answers, and want to say that what you are doing is excellent.

I have two other questions/possible solutions.

First of all, can problem discussion threads be added? I know this would be hard to create, but it would go far in solving some problems.

Secondly, could appointing moderators solve this? Knowledgeable moderators could find and fix erroneous solutions, and make allowing multiple solutions feasible.

Daniel Chiu - 7 years, 10 months ago

Log in to reply

While I agree with your second point, I'm not sure if that first point is such a good idea. Well, having a discussion thread is, but but posting solutions there as well would mess things up probably, because there is already a solution section. What if the best solution was submitted in the discussion thread? That would just be awkward. The solution thread would be great for talking about related problems, ideas on maybe solving it in an alternate way (but not a full solution), things like that.

Tim Vermeulen - 7 years, 10 months ago

Log in to reply

@Tim Vermeulen Well, that would reduce the number of submitted solutions, but I suppose those solutions might also have to be checked (I changed my comment). You have a good point.

Either way, problem discussion forums would still be helpful for many other things (possible extensions, advice on solving similar problems, etc.) as I said before.

Daniel Chiu - 7 years, 10 months ago

Very encouraging to me, as I have problems and frustration typing everything special when I just want to communicate my proof. I agree with the last statements; a well-written but unformatted solution is more annoying than one average solution with average formatting. PS- You should host self-confidence seminars, Peter.

Justin Wong - 7 years, 10 months ago

OH man.... I just put some effort into a little solution this week for a proof in Geometry and made sure to format it like you said... and I am currently number one proof! woot! Not to say it is a perfect proof or the others stink... but I think your advice is true.

Justin Wong - 7 years, 10 months ago

Please explain the evolution of the voting dynamics?

So...In case anyone is new to the conversation. There are two basic ways in which voting on solutions discussions can be unfair, and they are related.

  1. Timing prejudice. People who submit earlier, have their solutions visible for longer and are more likely to receive votes. They are also more likely to receive votes by virtue of existing when there are fewer other solutions to compete for your attention.

  2. Bandwagon effect. You are psychologically more likely to vote for solutions that already have a lot of votes and be prejudiced against solutions that don't. You are also more likely to vote on the thing at the top of the page, because you are more likely to see the thing at the top of the page.

As you can see, the bandwagon effect can compound on the timing prejudice to unfairly promote solutions that do not deserve to be the best. Our first week of solution discussions saw people voting vigorously, but early submitters rode the bandwagon effect to undeserved glory. The following week, we experimented with not having voting open until partway through the week, trying to eliminate the preference given to early solutions. This was more fair, but overall people voted much much less and sorted solutions must less effectively.

To increase voting participation, the next week (last week) we implemented a model where you could vote instantly, but the ordering on the page of solutions was random and you could not see the vote count till mid-week. This seemed to fairly sort solutions,and overall voting was more prolific but still not as high as we wanted it to be, to be a fine quality sorter.

This week, we are curious whether having the vote counts visible, but the ordering random, is the sweet-spot for fairness, effectiveness, and engagement. The week is young. We will see. But in general, expect to see a lot of different experiments like these thrown at you week over week. If you have a strong preference over any of the ways of using Brilliant you have seen so far, please let us know.

Peter Taylor Staff - 7 years, 10 months ago

Log in to reply

This week, we are curious whether having the vote counts visible, but the ordering random, is the sweet-spot for fairness, effectiveness, and engagement.

This is definitely something I'd personally really look forward to on the long run. It may get rid of a LOT of problems with the current system. I was wondering why the order of the solutions kept changing :) Keep it up guys!

PS: Have you considered making both the vote count invisible and the order random?

Ivan Sekovanić - 7 years, 10 months ago

Log in to reply

That is exactly what they did last week, as he noted. :)

@Peter, the system from last week was by far the best in my opinion. It completely got rid of the bandwagon effect, and the random order gave a more equal opportunity to people in bad time zones to have their solutions seen and voted on.

Sotiri Komissopoulos - 7 years, 10 months ago

Log in to reply

@Sotiri Komissopoulos Ah, I didn't notice the order was random last week. In that case, that's indeed a superior solution. :)

Ivan Sekovanić - 7 years, 10 months ago

Ohh and thanks for the bug notifications in the post by Tim, and brought up in comments in this thread. We will start tending to them. All of the feature requests will come under consideration, especially the request for a general "problem discussion."

Also we would all like to apologize about the ambiguity of "You may post one solution this week." It sounds like an extra sentence could have saved a lot of you from much frustration.

Peter Taylor Staff - 7 years, 10 months ago

Do we get points for our solutions?

Points for the featured solutions have been removed for the moment. Eventually, as a motivator, we are considering offering rewards based on the quality of your solution and its value to others who are reading it. This would be strongly tied to the number of votes that you receive, and the difficulty of the problem. Hence some version of "points" may make it back, and we hope that this will be the case. At this stage, we are still looking at voting behavior, to decide whether there is an accurate enough judgement of solutions to allow for this type of system. You will have noticed the voting rules changing a lot in the last weeks. We have been experimenting to find out which systems are fair and that people actually use.

Peter Taylor Staff - 7 years, 10 months ago

I have to agree with you about the 1 solution per problem set limit. I only discovered its existence after I finalized my 2nd solution for one of the problem sets. Although it is entirely possible that we were notified of this somewhere and managed to miss seeing it. Note that there is nothing stopping you from writing a solution to every problem that you solve. You just won't get to have people read them.

This limit also prevents correct solutions from being seen. One of the Geometry&Combinatorics problems currently has 2 solutions published, both of which have Challenge Master notes explaining why they are incorrect. Although one of the comments has a correct solution.

Also, when I try to write a solution, the textbox appears to the right of the "Skip and discuss solutions" button instead of below it. This causes most of the textbox to be to the right of the right margin on the page. This might be my computer's display settings, or a problem with the website. I'm not sure which.

The features you suggest sound helpful enough to be worth implementing assuming the Brilliant staff has time to do so.

Jimmy Kariznov - 7 years, 10 months ago

Log in to reply

yea I have the issue with the box too. There should be a scroll bar to the right to allow you to move the screen I believe.

Something that is curious is I was able to submit two solutions. I submit one on a question about the cards and points (Which was flawed :( ) and another one for "Arthur's code." The first is a number theory and the other is a combinatorics. Could that be why?

Update: I just tried to write a solution for another G&C problem and I was not allowed to because "[I] have already published one solution this week in this problem set." So yes, it is one per category.

Michael Tong - 7 years, 10 months ago

I haven't experienced that issue. Which browser do you use? I have noticed that when you make an expression too long, then it continues on the right rather than on the next line, but I'm assuming this is because it's LaTeX\LaTeX.

Tim Vermeulen - 7 years, 10 months ago

Log in to reply

I'm talking about the textbox for editing a solution, not a long LaTeX\LaTeX expression.

Jimmy Kariznov - 7 years, 10 months ago

Log in to reply

@Jimmy Kariznov Yeah I know, I was just pointing out something that I've encountered.

Tim Vermeulen - 7 years, 10 months ago

What happened to the Key Technique Tag next to the answer in solution discussions?

We are looking at how to effectively use the Key Techniques for our problem. We'd like people who wrote solutions to state the techniques that they use, which would make it easier to scan through their approach. We had previously tested using these techniques as a hint, and reached the conclusion that they were not useful, in the current 'raw state'. This will likely not come back in the near future.

Peter Taylor Staff - 7 years, 10 months ago

I think one of the reasons why people can't edit solutions is the same reason why you can't see other solutions before you submit yours -- so you can't just copy what other people said after you submit. If you got something wrong -- that's okay! The comments are there for a reason (it's also called a "discussion" for a reason). Use it! There's always a second chance!

Jiao Wang - 7 years, 10 months ago

Log in to reply

I have been using the comments whenever I flipped a sign or used an incorrect symbol or whatever, but still it really annoys me whenever I do my best on a solution and afterwards discover a stupid mistake. If we'd only be allowed 10 characters to edit, these typos can be fixed, and cheating is pretty difficult.

Tim Vermeulen - 7 years, 10 months ago

Yes, I completely agree . Few weeks ago, i have also submitted solution and they said if my solution will correct i will get up to 700 points. but I have not get points yet and have not received any email about this ....... If my solution was wrong they should must send me an email.

Kiriti Mukherjee - 7 years, 10 months ago

I support the one-solution-per-problem-set scheme. We can write the solution to as many problems as we like and then before we publish (whether privately or publicly), we have a chance to see the solutions of other users. Then we can select one of our solutions that we think is best to be shown publicly. Perhaps increasing from one to two solutions per problem set is nice too.

Aram Tangboonduangjit - 7 years, 10 months ago

I wish to write a solution to a problem but I got that problem wrong, due to a silly mistake I had made now can't i submit solution to that problem?

Racchit Jain - 4 years, 11 months ago

such long note!

jawwad shadman siddique - 7 years, 10 months ago
×

Problem Loading...

Note Loading...

Set Loading...