Problems with Ideascale

[Being]

If you propose an idea in the IdeaScale crowdforging tool everyone gets to see and consider the idea and upvote or down vote. You can filter the ideas to be considered by how new they are, by how many votes they have received, and how 'hot' they are.

However, if a community member decides your idea is a duplicate, or if they just want to suppress your idea they just have to click on the 'report duplicate'. Once enough people have done so it is removed from view for evaluation by an admin.

If it sits in this limbo long enough then even if the post is judged unique and made active once more it is no longer new and sinks to the bottom on the 'recent ' tab. It wasn't there to receive votes or comment so it sinks to the bottom of the 'popular' tab. It also has grown cold so it sinks to the bottom of the 'hot' tab.

Even then all it takes is enough votes to suppress it once more removing it from view again for the admins to eventually review it again, approve it again, and it reappears at the bottom of every list.

All it takes is enough community members of a company who want the game to develop only the way they want to effectively remove an idea they oppose from consideration.

I think this is a problem with the way Ideascale is designed. Ideascale, I believe, was built with an assumption that the participants would be working together to find ideas rather than working against one another.

I've had five ideas removed from view for reported duplication. This morning all five were returned (at the bottom of each list now) to community consideration by Admin adjudication. All five were also reported as duplicates yet again just minutes later.

The ideas once more 'pending approval' are:
1. Overpopulation: Should the population of any given settlement be limited by its ability to feed that population, whether abstractly or in terms of crafted food production?
2. Army Size Limitation: Should the armed forces that a settlement, alliance, or nation state (can field) be limited by the ability of the settlement, alliance, or nation to supply that army (whether by development indices or real food/materiel production)?
3. Embassies: A settlement should conditionally be able to build an embassy in another settlement, given permission is granted from the hosting settlement. DI afforded by the consulate-building settlement.
4. Mobile Resurrection Shrines: A disadvantage for the force attacking a settlement distant from their own settlement is the speed with which character respawns can return to the battle. The defenders probably resurrect in the defending settlement whereas the attackers must respawn in theirs or in a non-hostile location that will still be farther away than the defender. Should there be mobile respawn points to overcome this disadvantage?
5. Chosen Skill Perk: Should a new character have the option to choose a single perk at character creation which provides that character a chosen skill acquisition advantage? (explanation from comments area: To explain: an idea was proposed that has merit but not as the original post described. Should a new character have the ability to expend initial build points to make the acquisition of a particular tree of skills easier for them than normal? If my character is totally invested in archery, should I be allowed to engineer some advantage in the acquisition of archery skills at the expense of disadvantage in other skills?

I don't believe these are duplicates. I think some folks don't like these ideas and engineered the system to remove them from the table by fiat instead of letting them be downvoted. If they are bad ideas they should be downvoted, not avoided.

[Guurzak]

My hex interdict proposal went into an invisible/ moderated state night before last. I contacted Ryan with a heads-up as to possible bug or moderator error. He pushed it back to active yesterday- and a few hours later it was in moderation again.

If this is a deliberate gaming of the system... well, I don't imagine GW is going to respond positively.

[Being]

It might be a solution were admin 'duplicate' adjudication required before the idea is removed from view.

Belief and knowledge differ. I don't know if some faction is gaming the system but I sure do see a way the system could be gamed. I don't know whether GW can figure out whodunit, but it is a reportable problem, particularly in a competitive environment.

[Laik]

From what I can see from ideas already merged, most merging looks like basic human inattentiveness and laziness in reading, rather than some devious plan.

[Kemedo]

Well I posted a duplicated Idea yesterday and merged few minutes after. I guess it's working as intended.

One way to avoid those duplicated ideas is showing the related ideas before the new idea goes live (maybe antoher step before active it). Another is a possiblity to post a sub-idea inside a more generic one.

But both of those will complicated things a bit. And I guess must be implemented in Ideascale side, not GW-side. I see little problems, by being a 3rd party site. The ideal should be GW have their own system, alas I prefer the game ready first... :)

[Being]

Laik wrote:

From what I can see from ideas already merged, most merging looks like basic human inattentiveness and laziness in reading, rather than some devious plan.

The system can be gamed, whether it was or not. That is a problem in a competitive environment.

[Being]

Kemedo wrote:

Well I posted a duplicated Idea yesterday and merged few minutes after. I guess it's working as intended.

One way to avoid those duplicated ideas is showing the related ideas before the new idea goes live (maybe antoher step before active it). Another is a possiblity to post a sub-idea inside a more generic one.

But both of those will complicated things a bit. And I guess must be implemented in Ideascale side, not GW-side. I see little problems, by being a 3rd party site. The ideal should be GW have their own system, alas I prefer the game ready first... :)

These five ideas were judged no duplicates and returned to active status, only after so much time they were at the bottom of the lists. Even so they were once more thrown into the que awaiting adjudication and out of public view. Most likely that was a step one of the admins had to take for some other reason, as it is unlikely that a group of people were hanging around watching for their return to active status.

Nevertheless the presence of an idea for player evaluation can be removed by the method I outlined, and that is a problem. The community works at cross-purposes often. Not everyone is collaborative and cooperative.

[Guurzak]

Being wrote:

(since cooperation is obviously a socialistic sentiment).

Let's avoid the unnecessarily contentious political philosophy. The last thing we need here is an argument about what socialism is, what people misunderstand socialism to be, and whether cooperation is a part of either or both of those things.

[Dario]

Hmm, since this was posted, we went from 206 ideas to 195 ideas. Maybe somebody thought your idea was clever.

[Being]

Could be so, yep. It wasn't me.

[Being]

Guurzak wrote:

Let's avoid the unnecessarily contentious political philosophy. The last thing we need here is an argument about what socialism is, what people misunderstand socialism to be, and whether cooperation is a part of either or both of those things.

Ah, okay, I apologize.

[Audoucet]

Cheatle told me like yesterday, that a few ideas he just had posted disappeared.

[Being]

Count on this community to be at least as clever as I am.

[Kemedo]

Well, the auto-regulation system is a great one, since users use it properly. But, when it's proved it's abused by any form, must be turned off for the good of community, which seems the case. The abusers must be identified and actions must be taken. A healthy community takes effort to maintain.

[Audoucet]

Kemedo wrote:

Well, the auto-regulation system is a great one, since users use it properly. But, when it's proved it's abused by any form, must be turned off for the good of community, which seems the case. The abusers must be identified and actions must be taken. A healthy community takes effort to maintain.

Yep.

If there's REALLY some abuse, it should suffer big consequences.

[TEO Cheatle]

One of my ideas was merged, because I didn't realize it was very similar to another idea. Two more were removed, not merged, pending approval.

[Kemedo]

Cheatle, those numbers are impressive low, since you have about 124,324.5 Ideas over there :D

[Ryan Dancey]

We need to investigate. When an item is flagged as a duplicate, I get an email. I review the idea and the duplicate suggestions from Ideascale and if I judge there are close enough matches I merge the ideas. There have not been more than three or four a week after an initial surge of duplicates when we opened the tool. However yesterday after I was asked to investigate, I found 20+ ideas that had been set to a status other than "Active". I do not think I received email about duplicates regarding those ideas but I have not verified that yet.

Prior to investigating further I have some speculations:

The system may be reacting to people submitting a large volume of ideas in a short period of time

The system may be reacting to submissions of ideas by people who have previously submitted ideas with lots of negative votes

The system may be detecting what it thinks are sock puppets

The system may be flagging what it thnks are known problematic users or users from various blacklists and anti-fraud services, etc.

The problem may be people gaming the system to kill ideas they don't like

I will look into this today and see if I can determine what is happening and why.

[Ryan Dancey]

PS: if people are actually gaming the system that would be a sign that folks are taking our Crowdforging commitment seriously and that's good, even if that particular behavior is bad. :)

[T7V Jazzlvraz]

Thank you, as always, for the responsiveness, Ryan. It's a factor that continually separates you, in my mind, from your competitors--both direct and indirect.

[Kemedo]

Ryan Dancey wrote:

PS: if people are actually gaming the system that would be a sign that folks are taking our Crowdforging commitment seriously and that's good, even if that particular behavior is bad. :)

Good in a bad way? Or bad in a good way? ;)

[Being]

The sign is good because people believe in the concept. The behavior is bad because it suppresses the creative intent of the system.

[DeciusBrutus]

It would be good if people took the idea seriously enough to want to "win" it, but bad in that they don't understand the concept well enough to know what "winning" means.

[Kitsune Aou]

I had submitted an idea the other day, and but a couple hours later, I came back to find that it was merged with a very generic idea that was not even really the same as what I was suggesting. I'm still rather upset about it. I'm considering deleting the merged post and re-posting again.

[Hardin Steele]

FWIW, if you use att.net like I do, you can't get the system to send your authentication email message. This is not the first time I have had a problem with my att.net email address. That is not an Ideascale problem per se, but an att.net problem., Still it affects the efficacy of the system overall.

[Ryan Dancey]

Ok, there are now a bunch of ideas that have been flagged for moderation.

One is "Simple and Advanced Contract Mechanic Options", submitted by "Community Member". I assume that's some kind of placeholder for an account that has not been configured with a public name.

That idea has had the following transaction history:

Events [ 6 ] [-]
Flag Status Changed from Approved to Flagged 1 hour ago
Status Changed from Active to Pending Approval 1 hour ago
Status Changed from Pending Approval to Active 16 hours ago
Flag Status Changed from Default to Flagged 19 days ago
Status Changed from Active to Pending Approval 19 days ago
The idea was posted 21 days ago

Ideascale does not allow me to see who took these actions, although I'm pretty sure the status change 16 hours ago was me responding to the PM that kicked off this investigation.

I have altered the Ideascale defaults. It now requires more than 1 report of an idea as abusive, and more than 1 report of an idea as a duplicate to shift the idea into moderation. We'll see if that helps.

[Being]

Thank you for the effort, Ryan.

[Lam]

Hardin Steele wrote:

FWIW, if you use att.net like I do, you can't get the system to send your authentication email message. This is not the first time I have had a problem with my att.net email address. That is not an Ideascale problem per se, but an att.net problem., Still it affects the efficacy of the system overall.

I had same problem with pacbell.net with is an alternate att server. I when out to a third server which redirects to ma\y pac bell.net in order to get set up. I do not want that address public, so I am also a "Community Member" account.

Creating 3 accounts is contrary to policy, but first two are non-functional and I am not using.

[Kobold Catgirl]

My Idea about ingame voicechat mechanisms has been inactive for the LONGEST time. It got merged with a newer idea at some point a while back—could that be the cause?

EDIT: To clarify, it got fixed today.

[Nihimon]

Ryan Dancey wrote:

Ok, there are now a bunch of ideas that have been flagged for moderation.

When we see an idea posted that's for something that's already been announced to be in the game, how do we flag it as such? Is the proper procedure to "Report Abuse" or "Report Duplicate"? Or is there something else?

[Ryan Dancey]

Don't worry about it. I read all the ideas and if I think that something is too close to a feature that will be in the game I usually take action.

[randomwalker]

Ryan Dancey wrote:

Don't worry about it. I read all the ideas and if I think that something is too close to a feature that will be in the game I usually take action.

so.. is Ideascale then only for new and original ideas?

I thought the crowdforging was also (more!) about priorities ("I want this before that") and not just brainstorming.

Three quick points from the devil's advocate:
-If I believe the community would prioritize a "known idea" way higher than GW thinks, isn't Ideascale exactly the right tool for that?
-If the community absolutely loves an idea you were going to put in anyway, isn't that just good?
-If the community hates an idea you were planning... isn't it better to know?

[TEO Cheatle]

Random, while I agree with you for the most part, some ideas will have to be included whether we like it or not.

Partial Loot system for instance, as in when you kill someone you can take some/most of their stuff. A lot of people dislike this idea, I don't know if it is majority though, either way it is required because this game is quite a sandboxed.

[Kobold Catgirl]

I think the vast majority of people are in favor of it, honestly. I can't imagine working as a bandit without the ability to loot a corpse. :)

[Ryan Dancey]

Ideascale is a test of a system. The test seems to be going well.

There's a difference between something that we will put into the game no matter what, and something that has to be prioritized and balanced against other functions and features. We have a list of stuff we are going to put into the MVP and when someone suggests one of those things I tell them in the comments and close the idea.

There are lots and lots of ideas that people have suggested which are a part of our plan but which we have not prioritized or thought about the various tradeoffs that will be required and those have been left alone.

[Guurzak]

Ryan Dancey wrote:

We have a list of stuff we are going to put into the MVP and when someone suggests one of those things I tell them in the comments and close the idea.

I'd love it if we could see those ideas remain in the system (maybe set them to In Development?); it would give all of us on the outside that much more info on what we have to look forward to.

[Ryan Dancey]

We're actually building a list of what is NOT going to be in Early Enrollment, which may be more useful.

[TEO Cheatle]

I was wondering, Ryan, what your thoughts would be on a cutoff line for ideas to be removed once they hit a certain number of total votes? Perhaps a percent based on the total voters?

There are a few Ideas, mine included, that have either really failed or really succeeded, and now are hindering a lot of people from fighting ideas they haven't yet voted on.

[Ryan Dancey]

How are they hindering?

[<Tavernhold>Malrunwa Soves]

I just hit the recent button and go backwards till I see something I have voted on.

[TEO Cheatle]

Sorry, I meant finding ideas, I shouldn't type while having a bad headache.

And a lot of ideas are getting buried, it seems to me, especially when chatting with people. Right now the main ways of looking up an idea is if it is Popular, Hot(seems like lots of comments), or Recent. You can also look for specific ideas through tags/campaigns, but that seems to be used to check for similar ideas.

If it isn't fairly new, or if it isn't extremely popular, some ideas seem to be buried in the 7-8 pages between them. We have close to 400 voters right now, and there are things buried that have 10, 20, 30 votes sometimes split, or sometimes trending into the positive or the negative. The people that scroll through all 9 pages of ideas to look at all of them might be in the minority here.

I think after you post your list of what can/can not be developed into EE most will have a clearer picture for the crowdforging process.

[<Tavernhold>Malrunwa Soves]

I don't vote on things that have no context yet, like particular siege weapons (one example). Don't know about other people.

[Kemedo]

I'm trying to vote on anything I understand what it's been posted.

[Being]

<Tavernhold>Malrunwa Soves wrote:

I don't vote on things that have no context yet, like particular siege weapons (one example). Don't know about other people.

If the objective is to crowdforge, which is a gathering of ideas people have for the game whether positive (things they want), or negative (things they don't want), where those ideas and the popular sentiment about them is offered for the developers' consideration, then it seems to me important for prospective players to weigh in with an opinion on things that don't yet have context.

The idea is to give the developer an accounting of our collective opinion, weighted by means of both quantifiable vote and by verbose commentary.

[Guurzak]

"meh" is an opinion.

[Kemedo]

Guurzak wrote:

"meh" is an opinion.

The problem with no voting is how devs can interpret from "meh" or "unsaw"

[<Tavernhold>Malrunwa Soves]

Until I see a basic implementation I just don't know what I would want to prioritize as an add-on/change to the system.

[Lam]

<Tavernhold>Malrunwa Soves wrote:

Until I see a basic implementation I just don't know what I would want to prioritize as an add-on/change to the system.

Maybe tool need two vote per idea: up/down and hi priority/it can wait

[Kemedo]

I see priority as a concern of GW. We just tell then if want or not want to, at some moment.

[Being]

I concur. GW will know what they have on their plate and their project manager will set the future schedule. In the case of PFO part of that schedule will consider our input on ideascale. We don't get to dictate priorities, but we are being allowed to offer input on our preferences. But if we refrain from evaluating ideas they cannot know our preferences. The more of us who weigh in, the more accurate will be the information they have to work from. Many popular ideas are very evenly divided for or against.