|
I do not believe the requirements are onerous either.
If you're a DM regularly attending megacons you should be able to schedule enough games in between 4 and 5 stars without too much problem.
Online DMs ...I can probably get that many venture critters in my games on accident. The spray never works...
If you're a DM outside of north America this looks like its going to cost you a grand minimum. Get one table, maybe 2 per convention. I'm going to try to make new online tables I make with the idea in mind that they might get handed over to a meatspace DM with little online experience to try one of these.
Even I know venture officers at conventions are usually busy.. well conventioning. Getting 3 away and knocking it out in one convention is probably a lot to ask for.
Question, can Venture Critters doing the evaluation get a pass on the replay rules or do you need to find one that has the scenario available and is available to play?
Venture-Captain, Missouri—Columbia
|
Why the restriction to only one online VC adjudication?
If you don't think the requirements are onerous, please move to an area with no VC within 100 miles of you and no decent sized conventions within 100 miles of you.
Better yet, move overseas. This process is a slap in the face of the international community.
By the way, what's the process for challenging a bad review?
Are the VC's going to be required to perform the reviews? If not, then the pool of reviewers is going to be a smaller than the potential pool of reviewers.
|
|
| 1 person marked this as a favorite. |
Why the restriction to only one online VC adjudication?
Because stuntmonkey and ironhelix need to sleep eventually.
The restriction only applies to physical DMs. So if you're 100 miles away from anyone and most dm online it doesn't apply to you. You can line up venture critters for friday saterday sunday and bang it out in a weekend if you want.
If you don't think the requirements are onerous, please move to an area with no VC within 100 miles of you and no decent sized conventions within 100 miles of you.
*points up*
That's definitely something that needs a tweak.
By the way, what's the process for challenging a bad review?
Getting a good one in 6 months?
Venture-Captain, Online—VTT
|
| 2 people marked this as a favorite. |
Although there is certainly some clarity that we need in regards to that, if it's written as I can only assume then I can't see how there is any bias against the online community at all, I'd argue it's strongly the reverse. GMs who primarily run games in meatspace are limited to only getting one evaluation from an online VC, thus GMs who primarily run in online formats may get all three evaluations from VCs online since there is no such limit on them. This heavily favours online GMs in how easily they may find and arrange VCs to evaluate their games, the only other GMs who might be similarly lucky are those in areas with huge PFS/SFS populations and multiple local VCs present.
Understand, I'm not saying this is a good thing for the evaluations overall, under these new rules it will put more onus on being primarily an online GM than would otherwise naturally occur if a GM wants their 5th star. I'm obviously a big proponent of the online region and think that we have a ton to offer both individuals and the campaign as a whole, but only where people genuinely want to run, having a situation where people feel forced to primarily GM online so they can easily get their 5th star goes contrary to all expectations and desires.
The rubric has a ton of excellent points and I think is overall a very good thing for telling GMs what to aim for in running games and providing the best possible experience for everyone at the table, but the implementation of the evaluations could definitely use some more time to be worked into a form that best serves all of our GMs, online and off, both in populous areas and remote locations, fairly. Which is presumably the entire point of this thread.
Ultimately, constructive criticism will only help improve this, vitriol and insults don't get us anywhere and are frankly entirely uncalled for. Also, please bear in mind that we've just had Paizocon and have plenty more big cons coming up, Tonya is going to be snowed under with travel and organisation, I'm sure there will be plenty of response to come but allow the time needed to also ensure everything else gets done. The new Guide won't be out until August at the earliest presumably, so there's still time.
|
|
| 1 person marked this as a favorite. |
It is inaccurate to combine all overseas venues and worse to equate them all with remote locations.
So the issue are GMing in remote locations and/or being a GM of limited means.
I think I have 1 VC, 1 VL, and 2 VAs in my city and nowhere else for at least 100 miles.
If I live downtown and don't own a car, I can't get to the venues to GM a game with my VC, let alone get to a convention.
Oh, wait... that also means that I haven't run 10 MTS, which have been a requirement forever, either. I'd best not get my cart before the horse.
EDIT: I left off my point. I'm not saying it should or should not be so, but it might be that 5-glyphs are not for everyone, no matter how many PFS(2) scenarios you buy and run.
|
|
It is inaccurate to combine all overseas venues and worse to equate them all with remote locations.
It's not that they're remote, it's just that their PFS presence tends to be a bit reduced and spread out. The person with 150 games in a country is more likely to be the person whos supposed to be doing the evaluations, so they're going to have to go to another country.
Statements that people outside of north america were looking at a grand minimum to do this are not intended suggest that it's not difficult for people in the US outside of major population areas (which I think i've mentioned a few times before)
In the US local transport is a problem but once you can get to a hub buses can get you anywhere cheap..ish. International flights though...
I'll let you know how well it works if i survive the bus trip to origins..
So the issue are GMing in remote locations and/or being a GM of limited means.
As one of the biggest proponents of the RSP you're definitely right there.
Oh, wait... that also means that I haven't run 10 MTS, which have been a requirement forever, either. I'd best not get my cart before the horse.
Go online, we're cheap. Erm.. to play...
I left off my point. I'm not saying it should or should not be so, but it might be that 5-glyphs are not for everyone, no matter how many PFS(2) scenarios you buy and run.
They're trying to make it a mark of quality and probably didn't INTEND to make it that onerous on people outside of venture critter central. There's probably something that can be worked out once they figure out there's a problem.
|
|
I'm a superfan of online play, but even I have to acknowledge the "play online" response to MTS (or anything, really) still ignores a considerable population, especially when you get to the complexity of GMing tables.
EDIT: And to demonstrate that I am not talking out of my tail-side, I have attempted online play from, in increasing degrees of frustration/difficulty: Lilongwe, Malawi; Zomba, Malawi; and Lodwar, Kenya.
Venture-Lieutenant, Netherlands—Leiden
|
| 5 people marked this as a favorite. |
I think it's good to keep a sense of perspective about earning five whoozits. This isn't graduating college on time or earning a certification you need for a job. If you're seriously contemplating blowing a grand on it, you should consider if you're not taking it too seriously.
In our lodge, the five star test so far has been mostly a celebration. The people doing it have been GMing for a while and are pretty confident of passing. Typically we invite friends who've been along for that ride and celebrate it as passing a milestone together.
If you look at the rubric and assess "how hard is this", it really isn't asking for the stars. "Acceptable" is a pretty modest bar and earning multiple "Exceeds" should be in the cards for most people taking the test. When you're planning a trip to some fun convention that also has some people who could administer the test, you should be able to be confident that you'll pass. This isn't a desperate gamble. Don't spend a grand to go to a convention just to get tested, but if you're going anyway, do make use of the opportunity.
---
Note that the proposal is now tuned to say that the same VC can't evaluate you more than once in a weekend. That means the VC could evaluate you again next month. That should help for places like Australia where the distances between VCs are extreme.
In areas where even getting once VC is a problem, but folks are creeping up on five shiny status, it might be time to have a talk with the RVC; if the area is seeing enough play that people are meeting the prerequisites (especially those 10 specials), maybe there ought to be an extra VC closer by?
The logistics of how to make tests happen definitely need working out, but I think that's something that has to be done regionally. A solution tuned for the US situation where conventions are very top-down and GMs just get assigned scenarios in return for a hotel room is not comparable to our boutique convention situation where we can do more bespoke scheduling.
So, broadly, make clear rules; but give enough flexibility to RVCs to fine-tune the logistics to their regions' needs.
Venture-Agent, Netherlands
|
| 1 person marked this as a favorite. |
A thought I had regarding Online play vs 6 month on-hold for a failed evaluation.
For Example
If you are running a PBP with a VC in it for evaluation.
You then run a meatspace table in the meantime for a VC and they fail you, and your 6 month on-hold kicks in.
What happens do your PBP table? Do you now need to stretch it over 6 months? Do you get to have it evaluated because you started before the 6 months? Does it suddenly become an invalid table?
Are there rules and a committee for deciding what happens in these cases?
|
Venture-Lieutenant, Finland—Turku
|
| 2 people marked this as a favorite. |
Oh, wait... that also means that I haven't run 10 MTS, which have been a requirement forever, either. I'd best not get my cart before the horse.
I want to point out that even if SFS doesn't have them -yet-, I think it's likely that they'll get their own 4* specials too (Like PFS currently has - you can get to 10 specials run without running a single MTS).
EDIT: I left off my point. I'm not saying it should or should not be so, but it might be that 5-glyphs are not for everyone, no matter how many PFS(2) scenarios you buy and run.
It might be that 5-glyphs aren't for *everyone*, but it should be that 5-glyphs are for *everyone who has run the required amount of scenarios and specials and meets the qualitative requirements, regardless of the VC population in their surrounding area.*
As a personal example, our town has a VL and a VA. Closest VC is about 540 miles away. Our city still has well over 20 active players (plus several semi-active ones) and we can sometimes draw couple of players from a few other nearby cities when running a con, and while a special isn't "always" an option, we've run a few with the population we have - it just takes a bit of coordination to ensure we can fill the tables. With the amount of specials PFS is currently offering, getting to 10 is just a matter of time for any GM that's interested in running specials.
This obviously is more of an issue with SFS (currently)(and PFS2 at the beginning) since there's just 2 different specials available. I assume I could run both specials once locally with some effort, and I could quite easily attend cons in the couple other nearby cities to get maybe 4 or 5 specials (assuming I played/GM'd starfinder/PFS2) - getting to 10 specials is, at this moment, nearly impossible because of low amount of specials available, but I'd rather have the discussion about the problematic VC population requirement now when the rubric is new and fresh, rather than later when the specials are more easily achieved.
|
Venture-Lieutenant, Finland—Turku
|
| 1 person marked this as a favorite. |
---
Note that the proposal is now tuned to say that the same VC can't evaluate you more than once in a weekend. That means the VC could evaluate you again next month. That should help for places like Australia where the distances between VCs are extreme.
This would resolve my concerns regarding the VC limit, but I don't think this is what Tonya's post at the beginning of the thread says. Where is this stated?
|
|
| 2 people marked this as a favorite. |
If you're seriously contemplating blowing a grand on it, you should consider if you're not taking it too seriously.
Much like the time I was four and picked up the "striped kitten" to bring home, that's the wrong take away.
The point is that the testing mechanism, which is perfectly reasonable if you're in a large city in the US, makes it rather difficult for people who aren't in that demographic. The mechanism might need a tweak to compensate.
The point if the 5th star is to recognize the best, not the geographically situated or (ironically) those who are online but NOT in meatspace.
|
Venture-Lieutenant, Netherlands—Leiden
|
| 1 person marked this as a favorite. |
This would resolve my concerns regarding the VC limit, but I don't think this is what Tonya's post at the beginning of the thread says. Where is this stated?---
Note that the proposal is now tuned to say that the same VC can't evaluate you more than once in a weekend. That means the VC could evaluate you again next month. That should help for places like Australia where the distances between VCs are extreme.
It's a bit buried in the text.
To keep this from being onerous for our less-populous areas, a GM may complete all three evaluations in one weekend, though no more than one evaluation per person may be done by any one Venture-Captain.
|
Venture-Lieutenant, Finland—Turku
|
This would resolve my concerns regarding the VC limit, but I don't think this is what Tonya's post at the beginning of the thread says. Where is this stated?---
Note that the proposal is now tuned to say that the same VC can't evaluate you more than once in a weekend. That means the VC could evaluate you again next month. That should help for places like Australia where the distances between VCs are extreme.
It's a bit buried in the text.
To keep this from being onerous for our less-populous areas, a GM may complete all three evaluations in one weekend, though no more than one evaluation per person may be done by any one Venture-Captain.
To me, that seems to say that the same VC can not complete more than one evaluation. It doesn't seem to suggest that this restriction would only apply if you try to complete all three games during the same con. If that is the intention, the wording could be clearer I think.
(I thought the purpose of that sentence was to make clear that you can get all your evaluations by three different people, during the same convention, instead of having to arrange three separate occasions for evaluations. I don't think it's purpose was that you only need three different VC's if you do all evaluations on the same weekend. I could be mistaken, though?)
Id love to come over and do some evaluations when the time comes.
Whoa! Thank you! I'll take you up on that offer if/when the time comes ^_^
Our lodge doesn't really do PFS2/SFS, yet. Even though I use our lodge as an example, I'm sure there are others that share a similar situation.
Venture-Captain, Illinois—Fairview Heights
|
| 3 people marked this as a favorite. |
I think getting VC support may be harder than people realize. I’ve asked for any VC willing to evaluate me at a table at Origins in the VO forums and not a single person has replied or contacted me. It’s gone completely ignored. I have 3 VCs within 2 hours of me, so I’m not worried about the ability to get my evaluations done. I’m at 96 tables with 4 scheduled before Origins, but I want to meet other VCs and hear feedback that isn’t from people who know and see me regularly. I want outside, unbiased feedback that I can use to improve because I enjoy trying to reach my highest potential in everything I do.
If I can’t get a single VC to be willing to even open a dialogue about it for Origins where I will be running 8 tables, I can’t help but have concern for the ability to do this at major cons.
|
|
| 1 person marked this as a favorite. |
If I can’t get a single VC to be willing to even open a dialogue about it for Origins where I will be running 8 tables, I can’t help but have concern for the ability to do this at major cons.
I can sympathize, but none of that is new, right?
The status quo: 150 tables, 50 unique, 10 specials and be evaluated by a VC who just goes with his gut.
The new mode: 150 tables, 50 unique, 10 specials, and be evaluated 3 times by a VC using the proposed rubric.
If you situation was such that you couldn't get 1 evaluation without overarching guidance for the evaluator, that situation hasn't changed for you and has nothing to do with using the rubric/guidance on how to measure adequacy.
With the exception of the number of evaluations, the process was objectively worse than the proposed process, unless you were friends with your VC. Now, at least, those of us not friends with our VCs can be confident that there's more guidance on pass/fail than a random stranger's gut feeling. We have benchmarks to aim toward.
|
|
| 2 people marked this as a favorite. |
Alright. I had an opportunity to see and use the rubric in action this past weekend.
It is a thing.*
I do not believe it is wholly necessary, either.
After participating, I firmly believe no one should ever have a qualification game at a large, major convention, totes if one must follow a rubric. My ability to judge one's "worthiness" of whatever marker under discussion is constrained by having a sample size of one. This is unfair to both the GM and the VO doing the evaluation. People have off days, people have differing styles of play/GMing, personality issues can emerge, etc. This process is one that should be between a GM and their local leadership. And for those that do not have local leadership or have never met said person**, by this time your style of play is set in stone. There is nothing anyone can do to change that barring abusive or combative behaviors. So, why should I as an outsider to both you and your region have any say in whether you qualify for Prize X? But if we are to appease the peeps who want Prize X to "mean something" beyond the time spent volunteering, then there needs to be a lot more discussion because of the nature of the campaign.
The reason for this is because the rubric as written leaves so much open to individual interpretation and personal biases. There was post game discussion about not having props/handouts for the game and thus this was a mark against "Exceeds Expectations". I was upset by this. Not everyone has the means and/or ability to produce said items. So if this is going to be the standard going forward, VOs will need to check their biases at the door and judge on the merits of GMing, not how much disposable income someone has.
*
**
Regional Venture-Coordinator, Appalachia
|
| 2 people marked this as a favorite. |
I think it's good to keep a sense of perspective about earning five whoozits. This isn't graduating college on time or earning a certification you need for a job. If you're seriously contemplating blowing a grand on it, you should consider if you're not taking it too seriously.
In our lodge, the five star test so far has been mostly a celebration. The people doing it have been GMing for a while and are pretty confident of passing. Typically we invite friends who've been along for that ride and celebrate it as passing a milestone together.
If you look at the rubric and assess "how hard is this", it really isn't asking for the stars. "Acceptable" is a pretty modest bar and earning multiple "Exceeds" should be in the cards for most people taking the test. When you're planning a trip to some fun convention that also has some people who could administer the test, you should be able to be confident that you'll pass. This isn't a desperate gamble. Don't spend a grand to go to a convention just to get tested, but if you're going anyway, do make use of the opportunity.
If this is the case - and I agree that it is and should be - I have to question what the purpose of the rubric and triplicate evaluations even is.
It just feels like bureaucracy for its own sake and I already work for the federal government man.
|
Venture-Captain, Missouri—Columbia
|
The problem with the three required adjudications and the parameters for obtaining them makes it more difficult to reach the desired status. It will be based on one's location in relationship to VCs who are willing to evaluate GMs. While the rubric is a nice set of suggested guidelines, it ultimately comes down to how VCs employ them.
As Shifty has said, this involves one's socioeconomic advantage outside the US. In the US it includes that as well if one does not have available VCs.
We still have the problem of whether or not VCs will be required to evaluate GMs. If they are not required to do so, and I am not suggesting they should be compelled to do so, then that exacerbates the VC shortage outside of major urban areas.
While desiring a qualitative measurement to the process is nice, this implementation falls short of making it possible without being detrimental to the organized play of Pathfinder/Starfinder/Pathfinder 2 overall.
|
| 3 people marked this as a favorite. |
Lots of informative stuff.
This is a single instance, so the smallest of sample sizes, but it illustrates much of my worry.
Speaking from a position of a professional educator, someone who has spent a lot of time around rubrics (especially here), and who engaged in hours of conversation and reflection on this one: the vitriol is neither warranted nor helping your cause.
Chris, I am not trying to diminish the work that you have done putting the rubric together, but there are flaws in this approach due to the nature of PFS. The people who will be implementing the rubric are not trained educators. Leg'o Lamb is one of the most respected members of this community and ran into issues with it (or observed them). Venture Officers, including Venture Captains, don't receive any appreciable level of consistent training across PFS. Other than whatever criteria is on the selection process, it's entirely up to the RVCs. That situation requires a lot more details and guidance than a rubric that would be handed over to an educator who has gone through pedagogy classes and has some basic framework from which to work.
I respect the Venture Captains we've had locally (we don't currently have one in the state -- different topic). I respect a lot of the VOs I've run into playing elsewhere around the country or online. I have also run into ones who are argumentative, get rules wrong and insist they are right, organize one game of PFS a month, but primarily play other systems, and who would generally fail several of the items on this rubric as GMs.
If you're looking to advance the conversation fruitfully, I recommend pulling back the polemic. I'm happy to respectfully and politely engage with anyone on all the work that went into this, but this hobby is fun for me and I'm keeping it that way.
This whole process and topic is making this hobby less fun for a lot of people.
|
Venture-Captain, Illinois—Fairview Heights
|
| 2 people marked this as a favorite. |
If I can’t get a single VC to be willing to even open a dialogue about it for Origins where I will be running 8 tables, I can’t help but have concern for the ability to do this at major cons.I can sympathize, but none of that is new, right?
The status quo: 150 tables, 50 unique, 10 specials and be evaluated by a VC who just goes with his gut.
The new mode: 150 tables, 50 unique, 10 specials, and be evaluated 3 times by a VC using the proposed rubric.
If you situation was such that you couldn't get 1 evaluation without overarching guidance for the evaluator, that situation hasn't changed for you and has nothing to do with using the rubric/guidance on how to measure adequacy.
With the exception of the number of evaluations, the process was objectively worse than the proposed process, unless you were friends with your VC. Now, at least, those of us not friends with our VCs can be confident that there's more guidance on pass/fail than a random stranger's gut feeling. We have benchmarks to aim toward.
I may have misrepresented my concern. I have frequent and regular access to at least 3 VCs, so the new system doesn’t pose a problem for me, personally. My concern is what appears to be a boycott by VCs to even discuss the possibility of doing a Starfinder evaluation at Origins. Of all the places someone should be able to have that discussion, especially for those with limited VC access, Origins should be that place. I am currently at 7 specials, 42 unique scenarios and 96 tables. I will have all but the 150 table count by January with no trouble using my local VCs.
If exactly zero VCs are even willing to open a dialogue at doing a Starfinder evaluation at Origins in some sort of act of protest, that’s a problem. That’s their right, but at what cost and to who? Refuse to entertain the idea to force change at the expense of your GM core? That seems to be a problem that needs to be addressed. “I don’t like it, so I will passively contribute to preventing anyone else from achieving this.” is a poor way to protest and isn’t victimless. That’s not the behavior that seems appropriate for a VC.
|
|
This whole process and topic is making this hobby less fun for a lot of people.Chris Marsh wrote:
If you're looking to advance the conversation fruitfully, I recommend pulling back the polemic. I'm happy to respectfully and politely engage with anyone on all the work that went into this, but this hobby is fun for me and I'm keeping it that way.
Sorry, too late to go back and edit. I realized that could come across as justifying people being rude to you, or seem like I’m saying you shouldn’t be able to have fun. That was not my intention.
I am making the point that this kind of qualitative evaluation is not fun for a lot of people. That includes those being evaluated and those doing the evaluation.
|
Venture-Captain, Illinois—Fairview Heights
|
| 2 people marked this as a favorite. |
I should also add context to my previous post. I originally asked if someone would be willing to work with me on this a month ago under the understanding that this was coming and I would like to be proactive rather than procrastinate. Any attempt to open a dialogue with the VC core has been completely ignored. I don’t say that out of contempt or frustration, but as a point of fact.
If requests for evaluations are to be met with silence by the VC core then then their inaction will halt all access for everyone who doesn’t have the geographical luxury that I have.
Wether they like it or not, the evaluation has moved from question to statement. The question now is how many people they deny progression through intentional inaction and refusal to support an official part of the Organized Play program. Wether or not you agree, this is now set forth by Paizo as the official way ahead.
It is not appropriate to refuse to even discuss the possibility of supporting the program at the expense of others because of your personal stance on what Paizo has officially set forth.
The evaluations are already taking place and even conducted at PaizoCon. Why attempt to penalize everyone else over your disagreement? You can disagree out loud without holding everyone back. Evaluate under protest in an effort to not penalize everyone else.
I’m sure that absolutely zero VCs will be willing to evaluate me at Origins anymore now than they were in the last 30 days, but again, I don’t need the VCs at large. I’m voicing my concern over the impact to everyone else who will have trouble finding a VC in the first place, much less three times, when requests are met with silence or completely ignored.
Is it really too much to even discuss the possibility of scheduling an evaluation table at “the Starfinder con”? It’s not like we don’t have precedent of evaluations before the guide update, because that’s already happening and the criteria has already been published.
|
Venture-Captain, Online—VTT
|
| 1 person marked this as a favorite. |
If exactly zero VCs are even willing to open a dialogue at doing a Starfinder evaluation at Origins in some sort of act of protest, that’s a problem. That’s their right, but at what cost and to who? Refuse to entertain the idea to force change at the expense of your GM core? That seems to be a problem that needs to be addressed. “I don’t like it, so I will passively contribute...
I think you may be rather jumping to conclusions here, I'm not saying that you're wrong or that it doesn't mean it won't be difficult to get evaluations under the proposed system if your experience is borne out so please don't misunderstand me. But I do think that saying there is some massive, organised secret protest or boycott from VCs because no-one replied to one post you made is... stretching things.
Your target audience is VCs who regularly have time to read the forums, feel they are knowledgeable enough to run an evaluation, are going to the specific con you are at, have free time at that con or can cancel other obligations to help. That's a lot of hoops for someone to jump through (which is indeed part of the problem) and more likely the cause of your lack of offers, rather than a secret cabal of VCs protesting in silence, I'm pretty sure many of the VCs will (and some have) speak up about something like this they see problems or issues with.
|
Venture-Captain, Illinois—Fairview Heights
|
| 1 person marked this as a favorite. |
I think you may be rather jumping to conclusions here, I'm not saying that you're wrong or that it doesn't mean it won't be difficult to get evaluations under the proposed system if your experience is borne out so please don't misunderstand me. But I do think that saying there is some massive, organised secret protest or boycott from VCs because no-one replied to one post you made is... stretching things.
If exactly zero VCs are even willing to open a dialogue at doing a Starfinder evaluation at Origins in some sort of act of protest, that’s a problem. That’s their right, but at what cost and to who? Refuse to entertain the idea to force change at the expense of your GM core? That seems to be a problem that needs to be addressed. “I don’t like it, so I will passively contribute...
Please don’t mistake me for thinking our VCs are united in a coordinated effort to stop this. I don’t believe that at all. I believe that one of them is. And another one is. And another one is. It appears that it is more an aggregation of individual refusal to participate.
Your target audience is VCs who regularly have time to read the forums, feel they are knowledgeable enough to run an evaluation, are going to the specific con you are at, have free time at that con or can cancel other obligations to help. That's a lot of hoops for someone to jump through (which is indeed part of the problem) and more likely the cause of your lack of offers, rather than a secret cabal of VCs protesting in silence, I'm pretty sure many of the VCs will (and some have) speak up about something like this they see problems or issues with.
I’m not concerned at a lack of offers or agreement to help me, personally. I would LOVE to have other people who don’t know me give me the feedback to improve or the kudos to congratulate me. I always strive to be better than I am in everything I do.
My concern is that I see a LOT of active VCs, many who have stated they will be at Origins. Not one has even said hey, let’s meet up and see if this is possible. That’s more than any have been willing to do so far.
No one said you need to be a Starfinder expert to evaluate. In fact the criteria doesn’t require the evaluator to be one. Excessive reference, excessive delays, things like that aren’t Starfinder specific. Good GMing isn’t Starfinder specific. If they don’t know the system, they can be an asset in ways others can’t. They get to see how you are with players new(ish) to the system.
|
|
| 3 people marked this as a favorite. |
I would also say that I do not think it is necessary for a 5 star GM to exceed expectations in all categories to make the grade. They should certainly meet expectations in all, and have a few areas that they generally excel in. For example the GM that makes the awesome maps and always has the right minis. Or the master story teller...
Again, the bolded section represents an aspect of this process I really want to see eliminated. We should not judge a GM based on whether or not they have the right mini. Hells, we shouldn't even be discussing minis as they have zero bearing on how a GM runs a game. By this logic, should we penalize the GM that uses tasty candy to represent monsters? Or, the GM who sucks ass at drawing maps yet does the best to their ability yet still penalized because it is not an accidental renaissance masterpiece? I started printing my own maps because I got tired of people reminding me every game my map drawing skillz are teh suxorz. Would I not exceed expectations in this area because I use a pawn that is relatively close enough to the critter put down on a craptastic map drawn on a blank flip map? Again, these criteria are not within everyone's time, means, or ability to provide.
Master storytelling I can get behind, though.
|
Venture-Captain, Illinois—Fairview Heights
|
| 2 people marked this as a favorite. |
Just a quick update. A VC did reach out to me yesterday about sitting in with another VC to observe me due to their general inexperience with high level Starfinder. I did not see this message until recently, because I didn’t notice that I had a PM. While I still believe my concern has merit, this individual is willing to make an effort and I find that to be utterly fantastic!
I truly hope we see VCs who will be present at events people are looking for evaluations at step up like this. Wether we agree or disagree with the program implementation, that is no longer our choice. Being a community that still works to support each other through the process and its revisions is exactly what we need.
Thank you, kind VC for stepping up and helping others move forward in this process. Even if you don’t feel comfortable signing the evaluation without more SF experience, you can provide valuable feedback that players are often uncomfortable providing and that means something to me.
Thank you.
|
|
| 1 person marked this as a favorite. |
Well you could ask, but you might be waiting 6 months if you drop the ball, and the VC might not be so keen to keep pencilling you in for future cracks...
This system really relies on generating good will with your local VC I suppose.
I know that I will almost certainly end up prioritising some people ahead of others.
|
Venture-Agent, Netherlands
|
Id love to come over and do some evaluations when the time comes.Whoa! Thank you! I'll take you up on that offer if/when the time comes ^_^
Our lodge doesn't really do PFS2/SFS, yet. Even though I use our lodge as an example, I'm sure there are others that share a similar situation.
Well...if your lodge needs help with getting SFS off the ground...I've not been to Finland yet!
My ability to judge one's "worthiness" of whatever marker under discussion is constrained by having a sample size of one. This is unfair to both the GM and the VO doing the evaluation.
And the same I think goes the other way. Should my worthiness be judged by VC's I'm in constant contact with, have run games for before and discuss rules/scenarios with all the time? Is there a bias towards me because of that?
In this case I trust the VC's to be honest with me, my own VC and the German VC's I know aren't afraid to tell it like it is. But I'm not sure the same can be said for everyone around the world.
But do my evaluations then need to be monitored? As I have VC's close who already know me. Do one of the three have to be done by a random VC at a convention (say PaizoCon2020?) Is there going to be a process for that? Or just getting 3 is enough and that's it (as long as I pass all 3)
And while we're at it, one more thing...I know the VCs close to me all have lots of rules knowledge about Starfinder. I know this is supposed to be an evaluation on pure GMing alone, but I can't help but think, does rules knowledge help/hinder an evaluation?
|
| 3 people marked this as a favorite. |
I’m glad that this thread has finally emerged! I’m definitely in the “Qualitative Aspect” camp—not so much as a gatekeeping function than as a means to spur us to get better at what we do. So, cheers to the Powers that Be for putting this scheme together! This is not to say that I don’t appreciate many of the concerns that have been raised. The most important question for me is whether the new standards move us to collectively raise our game as GMs, or whether they just add more hoops and create friction. This concern motivates all of the following.
It certainly does seem as if there will be issues getting timely reviews of some of our more far-flung GMs. Would it be possible to cut those folks a break? I don’t have an issue if GM Freezerburn at McMurto Station has to do fewer observation tables than I. They have enough problems. Dang penguins never manage to keep their chronicle sheets in order.... And, yeah, that’s unfair to the GMs who can easily schedule three reviews. But there doesn’t seem to be a fair option available, so why not choose the one that at least gives everyone a shot?
There might also be some crazy solutions involving prioritizing VC time at conventions (which carries its own drawbacks) or, I dunno, telepresence. Not saying that’s realistic, but maybe worth thinking about.
We should also consider the obvious concern that any qualitative assessment will be inevitably subjective. Of course, that’s why you have a rubric (which I’ll talk about in a bit), but it’s going to be open to interpretation. Expect table variation, as they say. What we don’t know—and won’t know until the process is up and running—is how problematic subjectivity is going to be. What we also don’t know is how much of a positive impact the reviews will have on the quality of GMing by aspiring 5-stars. Given that lack of knowledge, I ask the Powers That Be to periodically review the system to see whether it is functioning as intended.
It is certainly possible, in principle, to train the reviewers in order to mitigate variation. This begins with making the rubric as clear and detailed as possible. The next step might be something like having multiple reviewers observe the same game—even a recorded one, or excerpts thereof—and compare notes on their reactions. Whether this is a wise use of the VCs’ limited time I leave as an open question.
As for the rubric, it’s definitely headed in a good direction. There are a few additional items which I find essential to a good game that I’d like to see included. I get grumpy when they aren’t handled well—particularly if I’m the GM, when I grump mercilessly at myself! Here’s a list; I’ll discuss them in more detail afterward.
1. Coherent presentation: Did the GM present the story, setting, critters and NPCs in a way that the players could understand?
2. Adjust to the table: Did the GM modulate the game (within appropriate limits) to meet the play style and experience level of the players?
3. Monitor player engagement: Did the GM ensure that all players had a chance to contribute? Were certain players allowed to dominate the table? Did the GM encourage the reticent players? Were the players given the space to introduce themselves and their characters early in the session? If some players are repeating the scenario, did the new kids get the chance to drive?
Discussion:
1. This item could probably fit in Category 3. I want to mention that successful presentation is sometimes at odds with Category 4 (run as written) in the following way: sometimes, scenarios do a terrible job of presenting information to players in a coherent fashion at appropriate times. For example, I recently ran a table of 10-16 that I am not proud of. I’d prepped it, but hadn’t realized, until I was deep into the session, that it gives out information in the wrong order. For the story to work, the players should start with the recorded version of events, then uncover the truth. The scenario isn’t written that way, and, worse, gates the few drabs of the “official” story behind knowledge checks. Run-as-written is important, but the next time I’m running this scenario, I’m doing it differently. Otherwise, the story falls flat. And, yeah, I’ll die on this hill.
More generally, this category covers things like going beyond box text to make sure that players understand what they’re looking at, especially in complex environments; or giving extra background on NPCs, events, or locations, so that the players have appropriate context for the adventure.
2. Unlike point (1), I’m really not talking here about deviating from the published adventure. But a GM can (and should) adjust tone and (to a certain extent) challenge to fit the group and the scenario. By “tone,” I mean things like the amount and depth of roleplaying and the level of silliness and meta-references the GM allows or instigates. As for “challenge,” it is obviously verboten to change encounter design, but it is often possible to make things more or less deadly while still following stated tactics. And, please, try not to kill the one new player at the table right off the bat.
3. This probably goes in Category 3. Though that category is starting to feel a bit bloated. Oh, and it’s important that we not ding GMs for getting saddled with problem players. Y’all know what I’m talking about.
On that note, I’d like to make a general plea for leniency. Some issues are beyond a GM’s control. The real measure is how they dealt with the challenge. And let’s be nice to the folks who, for example, got a con schedule of a whole bunch of Pathfinder prior to running a Starfinder review table, then try for a 5-foot-step. It happens.
Finally, I think Category 4, “Run As Written,” calls for some discussion. Let’s crank open that can of worms. How do we prioritize running a good game vs. sticking to the script, particularly when faced with incoherent, contradictory, or plot-hole-laden scenarios? To what extent ought we to stick to complex special mechanics rather than taking a more freeform approach, when we know the former is likely to be frustrating? There’s a reason, for example, why social combat encounters have become increasingly pared down over time. Is it acceptable—better, even—to apply that strategy when running earlier scenarios? Should we disregard tactics in order to spare the characters of inexperienced players? That sort of thing. I’m curious to see what people think.
|
Venture-Lieutenant, Netherlands—Leiden
|
| 1 person marked this as a favorite. |
@Aaron: thank you for an extremely on-point, interesting discussion of the guts of the rubric.
I think you bring up important points, which I'll (with extreme generalization) describe as "GM in adversity".
Last 5-star evaluation I was at was a GM running for me, for a fairly fresh player who'd joined our lodge that year and was playing at the 10-11 tier for the first time, another 4-star GM from my lodge, a 5-star VC, and two 5-star RCVs.
The scenario was said to be challenging (Ghol-Ghan Heresy), but we had no real difficulty. Soiler: 5-star RVCs generally know how powerplaying is done. People had answers for everything, we had buffs and power to spare. But the GM soldiered on, and kept a steel face and made us think there were challenges ahead that we should be worried about.
We had a fun game, felt good about ourselves afterwards. Looking at the rubric standards, there would have been no hiccups. But the particular challenge for the GM really was to present this game to this particular audience, which was OP as heck. He passed with flying colours, he delivered a good story, didn't get frazzled at our ridiculous numbers, kept the ball in our court by instead of challenging us with combat challenging us with a scenario-choice that we had to agonize over.
My point about it is: I agree with you that it's good that GMs get tested in adaptability. It's nice that you prepared extremely well; it's even better when you go beyond executing the script, and really adapt to who's at your table.
|
Regional Venture-Coordinator, Appalachia
|
I have no huge issues with the rubric - a few quibbles here and there but Jon and Aaron really nailed it.
Its the three evaluations that strike me as placing an undue administrative burden on our volunteer corps and frankly on a number of our GMs.
I WANT the 5 thingy evaluation to retain that celbratory nature and that milestone feel. It was a special evening the night we did mine.
I feel that going to 3 evaluations makes the whole thing into a chore for everyone and robs us of that whole aspect.
|
It certainly does seem as if there will be issues getting timely reviews of some of our more far-flung GMs. Would it be possible to cut those folks a break? I don’t have an issue if GM Freezerburn at McMurto Station has to do fewer observation tables than I. ... And, yeah, that’s unfair to the GMs who can easily schedule three reviews. But there doesn’t seem to be a fair option available, so why not choose the one that at least gives everyone a shot?
I just want to amplify something that I feel is important that Aaron alluded to here: equity vs. equality. Using the same rules and standards for all GMs to receive their 5th Symbol is equal and just, but it also puts additional strain on GMs without local VOs, or at least without easy & consistent access to local VOs. These extra impediments and logistical challenges make it harder, effectively, to get that 5th Symbol. The rules here are equal, but they are not equitable.
I don't really have any suggestions, per se, which I realize isn't helpful...but I nonetheless fully support Paizo changing or amending the rules & guidelines such that the effort you need to put in to get a shot at that that 5th Symbol is the same, regardless of where you happen to live, and regardless of what the local VO situation is there.
Is there a way to do that? Honestly, I don't know. But I don't think that the "that’s unfair to the GMs who can easily schedule three reviews" argument carries water. The presence & availability of your local VOs is definitely a huge advantage to a region's gaming scene in general, but it shouldn't be the chief contributor to the determination of who is or isn't 5th Symbol worthy. I support anything Paizo can do to address that, even if it is, technically, unfair to GM "My RVC And Four VOs Live In The Same Neighbourhood."
|
Venture-Captain, Minnesota
|
| 2 people marked this as a favorite. |
I completed two evaluations at PaizoCon. Both were cases where the GM requested an evaluation and with whom I had no prior experience with as GM or a player. I liked that both were from outside my region. One was a table planned a few weeks in advance with invited participants. The other was a pickup game that we only decided to do the day before, with random players added to the table by headquarters. So I got to see two very different eval tables!
Both GMs were dedicated and enthusiastic members of the Starfinder community. Their games went smoothly and were great fun. They had talents in very different areas, which was interesting to see as an evaluator. Both GMs passed their first evals.
I do think that most of the Rubric is comprehensible and a fair set of criteria for an evaluation. I see the nova eval process as a celebration of your skills, and as a place to get some gentle and affirming feedback. Depending on how specials are counted, it is very possible that neither of the people that I evaluated fully qualify for their fifth nova yet.
Is this process subjective? You bet. I am looking forward to later in the process when we all have more guidance on how to evaluate each set of questions -- on the other hand, I think we’ll need to accept some table variance in how evaluators adjudicate the different areas. This is more art than science, but since you will be hearing from 3 different people, you will get different insights from each, and I think that’s good thing.
Because this was an evaluation, I was in a much more critical mindset than I would be in a 5 star PFS game. I looked for stuff to praise but also places where I could recommend that a person can build and grow. I tried to take into account that there are different ways to excel, and to understand that not everyone has the same talents / gifts. I did not want to be a rubber stamp of automatic approval but neither did I want to be harsh.
Complicating Factors
The GM's preparation allowed for smooth game flow.
The GM was able to keep the flow of the game consistent, and the GM dealt with unforeseen challenges by exercising skilled time management.
This criteria focuses on flow and pacing, two things essential for running a good game.
There was some discussion at our tables of what makes a game exceed expectations when it comes to prep. Though not stated, I do think that there are multiple ways that one can exceed expectations:
Don’t expect everything, but allow different GMs to show off their prep strengths.
The GM had a solid understanding of the rules to the game.
The GM had solid rules knowledge, and kept the game flowing while handling questions. GM acknowledged when a rule is unclear or when the GM made a mistake. GM did not have confusion between game systems. If a rules challenge arose, the GM handled it fairly and consistently.
I do like that the phrasing of this allows a GM to exceed expectations if they have to look up a rule -- so long as they do not let it affect the flow of the game. Since that was the standard I was applying in my evaluations, and I am happy to see this here.
The GM took efforts to make the game distinct and interesting.
The GM made a reasonable effort to make the game distinct in at least one meaningful way, such as deeply roleplaying the NPCs, using setting specific terms and lore to increase immersion, or using words with imagery to describe the environment, situations.
The GM put in an excellent effort to make the game distinct, using multiple techniques off the "meets expectation" list.
I really think this one is solid. I like that there are multiple paths to this one,
GM presented the scenario as written.
The GM stayed true to the storyline while allowing for creative solutions and player interest.
I love this criteria, because it puts creative solutions and player interest in the exceeds category. Follow the scenario, and keep the heart of the story, but also allow your players options.
The GM understood and applied the rules of the Organized Play Program.
The GM was markedly familiar with the majority of organized play concepts and applies the rules of organized play consistently. GM knew where to find obscure corner case answers in the Guide.
I feel that as is, this is an aspect that is challenging to exceeds expectation in. It’s hard for GMs to demonstrate knowledge of obscure organized play rules and corner cases when those corner cases do not come up in play. I gave one of my GMs a ‘Meets Expectations’ here, but I am tempted to revise it to ‘Exceeds’ because I don’t think a criteria should be graded on something entirely out of the GM’s control.
|
Venture-Captain, Minnesota
|
| 4 people marked this as a favorite. |
For me, the rubric is about more than just making someone a 5-nova and increasing the alphabet-soup floating around their heads. It's about helping GMs grow, and finding their talents. It's about mentoring. I've already been asked locally to provide a practice rubric to a GM that wants the feedback, and that's awesome.
I'm hoping that we all find a way to hone this and make it work for all of us in our global community.
Hmm
|
Venture-Captain, Missouri—Columbia
|
| 2 people marked this as a favorite. |
Drop the VC adjudication back down to one session and you solve most of the problems with the proposed changes. At that point we can really begin to explore the implementation and use of the proposed rubric which has a lot of potential. Until then, we're just stuck in the starting blocks.
