| 1 person marked this as a favorite. |
When I talk to the various people who make virtual tabletops and other RPG tools, one of the things I mention is a sort of unified API for this sort of thing. Generally they pause for a bit, and then give me the "I think this guy may actually be really crazy" look....
Where do we start?
UML? ERD? Back of a napkin?
| 1 person marked this as a favorite. |
How about this... what is the intended scope of this API? Are we talking about primarily web-oriented stuff, for VTTs (GameSpace, MapTool, Roll20.net, D20Pro, FantasyGrounds, et al.) and character management software (HeroLab, PCGen, etc.)?
Do we need to choose a format (e.g. XML, JSON, ADO) or do we only want to define the structure so that individual applications can convert stuff as needed? If I had my druthers, it might well be JSON because that most ideal... but I've never specified an API before.
Are there any common latin abbreviations which I left out?
Some tech-savvy folks could really do something big here, if they came up with a plan and presented it to Paizo.
Some tech-savvy folks could really do something big here, if they came up with a plan and presented it to Paizo.
I'm really interested in doing something, but I'm really an extroverted technologist. If at least one or two knowledgeable other people get involved in the discussion, I'm sure we can at least make a foundation. The players have a good sense of what is needed.
I've been thinking about creating some web-based character management tools, lately. I've taken a lot of database courses in the last few months, and imagining PF applications is the only thing that kept me conscious during the whole ordeal. I would be 1000% more excited if I had something like this API that I knew would get support to work with other people's stuff.
Consider if a random treasure generator had the option to output JSON objects in an OpenPF format. You could then import that into your character management system, or into a MapTool framework that parses JSON (the functions are all in there...)
It's a really good idea, but I don't see it happening anywhere but here. I can understand why the individual application developers don't see much value in it, but if we fans define the standard and petition them to use it, mayhaps they will?
|
This sounds absolutely awesome, and I'm totally down for helping in any way I can. I've got a bit of a tech-background, but likely only enough to "know what's going on". Unless something needs to be done in Python... then I'm your guy :)
The only problem I see right off the bat, is that there's very few applications that have much overlap in the specific things they support: a Herolab user may only have access to the Core Rulebook data pack, so exporting their Summoner from PCGen wouldn't work regardless of what format it's in. And vice-versa: trying to import my Magus from Herolab into PCGen wouldn't work since PCGen only supports Core and APG.
@LazarX — There's truth in this. That's why we can't expect them to define a standard. But we can combine the "business logic" of Pathfinder with some standard object types, and say "gee wouldn't it be nice if you supported this?"
Open standards don't "only work when companies get together at the start of a branch of technology" ... that is one time when they can work. But they also emerge from new developments that then benefit from interoperability and force the laggards to get onboard. For example, what becomes of HeroLab if someone comes onto the scene with a HTML5-JS character database that has Roll20's user-friendliness (and general prettiness) ... and if Roll20 (or anyone else) picks up the standard? At that point, it's in HeroLab's interest to pick it up as well, or at least bridge it over to their standard.
It's complex, and I'm not saying it /will/ go that way, but standards can create pressure that benefits the users. That's what I'd like to see.
@Austin — You're right as well. Standards won't (and oughtn't to) necessarily work with what exists, they're there for future developers to comply with, and for existing ones to patch toward, right?
Your point is really important, there needs to be a defined error for imports that aren't supported by the destination app. This would, presumably, be the same (or similar to) the way House Rules are handled. And I firmly believe house rules should be supported, which may mean that each object needs a rule reference URL (the PRD, d20pfsrd, or someone's house rule page).
Y'see, while that may play havoc with validation, it's not like a character changes to suit the software it's imported to. It would be great if the standard gave meaningful feedback so that applications could be writ smart enough to say "Beneficent Weasel-master Feat doesn't validate, rule referenced *here*" with a link. Or something.
Oh, and Gary, if you're out there, a little guidance would be great. You've probably considered this more than anyone, let's get the users to put some force ($$$) behind it!
| 2 people marked this as a favorite. |
In addition to LazarX's insightful points, there is another key factor to consider. This industry is tiny, so there's not much money to be made in it. Consequently, lots of the software is written by part-timers and/or volunteers. Heck, many of the full-fledged companies writing software for this industry are staffed by predominantly part-timers. For the "big boys", they might have full-time staff, but you can count the size of their development teams for a given product on one hand - and still have fingers left over.
Think about what you're asking for here. Coming up with a standard takes lots of time. You end up with lots of different stakeholders, and each has different objectives for their particular products. Due to these differing objectives, they have different priorities and requirements. All of these issues need to be reconciled, typically resulting in either a standard that tries to do everything or the lowest common denominator. The former is unwieldy and the latter is useless to everyone. So yet more work is needed to find a balance, which means lots more time.
Let's say the folks behind each major product/tool allocates someone to participate on this standards team. Working on a standard can easily consume more than half of each team member's time (trust me on this, as I've worked on international standards committees before). If the average product has a staff of effectively 2-3 full-time people working on it, then working on a standard slows down their overall progress by roughly 20-25%. That slowdown directly impacts the tasks that lead to the revenue that pays their bills. I think it's safe to assume that a 25% drop in revenue will put most companies out of business. Now consider the products that have even fewer staff working on them normally. The impact to them reaches 50% or more.
You may argue that it's an investment in the future. You'll get no argument from me on that point. However, investing requires there to be excess that can be invested. In the case of our industry, there's not a lot of excess, which means there's not much investment that's even possible.
So what can we do? In my opinion, and by extension Lone Wolf's view, the best approach is for the industry leaders to work with each other and interatively evolve standards through partnerships. That's what we've been striving towards with Hero Lab. In a few weeks, we'll be releasing a major change to our saved portfolio format that represents an important step forward towards an open standard. We've made these changes based on conversations with multiple other tools developers, and that includes Paizo [Hi Gary!] so we can work well in conjunction with their new gaming space. Is this going to be the ideal solution? Heck no. But it's a step forward, and certainly won't be the last.
We have another leap planned for next year with Hero Lab, and it's a major one. Could we do it sooner? Yep. But our first priority is to get tablet support out the door, so the next installment in our effort towards an open standard has to wait a bit longer.
Quick show of hands here. How many of you reading this would rather we focus on a Hero Lab tablet version first or evolve an open standard? Now, ask yourselves how the vast majority of GMs and players out there would answer that question. Now consider it from our business reality. If we get a tablet version into place, we bring in more revenue, which means we can maybe afford another hire, which means we can accomplish more and do it faster. And one of the many things we can accomplish is a better open standard format. So it may not be the fastest way towards the goal you're seeking, but it's definitely the most prudent from our perspective.
Lone Wolf also has a completely new product coming out in a few months - Realm Works. Based on the feedback we received at GenCon, this product could have a hugely positive impact on tabletop gaming in many different ways. [When Gary Teter escalates from calling you "crazy" to "Insane!", you take it as a serious complement. :)] Realm Works is a product I've been wanting to create for 30 years. Yes, you read that right. I've had this vision of a tool for RPGs since I started writing PC software. And I think this product will have vastly more impact than just creating an open standard for what's already been done before. If we succeed with this, the Realm Works framework will become a piece of the standard moving into the future.
Alright, now I'm just rambling. The bottom line is that, given the nature of this industry, a standard is not going to be successfully developed by dozens of participants dedicating time towards the effort. Any standard will need to be driven by the major players and fueled by their own enlightened self-interest to work together. Everyone else will then benefit by having something they can also leverage. Is this the ideal way of accomplishing the goal? Nope. But it's the only way I see as viable given the realities of this industry we all love.
I just like that this is being discussed.
interatively
I do believe that you have just accidentally invented a new word, meaning "through interactive iterations."
Lonewolf-Rob, that all makes a lot of sense from the perspective of an existing developer with an established product. As I mentioned above, those particular stakeholders don't stand to benefit much.
An open standard would be far more useful to independent creators and new entries into the (admittedly not-so-lucrative) domain. It may (or may not) create pressure on companies like Lonewolf to adopt the standard if those products developed to work with the standard are any good. Pressure is—by definition—going to inconvenience developers.
You mention that a lot of the software in this domain is written by part-timers and volunteers. In that, I see yet another reason an open standard would benefit all; developers wouldn't have to repeatedly reinvent the wheel or sink most of their time into data entry.
Think about what you're asking for here. Coming up with a standard takes lots of time. You end up with lots of different stakeholders, and each has different objectives for their particular products. Due to these differing objectives, they have different priorities and requirements. All of these issues need to be reconciled, typically resulting in either a standard that tries to do everything or the lowest common denominator. The former is unwieldy and the latter is useless to everyone. So yet more work is needed to find a balance, which means lots more time.
Let's say the folks behind each major product/tool allocates someone to participate on this standards team. Working on a standard can easily consume more than half of each team member's time (trust me on this, as I've worked on international standards committees before). If the average product has a staff of effectively 2-3 full-time people working on it, then working on a standard slows down their overall progress by roughly 20-25%. That slowdown directly impacts the tasks that lead to the revenue that pays their bills. I think it's safe to assume that a 25% drop in revenue will put most companies out of business. Now consider the products that have even fewer staff working on them normally. The impact to them reaches 50% or more.
This analysis is likely an accurate description of your firm developing the standard and sharing it out to others. That's not really what I'm envisioning. I should think we would start with a Javascript API for hobbyists to develop Pathfinder tools (some of which are bound to tread awfully close to HeroLab in function) that are interoperable.
If such a tool came along, I would imagine Lonewolf's decision to adopt it, or not, would be entirely a matter of its prevalence and the utility it could provide to your service. It may be that Lonewolf shouldn't adopt the standard. The idea of an Open API is still worth discussion for us hobbyists.
I'll just point out that we already have one standard - the stat block - and that many products are already using it.
Hero Labs/PC Gen/PFRD have stat blocks which you can import into DM's Familiar for your face to face game or import that to Maptools for your VTT game. (and other products may be doing similar things, those are just some I'm familiar with).
I'd hope that Paizo would be supporting that at least - the ability to import a stat block and have it generate your dice-rolling button.
Beyond that, I'm not really sure what an API would be doing.
| 1 person marked this as a favorite. |
I do a huge amount of data parsing in my day job. One of our apps pulls data from XML, tab-delimited flat files, web service calls (which use XML behind the scenes but never expose it), JSON, and SQL queries against a remote database, and parses it all into a form we can use and store in our database.
All of these formats convey the same basic data structure. None of them follow any standard (other than being correctly-formatted for their type), although they are usually well documented. I have wished they would all agree on something, but it's never going to happen. On the other hand, if you know you're going to be dealign with this as a consumer of the data, it's relatively easy to write something where the central part says "I need this data" and depending on which source you're communicating with, a specific separate part says "Ah, I speak that language, here's your data." .
Generating data in lots of formats is harder, but you can use the same model. The central part says "Here, add this data to what you're sending" and the appropriate part translates it on its way out.
On the other hand, this is my day job, or at least a part of it. Like Rob said, it takes time and resources away from the other things I could be doing. Setting up the central processors and our first two sources took a lot of initial setup time, although it's been amply repaid by the time we've saved as we've expanding the sources we communicate with from the original 2 to the current 9 or 10. But if I was only ever going to be talking to one or two sources, the time wouldn't have been worth it. I said it was relatively easy, and it is - I probably spent 50% longer on the initial two sources to create the central processor along with them - but it's still a time investment which might not pay off.
Unrelated to that, I used to be active on the dumpshock forums, and at one point I created an entire XSD for inter-program communication of Shadowrun characters. The intent was to let you take your character from either the main Shadowrun chargen spreadsheet or any of the smaller GUI-based builders, and plug it into a program which would print nicely formatted character sheets. Unfortunately, there was no real buy-in from the maker of the main Shadowrun chargen spreadsheet, and I didn't have the skills to code the "pretty-printer" app. So it faded into obscurity (as far as I know), and is probably long lost.
-------------
What's the point to all this? Basically, it's twofold, to go with the two-part story.
First, to say that adding support for an existing specified standard would not require the same amount of overhead that Rob was discussing, but would still require some developer time and energy that could be spent on other things. This means that by offloading the creation of a standard to the forums, or a volunteer working group pulled from them, or anyone other than the ones who would use it, people like Gary and Rob won't need to pull nearly as much time away from other features. They'll still need to take the time to implement support for it, but in theory it would be relatively easy to add into an existing app. (Read standard -> Create in-memory character -> Save character into storage; Load character -> Export character -> Write standard).
Second, to point out that a standard developed by people other than the ones who would use it it won't ever get used, unless those people start using it. And they are going to need a reason to invest the time and energy into starting to use it.
I'd love to see an interop standard. I'd love to have a hand in creating it. But even if someone posted one that could handle everything that could possible be thrown at it, and was still somehow easy to use, it's unlikely to be adopted unless either two major established players in the market or one major player and one newcomer who has a gap in the first product they can fill decide to support it. The first major player can put pressure on others to be the second, but you need interest from at least someone established or there's no point.
Sell the idea of a community-built standard to someone who already has a product people use, then construct it. Otherwise, it won't happen.
Reading this post somehow gives me hope there is interest for a real API here, which is what I was looking for. The starting post is like 3 years old though so it would be nice to know who does still have interest on it, and who would like to help in anyway they can, even if you are not a programmer you could help.
But first of all, there have been several different things suggested here, so I would like to point out what I think will make more sense to start with, and from there it could be easy for anyone else to continue building on top of it.
1 - Have all the Pathfinder SRD content in a database using the database structure that paizo is already using
2 - Create an online service API, with endpoints for accessing all this data from any other tool(digital charactersheets, web spellbooks, character builders, etc)
- Contact paizo to see if they would be willing to share with us their DB of the SRD, or at least the architecture.
- Plan B: If they would not be willing to help, we can create the structure ourselves and scrape the SRD. Its a lot of effort, but it can be done gradually, starting with small things, like the drawbacks and traits, then moving to feats, spells, and from there classes and archetypes, and so on, and so on.
- A free hosting service(we can use heroku for starters)
- Once free hosting is not enough I myself can commit to pay for the 75 USD monthly that are needed, until we can have some donations or so in-place to pay for it.
- We need developer/s to create the first simple approach of the API. There are several techonologies that could make this really an easy task, after having the database.
- Other tools like Character Builders could hook up this API and enhance their services by offering searchable list of spells, drawbacks, clases, archetypes, or anything we offer from our database.
- Having this API being open source will give other developers the opportunity to help enhance the API and continuously create new functionality for it
- As we move forward, we could document the standard database structure, and schemas, that other people can use when they are creating third party rules for the system, allowing this API to also serve third party content.
- Having standards for third party contents means that it will be easy to review, and playtest third party rules, characters, archetypes, traits, etc. So the community would greatly help into having third party content of a higher quality, the same or even better than paizo itself.
To be honest I could try to do all that myself, I'll just be a lot slower, but having seen that there is a lot of people interested, It gives me hope that I could have some help and we could all benefit.
So if there is someone out there that still wants to participate in something like this, please let me know, send me your email, and we can coordinate to work together.
My email is matismasters@gmail.com feel free to contact me directly if you prefer.
Chief Technical Officer
|
1 - Have all the Pathfinder SRD content in a database using the database structure that paizo is already using
...
- Contact paizo to see if they would be willing to share with us their DB of the SRD, or at least the architecture.
Our rules content doesn't live in a database. The latest "official" version of most rules content lives in the most recent InDesign file for whatever book it appears in. The PRD is effectively just a marked up version of that. (When we go to reprint a book, we don't work backwards from the PRD—we update the InDesign files if needed, then update the PRD from that.)
I'm currently working on a Pathfinder Character Manager but struggle to find the time for it.
I try to spent at least 3 evenings out of my week on it.
I'm good on the technical side but what I really like is a good design, if yous want some JSON blobs I could upload what I have somewhere if you like?
So far I've got some json files for equipment, feats, naturalDamage, races, sizeModifiers, spells and traits.
I am a programmer working mainly with C / C++. Several years ago, I developed a stand-alone database program to keep track of large anime collections. The first thing I did was design the structure of the generic database entry. Then I got some graph paper and did some "paper prototypes", which is how every screen is supposed to look. I then wrote the "usual" database manipulation routines and bound them to the buttons on the appropriate screen. If anyone is interested, I'm willing to provide free advice for any sub-project for this thread of discussion.
Any idea on a good way to store errata? Or should that be left as an afterthought and just focus on the core content?