An Open API For Pathfinder RPG


Technology


1 person marked this as a favorite.
The Postmonster General wrote:
When I talk to the various people who make virtual tabletops and other RPG tools, one of the things I mention is a sort of unified API for this sort of thing. Generally they pause for a bit, and then give me the "I think this guy may actually be really crazy" look....

Link

Where do we start?

UML? ERD? Back of a napkin?


1 person marked this as a favorite.

How about this... what is the intended scope of this API? Are we talking about primarily web-oriented stuff, for VTTs (GameSpace, MapTool, Roll20.net, D20Pro, FantasyGrounds, et al.) and character management software (HeroLab, PCGen, etc.)?

Do we need to choose a format (e.g. XML, JSON, ADO) or do we only want to define the structure so that individual applications can convert stuff as needed? If I had my druthers, it might well be JSON because that most ideal... but I've never specified an API before.

Are there any common latin abbreviations which I left out?


Pathfinder Card Game, Companion Subscriber; Pathfinder Roleplaying Game Charter Superscriber

Some tech-savvy folks could really do something big here, if they came up with a plan and presented it to Paizo.


Elorebaen wrote:
Some tech-savvy folks could really do something big here, if they came up with a plan and presented it to Paizo.

I'm really interested in doing something, but I'm really an extroverted technologist. If at least one or two knowledgeable other people get involved in the discussion, I'm sure we can at least make a foundation. The players have a good sense of what is needed.

I've been thinking about creating some web-based character management tools, lately. I've taken a lot of database courses in the last few months, and imagining PF applications is the only thing that kept me conscious during the whole ordeal. I would be 1000% more excited if I had something like this API that I knew would get support to work with other people's stuff.

Consider if a random treasure generator had the option to output JSON objects in an OpenPF format. You could then import that into your character management system, or into a MapTool framework that parses JSON (the functions are all in there...)

It's a really good idea, but I don't see it happening anywhere but here. I can understand why the individual application developers don't see much value in it, but if we fans define the standard and petition them to use it, mayhaps they will?

Grand Lodge

Pathfinder Adventure Path, Starfinder Roleplaying Game Subscriber

The problem with this is that it's going require at least someone in your grand alliance to scrap a lot of the work they've done and redo from scratch in order to pull this off.

And quite frankly, I don't see any return they'd get to justify the effort, especially since it means spending money their competition is not.

These things only work when companies get together at the start of a branch of technology, not after it's been around for a good while, and they always run the risk of someone stabbing them in the back, the way Sony did with the DVD Consortioum when they tried going to High Def. Where are all those HD-DVD partners now?

Liberty's Edge

This sounds absolutely awesome, and I'm totally down for helping in any way I can. I've got a bit of a tech-background, but likely only enough to "know what's going on". Unless something needs to be done in Python... then I'm your guy :)

The only problem I see right off the bat, is that there's very few applications that have much overlap in the specific things they support: a Herolab user may only have access to the Core Rulebook data pack, so exporting their Summoner from PCGen wouldn't work regardless of what format it's in. And vice-versa: trying to import my Magus from Herolab into PCGen wouldn't work since PCGen only supports Core and APG.


@LazarX — There's truth in this. That's why we can't expect them to define a standard. But we can combine the "business logic" of Pathfinder with some standard object types, and say "gee wouldn't it be nice if you supported this?"

Open standards don't "only work when companies get together at the start of a branch of technology" ... that is one time when they can work. But they also emerge from new developments that then benefit from interoperability and force the laggards to get onboard. For example, what becomes of HeroLab if someone comes onto the scene with a HTML5-JS character database that has Roll20's user-friendliness (and general prettiness) ... and if Roll20 (or anyone else) picks up the standard? At that point, it's in HeroLab's interest to pick it up as well, or at least bridge it over to their standard.

It's complex, and I'm not saying it /will/ go that way, but standards can create pressure that benefits the users. That's what I'd like to see.

@Austin — You're right as well. Standards won't (and oughtn't to) necessarily work with what exists, they're there for future developers to comply with, and for existing ones to patch toward, right?

Your point is really important, there needs to be a defined error for imports that aren't supported by the destination app. This would, presumably, be the same (or similar to) the way House Rules are handled. And I firmly believe house rules should be supported, which may mean that each object needs a rule reference URL (the PRD, d20pfsrd, or someone's house rule page).

Y'see, while that may play havoc with validation, it's not like a character changes to suit the software it's imported to. It would be great if the standard gave meaningful feedback so that applications could be writ smart enough to say "Beneficent Weasel-master Feat doesn't validate, rule referenced *here*" with a link. Or something.


Oh, and Gary, if you're out there, a little guidance would be great. You've probably considered this more than anyone, let's get the users to put some force ($$$) behind it!

Lone Wolf Development

2 people marked this as a favorite.

In addition to LazarX's insightful points, there is another key factor to consider. This industry is tiny, so there's not much money to be made in it. Consequently, lots of the software is written by part-timers and/or volunteers. Heck, many of the full-fledged companies writing software for this industry are staffed by predominantly part-timers. For the "big boys", they might have full-time staff, but you can count the size of their development teams for a given product on one hand - and still have fingers left over.

Think about what you're asking for here. Coming up with a standard takes lots of time. You end up with lots of different stakeholders, and each has different objectives for their particular products. Due to these differing objectives, they have different priorities and requirements. All of these issues need to be reconciled, typically resulting in either a standard that tries to do everything or the lowest common denominator. The former is unwieldy and the latter is useless to everyone. So yet more work is needed to find a balance, which means lots more time.

Let's say the folks behind each major product/tool allocates someone to participate on this standards team. Working on a standard can easily consume more than half of each team member's time (trust me on this, as I've worked on international standards committees before). If the average product has a staff of effectively 2-3 full-time people working on it, then working on a standard slows down their overall progress by roughly 20-25%. That slowdown directly impacts the tasks that lead to the revenue that pays their bills. I think it's safe to assume that a 25% drop in revenue will put most companies out of business. Now consider the products that have even fewer staff working on them normally. The impact to them reaches 50% or more.

You may argue that it's an investment in the future. You'll get no argument from me on that point. However, investing requires there to be excess that can be invested. In the case of our industry, there's not a lot of excess, which means there's not much investment that's even possible.

So what can we do? In my opinion, and by extension Lone Wolf's view, the best approach is for the industry leaders to work with each other and interatively evolve standards through partnerships. That's what we've been striving towards with Hero Lab. In a few weeks, we'll be releasing a major change to our saved portfolio format that represents an important step forward towards an open standard. We've made these changes based on conversations with multiple other tools developers, and that includes Paizo [Hi Gary!] so we can work well in conjunction with their new gaming space. Is this going to be the ideal solution? Heck no. But it's a step forward, and certainly won't be the last.

We have another leap planned for next year with Hero Lab, and it's a major one. Could we do it sooner? Yep. But our first priority is to get tablet support out the door, so the next installment in our effort towards an open standard has to wait a bit longer.

Quick show of hands here. How many of you reading this would rather we focus on a Hero Lab tablet version first or evolve an open standard? Now, ask yourselves how the vast majority of GMs and players out there would answer that question. Now consider it from our business reality. If we get a tablet version into place, we bring in more revenue, which means we can maybe afford another hire, which means we can accomplish more and do it faster. And one of the many things we can accomplish is a better open standard format. So it may not be the fastest way towards the goal you're seeking, but it's definitely the most prudent from our perspective.

Lone Wolf also has a completely new product coming out in a few months - Realm Works. Based on the feedback we received at GenCon, this product could have a hugely positive impact on tabletop gaming in many different ways. [When Gary Teter escalates from calling you "crazy" to "Insane!", you take it as a serious complement. :)] Realm Works is a product I've been wanting to create for 30 years. Yes, you read that right. I've had this vision of a tool for RPGs since I started writing PC software. And I think this product will have vastly more impact than just creating an open standard for what's already been done before. If we succeed with this, the Realm Works framework will become a piece of the standard moving into the future.

Alright, now I'm just rambling. The bottom line is that, given the nature of this industry, a standard is not going to be successfully developed by dozens of participants dedicating time towards the effort. Any standard will need to be driven by the major players and fueled by their own enlightened self-interest to work together. Everyone else will then benefit by having something they can also leverage. Is this the ideal way of accomplishing the goal? Nope. But it's the only way I see as viable given the realities of this industry we all love.


I just like that this is being discussed.


lonewolf-rob wrote:
interatively

I do believe that you have just accidentally invented a new word, meaning "through interactive iterations."


Lonewolf-Rob, that all makes a lot of sense from the perspective of an existing developer with an established product. As I mentioned above, those particular stakeholders don't stand to benefit much.

An open standard would be far more useful to independent creators and new entries into the (admittedly not-so-lucrative) domain. It may (or may not) create pressure on companies like Lonewolf to adopt the standard if those products developed to work with the standard are any good. Pressure is—by definition—going to inconvenience developers.

You mention that a lot of the software in this domain is written by part-timers and volunteers. In that, I see yet another reason an open standard would benefit all; developers wouldn't have to repeatedly reinvent the wheel or sink most of their time into data entry.

lonewolf-rob wrote:

Think about what you're asking for here. Coming up with a standard takes lots of time. You end up with lots of different stakeholders, and each has different objectives for their particular products. Due to these differing objectives, they have different priorities and requirements. All of these issues need to be reconciled, typically resulting in either a standard that tries to do everything or the lowest common denominator. The former is unwieldy and the latter is useless to everyone. So yet more work is needed to find a balance, which means lots more time.

Let's say the folks behind each major product/tool allocates someone to participate on this standards team. Working on a standard can easily consume more than half of each team member's time (trust me on this, as I've worked on international standards committees before). If the average product has a staff of effectively 2-3 full-time people working on it, then working on a standard slows down their overall progress by roughly 20-25%. That slowdown directly impacts the tasks that lead to the revenue that pays their bills. I think it's safe to assume that a 25% drop in revenue will put most companies out of business. Now consider the products that have even fewer staff working on them normally. The impact to them reaches 50% or more.

This analysis is likely an accurate description of your firm developing the standard and sharing it out to others. That's not really what I'm envisioning. I should think we would start with a Javascript API for hobbyists to develop Pathfinder tools (some of which are bound to tread awfully close to HeroLab in function) that are interoperable.

If such a tool came along, I would imagine Lonewolf's decision to adopt it, or not, would be entirely a matter of its prevalence and the utility it could provide to your service. It may be that Lonewolf shouldn't adopt the standard. The idea of an Open API is still worth discussion for us hobbyists.

Grand Lodge

Pathfinder Adventure Path, Starfinder Roleplaying Game Subscriber
lonewolf-rob wrote:
Quick show of hands here. How many of you reading this would rather we focus on a Hero Lab tablet version first or evolve an open standard? Now, ask yourselves how the vast majority of GMs and players out there would answer that question. Now consider it from our business reality. If we get a tablet version into place, we bring in more revenue, which means we can maybe afford another hire, which means we can accomplish more and do it faster. And one of the many things we can accomplish is a better open standard format. So it may not be the fastest way towards the goal you're seeking, but it's definitely the most prudent from our perspective.

Here's really what I'd rather have on a tablet than what some folks think should be on there. (apologies for the bit of a threadjack, but it seems an appropriate opening)

I'm using Herolab quite happily on my desktop. I don't want to work Herolab only off a tablet but if you provide such software to people who want to forswear their Macs and PC's forever that's fine. What I want is something I call Herolab Mobile, where you create characters on your desktop, export them to Mobile, where I could do two things.

1. Print out standard type character sheets on an AirPlay printer.

2. Run the characters from Mobile, i.e. cast spells, roll dice, the usual stuff with maybe some options to export data change back to the PC application via wired or wireless data transfer.

The base tech to do this is present in IOS, it might be in Android in somewhat less polish, but I don't think for the least that it would be a trivial task. This is something I'd be willing to pay for as a supplement to my desktop Herolab purchase. What I'm not interested in doing is transferring my Herolab operations totally onto a tablet, at least not in the present state of tablets.

Grand Lodge

Pathfinder Adventure Path, Starfinder Roleplaying Game Subscriber
Evil Lincoln wrote:

@LazarX — There's truth in this. That's why we can't expect them to define a standard. But we can combine the "business logic" of Pathfinder with some standard object types, and say "gee wouldn't it be nice if you supported this?"

We can say that "it'd be nice" but coming down to brass tacks, people need to be paid to do this work because.... they gotta eat. And there simply isn't the spendable money by consumers in this still very niche hobby to make it worthwhile. The freeware folks like the PCGen crowd won't go on board because quite frankly trying to rope freeware authors of different camps is like trying to wash a herd of cats in a bathtub.


I'll just point out that we already have one standard - the stat block - and that many products are already using it.

Hero Labs/PC Gen/PFRD have stat blocks which you can import into DM's Familiar for your face to face game or import that to Maptools for your VTT game. (and other products may be doing similar things, those are just some I'm familiar with).

I'd hope that Paizo would be supporting that at least - the ability to import a stat block and have it generate your dice-rolling button.

Beyond that, I'm not really sure what an API would be doing.


IMHO the real 'killer' app for a standardised API would be to allow the various products you are using to interact in real time. So in Roll20 you choose to attack with your two handed weapon and it requests from HeroLab what the current dice roll should be. Or an area effect spell such as bless can be broadcast to all players.

The individual stat block is one aspect, but you also want to transmit state transitions.

The point also being that you want the manufacturers to concentrate on their specialities.

However such a standards implementation is not an easy task. It requires a certain amount of generalisation whilst at the same time keeping focus on the key requirements to make for cool gaming. This is a balancing act which most standards hardly ever get right. As lonewolf-rob says it would place a big drain on most games companies with limited resources.

So ultimately this would have to be a labour of open-source love, which just might gain some momentum and then some sponsoring.

But I would be happy to take part if there was some kind of working group.


1 person marked this as a favorite.

I do a huge amount of data parsing in my day job. One of our apps pulls data from XML, tab-delimited flat files, web service calls (which use XML behind the scenes but never expose it), JSON, and SQL queries against a remote database, and parses it all into a form we can use and store in our database.

All of these formats convey the same basic data structure. None of them follow any standard (other than being correctly-formatted for their type), although they are usually well documented. I have wished they would all agree on something, but it's never going to happen. On the other hand, if you know you're going to be dealign with this as a consumer of the data, it's relatively easy to write something where the central part says "I need this data" and depending on which source you're communicating with, a specific separate part says "Ah, I speak that language, here's your data." For the technical, C# with an abstract base class with the entry point to the central logic and a lot of small abstract/virtual functions, with each implementing class 'speaking' one format by implementing those..

Generating data in lots of formats is harder, but you can use the same model. The central part says "Here, add this data to what you're sending" and the appropriate part translates it on its way out.

On the other hand, this is my day job, or at least a part of it. Like Rob said, it takes time and resources away from the other things I could be doing. Setting up the central processors and our first two sources took a lot of initial setup time, although it's been amply repaid by the time we've saved as we've expanding the sources we communicate with from the original 2 to the current 9 or 10. But if I was only ever going to be talking to one or two sources, the time wouldn't have been worth it. I said it was relatively easy, and it is - I probably spent 50% longer on the initial two sources to create the central processor along with them - but it's still a time investment which might not pay off.

Unrelated to that, I used to be active on the dumpshock forums, and at one point I created an entire XSD for inter-program communication of Shadowrun characters. (An XSD is the file which tells you whether your XML structure is valid or not, so it can be used to generate correct XML). The intent was to let you take your character from either the main Shadowrun chargen spreadsheet or any of the smaller GUI-based builders, and plug it into a program which would print nicely formatted character sheets. Unfortunately, there was no real buy-in from the maker of the main Shadowrun chargen spreadsheet, and I didn't have the skills to code the "pretty-printer" app. So it faded into obscurity (as far as I know), and is probably long lost.

-------------

What's the point to all this? Basically, it's twofold, to go with the two-part story.

First, to say that adding support for an existing specified standard would not require the same amount of overhead that Rob was discussing, but would still require some developer time and energy that could be spent on other things. This means that by offloading the creation of a standard to the forums, or a volunteer working group pulled from them, or anyone other than the ones who would use it, people like Gary and Rob won't need to pull nearly as much time away from other features. They'll still need to take the time to implement support for it, but in theory it would be relatively easy to add into an existing app. (Read standard -> Create in-memory character -> Save character into storage; Load character -> Export character -> Write standard).

Second, to point out that a standard developed by people other than the ones who would use it it won't ever get used, unless those people start using it. And they are going to need a reason to invest the time and energy into starting to use it.

I'd love to see an interop standard. I'd love to have a hand in creating it. But even if someone posted one that could handle everything that could possible be thrown at it, and was still somehow easy to use, it's unlikely to be adopted unless either two major established players in the market or one major player and one newcomer who has a gap in the first product they can fill decide to support it. The first major player can put pressure on others to be the second, but you need interest from at least someone established or there's no point.

TL;DR: Sell the idea of a community-built standard to someone who already has a product people use, then construct it. Otherwise, it won't happen.


Hello guys,

(I'm skipping the startup opportunity talk.)
I'm humbly trying to develop a character page hosting service, with automated score calculation and stuff. I'm on the very start and doing it as a personal project for now. I did it with Ruby/Rails + stored on mongodb for now, as the target is to have it offered on the "cloud". And I think It could be offered as a web API
I'm on the very start for now, but does anyone think it could be worth going forward ? Or just interested on the stuff ?
More on the technical part :
Problem one, coding the rules. There's a quick path here.
Regarding the rules material, I'm using the rule files used by the open source PCGen project. PCGen is a Java software running on desktop, able to manage your character based on D20 and calculate all the scores (Attack bonus, Saving throws, weight, etc). Classes, feats, etc, are described with their impact on the character scores. So my current point is being able to parse the rule files and process them. When that's done, I can handle all rule set described in PCGen : Pathfinder, DD3, 3.5,4 and lotsa rulesets are already available.
Second part of the problem is to offer a proper web interface to that, and why not excel / pdf export/import, stuff like that. But regarding your thread subject, there would easily be a Rest web API if there's a web interface. In case it'll go public, this would be a king of open API.

Is this the kind of service you're looking for ?


I am intersted in this. The free content for the pathfinder websites has to be on a database somewhere.

What we need as developers is a way to query that database or at least gain access to download a live copy of all of the data at once(and keep the structure). Certain pieces of information would be good to be able to download on the fly, like the errata and faq updates.
The problem is that all of this data output has bandwidth costs to whomever runs the Paizo website (http://paizo.com/pathfinderRPG/prd/) with little to no pay out, unless the API was a paid for resource. I would certainly pay for a database dump of all static information from the srd. I would also pay a monthly fee for access to a web based (like REST) API that could update users with the newest rule changes and explanations of those rules.

If this already exists, could someone point it out to me?

Currently (as far as I know), the only way to access this information would be web-site scraping and data mining techniques that I don't think Paizo would be happy with.


From reading this, it doesn't appear that people really want an API. What they actually appear to want is easy and open access to the data set. If all the data related to the races, classes, skills, magic, creatures, and even backstory could simply be presented to the public as JSON or XML documents then developers could focus on writing the app logic and just tap into the data. The schema already exists. Here is a partial example of an Aasimar as presented in the online bestiary.

{"Creatures":[
{"Creature":"Aasimar",
"XP":"200",
"Allignment":"Neutral Good",
"Size":"Medium",
"Type":"Outsider (native)",
"Initiative Modifier":"+5",
"Senses":
[
{"Sense":"Darkvision", "Range":"60ft"}
],
"Skills":
[
{"Skill":"Perception", "Skill Modifier":"+5"}
]
}
]
}


Reading this post somehow gives me hope there is interest for a real API here, which is what I was looking for. The starting post is like 3 years old though so it would be nice to know who does still have interest on it, and who would like to help in anyway they can, even if you are not a programmer you could help.

But first of all, there have been several different things suggested here, so I would like to point out what I think will make more sense to start with, and from there it could be easy for anyone else to continue building on top of it.

Main objectives:
1 - Have all the Pathfinder SRD content in a database using the database structure that paizo is already using
2 - Create an online service API, with endpoints for accessing all this data from any other tool(digital charactersheets, web spellbooks, character builders, etc)

What do we need to achieve Objective (1):
- Contact paizo to see if they would be willing to share with us their DB of the SRD, or at least the architecture.
- Plan B: If they would not be willing to help, we can create the structure ourselves and scrape the SRD. Its a lot of effort, but it can be done gradually, starting with small things, like the drawbacks and traits, then moving to feats, spells, and from there classes and archetypes, and so on, and so on.

What do we need to achieve Objective(2):
- A free hosting service(we can use heroku for starters)
- Once free hosting is not enough I myself can commit to pay for the 75 USD monthly that are needed, until we can have some donations or so in-place to pay for it.
- We need developer/s to create the first simple approach of the API. There are several techonologies that could make this really an easy task, after having the database.

If we could achieve these two objectives, what would be the benefit of having this online?
- Other tools like Character Builders could hook up this API and enhance their services by offering searchable list of spells, drawbacks, clases, archetypes, or anything we offer from our database.
- Having this API being open source will give other developers the opportunity to help enhance the API and continuously create new functionality for it
- As we move forward, we could document the standard database structure, and schemas, that other people can use when they are creating third party rules for the system, allowing this API to also serve third party content.
- Having standards for third party contents means that it will be easy to review, and playtest third party rules, characters, archetypes, traits, etc. So the community would greatly help into having third party content of a higher quality, the same or even better than paizo itself.

To be honest I could try to do all that myself, I'll just be a lot slower, but having seen that there is a lot of people interested, It gives me hope that I could have some help and we could all benefit.

So if there is someone out there that still wants to participate in something like this, please let me know, send me your email, and we can coordinate to work together.

My email is matismasters@gmail.com feel free to contact me directly if you prefer.

Paizo Employee Chief Technical Officer

Matis Masters wrote:

1 - Have all the Pathfinder SRD content in a database using the database structure that paizo is already using

...
- Contact paizo to see if they would be willing to share with us their DB of the SRD, or at least the architecture.

Our rules content doesn't live in a database. The latest "official" version of most rules content lives in the most recent InDesign file for whatever book it appears in. The PRD is effectively just a marked up version of that. (When we go to reprint a book, we don't work backwards from the PRD—we update the InDesign files if needed, then update the PRD from that.)


I'm currently working on a Pathfinder Character Manager but struggle to find the time for it.
I try to spent at least 3 evenings out of my week on it.

I'm good on the technical side but what I really like is a good design, if yous want some JSON blobs I could upload what I have somewhere if you like?

So far I've got some json files for equipment, feats, naturalDamage, races, sizeModifiers, spells and traits.


I am a programmer working mainly with C / C++. Several years ago, I developed a stand-alone database program to keep track of large anime collections. The first thing I did was design the structure of the generic database entry. Then I got some graph paper and did some "paper prototypes", which is how every screen is supposed to look. I then wrote the "usual" database manipulation routines and bound them to the buttons on the appropriate screen. If anyone is interested, I'm willing to provide free advice for any sub-project for this thread of discussion.


John Napier 698 wrote:
I am a programmer working mainly with C / C++. Several years ago, I developed a stand-alone database program to keep track of large anime collections. The first thing I did was design the structure of the generic database entry. Then I got some graph paper and did some "paper prototypes", which is how every screen is supposed to look. I then wrote the "usual" database manipulation routines and bound them to the buttons on the appropriate screen. If anyone is interested, I'm willing to provide free advice for any sub-project for this thread of discussion.

Any idea on a good way to store errata? Or should that be left as an afterthought and just focus on the core content?


Laravel Dusty wrote:
John Napier 698 wrote:
I am a programmer working mainly with C / C++. Several years ago, I developed a stand-alone database program to keep track of large anime collections. The first thing I did was design the structure of the generic database entry. Then I got some graph paper and did some "paper prototypes", which is how every screen is supposed to look. I then wrote the "usual" database manipulation routines and bound them to the buttons on the appropriate screen. If anyone is interested, I'm willing to provide free advice for any sub-project for this thread of discussion.
Any idea on a good way to store errata? Or should that be left as an afterthought and just focus on the core content?

Perhaps create a Hypertext-like link that redirects the user to the errata screen. But other than that, get the core working properly. Then make incremental improvements, and release them after thorough testing.


This thread has been around a while, but nothing has happened recently. Is there still an interest in getting something working?

My take on it would be a RESTful API that returns JSON objects. This would allow access to the data from almost anywhere.


1 person marked this as a favorite.

I'm very keen on doing this.
I've started designing some tools to help my pathfinder group and I'm looking for developers as I've moved from code to ux design). I'm thinking that since I plan to use GraphQL/REST to fetch data, why not make an open API at the same time.

So if anyone with some js/web development skills are keen to join me, don't hesitate to reach out.

Community / Forums / Gamer Life / Entertainment / Technology / An Open API For Pathfinder RPG All Messageboards

Want to post a reply? Sign in.
Recent threads in Technology