Around 2010 I started playing a browser game called Grepolis. Back then it was very simple only having a browser client to play the game. When I started getting in to web related development I wrote a couple of scripts to gather the public data for the game that is still hosted in automatically updating gzipped csv files to make it easier for developers to use the data since even back then I thought that using gzipped csv files was not a very developer friendly way to share the game data. Since then I have tried this idea with a couple languages and eventually released a small lightweight NodeJS library that simply acted as a middle man to gather the data and transform it into JSON arrays. I quickly lost interest in this project because it's really simple and I didn't see a way to expand upon it at the time.
Then a couple weeks ago someone reached out to me via email and we chatted on Discord about the library. They asked me how I was able to reverse the Grepolis API which I explained that I didn't have to do for this as this data is known from a forum post many years ago. This sparked a new interest for me though because I thought "Grepolis has an app now, so they must have an API" and this thought gave me a way to expand upon my library and make it way more useful. Up until this point you could only read public data such as large lists of players, cities, battle points, etc. but if I could reverse the private API then I could create a full API wrapper for Grepolis which had not yet been done.
I set out on this task knowing a bit about sniffing communication between apps and their backend as I had done this once before using Charles Proxy but I never really liked that tool so I wanted to find something new to use that is easier and more robust. I landed on Proxyman which has all of the features I needed, in fact I only learned enough about the program to do what I needed which is sniffing the connection and decrypting the encrypted HTTPS traffic.
After configuring Proxyman on my computer and setting up the proxy on my iPhone I started seeing packets from
en45.grepolis.com I play on the
en45 "Hyperborea" world now so I knew this is what I was looking for. Inspecting the packet data it was clear that the structure for authentication and entering a game world went something like this:
- Login to Grepolis via
en0which passes back a
sessionidthat we save for later
- Login to game world with the
sessionidfrom earlier, this gives a game session id (aka
session_id) that we can use with the game server
- Finalize game world login (not sure if this is required) this seems to just be something that the app does, i'm not sure that it provides value for the library other than giving us an initial town id
initial_town_idbecause we do need a town id to start talking with game world endpoints
- Start sending requests as if you're a user on the app!
I also noticed a typical pattern with this API at this point, now that I had a couple of requests written and tested I looked at the structure of each
POST body to try and recognize these patterns to reduce redundancy in the codebase, and there was quite a bit of redundancy since one of the goals of this project is to not use any third party runtime libraries.
I noticed that there were two main sections for this API there are the non-world-specific ones such as login which use
en0.grepolis.com and world specific ones that communicate with the game server itself, in this case
en45.grepolis.com aside from the host being different the path is the same for both
/api albeit with various args depending on the server and what you're trying to do. Some common args are
ssid which is our session id token from login that's used to communicate with non-game-specific API's and
gsid which is the game session token used to communicate with world specific servers. The body content typically had the following structure:
and the headers for each request had some items of note as well:
'Accept-Encoding': 'gzip, deflate, br',
With this information I decided to write a
Util.js which contains generic
POST functions and I decided to match the
method_name with the library to make it more understandable. This means that if we have a request like this:
You can call it in the library with
I ran in to a couple of endpoints that gave me some weird data so I assumed from the headers that some of the responses were gzipped so I wrote gzip decompression into some of the
Util.js functions to properly handle this from the endpoints I noticed.
This project is still a work in progress but it's been a neat and interesting experience so far. You can check out the repository for updates as I go through the private API and write wrapper functions for it. I think with this article it's important to remember if you're writing an API that's open to the public no matter how you attempt to secure communication between your frontend and backend there is always a way around it and as such you should focus your backend development on writing secure API's.