How to circumvent the paywall of “El Español”
Spanish new digital newspaper “El Español”, born this year, lets you read only 25 articles per month unless you pay. Reading it in incognito mode or disabling JavaScript on the page are the simplest ways to get past that limitation (as they are for other paywalls too). Here, for the sake of learning, is an alternative way.
tl;dr
- Install one of these two extensions in your browser: Tampermonkey (for Chromium or Chrome) or Greasemonkey (for Mozilla Firefox). You may need to restart the browser after it’s installed.
- Install the user script Lily Luna Potter (click “Install this script”, and in the next page, “Install”).
- Browse “El Español”. That pop-up should not hide the content any more!
How it’s done (boring technical stuff)
First, we browse the site and open several articles until we reach the limit. In a different browser session (for example, in a private window), open another article. Using the developer tools and looking at the HTTP traffic, notice the differences.
There is a little chunk of JSON being loaded dynamically that looks interesting. The URL (http://www.elespanol.com/static/sortingHat.js?stigma=xxx) includes, as a parameter, what looks like a cookie ID or a session ID. The content returned is little more than a boolean flag, and looks suspiciously related to the paywall.
There is also an XMLHttpRequest of the URL http://www.elespanol.com/usuarios/azkaban/promotions/. The preview shows that that’s precisely the obscuring pop-up. It’s very likely that there is some JavaScript on the pages loading that content iif the user is not allowed to continue reading.
(“Azkaban”…?)
Next, let’s examine the JavaScript that those pages load dynamically. Looking at their names and parameters, there are quite a few that are probably related to analytics, tracking or ads. We’re not interested in those. (I mean, of course we’re interested — we want them to disappear from the web. But that’s for a different post…)
Let’s download some of the others; in particular, the first three seen above.
Fortunately, those sources are not minified nor obfuscated. They just look ugly because superfluous white space has been collapsed, so every script is one very long line.
It is easy to fix that. You could open the files in your favourite editor and use its features to format and indent the code. An alternative, if you have a Node.js environment, is to install the package js-beautify globally, and invoke it from the command line.
With the scripts in a readable form, now it’s a matter of pulling the thread. We already found those curious names before— “sortinghat” and “azkaban”; but if that were not the case, we would scan the code and look out for key words (“paywall”, “register”, etc), cookies or asynchronous HTTP.
While looking at instances of those words, we notice something that might help. “browser.js” defines a global object called “jeef”, with a property “debug” set to “false”.
What would happen if we fire up the console in Chromium and toggle that flag…?
As one would expect, once we enable debug output and continue interacting with the page (without reloading) the console starts filling with interesting information.
The problem with this is that we can enable debugging only after the scripts have loaded and executed. Yes, there are events being logged after that (as you can see above), but it is likely that the most important initialisations and checks are done (and not logged to the console) before we can possibly toggle the flag.
To start dumping the log immediately after “browser.js” is parsed and executed synchronously, I added that one-liner (“jeef.debug = true”) to my Tampermonkey extension in Chromium as a new user script.
Now, when I reload the page, the really interesting stuff appears at the beginning.
“The wizard is muggle”… “Obtaining wand for wizard xxx”… “Initialising Azkaban.”… By now, the naming scheme is obvious, even for someone like me, who hasn’t read any of the books!
Further inspection looking for “muggle” discovers a bunch of related functions with revealing names.
These funny-named roles, or permissions, are granted to users depending on their status with the newspaper (I’m guessing): casual visitor, registered visitor, subscriber. All these functions return a boolean indicating whether the current user belongs to a particular category.
Again, I haven’t read any of the Harry Potter books, nor watched any of the films. But I vaguely remember learning that muggles were the lame type, while the other houses sound much cooler (and a quick visit to Wikipedia or Wiktionary confirms that suspicion).
So the code of the final user script is this:
And that works like… well, like a charm.
Originally published on Medium