Data & Analytics

PocketQuery for Confluence - How to REST Wikipedia

PocketQuery for Confluence - How to REST Wikipedia

In February, we released version 3.0 of our add-on PocketQuery for Confluence. One major new feature was the introduction of REST APIs as datasources. Now, we would like to demonstrate some catchy and practical example usages, that you can copy, adjust to your needs or just use as an inspiration. We will start with one example in this blog post and continue with a second one within the next few days.

You should already be familiar with PocketQuery in general and its handling of REST APIs. Otherwise, please have a look at the Getting Started Guide first.

Wikipedia: Integrating Knowledge

If you are using Confluence, chances are that you are collecting knowledge. This knowledge is contributed by everyone who creates and edits pages. With the help of PocketQuery, you can also integrate external knowledge sources to enrich and further complete the contents of your knowledge base.

How about adding a whole Wikipedia article to your Confluence, that is styled correctly and even integrates into Confluence Search?

datasources

Datasource

Datasource type REST Custom
Datasource URL "https://en.wikipedia.org/w/api.php"


The Wikipedia REST API is a charming example: it’s very basic, yet very powerful. As the data is publicly available, there is no need for authentication.

We’ll start by adding a new datasource of the type “REST Custom”, but we can neglect all of the advanced options. All we need is a meaningful name and the REST base URL “https://en.wikipedia.org/w/api.php”. This is even sufficient for a successful connection test- we do not need to append a test URL.

Query

Query URL ?action=parse&page=:title&format=json
Query JSON path $.parse.text['*']
Query settings Enable "Add results to Confluence search"


Again, nothing fancy here. We need to select our Wikipedia datasource, a name, the REST URL that will be appended to the base URL defined in the datasource (in this case ?action=parse&page=:title&format=json) and the JSON Path to the data of interest within the result. Note that we use a parameter in the Query which allows us to set the name of the Wikipedia article (:title).

The JSON Path can be found easily by having a look at the result from the Wikipedia REST API. Simply call an URL like “https://en.wikipedia.org/w/api.php?action=parse&page=Atlassian&format=json” by e.g. pasting it into a browser and use a JSON formatter to get an idea of the structure of the result:

{"parse":
    {"title":"Atlassian",
     "pageid":26569739,
     "revid":765016913,
     "text":{
        "*":"<table class=\"infobox vcard\"style=\"width:22em\">\n<caption class=\"fn org\">Atlassian Corporation Plc</caption>\n<tr>\n
            (...)

As we can see there is lots of meta data while the actual content of the article itself is wrapped within an element named “*” below the elements “text” and “parse”. In other words, the JSON Path that leads to the data of interest is $.parse.text[‘*’].

Converter

function convert(json) {
    var parsedJsonObject = JSON.parse(json);
    return [{ "wikipediaContent": parsedJsonObject }];

As a default, PocketQuery expects to work with simple tables, but after applying our JSON Path we are left to deal with a single JSON element. So we need to process our result using a simple converter. All we need to do is parse the object we extracted from our JSON (the parsing is necessary to properly handle special characters) and wrap it into a Javascript object.

Template

<link rel="stylesheet" href="https://en.wikipedia.org/w/load.php?debug=false&lang=en&modules=site.styles&only=styles&skin=vector">

$result.get(0).wikipediaContent

The template is only two lines. The first one includes the original stylesheet from Wikipedia while the second one prints the element our converter forwarded.

Enable Indexing for Search

datasources

Apply converter and template to the query and make sure to enable the option “Add to Confluence Search”. Et voilà, you can include any Wikipedia article you like and its content will even be found using the usual Confluence search!

What´s next?

In the next blog post, we will show you a more complex example using a REST API. It will include some preprocessing logic within the converter to do some simple calculations and build a dashboard.

Where to go?

And there is even more: