How to read and write data from a Python script using Forsta Plus public REST APIs, Jupyter Notebook and Quantipy

With the Forsta Plus REST APIs for reading and writing survey data and custom data, some new possibilities have opened up for more easily performing off-site data processing and analytics using Python. There is a wide range of Python libraries available for different types of data manipulation.

Quantipy is a library for survey data, that can read and process dat from Dimensions, SPSS, Decipher, Ascribe and Forsta Plus. It has an open metadata format to describe and manage datsets, and supports cleaning, editing, recoding and transformation of datasets, computation and assessment of data weights, data aggregation, analysis and reporting.

There are various other Python libraries available for operations like cleaning (wrangling, munging), editing, recoding, transformation, merging, sorting, aggregation, statistics, visualization, machine learning & AI, and NLP.

For now, Python scripts and programs will need to be hosted off-site, running for example ad-hoc on a desktop using Jupyter Notebook.

This short guide explains how to get started with Python on desktop, with tools we recommend using. Follow the steps below to install Jupyter Notebook, access the REST API, and learn how to use the REST APIs to both read survey data and write data back to Forsta Plus.

Installation

Anaconda

The easiest way to get started with Jupyter Notebook is to install Anaconda, a toolkit to work with various open source packages and libraries.

Install Miniconda

1. Download and install Miniconda

2. If on Windows, install the Miniconda3 Windows 64-bit (including python 3.8) using default settings





3. At the end of the process, click Finish to complete the installation

Python

Python 3.8 is already installed as part of the Miniconda installation in the "base" environment. However, it is recommended you create separate environments so you have control of which versions of 3rd party libraries are installed. Note that to run the quantipy3 example below, specific versions of libraries are required that might be different from those other scripts on the same machine require. Separate environments allow you to have different versions of the same library on the same machine.

Create environment and install python libraries

1. Open Anaconda Prompt (Miniconda3) (this is located in the start menu).



2. Create a new environment named "quantipy3" with a specific version of Python. We will use Python 3.7 for this example.

conda create -n quantipy3 python=3.7

3. Verify that the environment was created.

conda env list

4. Active the environment you have just created.

conda activate quantipy3

5. Install Jupyter Notebook.

pip3 install jupyter

6. Install the required Libraries.

pip3 install requestspip3 install quantipy3pip3 install scipy

Visual Code (optional)

As an alternative to running the Jupyter Notebook from the browser, you have the option of running the same notebooks from Visual Code.

Install and configure VS Code

Download and install VS Code - VS Code

Install the Python Extension. VS Code will also recommend this extension if you open the file.

When opening the notebook file .ipynb, VS Code will start a Jupyter Notebook server automatically. The script can be executed in the same way as from the web.

The first time you run this you must make a couple of selections:

When you open the notebook, click "Trust" or "Trust all notebooks".

Select Python Interpreter and select the one you just created.

Note: You can change this by clicking the "Python…​ " text in the lower-left corner:

Jupyter Notebook

Starting Jupyter Notebook

The installation instructions above are only required the first time you run the notebook.

To start Jupyter Notebook later do the following:

1. Open Anaconda Prompt (Miniconda3) (this is located in the start menu).



2. Activate the environment and start Jupyter Notebook

conda activate quantipy3
jupyter notebook

Creating, uploading or opening a notebook

1. Navigate to the folder where you would like to store your notebook, and click New > Notebook: Python 3



2. A blank notebook opens. Click the + button to start adding cells with code.



3. Click Ctrl + Enter to run the cell, or click the Run button.

Access to Public APIs

Contact [e-mail] to obtain a valid client ID and Client Secret so you can access the public REST APIs. You will need a valid Forsta Plus professional user with SYSTEM_API_ACCESS.

Client ID/Secret example

ClientID: aa55d014-829c-454b-aaf9-dc4549d9fa80

Client Secret: 788e2edd-6084-4094-9b9d-1e0dc1979129