Start a new topic

Connecting to an API using file input (or Python?)

I’m looking to connect to an API endpoint that requires a token passed in the header in the form of:

All calls to the API require this token to be passed in the Authorization header:

Authorization: Token TOKEN_VALUE

I’ve successfully done a POST request to the auth url to get the token (and parsed the json) but I can’t see how to then use that token downstream as the auth.


I need to pass that as the header in all requests.

Ideally I'm looking to request a list of datasets (it also gives download URLs) behind the API then filter that list and request each download link with that auth header to download the files to a local folder then process each one in whatever Omniscope job it's intended for.


Any help massively appreciated.


Thanks,


Simon


Just to add that it’s now possible to add HTTP headers and parameters to the HTTP requests in the File block, so effectively you don’t need a custom block, but just a File block to connect to any REST API out there.

Hi Simon,


We've put together a workflow with a Python block that demonstrates how to do this:


https://omniscope.me/Forums/POST+to+get+auth+token+before+downloading.iox/


You won't be able to execute this locked project, but feel free to copy/paste the Python into your own workflow, or download an IOZ of this project and import it locally.


For convenience here's the python code from inside that project; see comments inline:


# This script simulates doing a POST request to obtain an auth token
# then doing a GET request to retrieve a CSV, passing the auth token as a HTTP header.

# This is a common lifecycle for restricted-access API requests.

# In both cases we fake-up the remote server by using the handy postman-echo service
# and actually sending the data we are expecting to be returned.
# See https://docs.postman-echo.com/

# In the real world, for the first request, you'll be sending username and password,
# and getting back a token which you can't predict. And for the second request,
# you'll be hitting a URL which retrieves CSV data held remotely.

import urllib.request
import json
import pandas as pd

# Request the auth token; store in 'token' variable:
req = urllib.request.Request(url="https://postman-echo.com/post")
req.add_header('Content-Type', 'application/json; charset=utf-8')
responseData = urllib.request.urlopen(req, json.dumps({'token': 'abcd1234'}).encode('utf-8')).read().decode()
token = json.loads(responseData)['data']['token']

# Request the CSV data, passing the auth token:
req = urllib.request.Request(url="https://postman-echo.com/get?data=a%2Cb%2Cc%0A1%2C2%2C3%0A4%2C5%2C6%0A")
req.add_header('Authorization', 'Token '+token)
responseData = urllib.request.urlopen(req).read().decode()
csv = json.loads(responseData)['args']['data']

# Parse and return the CSV:
output_data = pd.read_csv(pd.compat.StringIO(csv))




1 person likes this
Login or Signup to post a comment