Connecting to an API using file input (or Python?)

Posted almost 6 years ago by Simon Wiggins

Post a topic

Solved

Simon Wiggins

I’m looking to connect to an API endpoint that requires a token passed in the header in the form of:

All calls to the API require this token to be passed in the Authorization header:

Authorization: Token TOKEN_VALUE

I’ve successfully done a POST request to the auth url to get the token (and parsed the json) but I can’t see how to then use that token downstream as the auth.

I need to pass that as the header in all requests.

Ideally I'm looking to request a list of datasets (it also gives download URLs) behind the API then filter that list and request each download link with that auth header to download the files to a local folder then process each one in whatever Omniscope job it's intended for.

Any help massively appreciated.

Thanks,

Simon

0 Votes

5 Comments

Simon Wiggins posted almost 3 years ago

Perfect Antonio thanks!

0 Votes

Antonio Poggi posted almost 3 years ago Admin

Hello Simon,

Yes, this is possible with the new HTTP Batch block we've recently added. You can compose a series of data driven HTTP calls, using input data to compose the request (url, headers, authorization (bearer) http method, body, etc) and getting the response data body and headers as input of another Http request.

Feel free to play on our daily sandbox first to try stuff out http://daily.omniscope.me/Demo/API/HTTP+Batch.iox/

Please confirm this suits your case, happy to jump in a call to further assist you.

1 Votes

Simon Wiggins posted almost 3 years ago

Rounding back on this as I don't see anything else on the forum.

Do you have a good solution to doing a token exchange with an API?

I realise it can be fairly easily done in Python but I'm trying to see if it can be done without.

So essentially connecting to the auth/login url with username and password with a HTTP file block.

Receive back the token

Use that token in the next call in a header "Authorization": "Bearer tokenheredkhfakfjdh"

query the API as normal with that header.

Another extension to this, do the token refresh as well?

0 Votes

Antonio Poggi posted over 4 years ago Admin

Just to add that it’s now possible to add HTTP headers and parameters to the HTTP requests in the File block, so effectively you don’t need a custom block, but just a File block to connect to any REST API out there.

1 Votes

Steve Spencer posted almost 6 years ago Admin

Hi Simon,

We've put together a workflow with a Python block that demonstrates how to do this:

https://omniscope.me/Forums/POST+to+get+auth+token+before+downloading.iox/

You won't be able to execute this locked project, but feel free to copy/paste the Python into your own workflow, or download an IOZ of this project and import it locally.

For convenience here's the python code from inside that project; see comments inline:

# This script simulates doing a POST request to obtain an auth token
# then doing a GET request to retrieve a CSV, passing the auth token as a HTTP header.

# This is a common lifecycle for restricted-access API requests.

# In both cases we fake-up the remote server by using the handy postman-echo service
# and actually sending the data we are expecting to be returned.
# See https://docs.postman-echo.com/

# In the real world, for the first request, you'll be sending username and password,
# and getting back a token which you can't predict. And for the second request,
# you'll be hitting a URL which retrieves CSV data held remotely.

import urllib.request
import json
import pandas as pd

# Request the auth token; store in 'token' variable:
req = urllib.request.Request(url="https://postman-echo.com/post")
req.add_header('Content-Type', 'application/json; charset=utf-8')
responseData = urllib.request.urlopen(req, json.dumps({'token': 'abcd1234'}).encode('utf-8')).read().decode()
token = json.loads(responseData)['data']['token']

# Request the CSV data, passing the auth token:
req = urllib.request.Request(url="https://postman-echo.com/get?data=a%2Cb%2Cc%0A1%2C2%2C3%0A4%2C5%2C6%0A")
req.add_header('Authorization', 'Token '+token)
responseData = urllib.request.urlopen(req).read().decode()
csv = json.loads(responseData)['args']['data']

# Parse and return the CSV:
output_data = pd.read_csv(pd.compat.StringIO(csv))

1 Votes