How to run python on Omniscope?

Thanks for your reply.

However, I know how to use record filter indeed.

Let's go back to the example. So for a limited amount of data record from one table. It's doable certainly to enter manually using filter block, while it isn't efficient to check via filter block if you have 100 K of records  

Is there any way you can run automatically?  And also, it's not 100% equal because there are lots of entries containing something like "David_zhong" or "Zhong_David".

And hence I was thinking I can write a script in python to do it for me. I still try to get my head around the customer block and there is not a lot of tutorials and discussion online. It's really headache to make it work. Can you explain a little bit that why input_data=output_data in default script and what does output_data refer to? 

Thanks

I'm familiar with writing scripts on Jupyter notebook. 

Thanks for your reply. really helpful.

correct me if I'm wrong. For the article you share,  you can write SQL queries on the file once you created database that is connected (In my case, once I filled in host, etc for MySQL). Regarding SQL queries, I'm wondering how good it is. So can I write any SQL queries? 

Let me give you more details. Any advice would be super appreciated.

In the article you shared,  the scenario objective is to create  a table to filter out those are in the categories. However,  what if the scenario objective is to check whether a data entry is in our established database? 

My idea is to use SQL Like to match them and return some sort of result to show me % they're likely in database.  The difference between the dataset Fifa 2019 football player  from Kaggle and the social media dataset we're dealing with is whether it is factual or not. In other words, any Fifa dataset you get is nearly 100% matching with data stored into database, because it contains fields such as player name, age, nationality, value, club.  

What if there is a dataset containing double idenity like social media.

Do you have any idea based on what I described? 

Thanks

David

Hi, 

I'm not a developer but I'm exploring some ideas for the recent project. One of them is to write script to automate some sort of process.

I've read all the guideline and those are less applicable to my project. But is it not like writing jupyter notebook?

For connector, I was asking for write something like

!pip install ipython-sql
!pip install pandas

#load in SQL module you just installed
%load_ext sql
import pandas as pd

%sql  mysql://[IP address]


%%sql

#select one column from the database

real_name= %sql SELECT Name FROM view_authors
df = real_name.DataFrame()

So I can write SQL via python but actually is there any way like you said we can write SQL to interact with database since we don't have problem with connecting database. 

To answer your question, it's MySQL. 

Just to add that In general to connect to a database you can exploit any JDBC driver and have Omniscope Database Source block dealing with connection / querying etc.  Simpler than writing your Python connector, of course unless the JDBC driver for that DB is not available. 

Please let us know if you still need some help.

Hello David,

Do you refer to this guide ? https://help.visokio.com/support/solutions/articles/42000071270-getting-started-custom-scripts-in-omniscope-evo 

Also notice this is our GitHub repo for custom blocks, some written in Python. You could check the code there as examples https://github.com/visokio/omniscope-custom-blocks

By the way, to which database your are trying to connect? 

Sorry, I realized I have to provide more details. I want to write script to connect with our database. So I'm not clear how it works. In a normal situation, the user guide for writing python script is not that useful.