Omniscope's Custom Blocks come in two varieties and two execution modes: Python and R, and host execution and Docker execution. In order for its usage to be seamless, these varieties and modes need to be setup in an orderly fashion, otherwise you might experience problems. The source of most of the problems experienced with Custom Blocks is package and library conflicts. Omniscope itself tries to mitigate potential conflicts, but if the underlying system is in a problematic state then any mitigation might fail.

In case you run into problems, this guide will hopefully help you solve them:

Logs

Both R and Python Custom Blocks have specialised logs that can help identify the cause of the problems experienced. You can find them in the admin section "Logs". Of particular importance are the R and Python host package installation logs, which contain information about potential problems during package installation on your host system, and the custom block execution log, which will tell you what happened during execution. This file is especially important when you run Custom Blocks in Docker containers because problems during package installation in Docker containers are reported there.

Old blocks

In case you execute a workflow containing Community Blocks, which was created at an earlier point in time, and the workflow execution now fails, it's possible that it is because those Community Blocks are out of date. Over time, the R and Python landscape changes. New R and Python being updated packages, which might deprecate functionality used within these blocks. If this is the case, the solution is easy: Simply replace the Community Block in the workflow, with a fresh one from the Block palette. This is especially true for print-related blocks which has seen a lot of change in recent years.

Using Docker

Most problems with Custom Block executions come from package conflicts. This can mean, conflicts between packages, conflicts between packages and the version of the script language, conflicts between packages and system libraries, and even conflicts between the host system and the packages / script language itself.

One convenient way to leave these kind of problems behind is to utilise Docker in Custom Blocks. Docker is a tool to containerise a Custom Bock execution which provides a well defined and fresh environment in which the execution is performed. This excludes automatically all sort of conflicting scenarios and ensures that your Custom Blocks run in the same manner regardless on the type of machine they are used on.

Please have a look at this article in order to get started with Docker containers in Omniscope.

System libraries

It is possible that your Custom Block contain some Python or R package dependencies which themselves depend on non-R or non-Python system libraries. These are libraries that are not installed with the Python or R package installers, but need to be installed from the outside. E.g. in Linux systems this would amount to install a library using apt install, or apt-get install. On Mac OS this could be done via Homebrew, and on Windows this could mean downloading an installer for a certain library and executing it.

These libraries, once installed, are not automatically updated. Over time, it is possible they run out of sync with newer Python or R libraries and lose compatibility. The installation logs / custom block execution logs should be able to shed light on whether that is, or is not the case. If it is the case, then affected system libraries need to be upgraded to a compatible (or latest) version.

Upgrading Python & R

Most R and Python packages are maintained with respect to the most recent version of either R or Python. Once R or Python are not upgraded in a while, they might become incompatible with current package version. Therefore it is highly recommended to keep R and Python up to date.

Upgrading Docker

Docker is an application which can be downloaded and installed from here. With Docker, Omniscope is able to execute other software (e.g. R or Python) in a container virtual environment. This not only brings with it enormous security advantages, but it also lets Omniscope completely control the environment in which R and Python is executed. That means it can create a setup containing exactly the R and Python version it needs, with exactly the libraries and packages that the block depends on.

When using Docker, it is not necessary to keep Python and R up to date, or worry about packages and package management at all. Omniscope makes sure that a docker image is used which is compatible, and which contains everything needed.

That being said, it is still possible for Custom Block executions to go wrong. Typically, new images also need new Docker versions, so if you encounter unexplained issues, upgrade Docker to its newest version.

Purging Docker containers and images

Once a Docker image is first built, it will stay as it is. This provides great continuity, but at some point the images might become outdated. Omniscope removes outdated containers itself, but it is nonetheless advisable to remove images and containers yourself in case you encounter unexplained issues. 

In case you are using Docker exclusively for Omniscope, it is sufficient to stop Omniscope, open up a console and run the following command:

docker system prune -a

Purging local Python & R libraries

In case you are running Python and R locally on your computer, and not in a Docker environment, it can be helpful to purge your Omniscope-installed R and Python packages. Omniscope cleans up its environment when new versions are installed, but if an Omniscope installation is used for a long time, it is possible that package version conflicts creep into the system. A simple solution is to locate the folder where these packages are installed, remove them, and restart Omniscope. Omniscope will then re-install packages whenever needed with their newest, and hopefully more-compatible versions.

The locations in which Omniscope installs R and Python packages are as follows (where XXXXX stands for the user running Omniscope):

• Windows:  C:\Users\XXXXX\AppData\Local\Visokio
• MacOS: /Users/XXXXX/Library/Application Support/Visokio
• Linux: /home/XXXXX/.visokioappdata/Visokio

In this folders you will see various subfolder named python3.6.15, or R4.3.0. It is best to remove all of the folders with this naming schema.

Removing conflicting global packages

Sometimes you have R or Python packages installed globally which conflict with those that Omniscope installs locally. E.g. if a globally installed Python package is available, Omniscope will use it and not install a local version. If that global package has a version which is very old, then this can lead to versioning conflicts, or other instabilities which lead to errors. It is difficult to spot these kind of conflicts, but the logs should help figuring out if e.g. a Custom Block dependency cannot be installed, because no compatible version could be located.

To solve this problem, it is best to remove (or upgrade) the conflicting global packages. Afterwards Omniscope needs to be restarted for the new environment to take effect.

Python Hyphen / Underscore issue

Frequently a Python error appears after installing a dependency, that a package was installed, but Omniscope can't find it. This usually happens when the package name contains a hyphen and is misspelled in the dependency section of the Custom Block configuration.

E.g. let's say there is a package called "test-package". Now, when I specify the dependencies, and put "test_package" (notice the underscore) into the section, unfortunately Python will happily correct the underscore to a hyphen and install the package nonetheless. When Omniscope later looks for "test_package", it will not find it, because it was installed under its proper name "test-package". It is easily corrected by verifying all dependencies are actually correctly specified.