Planning an Omniscope Server Deployment: Hosting, Sizing and Setup Checklist

This article helps you plan an Omniscope server deployment for shared, multi-user access, hosted reports and scheduled workflows.

It covers where to host Omniscope, how to size the server, storage and security considerations, installation options, and the checks to complete before going live.

Use this as your general planning guide. Follow the linked articles for detailed installation, configuration and operational instructions.

TABLE OF CONTENTS

• 1. Choose where to host Omniscope
  • Choose an on-premises server when
  • Choose a cloud server or virtual machine when
  • Recommended approach for most new deployments
• 2. Start with an appropriate server specification
• 3. Size the server for your workload
  • Workflow processing
  • Hosting reports
  • Memory allocation
  • Optional performance options
• 4. Plan your storage layout
• 5. Plan network access, security and user management
• 6. Choose an installation route
  • AWS and Azure Marketplace deployments
  • Self-managed servers and virtual machines
• 7. Configure Omniscope for multi-user access
• 8. Back up, monitor and maintain the server
• 9. Pre-go-live checklist
• Related articles

1. Choose where to host Omniscope

Omniscope can run on a physical or virtual server in your organisation, or on a cloud virtual machine, for example in AWS, Azure or Google Cloud.

There is no single right choice. Select the option that best matches where your data is held, how users need to connect, and how your organisation manages infrastructure.

Choose an on-premises server when

• Your data sources, databases and file shares are mainly inside your corporate network.

• You require low-latency access to internal systems.

• Your organisation has data-residency, compliance or security policies that require data to remain on-premises.

• Your IT team already manages local servers, backups, firewall rules and certificates.

Choose a cloud server or virtual machine when

• Users need secure access from several offices, home networks or external locations.

• Your organisation prefers cloud backup, snapshots, monitoring, resizing and disaster-recovery capabilities.

• Your workload changes over time, for example a large scheduled workflow that runs overnight.

• You already operate workloads in AWS, Azure, Google Cloud or another cloud platform.

For periodic, resource-intensive workloads, a cloud VM can be configured with the CPU and memory needed for the workload, then stopped when it is not required. See Installing in the cloud, for cost-effective periodic workloads.

Recommended approach for most new deployments

Start with one dedicated Omniscope server or VM. Place it close to the data sources it will use, protect it with your corporate network controls, and monitor real usage before increasing capacity or adding additional servers.

Do not run multiple Omniscope server instances against the same Sharing folder unless you have designed and tested that architecture carefully. For most teams, one central server with users connecting through their browsers is the preferred approach.

2. Start with an appropriate server specification

Omniscope runs on 64-bit Intel architecture. The following is a typical starting specification for a small or moderate server deployment:

This is a starting point, not a fixed capacity requirement. The right size depends on the size and number of datasets, workflow complexity, scheduled jobs, concurrent users, and the number of reports being hosted.

For supported operating systems and the latest baseline requirements, see Omniscope system requirements.

3. Size the server for your workload

When planning capacity, consider:

• The largest dataset that will be processed or hosted.

• The number and complexity of workflow steps, especially joins, transformations, profiling and custom blocks.

• The number of workflows expected to run at the same time.

• The number of editors and report viewers expected at peak usage.

• The amount of report data held by the Omniscope Data Engine.

• Available temporary disk space during workflow execution.

• The capacity and response time of any external database used for live or direct-query reporting.
  
  

Workflow processing

Omniscope workflows use available memory and temporary disk space. Fast local SSD or NVMe storage is important, particularly for large ETL workloads.

As an example, a workflow processing one billion records with 20 fields may require at least 200 GB of free disk space. Actual requirements vary depending on the workflow blocks and data types in use.

By default, Omniscope runs one workflow job at a time. Additional workflow jobs are queued. You can configure concurrent execution, but each additional job shares CPU, memory, disk and Data Engine resources with other workflows and reports.

Start with one concurrent workflow job unless there is a clear operational reason to increase it. Use the Admin application to review the Workflow Job Queue and Resource Monitor before changing this setting.

For more information, see Concurrent Workflow jobs execution.

Hosting reports

Report capacity depends on both available RAM and Data Engine disk allocation. The following examples are useful starting points for report hosting with the bundled Data Engine:

These figures are based on sample datasets and should be validated with your own data, report design and user activity.

Where reports use live or direct queries against an external database, the database must also be sized and tested for interactive use. As a general guide, aim for query response times of less than 10 seconds for a satisfactory user experience.

Memory allocation

Omniscope is a Java-based application and uses the bundled Data Engine for report hosting. The Java process and Data Engine both require memory, alongside the operating system and any other services running on the server.

Leave the default Java memory allocation in place unless you have measured a specific need to change it. In most cases, increasing the memory available to the server or VM is preferable to reducing the memory available to other components.

Optional performance options

For some deployments, you may be able to reduce local storage requirements or improve performance by choosing a different data-processing architecture:

• Live query reporting: run report queries directly against your database or cloud data warehouse, rather than loading the full dataset into the local Data Engine. See Database source - querying data in live query mode.

• In-memory workflow execution: keep workflow processing data in memory rather than using temporary disk storage. See In-Memory Workflow Execution Mode.

• Fully in-memory processing and reporting: run both workflow processing and report querying in memory on suitably sized infrastructure. See Running Omniscope Fully In-Memory: Processing and Reporting.

These are optional advanced configurations. Start with the standard deployment unless you have a measured performance or architecture requirement.

4. Plan your storage layout

Use persistent local storage for Omniscope projects, configuration and operational data.

For best performance:

• Store Omniscope projects and the Sharing folder on a local SSD attached to the Omniscope server.

• Use SSD storage at minimum; use NVMe where large workflows or high report activity are expected.

• Keep adequate free capacity for workflow temporary files and Data Engine data.

• Consider separate fast disk locations for workflow temporary data and Data Engine storage on larger deployments.

• Include Omniscope project files, configuration and logs in your storage and backup design.

• Avoid relying on a network drive for the main Sharing folder where possible, as network latency or connectivity issues can affect performance and reliability.

The Sharing folder contains the projects, workflows, reports and files served by Omniscope. By default, it is called files and is usually located within the omniscope-server folder in the account running Omniscope, for example:

/home/myusername/omniscope-server/files

You can change the Sharing folder location in the Admin application. Plan this before go-live, as moving the folder requires care when transferring existing files and restarting Omniscope.

For more information, see:

• Disk space and performance

• Projects folder / Sharing folder configuration

5. Plan network access, security and user management

Before installing Omniscope, agree how users will reach the server and how access will be protected.

Plan the following:

• A stable DNS name for production access.

• Whether users will connect only from the corporate network, through a VPN, or from outside the organisation.

• Firewall, private-network, reverse-proxy or identity-aware access controls.

• HTTPS and the management of a trusted SSL/TLS certificate.

• The authentication method for users and administrators.

• User groups and folder permissions.

• Whether external users will be allowed to access projects or reports.

Use HTTPS for production access, particularly when accessing the Omniscope Admin application. A self-signed certificate may be suitable for testing, but production users should access the server using a trusted certificate.

Restrict access using your firewall, VPN, private network or reverse proxy. Do not expose the Omniscope administration interface directly to the public internet.

Omniscope supports local user accounts as well as identity-provider and directory-based authentication methods, including OpenID Connect, LDAP and SPNEGO. Choose an approach that fits your organisation’s identity-management policy.

For more information, see:

• Security: Web Server SSL configuration

• Setup HTTPS using Let’s Encrypt on Linux

• User authentication mechanisms

• Sharing Omniscope reports and projects: authentication and permissions

6. Choose an installation route

Pre-built cloud images

AWS, Azure and Google Cloud provide pre-built Omniscope server images, so a separate manual Linux installation is not required.

After provisioning the virtual machine, follow the relevant guide to activate Omniscope, create the administrator account and configure server access:

• AWS Marketplace Omniscope setup
• Azure Marketplace setup
• How to launch Omniscope on Google Cloud

For Google Cloud, the pre-built image is launched from Visokio’s public image project using the Google Cloud command-line tools; it is not selected from the Google Cloud Marketplace or standard image picker.

Do not also follow the manual Linux installation guide on a VM created from one of these pre-built images.

AWS and Azure Marketplace deployments

AWS and Azure Marketplace provide pre-built Omniscope server images. This is usually the simplest route for a new deployment in either cloud, because Omniscope is already installed on the virtual machine image.

After provisioning the VM, follow the relevant Marketplace guide to activate Omniscope, create the administrator account and configure server access. The Marketplace guides cover the first-time setup process, including the temporary SSH connection required by the image.

• AWS Marketplace Omniscope setup

• Azure Marketplace setup

A suitable Omniscope server licence is required to activate the Marketplace image. Preserve the VM’s persistent disk after activation, as activation and configuration information are stored on the server disk.

Do not follow the manual Linux installation guide as well as the Marketplace guide on the same Marketplace VM.

Self-managed servers and virtual machines

Choose a manual installation when Omniscope will run on an existing or customer-managed server or virtual machine, rather than on an AWS or Azure Marketplace image.

Use the relevant guide for the selected platform:

• Install Omniscope on Windows

• Linux Installation

• Install Omniscope as a Windows Service

• Omniscope on Docker Hub

7. Configure Omniscope for multi-user access

After Omniscope has been installed and activated, configure it for browser-based multi-user access.

The main configuration tasks are:

1. Create and securely store the Omniscope administrator credentials.

2. Enable the Omniscope Web Server when external user access is required.

3. Configure the server’s external address, DNS name and HTTPS settings.

4. Confirm that firewall, VPN, reverse-proxy or cloud network rules permit access for the intended users.

5. Configure the Sharing folder.

6. Configure user authentication, groups and folder permissions.

7. Test access from a representative user device and network.

By default, Omniscope only permits access from the machine where it is running. When the Web Server is enabled for external use, users can access hosted projects and reports through the server address and configured port. The default HTTP port is 80.

For cloud deployments, configure the external base URL in Omniscope so that shared links use the correct public domain name or address.

Follow Configure Omniscope as Server for multi-user access for the detailed configuration steps.

8. Back up, monitor and maintain the server

Include Omniscope in your normal server operations processes.

At a minimum:

• Back up the Omniscope server configuration and Sharing folder regularly.

• Use VM or disk snapshots before an upgrade or migration.

• Test that backups can be restored.

• Monitor free disk space, CPU, RAM and disk I/O.

• Monitor workflow duration, queued jobs and concurrent user activity.

• Review server logs when troubleshooting.

• Keep the operating system, certificate configuration and Omniscope version under regular maintenance.

• Review antivirus settings and configure the appropriate exclusions where this is permitted by your security policy.

Before an upgrade, back up Omniscope project files, server configuration, internal application data and any custom system configuration. Keep the backup until the upgraded server has been tested successfully.

Before migrating to a new server, build and test the new environment before removing the existing one. Take a full backup or VM snapshot of the original server first.

For more information, see:

• Upgrading Omniscope to a latest version

• Migrate an Omniscope server installation from one machine to another

• Antivirus exclusions

9. Pre-go-live checklist

Before making the server available to users, confirm that:

•  The server or VM meets the expected CPU, RAM and SSD capacity.

•  The largest expected workflow and report have been tested successfully.

•  There is sufficient free disk space for projects, Data Engine data and workflow temporary files.

•  The Sharing folder is on persistent storage and is included in backups.

•  Administrator credentials have been configured and secured.

•  HTTPS is enabled with a trusted certificate.

•  Firewall, VPN or reverse-proxy rules limit access to the intended users.

•  Authentication, groups and folder permissions have been tested.

•  Scheduled workflows have been tested where applicable.

•  Monitoring, backup and recovery procedures are in place.

•  A rollback plan is available before upgrades or major configuration changes.

•  The external base URL has been configured where users will access Omniscope through a public domain name or reverse proxy.

•  A representative user can open the required projects and reports from their intended network location.

Related articles

• Omniscope system requirements

• Configure Omniscope Evo Business or Enterprise as Server for multi-user access

• AWS Marketplace Omniscope setup

• Azure Marketplace setup

• How to launch Omniscope on Google Cloud

• Installing in the cloud, for cost-effective periodic workloads

• Disk space and performance

• Projects folder / Sharing folder configuration

• Concurrent Workflow jobs execution

• Security: Web Server SSL configuration

• Setup HTTPS using Let’s Encrypt on Linux

• User authentication mechanisms

• Sharing Omniscope reports and projects: authentication and permissions

• Upgrading Omniscope to a latest version

• Migrate an Omniscope server installation from one machine to another

• Antivirus exclusions