Slow folder listings on network or cloud file storage (Azure Files, NFS, SMB)

If your Omniscope sharing folder lives on network or cloud file storage, such as an Azure file share, an NFS export or a mapped SMB drive, you may find that folders containing many projects take several seconds to open, and that sorting a folder by size takes minutes.

What you might see

• Opening a folder shows the loading spinner for a long time, and the delay grows with the number of projects in it. For example, a folder of 40 projects might take 6 seconds to open, and one of 200 projects 14 seconds or more.
• The wait happens again every time you return to the folder, for instance when you leave a project.
• Sorting the folder by size is much slower again, sometimes several minutes.
• The same folders open in well under a second on a server with local SSD storage.

Why it happens

To build a folder listing, Omniscope reads some file information for every project: its modified date, its file size, its description and whether it has a thumbnail. It also shows the size of each project's data, which it works out by adding up every file in the project's data folder. A project that has been executed can hold hundreds of files there.

On a local disk each of those lookups takes a fraction of a millisecond, so even a large folder lists quickly. On network storage, every lookup is a round trip to the file server. Azure Files in particular is slow at this kind of work: Microsoft describes listing directories and reading file information as "metadata-heavy" operations, with higher latency than ordinary reads and writes. A folder of a few hundred executed projects can mean tens of thousands of round trips.

Sorting by size needs the size of every project before anything can be shown, and the sort compares each project with several others, and each comparison measures both projects' data folders again, so each data folder is measured many times over.

Confirm the storage is the cause

On the server (or inside the Omniscope container), time a plain listing and a size count of one of the slow folders, then do the same for a copy of that folder on local disk:

time ls -l "/path/to/files/Slow folder" > /dev/null
time du -s "/path/to/files/Slow folder"/*.iox.data

Project data folders are named after the project with .data added, for example Sales.iox.data. If you don't want to copy a large folder, compare with any local folder holding a similar number of files. If these take seconds on the share and a fraction of a second locally, the storage latency is what you are seeing in Omniscope. In the Azure portal you can also look at the storage account's Transactions metric split by API name; a slow listing shows up as large numbers of QueryDirectory, QueryInfo, Create and Close calls.

If the timings are fast and Omniscope is still slow, please capture diagnostics while it is slow, as described in Diagnostics for poor server performance, and send them to us.

Option 1: switch off the data size calculation

This gives the biggest improvement on slow storage. Starting Omniscope with the following JVM argument stops it adding up the data folder of each project:

-DdisableDataFolderSizeDetermination=true

With it set:

• Folder listings no longer read every file in each project's data folder.
• The data size shows as <disabled> in the size information for each project.
• Sorting by size still works, but uses the size of the project file only, not its data.

To confirm it has taken effect, open a folder in list view: the size information for each project shows <disabled> for its data.

The argument must be set on every Omniscope server process that serves folder listings. In a multi-node deployment, that means the editor node and every viewer node. How you add it depends on how Omniscope is installed.

Docker and Docker Compose

Add it to the JAVA_TOOL_OPTIONS environment variable of each Omniscope service. If the service already sets JAVA_TOOL_OPTIONS (the multi-node reference deployment uses it for java.io.tmpdir and, on viewers, omniscope.nodeType), add the new argument to the existing value, separated by a space. For example, for a viewer node:

environment:
  JAVA_TOOL_OPTIONS: >-
    -Domniscope.nodeType=viewer
    -Djava.io.tmpdir=/home/omniscope/omniscope-server/files/.tmp/viewer-1
    -DdisableDataFolderSizeDetermination=true

If your service uses the list form of environment:, put the whole value on one line instead:

environment:
  - JAVA_TOOL_OPTIONS=-Domniscope.nodeType=viewer -Djava.io.tmpdir=/home/omniscope/omniscope-server/files/.tmp/viewer-1 -DdisableDataFolderSizeDetermination=true

Keep your own existing values; the paths above are from the reference deployment. Recreate the containers (docker compose up -d) and check each container's log for the line Picked up JAVA_TOOL_OPTIONS: followed by your arguments.

Kubernetes

Set the same JAVA_TOOL_OPTIONS environment variable in the container spec of each Omniscope deployment or stateful set (or in your Helm values or ConfigMap, if that is where it is set), adding to any value already there, and roll out the change.

Linux (standard installation)

Edit _launch.sh in the installation folder. Find the JAVA_VM_PARS line and add the argument at the end of the existing value, separated by a space, just before the closing quote. For example, if the line reads:

JAVA_VM_PARS='-Xms64M -Xmx8G'

change it to:

JAVA_VM_PARS='-Xms64M -Xmx8G -DdisableDataFolderSizeDetermination=true'

Restart Omniscope. Note that upgrading Omniscope may replace this file, so check it after each upgrade.

Windows

Edit installconfig.properties in the installation folder (for example C:\Program Files (x86)\Visokio Omniscope\ or C:\Users\[your name]\AppData\Local\Visokio Omniscope app\). Find the line beginning ADDITIONAL_JVM_ARGS=, removing the # at the start if it is commented out, and add the argument:

ADDITIONAL_JVM_ARGS=-DdisableDataFolderSizeDetermination=true

If the line already has other arguments, add this one at the end, separated by a space. If Omniscope runs as a Windows service, also add it to wrapper.conf in the service subfolder of the installation folder, as a new wrapper.java.additional line numbered one higher than the highest existing one (the numbers must not have gaps). For example, if the last existing line is wrapper.java.additional.9:

wrapper.java.additional.10=-DdisableDataFolderSizeDetermination=true

Restart Omniscope or the service.

Option 2: tune the Azure file share

These changes speed up all of Omniscope's file access on Azure Files, not only folder listings. Most are Microsoft's own recommendations for metadata-heavy workloads; see Microsoft's Azure Files performance troubleshooting guide for the full list.

Use SSD (premium) file shares

Azure file shares come in SSD (premium) and HDD (standard) tiers. SSD shares have much lower latency per operation, which is what matters here. Note that the share's tier is separate from the disks attached to your virtual machines: premium SSD VM disks do not make an HDD file share faster. You can check the tier on the storage account's overview page in the Azure portal.

Enable metadata caching

For SSD SMB shares, Azure offers metadata caching, which caches file and directory lookups and, according to Microsoft, reduces metadata latency by 30% or more. Microsoft's page describes how to register for it: Improve SMB Azure file share performance.

Check the Linux mount options

The Linux SMB client caches file and directory information for only 1 second by default, so a listing repeated a few seconds later goes back to the server for everything. Microsoft recommends mounting Azure file shares with a longer cache time. Check your current options with:

mount | grep cifs

Make sure the options include actimeo=30 (Microsoft recommends 30 to 60 seconds). serverino and cache=strict should also be present; they are the default on current Linux kernels, but older systems may lack them. A typical /etc/fstab entry looks like this:

//ACCOUNT.file.core.windows.net/SHARE /mnt/omniscope-files cifs _netdev,nofail,credentials=/etc/smbcredentials/ACCOUNT.cred,serverino,nosharesock,actimeo=30,mfsymlinks,cache=strict 0 0

Keep any uid, gid, file_mode or dir_mode options you already use, so Omniscope can still write to the share. Unmount and remount the share (or reboot) for new options to take effect, then confirm with mount | grep cifs again.

If you mount the share through a Docker volume rather than on the host, put the same options in the volume's o driver option. On Azure Kubernetes Service, put them in the mountOptions of the Azure Files storage class; Microsoft's AKS Azure Files guide shows the recommended set, which includes actimeo=30 and cache=strict. For a statically provisioned volume, the options go in the persistent volume's mountOptions instead.

A longer cache time means a change made by one server can take up to that long to appear on another. For Omniscope's shared projects folder, 30 seconds is a reasonable balance.

Keep everything in one region

Make sure the storage account and the Omniscope servers are in the same Azure region. Every lookup pays the network latency between them, so a share in a different region multiplies the delay.

Option 3: organise large folders into subfolders

Listing time grows with the number of projects in the folder being opened, not with the total in the sharing folder. Splitting a folder of several hundred projects into subfolders, for example by team, client or year, keeps each listing quick. It needs no restart or configuration change, so it can also help while you make the other changes.

Other storage choices

Where you have the choice, local SSD storage gives the best performance for the sharing folder. Multi-node deployments need storage that every node can reach, so a network share is unavoidable there; in that case SSD file shares with metadata caching, or an NFS share on the SSD tier, are the best-performing Azure options. The same principles apply to other network storage, such as Amazon EFS, Google Filestore or an on-premises NAS: compare them on the latency of each operation rather than on throughput figures.

See also Disk space and performance and Diagnostics for poor server performance.

Last updated on 5 November 2026