Vortex Qlik Harvester
What It Is
The Vortex Qlik Sense Metadata Harvester is a portable command-line tool designed to extract comprehensive metadata from Qlik Sense environments. It supports both Qlik Sense Enterprise on Windows (QSEoW) and Qlik Cloud deployments.
The tool is a 64-bit Windows executable that requires minimal installation—you simply download, extract, and run it.
Purpose
The harvester serves as the first step in any BI modernization project supported by Vortex. Its core purpose is to collect metadata that enables data-driven decisions about how to approach consolidation, modernization, or migration efforts.
What It Extracts
The harvester queries over 40 different metadata object types, including:
- Application metadata — apps, sheets, objects, measures, dimensions
- Environment information — streams/spaces, users, data connections
- Script and lineage information
- Session and reload logs
- Folder data connection contents
- Security and governance information
- Engagement patterns and usage statistics
How This Metadata Is Used
Once collected, the metadata enables several key activities:
Use Case | Description |
Rationalization | Uncover insights about connections, lineage, and engagement patterns to prioritize what matters
|
Duplicate Detection | Identify derivative or duplicative content across your analytics landscape |
Complexity Assessment | Factor in structural complexity to estimate conversion effort |
Automated Conversion | Power Vortex's automated solution conversions to target BI platforms |
Project Planning | Generate inventories, project scaffolding, and runbooks |
Migration Approaches Supported
Whether you're doing a "lift-and-shift," a "rationalize-and-reimagine," or something in between, the harvester provides the foundational data needed to accelerate delivery with confidence.
Profile (Configuration File)
A profile in the Vortex Qlik Sense Harvester is the configuration file that stores all the settings needed to connect to and harvest metadata from a Qlik Sense environment.
File Details
- Filename: config.local.json
- Location: Stored in the harvester utility's directory
- Format: JSON
What It Contains
The profile defines all the parameters for a harvesting run, including:
Connection Settings
Parameter | Purpose |
Uri | Qlik Sense server hostname or Qlik Cloud tenant URL |
Type | Environment type (Enterprise or Cloud) |
CertificateLocation | Path to the .pfx certificate file (Enterprise only) |
CertificatePassword | Certificate password or API key (Cloud) |
RunAsUserDir | User directory for authentication |
RunAsUserName | Username or service account |
Output & Security Settings
Parameter | Purpose |
OutputLocation | Where harvested metadata is saved |
EncryptionKeyFilePath | Path to PGP public key for encrypting output |
Scope & Filtering Settings
Parameter | Purpose |
StreamsToSearch | Limit harvest to specific streams |
StreamsToIgnore | Exclude specific streams |
AppIds | Harvest only specific applications |
IncludeUnpublishedApps | Whether to include unpublished apps |
Performance Settings
Parameter | Purpose |
MaxConcurrentEngineAppConnections | Number of simultaneous engine connections |
HarvestFolderDataConnectionData | Enable folder data connection scanning |
MaxFileSearchDepth | Directory depth for folder scanning |
How to Configure
You can update the profile only in the UI:
- Through the UI — When you launch the harvester, enter the parameters required.
Inventory
Definition
The Inventory is a comprehensive catalog of all objects discovered during the Vortex harvesting process from a Qlik Sense environment. It serves as the foundational dataset for understanding the scope and composition of your analytics landscape.
What the Inventory Contains
Applications
- Application name and ID
- Owner of the application
- Published or unpublished status
- Last modified date
- Associated reload logs (if available)
- Structural composition including:
- Sheets and sheet objects
- Measures and dimensions
- Scripts and data models
- Expressions
Streams/Spaces
- Stream or space name and ID
- Applications published to each stream
- Stream ownership information
Data Connections
- Connection names and types
- Associated applications using each connection
Users
- Licensed and allocated users
- User directory information
Tasks
- Reload tasks configured in the environment
- Execution results and script log information
Purpose of the Inventory
Purpose | Benefit |
Scope Assessment | Understand the full extent of your Qlik environment |
Rationalization | Identify redundant, unused, or duplicative content |
Lineage Analysis | Trace data connections and dependencies across applications |
Engagement Insights | Analyze usage patterns through session and reload logs |
Migration Planning | Prioritize applications for conversion based on ownership, usage, and complexity |
Resource Allocation | Estimate effort for manual vs. Vortex-assisted conversions |
How the Inventory Is Generated
The Inventory is built from metadata extracted via Qlik's official APIs, including the Repository Service API and Engine JSON API. The harvester collects this information and stores it as encrypted files (.pgp) in the configured output location.
Harvest
Overview
The Harvest is the process of scanning and extracting metadata from a Qlik Sense environment using the Vortex command-line tool.
Important: A valid Vortex license (public key file) supplied by Axis Group is required to run the harvester. Without this license, the tool will not function.
What the Harvest Does
The harvest process connects to your Qlik Sense environment and systematically extracts metadata using officially available APIs. This includes:
- Application metadata — apps, sheets, objects, measures, dimensions, scripts, data models, and expressions
- Environment information — streams/spaces, users, data connections
- Execution data — reload tasks, script logs, session logs
- Lineage information — connections and dependencies between objects
- Usage statistics — engagement patterns and last modified dates
All extracted metadata is encrypted using a PGP public key and stored in the configured output location.
Harvest Requirements
Licensing
- Valid public key / license file supplied by Axis Group
System Requirements
Requirement | Specification |
Operating System | Windows 64-bit |
CPU | 4-core (recommended) |
RAM | 8 GB (recommended) |
Disk Space | 2-3 GB for extracted metadata |
Framework | .NET Framework 4.7.2 |
Qlik Sense Access
| Environment | Authentication |
Enterprise (QSEoW) | Certificate-based authentication via exported .pfx file |
Cloud | API key generated by a Tenant Admin |
Network Ports (Enterprise)
- 443 — Qlik Sense Proxy Service (QPS)
- 4242 — Qlik Sense Repository Service (QRS)
- 4747 — Qlik Sense Engine Service (QES)
Harvest Process
- Configure — Set up the profile (config.local.json) with connection details, output location, and scope filters
- Execute — Launch the harvester and press Enter to begin
- Monitor — Watch progress as the tool reports App Harvest, QRS Harvest, and Folder Harvest status
- Validate — Review the output directory to confirm metadata was captured correctly
- Transfer — Send encrypted output to Axis Group via secure file transfer (psend.axisgroup.com)
Performance Characteristics
- Processes approximately 3 apps/minute at default concurrency settings
- Concurrency can be adjusted via MaxConcurrentEngineAppConnections to balance speed vs. server load
- The harvest consumes memory and CPU capacity from your Qlik environment during operation
Who Can Run the Harvest
The harvest can be executed by:
An Axis Group engineer as part of a services engagement
A representative analytics administrator from your organization with support from Axis Group
Note: At customer discretion, the harvest can initially be performed against a development or test environment to evaluate impact before running against production.
Harvest History
Overview
Important: A valid Vortex license (public key file) supplied by Axis Group is required to run the harvester and generate harvest history. Without this license, the tool will not function.
Harvest History is a record that combines two key elements: a log of previous harvest executions and the historical metadata captured during each harvest run.
What Harvest History Contains
Execution Logs
- Record of each harvest run performed
- Timestamps of when harvests were executed
- Status and completion information
- Any errors encountered during harvesting (captured in Harvest Errors.json)
- Log files stored in the Logs directory within the utility folder
Historical Metadata Captured
Data Type | Description |
Reload Logs | Execution history of application data reloads (limited by automatic cleanup of historical logs) |
Session Logs | User session activity and engagement patterns |
Execution Results | Script log execution information for lineage analysis |
Last Modified Dates | Timestamps showing when applications were last changed |
Task History | Information about scheduled reload tasks and their execution |
Purpose of Harvest History
Harvest History enables organizations to:
- Track Changes Over Time — Compare metadata across multiple harvests to identify environment changes
- Analyze Usage Patterns — Understand application engagement through historical session data
- Support Lineage Analysis — Use reload and script execution history to trace data flows
- Audit Harvest Operations — Maintain a record of when and how harvests were performed
- Troubleshoot Issues — Review logs to diagnose problems with specific harvest runs
Where Harvest History Is Stored
- Execution Logs — Logs directory within the Vortex harvester utility folder
- Historical Metadata — Output location configured in the profile, including:
- Reload Logs subfolder
- Session Logs subfolder (QlikView)
- Harvest Errors.json file
Harvest Results
Overview
Important: A valid Vortex license (public key file) supplied by Axis Group is required to run the harvester and generate harvest results. Without this license, the tool will not function.
Harvest Results are the complete output generated from a Vortex harvest run. This includes all extracted metadata, reports, and logs from a scanned Qlik environment, stored in the profile's configured output directory.
What Harvest Results Contain
Output Type | Description |
Application Metadata | Encrypted .pgp files containing scripts, sheets, sheet objects, measures, dimensions, and data models |
Repository Information | QRS data including streams, users, data connections, and tasks |
Reload Logs | Execution history of application data reloads |
Session Logs | User session activity and engagement data |
Harvest Errors | Harvest Errors.json file documenting any issues encountered during the harvest |
Execution Logs | Harvester operation logs stored in the Logs directory |
Output Directory Structure
- OutputLocation\
- ├── Apps\ # Application metadata by app
- │ └── [App folders]\
- │ ├── script.json.pgp
- │ ├── sheets.json.pgp
- │ └── sheetObjects.json.pgp
- ├── QRS\ # Repository service data (.pgp files)
- ├── Reload Logs\ # Historical reload execution logs
- ├── Folder Data Connections\ # File metadata (if enabled)
- └── Harvest Errors.json # Error log (if any issues occurred)
Transmit
Transmitting Harvest Results
Once a harvest is complete, the results must be securely transmitted to Axis Group for decryption, analysis, and conversion.
Packaging Process
- Select Profile — Choose the profile whose output you want to package
- Verify Output Directory — Confirm the correct output location is selected
- Create Package — Generate an encrypted package containing all metadata, reports, and logs
- Upload — Transmit the package to Axis Group via the secure upload portal (psend.axisgroup.com)
What Gets Packaged
- All contents of the output folder (Apps, QRS, Reload Logs, etc.)
- Harvester logs from the Logs directory
- Any additional files in the base output directory
Secure Transfer Methods
Method | Details |
Primary | |
Alternatives | SFTP, BLOB storage, or SendRED (if psend is not accessible) |
Purpose of Harvest Results
Harvest results enable Axis Group to:
- Analyze your environment — Build a complete inventory of your Qlik landscape
- Perform rationalization — Identify redundant or duplicative content
- Execute automated conversions — Transform Qlik assets to target BI platforms
- Generate project deliverables — Create inventories, runbooks, and project scaffolding
Getting Help
For additional assistance or questions about the Vortex Harvester:
- Email: support@axisgroup.com
- File Transfer: psend.axisgroup.com
Additional Resources
- For detailed configuration information, see the Vortex Harvester Configuration Help documentation
- For system requirements and troubleshooting, consult your Axis Group representative
© 2024 Axis Group - Vortex Menu Definitions Documentation
Last Updated: 2025-12-30