Spark Engine Configuration | Collate Spark Profiling Setup

Learn how to configure your profiler pipeline to use Spark Engine for distributed data profiling.

Spark Engine Configuration

Overview

There are two ways to configure Spark Engine in Collate:

UI Configuration

Configure Spark Engine through the Collate UI.

External Configuration

Configure Spark Engine using YAML files for external workflows.

Before configuring, ensure you have completed the Spark Engine Prerequisites and understand the Partitioning Requirements.