As organizations scale their data operations, the traditional “single-project” cloud model often becomes a bottleneck for security, governance, and financial management. Syntasa addresses these challenges through a Cross-Project Deployment Architecture, which physically and logically separates the platform’s management functions from its data processing activities.
This architecture is defined by the isolation of the Control Plane and the Data Plane across distinct Google Cloud Platform (GCP) projects.
The Architectural Blueprint
In a cross-project deployment, the Syntasa environment is divided into two primary domains.
The Control Plane (Management Project)
The Control Plane serves as the command and control center of the Syntasa platform. It hosts the core application stack and orchestrates workflows across the environment.
Hosting
A dedicated GCP project such as syntasa-mgmt-prod.
Core Components
| Component | Description |
|---|---|
| GKE Cluster | Hosts Syntasa microservices and platform services. |
| Syntasa UI | User interface for building and managing workflows. |
| Job Orchestrator | Handles workflow scheduling and execution management. |
| Metadata Databases | Stores configurations, workflow definitions, and platform metadata. |
| Kafka Event Bus | Central messaging layer for orchestration and status communication. |
Primary Responsibilities
- User authentication and authorization
- Workflow scheduling and orchestration
- Monitoring and operational visibility
- API and service management
The Data Plane (Compute & Storage Project)
The Data Plane acts as the execution engine where workloads run and data assets reside.
Hosting
A customer-owned or business-unit-specific GCP project such as data-science-production.
Core Components
| Component | Description |
|---|---|
| Dataproc Clusters | Execute Spark jobs and compute-intensive workloads. |
| GCS Buckets | Store staging data, temporary artifacts, and final outputs. |
| BigQuery Datasets (EventStore) | Support querying, analytics, and event storage. |
| Pub/Sub Topics | Enable messaging and event-driven workflows. |
Primary Responsibilities
- Large-scale data processing
- Data storage and persistence
- External system integrations
- Runtime workload execution
Key Benefits of Isolation
Enhanced Security and Reduced Blast Radius
Separating the Control Plane and Data Plane creates a strong Zero-Trust security boundary.
Security Advantages
- Least Privilege Access:
The Control Plane requires only scoped IAM permissions such asdataproc.editorwithin the Data Plane project rather than broad administrative access. - Data Sovereignty:
Sensitive datasets remain fully isolated within the Data Plane project. Even if the Control Plane environment is compromised, project-level IAM boundaries continue protecting the underlying data assets.
Precise Cost Attribution and Billing
Separating platform operations from compute and storage workloads simplifies cloud financial management.
Financial Benefits
- Direct Billing:
Dataproc, BigQuery, and storage costs are billed directly to the Data Plane project, making departmental chargebacks and cost attribution significantly easier. - Quota Management:
Heavy data-processing workloads in the Data Plane do not compete with platform services in the Control Plane for API quotas or infrastructure resource limits.
Regulatory Compliance
For industries such as Finance and Healthcare, separating application management from data processing environments is often required for compliance.
Compliance Advantages
- Audit Integrity:
GCP Cloud Audit Logs provide a clear separation between administrative actions in the Control Plane and data-access activities within the Data Plane. - Regional Compliance:
Organizations can deploy the Control Plane in one region while maintaining the Data Plane in a different region to satisfy data residency and sovereignty regulations.
Technical Implementation Details
Cross-Project Communication
Syntasa securely connects the Control Plane and Data Plane using a high-performance integration architecture.
IAM Identity Federation
The Syntasa Service Account located in the Control Plane project is granted IAM roles within the Data Plane project. This enables secure provisioning of resources and submission of workloads across projects.
Kafka via TCP Load Balancer
To support real-time monitoring and status reporting, Syntasa exposes Kafka brokers through a Regional TCP Load Balancer (NLB).
This provides a stable cross-project virtual IP address that allows Dataproc nodes in the Data Plane to communicate back to the Control Plane, even across separate VPC networks.
Two-Tier Resolution Logic
Syntasa provides a flexible routing model to determine which Data Plane should be used for a workload.
Platform Default
A global GCP Project ID configured within the Infrastructure Settings acts as the default Data Plane destination for workloads and storage operations.
Runtime Override
Individual Runtime configurations can specify their own GCP Project ID. This enables a single Syntasa deployment to manage multiple independent Data Planes simultaneously, such as:
- Marketing workloads running in one project
- Finance workloads running in another project
Summary
The Cross-Project Deployment architecture provides the foundation for enterprise-grade data engineering within GCP environments.
By isolating the Control Plane and Data Plane, Syntasa delivers the security, scalability, governance, and financial transparency required by modern data-driven organizations while ensuring that data processing environments remain fully isolated and under customer control.