Big Data Integrations

One of several reasons Syntasa is able to greatly speed up the building of data pipelines is the number and variety of integrations available from the platform. The variety of integrations include big data, martech, and adtech, each group having numerous examples that are being added to on a regular basis.

This article provides a quick overview of the various big data integrations available:

Athena

Amazon Athena is a serverless, interactive query service that makes it easy to analyze big data in S3 using standard SQL. Athena integration allows users to run complex queries directly on Athena tables. It also provides access to load the data from Athena tables and perform more complex transformations and modeling. This integration also has the capability to make the processed data available back again in your Athena tables and views.

BigQuery

BigQuery is a serverless, highly scalable, and cost-effective data warehouse designed to help turn big data into informed business decisions. BigQuery integration provides easy access to enterprise data available in BigQuery. It allows users to run complex queries directly on BQ tables. It also provides access to load the data from BQ tables and perform more complex transformations and modeling. This integration also has the capability to load the processed data back to BQ tables and views.

Databricks

Databricks is a unified data analytics platform on the cloud for massive-scale data engineering and collaborative data science. Databricks integration provides the ability to launch Runtimes that are powered by Databricks clusters. This allows the users to seamlessly work with Databricks clusters with the click of a button. This integration automates the entire life cycle of cluster creation, job submission, and cluster termination as part of job workflow.

Dataproc

Dataproc is a GCP cloud-native distributed data processing engine for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Jupyter notebooks, Zeppelin notebooks, etc.

This Dataproc integration provides the ability to launch Runtimes that are powered by GCP Dataproc clusters. This allows the users to seamlessly work with Dataproc clusters with the click of a button. This integration automates the entire life cycle of cluster creation, job submission, and cluster termination as part of job workflow.

Elastic MapReduce (EMR)

EMR is an AWS cloud-native distributed data processing engine for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Presto, Jupyter notebooks, Zeppelin notebooks, etc.

This EMR integration provides the ability to launch Runtimes that are powered by AWS EMR clusters. This allows the users to seamlessly work with EMR clusters with the click of a button. This integration automates the entire life cycle of cluster creation, job submission, and cluster termination as part of job workflow.

HDInsight

HDInsight is an Azure cloud-native distributed data processing engine for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Jupyter notebooks, Zeppelin notebooks, etc.

This HDInsight integration provides the ability to launch Runtimes that are powered by AWS EMR clusters. This allows the users to seamlessly work with HDInsight clusters with the click of a button. This integration automates the entire life cycle of cluster creation, job submission, and cluster termination as part of job workflow.

Qubole

Qubole is a cloud-native data platform for analytics, machine learning, and self-service AI. This Quoble integration provides the ability to launch Runtimes that are powered by Quoble clusters. This allows the users to seamlessly work with Quoble clusters with the click of a button. This integration automates the entire life cycle of cluster creation, job submission, and cluster termination as part of job workflow.

Redshift

Redshift is a cloud-native highly scalable data warehousing solution offered by AWS. Redshift integration provides easy access to enterprise data available in Redshift. It allows users to run complex queries directly on Redshift tables. It also provides access to load the data from Redshift tables and perform more complex transformations and modeling. This integration also has the capability to load the processed data back to Redshift tables and views.

Snowflake

Snowflake is an analytic data warehouse provided as Software-as-a-Service (SaaS). Snowflake provides a data warehouse that is faster, easier to use, and far more flexible than traditional data warehouse offerings.

Snowflake integration provides easy access to enterprise data available in Snowflake. It allows users to run complex queries directly on Snowflake tables. It also provides access to load the data from Snowflake tables and perform more complex transformations and modeling. This integration also has the capability to load the processed data back to Snowflake tables and views.

Teradata

Teradata Database provides the most flexible analytical engine in the most scalable and manageable database for your data warehouse. Teradata integration provides easy access to enterprise data available in Teradata. It allows users to run complex queries directly on Teradata tables. It also provides access to load the data from Teradata tables and perform more complex transformations and modeling. This integration also has the capability to load the processed data back to Teradata tables and views.