Available in Syntasa environments installed in GCP or AWS, the Kubernetes Container runtime type is launched on the same Kubernetes cluster the Syntasa platform is using. The Image Type field determines the underlying image used. Each image has predefined libraries needed for running different types of workloads.
The basic runtime attributes required for all runtime types are detailed in Creating Runtime Templates. When selecting Kubernetes Container for the Runtime Type field, the remaining settings are dependent on the Image Type selection.
Image Types
Similar in importance to the runtime type selection, when using Kubernetes Container, the image type is the driver for the type of compute resource. Most of the selections have no further settings beyond the image type, a few do have further configurations. The intended use and required fields for each image type are detailed below.
Syntasa Base Image
The Syntasa Base Image contains all the base libraries that are needed for running Java/Scala/Python programs. Runtimes with this image type should be used for running jobs utilizing the Container Code Processor process, which is used for running standalone Java/Scala/Python jobs (not Spark jobs).
Machine type
Selecting the Syntasa Base Image enables the field for the machine type required for the runtime. The various machine families and machine types can be reviewed in Google's Support Machine Types article and AWS's Supported Instance Types article.
Syntasa Interactive Query Image
The Syntasa Interactive Query Image contains all the libraries that are needed for connecting to interactive query engines supported by Syntasa, e.g. BigQuery, Athena, Snowflake, etc. Runtimes with this image type should be used for running jobs utilizing processes such as BQ Process, Athena Code Process, Snowflake Code Process, etc.
Syntasa Dash Image
The Syntasa Dash Image contains all the libraries needed for running custom Dash code on the underlying Dash server. Runtimes with this image type should be used for running jobs utilizing the Dash process.
Syntasa Google Cloud ML Image
The Syntasa Google Cloud ML Image contains all the libraries needed for connecting to Google Cloud ML services for both training and scoring. Runtimes with this image type should be used for running jobs utilizing the ML Engine Train and Score processes.
Syntasa Spark Image
The Syntasa Spark Image contains all the Spark libraries and other Spark Syntasa utilities that are needed for running Spark code. Runtimes with this image type should be used for running any of the Syntasa-supported Spark-based processes and the Spark Processor process.
Alternatively, the Syntasa-supported Spark-based processes and Spark Processor process can be executed using cloud-native runtimes.
Machine type and options
Selecting the Syntasa Spark Image enables several fields related to the machine type required for the runtime. The various machine families and machine types can be reviewed in Google's Support Machine Types article and AWS's Supported Instance Types article.
Configuration options
There are also Spark configurations available. Key settings related to the number of cores and memory are defaulted but can be adjusted as needed. Other values available for configuration are detailed in the Apache Spark documentation on Spark Configuration.