Runtimes are the computing resources used to run processes and logic in application jobs and notebooks within the Syntasa platform. Users can define runtime templates, and configure cluster, container type, size, and settings.
These defined runtime templates can be selected in jobs, notebooks, and the interactive mode of apps. Each selection in these areas creates a new instance of the runtime template. Therefore, multiple instances of a runtime template can be active simultaneously.
Categorizing Runtime Types
While there are several runtime types available, these are categorized into two primary groups for easier understanding. Also, a third group is a slight variation of runtimes found in the two primary groups.
Container runtimes
Container runtimes are launched on the same Kubernetes cluster the Syntasa platform is using. They have limited to no settings to choose from because they are defined by the subsequently selected image. The image names, e.g. Syntasa Base Image or Syntasa Dash Image, help identify the intended use that the image is optimized, which is detailed in the article Kubernetes Container.
Cloud-native runtimes
Cloud-native runtimes, i.e. those with "cluster" in the name, utilize the appropriate cloud-native service (EMR in AWS; Dataproc in GCP; Synapse in Azure) to run jobs, execute code in notebooks, etc. These runtimes have all the various settings you would see in the cloud-native service. For example, the master and worker instance types, number of worker instances, max uptimes, etc.
Streaming runtimes
Streaming runtimes should only be used for streaming jobs, i.e. continuously running jobs unless manually stopped, as opposed to non-streaming runtimes that are used for batch jobs and notebooks that will be shut down once the job completes or inactivity or max timeout is reached.
Runtime types
The types of runtimes available depend on the cloud platform the Syntasa environment is installed; the settings available for the runtime template depend on the runtime type that is selected. The following runtime types are available for GCP and AWS:
Google Cloud Platform (GCP)
Amazon Web Services (AWS)