Runtime is where the size and configuration of the data processing cluster are determined. They can either be assigned to a job or run directly via interactive mode. There are two runtime types: Container and Spark. They determine the number of resources allocated for a job, and once a job is run with them it will create those resources automatically. If using interactive mode, the resources will be allocated, and then any jobs pushed onto it will run with them.
This article includes the creation and review of runtimes:
- Navigating the Runtime screen
- Creating and reviewing existing runtimes
- Editing an runtime
- Reviewing where a runtime is used
Navigating the Runtime screen
The Runtime page is available from the main menu ( 
) and then selecting the Runtime menu via the runtime icon (
). This is where various runtimes for processing will be defined. Multiple runtimes can be configured based on memory and nodes required for processing.
Calculating the number of worker nodes, executor memory, driver memory, and executor cores are initially performed by Syntasa professional services and set up as the default runtimes. These values are dependent on the data being processed.
Creating and reviewing existing runtimes
Viewing the list of runtime templates currently created in the system display the following information:
Add new runtime - Clicking the plus icon (
) allows a user to add a new runtime. Once clicking the icon the user has two options for Runtime Type, i.e. Spark or Container, each having their own options on creation.
Name - Name of the runtime given by the creator. When creating or viewing a new job, this is the name that is shown for the runtime the process has or is to run on. This field can be altered when editing the runtime template.
Clicking on the name of the runtime template allows for the editing of the said runtime template.
Runtime Type - Chosen at the time of creating the runtime template, i.e. Spark or Container, which determines how/where the process of the job will run. This field cannot be altered after the runtime template is created.
In Use - Indicates if the runtime template is in use, i.e. utilized in a created/saved job.
Clicking on the Yes/No of a runtime template takes the user into the dependencies section of the said runtime template.
Defaults - One runtime template of each type, Spark and Container, can be indicated as the default runtime for both development and production. That is, a total of four defaults can be set: Spark development; Spark production; Container development; Container production. A single runtime template can be the default for both development and production.
Terminate on Completion - Option for Spark runtimes only, this indicates whether the toggle for the said option is on or off. If on then the cluster will be shut down on completion of the job.
Worker Instance Type - Option for Spark runtimes only, this indicates the type(s) of worker nodes as configured within the runtime template.
Worker Instance Count - Option for Spark runtimes only, this indicates the value of the said option within the configuration of the runtime template.
Created At - Displays the date and time the runtime template was created. The date format will be shown in the language of the browser.
Updated At - Displays the date and time the runtime template was updated. The date format will be shown in the language of the browser.
Modified By - Displays the username of the person that last modified the runtime template.
Editing an runtime
Clicking on the name of a runtime template takes the user to the configuration section of the runtime template seen by the configuration gear icon (
). A user can also reach the configuration section of a runtime template if already within the dependencies section of the said runtime template and clicking the same configuration gear icon.
The fields seen within the configuration section of a runtime template vary depending on whether the Runtime Type is Spark or Container. These fields are described in each of the articles on the creation of a runtime template for each type.
Reviewing where a runtime is used
Clicking on the value of the column In Use of a runtime template takes the user to the dependencies section of the runtime template seen by the dependencies icon (
). A user can also reach the dependencies section of a runtime template if already within the configuration section of the said runtime template and clicking the same dependencies icon.
The Runtime Template Dependencies screen shows all jobs that have been created that utilize the said runtime template.
App Name - The name of the app that has a job utilizing the said runtime template. Clicking on the name of the app takes the user to the workflow canvas of the app.
Job Name - The name of the job within the said app that is utilizing the runtime template. Clicking on the name of the job takes the user to the activity logs section within the operations screen of the app.
Job Step Name - The specific step of the job that is utilizing the said runtime template.
Environment - Development or Production, indicates the environment of the job that is utilizing the said runtime template.
Scheduled - Indicates whether the job noted that is utilizing the said runtime template is scheduled or not. Scheduled jobs are only available within production.