User-defined processes are custom-built templates that bundle up code processes such as spark processors or BQ Processes into a single cohesive reusable no-code template process. UDPs can incorporate their own user interface (UI) elements, allowing for easy editing and customization of parameters specific to the UDP. This flexibility enables efficient management and execution of complex processes within a unified framework. UDPs can be then utilized in apps by dragging them into the workflow
Creating a New Custom Process
The menu may differ depending on the version of your current version of Syntasa.
Pre 7.0 versions:
Post 7.0 versions:
- Pre 7.0 versions: Under Templates click Process
- Post 7.0 versions: Under Platform click User-Defined Process
- Click the (+) to create a new Custom Process
- Give your process a name
- Upload template (leave default off)
- Override Icon (leave default off)
- Click Create
Name - A user-friendly name for the UDP. It should be a unique name.
Key - The key field is a unique key used to identify the UDP. The default value of the key is the given name of the UDP with spaces removed, but the value of the key can be altered at the time of UDP creation. The field is locked after the UDP has been created.
Upload Template - In Order to attach an existing template, you should toggle on the Upload Template option
Override Icon - The override icon toggle allows the default icon shown on a component's card to be altered. The default icon is dependent on the type of component. When the toggle is turned on there are two options to associate a new icon with the component, depending on the position of the Existing Icon toggle:
- On - This allows the user to search icons that have already been uploaded to the environment and associate one with the current component.
- Off - This allows the user to upload a new icon for the component. When uploading a new icon it must be given a name so that it can be referenced and searched for and thus is available to be reused on different components.
Click on the Save button in order to save the details
To navigate to the details page, click on the Name of the UDP that you have created from the grid
Process Types
The below processes are available to the user. They are designed so that they can handle anonymous input and outputs, using the parameter values contained within them. This should allow a custom template to flexibly handle different data sources without having to modify the process itself once published.
Processes:
- BQ Process
- Container Code Process
- Custom Shell
- Dash
- Spark Learn
- Spark Processor
- Spark Score
Process Input
Any number of process inputs can be brought into a custom process, for each of them an input type must be chosen. The three types are:
- Connection - For any sort of connection to an outside dataset. Can range from anything such as BQ to S3.
- Dataset - For any sort of internal Syntasa-based data. So there is a choice between a table or a view.
- Model - For consuming an ML model. So there is a choice between Learning and Treatment
- Stream - Available Object Type is Kafka Topic
- Files - The available choice is File Object
- File Upload - Available object types are Data file, JAR file, Model, Other, Python, etc.
Parameters
The parameters are used to enable users to pass in parameter values to the custom process. It requires a field name from the input source, the parameter name itself for the code process, a default value, and a type. An example used for this might be if a process is created to filter a dataset with a set of custom values, they could be passed in via a global parameter.
Process Output
Drag any number of outputs from the process into a Process Output node. Upon selecting the process output, a hint will need to be given to each output table as a reference point.
Download
Within the () menu, there is the option to download a process as a .json file. This can be used to transfer custom processes between multiple instances of Syntasa.
Publish
Using the publish button will allow the template process to be used throughout the app. It is also a good way to ensure that the process has been configured properly and will give warnings concerning any issues. This is also where the version will be set if a rollback is needed.