Available in November 2021, the latest version of the Syntasa platform introduces Airflow as a new job execution manager that brings new capabilities in job creation and management. There are also many updates to security and connection options as well as numerous user experience improvements.
New Features
- Under the hood, Airflow - We are taking advantage of new and improved cloud technologies empowering Syntasa jobs with new features and efficiency. We are now supporting Airflow as a new option for job execution management. This feature allows advanced features like executing job steps in parallel.
- Job management options - We've implemented many wishlist items for job creation and management so you have total control over how steps within the job should be run. You can now drag and drop the steps to define the order, choose to run steps in parallel when possible, and we've smoothed out the user experience when editing and executing jobs.
- Execution and Task views - Your full suite of pipelines results in creating many apps and many jobs on top of those apps. The existing Job Tracker provides a nice view across those apps and jobs, but only for the current, latest runs of each. We've now added the ability to analyze all historical executions across apps, jobs, and timeframe.
- Additional authentication options - The Syntasa platform typically is sitting within an organization's suite of software solutions. Many times all of the various software solutions need to comply with a single standard, for example, a certain type of authentication. We've added GEOAxIS, PKI certificates, and CAC cards to the list of available authentication methods.
Improvements
- Audience folders - Syntasa 6.0 introduced Workspace bringing together apps and dashboards that could be organized into folders. The same organization features have now been brought to the Audiences module.
- Data connection improvements - Syntasa empowers you to build data pipelines and models quickly with apps utilizing the drag-and-drop workflow canvas. The canvas is powerful because of the many off-the-shelf connections and processes, pre-packaged with the needed configuration options and algorithms built-in so that you can get to results faster. We're constantly adding and improving connections and processes to expand the number of available ingredients at your disposal:
- Snowflake - Snowflake database connections are now supported in the From DB and To DB processes.
- Matomo - The Matomo Load process now supports S3 connections. Also, Matomo connections are able to authenticate using SSL and PLI.
- HDFS - HDFS connections now have the option to connect to a remote HDFS.
- User Defined Process (UDP) improvements - Process templates, also known as User Defined Processes (UDP), gives the ability for a user to create a new, custom process that then can be (re)used in apps by dragging them into the workflow canvas as can be done with the system defined processes. Several improvements have been made:
- Designer options - The UDP designer feature has been expanded to include additional input types like radio buttons, toggles, sliders, and file inputs.
- UDP publishing - The publishing functionality has been improved to display the current name/version when asked to provide the new name/version.
- Product assistant - With our improved product assistant, we're easing the effort of users by taking care of the small things automatically and providing real-time feedback to avoid common mistakes. Improved and added validations:
- Drop & Replace confirmation - We've included a user confirmation when the Drop & Replace option is chosen to ensure the user wants to drop the table.
- Dashboard options - Dashboards created and shared within the Syntasa platform present data in the form factor that you decide is best. We've now provided more control of dashboards with the option of opening a new tab by default.
- Python code tabs - We've addressed an issue where tabs within copied Python code were being converted to spaces.
- Job cleanup - The system will now remove processes from jobs when the process is deleted from the app workflow.
- Lookback and Lookahead parameters - We've clarified the units when using the Lookback and Lookahead configuration options. Previously they assumed the tables were partitioned in days, but that may not always be the case.
- BQ partitioned tables - The system will now create a partitioned table in BQ automatically if the user parameter is configured as such.
- Interactive Mode, not available on-prem - The system will not show the interactive mode option in on-prem installations as the feature is incompatible with such environments.