In today's fast-paced digital environment, automation, and real-time data processing are crucial for staying competitive. The Syntasa platform addresses these needs by offering powerful tools to monitor and react to specific events within data workflows. Among these tools are the "File Watcher" and "Adobe Feed Watcher" events, which enable users to automate downstream processes based on the presence of new data files.
User-Defined Events
User-defined events in Syntasa are designed to monitor specific locations for new files and trigger predefined actions when these files appear. This capability is essential for automating workflows and ensuring timely processing of incoming data. Users have the flexibility to define the frequency at which these locations are checked, making the system adaptable to various operational requirements. These events can be toggled between "Active" and "Inactive" states, allowing users to control their execution based on current needs.
Supported Event Monitoring Locations
Syntasa’s user-defined events support a wide range of storage and file systems, reflecting the diverse environments in which modern data operates. These include:
- AWS S3 (Amazon Simple Storage Service): A scalable object storage service used widely for data storage and management.
- GCP GCS (Google Cloud Storage): Google's scalable object storage service for a variety of data types.
- FTP (File Transfer Protocol): A standard network protocol used for transferring files between a client and server.
- SFTP (Secure File Transfer Protocol): An extension of FTP that adds a layer of security for file transfer over a secure shell (SSH).
- ONPREM (HDFS - Hadoop Distributed File System): A distributed file system designed to store large data sets across multiple servers, commonly used in on-premises environments.
These locations are critical for organizations leveraging cloud or on-premises storage solutions, providing flexibility and ease of integration into existing workflows.
File Watcher Events
The File Watcher event is a versatile tool designed to monitor specific directories for the arrival of new files. By configuring this event, users can set criteria to look for files that match a particular pattern within a designated directory. When a new file that meets these criteria is detected, the event automatically triggers a downstream process. This automation is particularly valuable in scenarios such as data ingestion, where it initiates data processing workflows immediately upon the upload of new data files.
Adobe Feed Watcher Events
For users working with Adobe clickstream data, the Adobe Feed Watcher Event offers a tailored solution. This event type keeps a watchful eye on a designated directory for specific manifest file patterns. These manifest files typically signal the completion of an hourly or daily Adobe clickstream data feed.
Once the Adobe Feed Watcher Event detects the arrival of a new manifest file matching the pattern, it triggers a processing event within your workflow. This automation ensures your data pipeline seamlessly integrates with Adobe's data delivery schedule, allowing you to process the latest clickstream data efficiently.
In essence, the Adobe Feed Watcher Event acts as a bridge between your Syntasa workflow and Adobe's data delivery, streamlining the processing of valuable customer behavior insights.
Taking Charge of Your Workflow
User-defined events empower you to take control of your Syntasa workflows. By leveraging File Watcher and Adobe Feed Watcher Events, you can automate data pipeline triggers based on specific file arrivals, ensuring your workflows react promptly to new information. This translates to increased efficiency, improved data processing times, and a more streamlined data pipeline experience within the Syntasa platform.