Triggering a production job each time a file is loaded into a bucket

Completed

Comments

6 comments

  • Official comment
    Avatar
    Mahesh Shenoy (Edited )

    Adam Neo : Please create a new event before it is added to trigger.

     

    In this 

    Connection : specifies from where files are getting added  - GCP/AWS etc

    FilePath : - Path where file is added. Since we specify the bucket name in the connection, no need to provide the bucket name here.

    File pattern : regular expression pattern for file getting added.

     

    Once the file that satisfies the pattern is added to mentioned location, it will trigger the event. If this event is added in the triggered jobs, will initiate a run.

    Shahdy Ali Hassan

    Comment actions Permalink
  • Avatar
    Adam Neo

    Screenshot from file watcher in the UI as a bit of context

    0
    Comment actions Permalink
  • Avatar
    Brian Pavlicek

    i'm not sure if this process has changed - but i believe you need to set up an event in the Events section that you can reference in the file watcher for the app.

    Brian

    0
    Comment actions Permalink
  • Avatar
    Adam Neo

    Thanks for the feedback Brian, much appreciated.

    Would you happen to know if the filename gets passed as a parameter and if so, what this parameter will be called? This will be great to know for configuring the code processes.

     

    0
    Comment actions Permalink
  • Avatar
    Brian Pavlicek (Edited )

    you will set the filename and path in the Events configuration and the name created for the Event will be the file watcher object in your scheduled task as far as I have ever used it. The problem I have had with the file watcher/event trigger is that if you have multiple files dropping into the folder it kicks off the process even though other files may be filling into the location. I have asked in the past if there was a way we could set up a sleep or pause for a specific timeframe to allow all files for the current date (for example) to be available before the process kicks off or you will loose that information as it does not check for subsequent files in the folder. Not sure if this has been added yet. Hope that makes sense.

    0
    Comment actions Permalink
  • Avatar
    Mahesh Shenoy

    Adam Neo : shall we move this post to completed if it works.

    0
    Comment actions Permalink

Please sign in to leave a comment.