How do I install "offline" dependencies at runtime?



1 comment

  • Avatar
    Mike Z

    This can be done by doing a combination of storing your package files in your data store (i.e. S3, GCS, Blob) and adding a parameter to the Spark runtime.

    1. Download the package files (i.e. .whl)
    2. Upload the files in the /syn-cluster-config/deps/python/ folder in your bucket
    3. Add two new Config Settings/Values to your runtime (syntasa.python.enable.dependencies=true and 


    4. When you start a cluster using the runtime with the new config settings the specified packages will be installed

    See sample screenshot below of the two settings.


    Comment actions Permalink

Please sign in to leave a comment.