I am doing a ML project on AWS EMR clusters and use a bootstrap to setup my environment. I am running into a very common problem where my modules (in this case .py file I built) are not installed on my worker nodes.
My workflow is to code in a .py file and implement in a Jupyter notebook, and continually updating a .py file will not be pushed to any of the worker nodes.
Is there any simple way for the worker nodes to track changes in the master node, or to submit py files to the worker nodes from the master node? From my understanding bootstrapping only occurs upon creation of the EMR cluster.
ModuleNotFoundError: No module named 'src'