当新的dag被推送到git repo时，airflow git-sync不会刷新dag

Question

我在Kubernetes上使用气流，通过git-sync选项进行DAGS部署。当有新的DAG推送到我们的git repo时，它的气流不会更新新的DAG，直到我们重新启动调度程序pod。还有什么其他方法可以在不重新启动调度程序窗格的情况下更新DAG？

Answer 1

我认为您可以使用Jenkins。

With Jenkins可以更轻松地管理dag的自动生成，在环境之间释放它们以及自动管理版本控制。

那么您不必使用调度程序

Answer 2

使用git-sync sidecar容器解决了此问题。

values.yaml文件中有一个extraContainers:选项。我们可以将kubernetes git-sync添加为sidecar容器，以持续从git进行同步。

extraContainers: 
  ## Additional containers to run alongside the Scheduler, Worker and Web pods
  ## This could, for example, be used to run a sidecar that syncs DAGs from object storage.
   - name: git-sync
     image: k8s.gcr.io/git-sync:v3.1.1
     volumeMounts:
      - name: ssh-key
        mountPath: /var/ssh
      - name: known-hosts
        mountPath: /var/knownhosts
      - name: git-sync-volume
        mountPath: /tmp/git

     env:
      - name: GIT_SYNC_REPO
        value: git@gitlab.example.com/test.git
      - name: GIT_SYNC_DEST
        value: git
      - name: GIT_SYNC_WAIT
        value: "10"
      - name: GIT_SYNC_SSH
        value: "true"
      - name: GIT_SSH_KEY_FILE
        value: /var/ssh/sshkey
      - name: GIT_SSH_KNOWN_HOSTS_FILE
        value: /var/knownhosts/known_hosts
  ## Additional volumeMounts to the main containers in the Scheduler, Worker and Web pods.
  extraVolumeMounts: 
     # - volumeMounts:
          - mountPath: "/var/ssh"
            name: ssh-key
            readOnly: true
          - mountPath: "/var/knownhosts"
            name: known-hosts
            readOnly: true
          - name: git-sync-volume
            mountPath: /tmp/git

  extraVolumes: 
     #- volumes:
        - name: ssh-key
          secret:
            secretName: dags-secret
            items:
            - key: key
              path: sshkey
        - name: known-hosts
          secret:
            secretName: dags-secret
            items:
            - key: known_hosts
              path: known_hosts

        - name: git-sync-volume
          emptyDir: {}