Rsync显示文件已同步但无法在AWS S3中找到它

时间:2017-01-11 09:30:36

标签: ruby-on-rails ruby amazon-s3 capistrano3

我正在使用 capistrano 3 Ruby on Rails 3.2.14 Ruby 2.1.0 。我写了一个配方来检查S3挂载状态,然后运行一组特定的操作:

  1. 如果已卸载存储桶且文件夹(public/system)为空,请使用s3fs
  2. 进行安装
  3. 如果已卸载存储桶且文件夹不为空,则使用rsync将文件从public/system移至s3存储桶,然后清空文件夹public/system并重新安装。
  4. rsync 显示文件已逐步发送,但我无法在存储桶中找到它们。这是我得到的输出:

      05 rsync -av /home/deploy/unmounted /home/deploy/mounted
      05 sending incremental file list
      05 unmounted/
      05 unmounted/ckeditor/
      05 unmounted/ckeditor/pictures/
      05 unmounted/ckeditor/pictures/16736/
      05 unmounted/ckeditor/pictures/16736/bike.jpg
      05 unmounted/ckeditor/pictures/16736/content_bike.jpg
      05 unmounted/ckeditor/pictures/16736/thumb_bike.jpg
      05
      05 sent 750,943 bytes  received 93 bytes  166,896.89 bytes/sec
      05 total size is 750,396  speedup is 1.00
    
    用于安装s3的

    配方,我还没有优化它,因为我首先想检查它是否正常工作,

    desc "Mounting S3"
        task :mounting_s3 do
            on main_server, in: :sequence, wait: 5 do |host|
                as 'deploy' do
                    within "#{code_folder_path}/current" do
                        code_folder = "#{code_folder_path}/current"
                        path = "#{code_folder_path}/current/public/system"
                        unmount_path = "#{code_folder_path}/shared/public/system"
    
                        # check first if public/system exists
                        puts "** Checking if public/system folder exists on #{host}"
                        puts "** testing if it is there #{test("[ -d #{path} ]")}"
                        if test("[ -d #{path} ]")
                            puts "** public/system folder exists.."
                            puts "** Checking if S3 is mounted"
                            result = capture "cd #{code_folder} && mountpoint public/system" , raise_on_non_zero_exit: false
                            puts "result #{result}"
                            if  result.match('is a mountpoint') != nil
                                puts "** S3 is mounted at #{path} on #{host}"
                            elsif result.match('is not a mountpoint') != nil
                                puts "** #{path} is not a mountpoint on #{host}"
                                result = capture "ls -A #{unmount_path}"
                                if result.length > 0
                                    puts "** public/system is not empty"
                                    if test("[ -d /home/deploy/mounted ]")
    
                                        puts "** mounted folder exists"
    
                                        puts "checking if folder is mounted"
                                        result = capture "mountpoint /home/deploy/mounted" , raise_on_non_zero_exit: false
                                        puts "result #{result}"
    
                                        if  result.match('is a mountpoint') != nil
                                            puts "** mounted folder is mounted so unmounting it"
                                            execute! "fusermount -u /home/deploy/mounted"
                                            puts "** mounted folder is unmounted now"
    
                                            puts "** Deleting mounted folder"
                                            execute! "cd /home/deploy/ && sudo rm -r mounted"
                                            puts "** mounted folder deleted"
    
                                        elsif result.match('is not a mountpoint') != nil
    
                                            puts "** Deleting mounted folder"
                                            execute! "sudo rm -r /home/deploy/mounted"
                                            puts "** mounted folder deleted"
    
                                        end
                                    end
    
                                    if test("[ -d /home/deploy/unmounted ]")
                                        puts "** deleting unmounted folder"
                                        execute! "cd /home/deploy/ && sudo rm -r unmounted"
                                        puts "** unmounted folder deleted"
                                    end
    
                                    puts "** mounting bucket"
                                    execute! "cd /home/deploy/ && mkdir unmounted && mkdir mounted"
                                    execute! "s3fs s3_bucket /home/deploy/mounted"
                                    puts "** bucket mounted on /home/deploy/mounted folder"
    
                                    puts "** copying files from public/system to unmounted"
                                    execute! "cp -r #{unmount_path}/* /home/deploy/unmounted/."
                                    puts "** files copied"
    
                                    puts "** syncing unmounted files to s3"
                                    execute! "rsync -av /home/deploy/unmounted /home/deploy/mounted"
                                    puts "** Files synced"
    
                                    puts "** unmounting mounted folder now for safety"
                                    execute! "fusermount -u /home/deploy/mounted"
                                    puts "** folder unmounted"
    
                                    puts "** deleting unmounted folder now"
                                    execute! "sudo rm -r /home/deploy/unmounted"
                                    puts "** folders deleted\n"
    
                                    puts "** emptying public/system"
                                    execute! "sudo rm -r #{unmount_path}/../system/*"
                                    puts "** folder is empty, mounting public/system"
    
                                    execute! "s3fs s3-bucket #{path}"
                                    puts "** s3 is mounted"
                                else
                                    puts "** public/system is empty"
                                    execute! "cd #{code_folder} && s3fs s3-bucket #{path}"
                                    puts "** s3 is mounted"
                                end 
    
                            end 
                        else
                            puts "** public/system folder does not exists.."
                            puts "** Mounting S3"
                            execute! "cd #{code_folder} && s3fs s3-bucket #{path}"
                        end 
                    end
                end
    

    运行cap staging deploy:mounting_s3 --trace后得到的输出:

    ** Invoke www_data (first_time)
    ** Execute www_data
    ** Invoke load:defaults (first_time)
    ** Execute load:defaults
    ** Invoke rvm:hook (first_time)
    ** Execute rvm:hook
    ** Invoke rvm:check (first_time)
    ** Execute rvm:check
    rvm 1.27.0 (latest) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]
    ruby-2.1.0
    ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-linux]
    ** Invoke bundler:map_bins (first_time)
    ** Execute bundler:map_bins
    ** Invoke deploy:set_rails_env (first_time)
    ** Execute deploy:set_rails_env
    ** Invoke deploy:set_linked_dirs (first_time)
    ** Execute deploy:set_linked_dirs
    ** Invoke deploy:set_rails_env 
    ** Invoke deploy:mounting_s3 (first_time)
    ** Execute deploy:mounting_s3
    ** Checking if public/system folder exists on webby
    ** testing if it is there true
    ** public/system folder exists..
    ** Checking if S3 is mounted
    result public/system is not a mountpoint
    ** /home/deploy/apps/web_cc_cap1/current/public/system is not a mountpoint on webby
    ** public/system is not empty
    ** mounted folder exists
    checking if folder is mounted
    result /home/deploy/mounted is not a mountpoint
    ** Deleting mounted folder
    00:00 deploy:mounting_s3
          01 sudo rm -r /home/deploy/mounted
        ✔ 01 webby 0.649s
    ** mounted folder deleted
    ** mounting bucket
          02 cd /home/deploy/ && mkdir unmounted && mkdir mounted
        ✔ 02 webby 0.631s
          03 s3fs s3-bucket /home/deploy/mounted
        ✔ 03 webby 0.619s
    ** bucket mounted on /home/deploy/mounted folder
    ** copying files from public/system to unmounted
          04 cp -r /home/deploy/apps/web_cc_cap1/shared/public/system/* /home/deploy/unmounted/.
        ✔ 04 webby 0.651s
    ** files copied
    ** syncing unmounted files to s3
          05 rsync -av /home/deploy/unmounted /home/deploy/mounted
          05 sending incremental file list
          05 unmounted/
          05 unmounted/ckeditor/
          05 unmounted/ckeditor/pictures/
          05 unmounted/ckeditor/pictures/16736/
          05 unmounted/ckeditor/pictures/16736/bike.jpg
          05 unmounted/ckeditor/pictures/16736/content_bike.jpg
          05 unmounted/ckeditor/pictures/16736/thumb_bike.jpg
          05
          05 sent 750,943 bytes  received 93 bytes  166,896.89 bytes/sec
          05 total size is 750,396  speedup is 1.00
        ✔ 05 webby 4.299s
    ** Files synced
    ** unmounting mounted folder now for safety
          06 fusermount -u /home/deploy/mounted
        ✔ 06 webby 0.631s
    ** folder unmounted
    ** deleting unmounted folder now
          07 sudo rm -r /home/deploy/unmounted
        ✔ 07 webby 0.621s
    ** folders deleted
    ** emptying public/system
          08 sudo rm -r /home/deploy/apps/web_cc_cap1/shared/public/system/../system/*
        ✔ 08 webby 0.649s
    ** folder is empty, mounting public/system
          09 s3fs s3-bucket /home/deploy/apps/web_cc_cap1/current/public/system
        ✔ 09 webby 0.623s
    ** s3 is mounted
    

    为什么不将文件发送到s3存储桶?

1 个答案:

答案 0 :(得分:0)

我确实犯了一个错误,我的意图是复制未安装的文件夹和而不是文件夹本身的内容。所以我修复了rsync代码:rsync -av /home/deploy/unmounted/* /home/deploy/mounted并按预期工作。