处理Elixir中的多个保管箱写入请求

时间:2019-05-08 10:47:53

标签: elixir

我正在尝试在一个日期循环中通过FFMPEG从NVR获取JPEG。

import numpy as np
import pandas as pd

example_data = [[-10, np.NaN], [np.NaN, 20], [-30, np.NaN]]

example_df = pd.DataFrame(example_data, columns = ['A', 'B'])

for index, row in example_df.iterrows():
    if pd.isnull(row['A']):
        row['A'] = row['B']

example_df = example_df.drop(columns = ['B'])        

example_df

然后在上载和插入部分中,我将jpeg上载为Dropbox

Porcelain.shell("ffmpeg -rtsp_transport tcp -stimeout 10000000 -i '#{stream_url}' -vframes 1 -y #{images_path}").out
spawn(fn ->
  File.exists?(images_path)
  |> upload_and_inject_image(config, images_path, upload_image_path, start_date, timezone)
end)

但是如果 defp upload_image(status, image_path, upload_image_path) when status in [true, "true"] do client = ElixirDropbox.Client.new(System.get_env["DROP_BOX_TOKEN"]) case ElixirDropbox.Files.upload(client, upload_image_path, image_path) do {{:status_code, _}, {:error, error}} -> Logger.debug "Error while uploading. Error: #{inspect error}" _ -> :noop end end defp upload_image(_status, _image_path, _upload_image_path), do: :noop 得到的结果慢一点,文件上传就可以正常工作。

如果该方法运行速度很快,Dropbox API会给出错误Porcelain.shell("ffmpeg -rtsp_transport tcp -stimeout 10000000 -i '#{stream_url}' -vframes 1 -y #{images_path}").out,我正在将此库用于dropbox

我试图在Porcelain.shell命令之前进入睡眠状态,但是结果是相同的。是否有可能通过e药同时处理多个并发操作到Dropbox。

2 个答案:

答案 0 :(得分:0)

您正在达到Dropbox所能做的极限。最好在一个upload_session中发送多个文件。

请考虑切换到upload_session/startupload_session/appendupload_session/finish_batch

答案 1 :(得分:0)

因此,经过无数次尝试和失败之后,我发现了一些解决方法,我只是在发布帖子,以征询是否有比此方法更好的解决方案。

创建保管箱客户端后,

ImageAnalysis.Analyzer

我首先要通过它来获取文件大小。

client = ElixirDropbox.Client.new(System.get_env["DROP_BOX_TOKEN"])

然后我分别为每个文件(例如,

)启动一个 defp get_file_size(image_path) do File.stat(image_path) |> stats() end defp stats({:ok, %File.Stat{size: size}}), do: {:ok, size} defp stats({:error, reason}), do: {:error, reason}
upload_session

在开始%{"session_id" => session_id} = ElixirDropbox.Files.UploadSession.start(client, true, image_path) write_sessional_values(session_id, file_size, upload_image_path, path) check_1000_chunk(path) |> length() |> commit_if_1000(client, path) 时,如果您以一个请求而不是以块形式上载文件,则upload_session标头应为close,它将告诉api不再有块相同的文件将要上传。

其他方法像这样

true

我必须记下所有会话值以记录它们,以防应用程序停止或再次启动以从那里恢复它。

文件大小很重要,因为您将在完成批处理时将其传递。

具有这样的值。

  defp write_sessional_values(session_id, file_size, upload_image_path, path) do
    File.write!("#{path}SESSION", "#{session_id} #{file_size} #{upload_image_path}\n", [:append])
  end

  defp check_1000_chunk(path) do
    File.read!("#{path}SESSION") |> String.split("\n", trim: true)
  end

  defp commit_if_1000(1000, client, path) do
    entries =
      path
      |> check_1000_chunk()
      |> Enum.map(fn entry ->
        [session_id, offset, upload_image_path] = String.split(entry, " ")
        %{"cursor" => %{"session_id" => session_id, "offset" => String.to_integer(offset)}, "commit" => %{"path" => upload_image_path}}
      end)
    ElixirDropbox.Files.UploadSession.finish_batch(client, entries)
    File.rm_rf!("#{path}SESSION")
  end
  defp commit_if_1000(_, _client, _path), do: :noop

那么您认为有更好的解决方案吗?将会话值保存到文件中?或代码中的任何内容?