Question

我想将每个元组X [p]存储在不同的文件中 Mini_batch2中的mini_batch1.jld X [2]中的X [1] ..... 但是我的代码在创建的文件中存储（复制）X [p]的所有元组。我们来看一个例子：

m= 100
k= 3 # number of tuples or partition
y=rand_gen(m,k)

(3,[[-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0],[1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,-1.0],[1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0]])

我想进来： mini_batch1第一个元组

[-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0]

mini_batch2第二个元组

[1.0,-1.0,-1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,-1.0]

等等。但是我的代码完成了创建mini_batches文件的工作，但是无法逐个存储元组。怎么能解决这个问题？

workspace()
using JLD, HDF5

function gen_random(m,k)

    # m the length of the vector , for instance m=100000 and k
    # the number  of partitions let's set k=16
    s = rand(m)
    # Pkg.add("JLD"), Pkg.add("HDF5") these two packages are needed
    # in order to store our vectors in files under the extension jld
    # allow to convert each random number to -1 or 1

    X=float_to_binary(s)
    parts= kfoldperm(length(X),k)
    # l want to store each tuple X[p] in a different file
    # X[1] in mini_batch1.jld  X[2] in mini_batch2.....
    # but my code below store all the tuple X[p] in the files created.
    for p in 1:length(parts)
        file =jldopen(@sprintf("my path to file/mini_batch%d.jld", p),"w")
        write(file, "X", [X[p] for p in parts])
        close(file)
    end
    return [X[p] for p in parts]

    function float_to_binary(s,level=0.4)
        for i=1:length(s)
            s[i] = s[i] > level ? 1.0 : -1.0
        end
        file = jldopen("/my path/mydata.jld", "w")
        write(file, "s", s)  # alternatively, say "@write file A"
        close(file)
        return s
    end


    function kfoldperm(l,k)
        n,r = divrem(l,k)
        b = collect(1:n:l+1)
        for i in 1:length(b)
            b[i] += i > r ? r : i-1
        end
        p = randperm(l)
        return [p[r] for r in [b[i]:b[i+1]-1 for i=1:k]]
    end

Answer 1

将所有数据设为1和-1会使示例更难以阅读，因此这里有一个具有更多可区分数字的示例：

julia> using JLD

julia> X = Vector{Int}[[1,2], [3,4]]
2-element Array{Array{Int64,1},1}:
 [1,2]
 [3,4]

julia> for i = 1:2
           jldopen("file$i.jld", "w") do file
               write(file, "X", X[i])
           end
       end

julia> X1 = load("file1.jld", "X")
2-element Array{Int64,1}:
 1
 2

julia> X2 = load("file2.jld", "X")
2-element Array{Int64,1}:
 3
 4

Julia：如何在不同的文件中存储元组？

1 个答案: