Python-来自计算中涉及的不同文件(不同长度)的数据

时间:2014-08-12 15:12:53

标签: python file astronomy

我正在尝试弄清楚如何通过Members.dat表格中的每一行读取我的代码,阅读其中的rasdecs,查找{{1} }和ra匹配(不完全,但在一定范围内,因为它们来自两个不同的调查)在另一个表中(对齐...),如果它找到一个使用适当的{{进行计算1}}和dec用于redshiftrbandmag匹配。

如果找不到匹配项,则对齐...'文件没有ra文件中每个decra的数据,只需转到decMembers.dat对中的radec对{1}}并搜索那个。

现在代码通过Members.dat表读取并返回所有Members.datredshifts,但是对于第二个print语句,它只使用它读取的最后一对使用其他文件中的数据进行所有计算(Aligned ....)。

@mdurant

!在/ usr / bin中/ Python的

导入csv 导入numpy为np

如果名称 ==" 主要":

rbandmags

所以有两个目标:

  1. 查找某个范围内的匹配
  2. 仅对匹配的fout = open('newfile.txt', 'w') group, rab, decb, Redshift, a, b, c, d, e = np.loadtxt('Members.dat', unpack=True) matched_id, fuv_mag, fuv_flux, nuv_mag, nuv_flux, e_bv, g, raa, deca = np.loadtxt('Aligned FULL MEMBERS WITH DATA', unpack=True) dist = np.subtract.outer(rab,raa)**2*np.cos(decb)**2 + np.subtract.outer(decb-deca)**2 match_bool = dist.min(axis=0) best_match = dist.argmin(axis=0) a = fuv_mag b = nuv_mag v = c*Redshift MFuv = a-5*(np.log(10*v/70)) MNuv = b-5*(np.log(10*v/70)) Ls=3.846*10**26 LFuv=(Ls)*(np.e**((MFuv-4.47)/-2.5)) LNuv=(Ls)*(np.e**((MNuv-4.47)/-2.5)) X = MNuv-Rbandmag MFuv = a-5*(np.log(10*v/70)) MNuv = b-5*(np.log(10*v/70)) Ls=3.846*10**26 LFuv=(Ls)*(np.e**((MFuv-4.47)/-2.5)) LNuv=(Ls)*(np.e**((MNuv-4.47)/-2.5)) X = MNuv-Rbandmag print >> fout, '{:.2e},'.format(float(LFuv)), '{:.2e},'.format(float(LNuv)), '%f, %f, %f, %f, %f, %f, %f, %f' % (Fuvmag, Nuvmag, MFuv, MNuv, g, X,Redshift, Rbandmag) fout.close() ras进行计算,并仅对相应的数据进行计算。
  3. 我的代码:

    decs

    #!/usr/bin/python import csv import numpy as np if __name__ == "__main__": fout = open('LGroupall', 'w') for line in open('Members.dat'): if not line.startswith('#'): cols = line.split() Redshift = float(cols[3]) Rbandmag = float(cols[4]) c = 2.9979*10**5 v = c*Redshift print >> fout,'%f, %f' % (Redshift, Rbandmag) for line2 in open('Aligned FULL MEMBERS WITH DATA'): if not line2.startswith('#'): cols2 = line2.split() Fuvmag = float(cols2[1]) Nuvmag = float(cols2[3]) g = float(cols2[6]) a = Fuvmag b = Nuvmag MFuv = a-5*(np.log(10*v/70)) MNuv = b-5*(np.log(10*v/70)) Ls=3.846*10**26 LFuv=(Ls)*(np.e**((MFuv-4.47)/-2.5)) LNuv=(Ls)*(np.e**((MNuv-4.47)/-2.5)) X = MNuv-Rbandmag print >> fout, '{:.2e},'.format(float(LFuv)), '{:.2e},'.format(float(LNuv)), '%f, %f, %f, %f, %f, %f, %f, %f' % (Fuvmag, Nuvmag, MFuv, MNuv, g, X, Redshift, Rbandmag) fout.close() 示例:

    Members.dat

    # Group rab decb Redshift a b c d e 2293 215.197952 4.370193 0.02667 -18.116 0.739 0 0.950 1.71 2293 215.186829 4.257247 0.02722 -18.379 0.660 0 0.950 1.59 2293 215.390656 4.447858 0.02743 -17.853 0.523 0 0.943 1.95 2293 215.540527 4.521144 0.02672 -19.289 0.925 0 0.943 1.87 2293 215.647522 4.547533 0.02639 -18.097 0.715 0 0.943 1.76 2293 215.861160 4.533366 0.02646 -17.987 0.330 0 0.943 1.73 2293 215.711914 4.665845 0.02584 -18.848 0.574 0 0.943 1.56 2293 216.152466 4.559171 0.02598 -20.265 0.666 0 0.987 1.64 2293 216.167786 4.570475 0.02643 -18.496 0.513 0 0.987 1.66 2293 216.474045 4.642701 0.02615 -20.271 0.770 0 0.987 1.61 2293 216.262772 4.364132 0.02686 -18.347 0.491 0 0.987 1.97 2296 220.980103 4.096979 0.02549 -17.933 0.611 0 0.974 2.79 2296 220.881287 4.309001 0.02549 -17.989 0.373 0 0.974 3.08 2296 221.061447 4.218551 0.02557 -20.525 0.939 0 0.974 2.93 2320 216.793182 4.757412 0.02698 -18.566 0.564 0 0.987 1.69 2320 216.884888 4.821614 0.02685 -20.531 0.849 0 0.987 1.68 2320 216.873398 4.779740 0.02753 -20.111 0.699 0 0.987 1.76 2320 216.691772 4.575934 0.02806 -19.103 0.833 0 0.987 1.92 示例:

    Aligned....

    #matched_id fuv_mag fuv_flux nuv_mag nuv_flux e_bv g raa deca 6383546990167328408 22.11728 5.165242 20.81406 17.15441 0.02696899 2293 215.197952 4.370193 2429377676145856182 22.33954 4.209049 20.93236 15.38349 0.02696899 2293 215.197952 4.370193 2429377676144817309 21.39752 10.02288 -999 -999 0.02578526 2293 215.186829 4.257247 6383546999829960446 22.0422 5.535058 -999 -999 0.02578526 2293 215.186829 4.257247 6383546990167328949 19.40973 62.53322 19.39661 63.29321 0.02752976 2293 215.390656 4.447858 2429377676145857353 19.61978 51.53339 19.34272 66.51405 0.02752976 2293 215.390656 4.447858 6383546990165232142 -999 -999 19.93376 38.59196 0.03139252 2293 215.540527 4.521144 2429377676145858738 20.62856 20.35057 19.28588 70.089 0.03139252 2293 215.540527 4.521144 2429553598006301900 20.53106 22.2628 20.17618 30.86948 0.02996568 2293 215.647522 4.547533 6383546989093587265 20.6668 19.64633 20.26241 28.51261 0.02996568 2293 215.647522 4.547533 6383546989093587234 18.51288 142.8394 18.3558 165.0736 0.03155341 2293 215.86116 4.533366 2429553598006303442 18.64594 126.3644 18.3048 173.0134 0.02757647 2293 215.711914 4.665845 6383546989093587956 18.84079 105.6055 18.36372 163.875 0.02757647 2293 215.711914 4.665845 2429553598006302429 17.40474 396.3743 17.00334 573.674 0.03030299 2293 216.152466 4.559171 6383546989093587283 17.45678 377.825 17.03622 556.5612 0.03030299 2293 216.152466 4.559171 6383546989093587400 19.14371 79.89501 18.63046 128.1783 0.03030299 2293 216.167786 4.570475 2429553598006302434 19.07138 85.39802 18.62435 128.9025 0.03030299 2293 216.167786 4.570475 6383546989093587315 18.24552 182.7237 17.59955 331.2669 0.03227024 2293 216.474045 4.642701 2429518413634214656 18.04468 219.8524 17.5183 357.0097 0.03227024 2293 216.474045 4.642701 6383546988019847429 18.03906 220.9926 17.47195 372.5806 0.03227024 2293 216.474045 4.642701 2429553598006303494 18.05234 218.3062 17.59173 333.6613 0.03227024 2293 216.474045 4.642701 2937827037201564330 18.11119 206.7875 17.5436 348.7877 0.03227024 2293 216.474045 4.642701 6383546989093586253 19.34859 66.15564 18.8084 108.8025 0.03201414 2293 216.262772 4.364132 2429553598006299849 19.16085 78.64319 18.7739 112.315 0.03201414 2293 216.262772 4.364132 6383546994462298969 19.61141 51.93227 19.43124 61.30603 0.03103101 2296 220.980103 4.096979 2429905441727194814 19.99832 36.36427 19.34747 66.22339 0.03103101 2296 220.980103 4.096979 6383546983724877047 18.67604 122.9098 18.44853 151.5609 0.03023296 2296 220.881287 4.309001 2430010994843455574 18.65218 125.6409 18.40909 157.1673 0.03023296 2296 220.881287 4.309001 2937827037201564850 19.31565 68.19363 18.83648 106.0248 0.03118853 2320 216.793182 4.757412 2937827037199466565 -999 -999 19.97776 37.05916 0.02839247 2320 216.691772 4.575934 6383546988019846679 21.08108 13.41437 20.09341 33.31459 0.02839247 2320 216.691772 4.575934 2429518413634213555 20.98475 14.65897 20.00091 36.27739 0.02839247 2320 216.691772 4.575934 6383546988019848025 22.73846 2.914851 20.76309 17.97894 0.03277461 2320 217.289642 4.769514 示例:

    LGroupall

2 个答案:

答案 0 :(得分:0)

我认为这可以通过巧妙地使用numpy.loadtxt和重新排列来完成(参见numpy reference for these以及关于数组索引的文档)

import numpy as np

colnames = tuple('Group rab decb Redshift a b c d e'.split())
coltypes = tuple(['i4'] + ['f4']*(len(colnames)-1))
Members = np.loadtxt('Members.dat', dtype={'formats':coltypes, 'names':colnames})

colnames = tuple('matched_id fuv_mag fuv_flux nuv_mag nuv_flux e_bv g raa deca'.split())
coltypes = tuple(['i4'] + ['f4']*5 + ['i4'] + ['f4']*2)
Aligned = np.loadtxt('Aligned.dat', dtype={'formats':coltypes, 'names':colnames})

Members['Group'] == Aligned['g']

for Group in set(Members['Group']):             # [1]

    # get indexes in Members and Aligned for    # [2]
    # this group index
    Midx = (Group == Members['Group'])
    Aidx = (Group == Aligned['g'])

    raa = Aligned['raa'][Aidx]                  # [3]
    deca = Aligned['deca'][Aidx]
    rab = Members['rab'][Midx]                  # [4]
    decb = Members['decb'][Midx]

    # print out the first two element of the data
    # for each group ID
    print 'Group:',Group
    print '    raa = ',raa[:2],'...'
    print '    rab = ',rab[:2],'...'
    print '    deca = ',deca[:2],'...'
    print '    decb = ',decb[:2],'...'

    # do some math with the values:
    print raa * rab

注意: [1]循环所有唯一的Group值 [2] Midx是该组成员数据的“索引掩码”     和Aidx是Aligned数据的这个组的掩码 [3]此时raa是所有raa值的数组     特别是小组 [4]类似地,rab是该组所有rab值的数组

我更喜欢以这种方式使用列类型来强制索引列为整数类型,以便掩码更加健壮。有了这个,您可以继续在组级别上进行所需的数学运算。请参阅上面脚本最后一行的输出。

答案 1 :(得分:0)

我建议你不要像往常那样循环浏览文件。虽然这会起作用,但它使代码难以阅读和纠正,并且可能导致您遇到的问题。相反,您可以将整个文件加载到数组中,然后处理这些:

group, rab, decb, Redshift, a, b, c, d, e = np.loadtxt('Members.dat', unpack=True)
matched_id, fuv_mag, fuv_flux, nuv_mag, nuv_flux, e_bv, g, raa, deca = np.loadtxt('Aligned FULL MEMBERS WITH DATA', unpack=True)

然后你有一个匹配时(rab-raa)** 2 * cos(decb)** 2 +(decb-deca)** 2<限制** 2(我假设您的单位在这里,请检查)。

您可以循环访问一个坐标集:

for i in range(len(rab)):
   dist = (rab[i]-raa)**2*cos(decb[i])**2 + (decb[i]-deca)**2)
   if dist.min()<limit**2:
       print raa[dist.argmin()],deca[dist.argmin()]

或形成(大)距离矩阵

dist = np.subtract.outer(rab,raa)**2*cos(decb[:,None])**2 + np.subtract.outer(decb,deca)**2
match_bool = dist.min(axis=0)
best_match = dist.argmin(axis=0)