pandas.DataFrame

时间:2017-11-02 11:14:46

标签: python pandas numpy dataframe pairwise

我想在DataFrame中的所有行对上创建操作结果的矩阵。

以下是我想要的一个例子:

df = pandas.DataFrame({'val':  [ 2,    3,    5,    7  ],
                       'foo':  ['f1', 'f2', 'f3', 'f4']},
                      index=   ['n1', 'n2', 'n3', 'n4'])

def op1(row1, row2):
    return row1['val']*row2['val']

def op2(row1, row2):
    return f"{row1['foo']}{row2['foo']}"

def apply_op_to_all_row_pairs(df, op):
    # what goes in here?

apply_op_to_all_row_pairs(df, op1)
#     n1  n2  n3  n4
# n1   4   6  10  14
# n2   6   9  15  21
# n3  10  15  25  35
# n4  14  21  35  49

apply_op_to_all_row_pairs(df, op2)
#         n1      n2      n3      n4
# n1  'f1f1'  'f1f2'  'f1f3'  'f1f4'
# n2  'f2f1'  'f2f2'  'f2f3'  'f2f4'
# n3  'f3f1'  'f3f2'  'f3f3'  'f3f4'
# n4  'f4f1'  'f4f2'  'f4f3'  'f4f4'

我已经看到很多解决方案都依赖于现有的计算距离矩阵的函数,但我想要更通用的东西。 例如,scipy.spatial.distance.pdist执行我想要的格式,但只处理浮点数,并且不允许您按名称选择列(或者至少我无法弄清楚如何)。

1 个答案:

答案 0 :(得分:1)

您可以使用广播的numpy操作:

pd.DataFrame(x, df.index, df.index)

      n1    n2    n3    n4
n1  f1f1  f1f2  f1f3  f1f4
n2  f2f1  f2f2  f2f3  f2f4
n3  f3f1  f3f2  f3f3  f3f4
n4  f4f1  f4f2  f4f3  f4f4

转换为数据帧非常简单,只需调用构造函数:

root@vps47509:~/test# for f in 480/*.mp4; do python makesprites.py $f; done
Making dir: /root/test/thumbs/1838448.mp_vtt
START [2017-11-02 12:32:17.246337] : ffmpeg -i 480/1838448.mp4 -f image2 -bt 20M -vf fps=1/10 -aspect 16:9 /root/test/thumbs/1838448.mp_vtt/tv%03d.jpg 
END   [2017-11-02 12:33:11.368050]
ffmpeg version 3.2.8-1~deb9u1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '480/1838448.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
  Duration: 00:10:00.02, start: 0.000000, bitrate: 2122 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2010 kb/s, 29.97 fps, 29.97 tbr, 11988 tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 103 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[swscaler @ 0x559cbf99d940] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/root/test/thumbs/1838448.mp_vtt/tv%03d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.56.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 0.10 fps, 0.10 tbn, 0.10 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.64.101 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=   61 fps=1.1 q=1.6 Lsize=N/A time=00:10:10.00 bitrate=N/A speed=11.3x    
video:4753kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Removing first image, unneeded
60 thumbs written in /root/test/thumbs/1838448.mp_vtt
START [2017-11-02 12:33:11.370139] : mogrify -geometry 100x /root/test/thumbs/1838448.mp_vtt/tv041.jpg /root/test/thumbs/1838448.mp_vtt/tv008.jpg /root/test/thumbs/1838448.mp_vtt/tv023.jpg /root/test/thumbs/1838448.mp_vtt/tv040.jpg /root/test/thumbs/1838448.mp_vtt/tv025.jpg /root/test/thumbs/1838448.mp_vtt/tv059.jpg /root/test/thumbs/1838448.mp_vtt/tv060.jpg /root/test/thumbs/1838448.mp_vtt/tv019.jpg /root/test/thumbs/1838448.mp_vtt/tv039.jpg /root/test/thumbs/1838448.mp_vtt/tv030.jpg /root/test/thumbs/1838448.mp_vtt/tv052.jpg /root/test/thumbs/1838448.mp_vtt/tv061.jpg /root/test/thumbs/1838448.mp_vtt/tv038.jpg /root/test/thumbs/1838448.mp_vtt/tv014.jpg /root/test/thumbs/1838448.mp_vtt/tv048.jpg /root/test/thumbs/1838448.mp_vtt/tv044.jpg /root/test/thumbs/1838448.mp_vtt/tv047.jpg /root/test/thumbs/1838448.mp_vtt/tv053.jpg /root/test/thumbs/1838448.mp_vtt/tv055.jpg /root/test/thumbs/1838448.mp_vtt/tv026.jpg /root/test/thumbs/1838448.mp_vtt/tv024.jpg /root/test/thumbs/1838448.mp_vtt/tv010.jpg /root/test/thumbs/1838448.mp_vtt/tv007.jpg /root/test/thumbs/1838448.mp_vtt/tv056.jpg /root/test/thumbs/1838448.mp_vtt/tv002.jpg /root/test/thumbs/1838448.mp_vtt/tv017.jpg /root/test/thumbs/1838448.mp_vtt/tv036.jpg /root/test/thumbs/1838448.mp_vtt/tv045.jpg /root/test/thumbs/1838448.mp_vtt/tv035.jpg /root/test/thumbs/1838448.mp_vtt/tv050.jpg /root/test/thumbs/1838448.mp_vtt/tv033.jpg /root/test/thumbs/1838448.mp_vtt/tv051.jpg /root/test/thumbs/1838448.mp_vtt/tv043.jpg /root/test/thumbs/1838448.mp_vtt/tv028.jpg /root/test/thumbs/1838448.mp_vtt/tv009.jpg /root/test/thumbs/1838448.mp_vtt/tv003.jpg /root/test/thumbs/1838448.mp_vtt/tv020.jpg /root/test/thumbs/1838448.mp_vtt/tv004.jpg /root/test/thumbs/1838448.mp_vtt/tv027.jpg /root/test/thumbs/1838448.mp_vtt/tv021.jpg /root/test/thumbs/1838448.mp_vtt/tv006.jpg /root/test/thumbs/1838448.mp_vtt/tv022.jpg /root/test/thumbs/1838448.mp_vtt/tv015.jpg /root/test/thumbs/1838448.mp_vtt/tv049.jpg /root/test/thumbs/1838448.mp_vtt/tv032.jpg /root/test/thumbs/1838448.mp_vtt/tv016.jpg /root/test/thumbs/1838448.mp_vtt/tv011.jpg /root/test/thumbs/1838448.mp_vtt/tv031.jpg /root/test/thumbs/1838448.mp_vtt/tv058.jpg /root/test/thumbs/1838448.mp_vtt/tv029.jpg /root/test/thumbs/1838448.mp_vtt/tv057.jpg /root/test/thumbs/1838448.mp_vtt/tv042.jpg /root/test/thumbs/1838448.mp_vtt/tv005.jpg /root/test/thumbs/1838448.mp_vtt/tv018.jpg /root/test/thumbs/1838448.mp_vtt/tv034.jpg /root/test/thumbs/1838448.mp_vtt/tv046.jpg /root/test/thumbs/1838448.mp_vtt/tv012.jpg /root/test/thumbs/1838448.mp_vtt/tv054.jpg /root/test/thumbs/1838448.mp_vtt/tv013.jpg /root/test/thumbs/1838448.mp_vtt/tv037.jpg 
ERROR   [2017-11-02 12:33:11.377429] An exception occurred
None
[Errno 2] No such file or directory
Traceback (most recent call last):
  File "makesprites.py", line 281, in <module>
    run(task)
  File "makesprites.py", line 248, in run
    resize(thumbfiles)
  File "makesprites.py", line 128, in resize
    doCmd("mogrify -geometry %dx %s" % (THUMB_WIDTH," ".join(map(pipes.quote, files))))
  File "makesprites.py", line 91, in doCmd
    raise e #todo ?
OSError: [Errno 2] No such file or directory