Python pandas - read_csv Dataframe正在从列

时间:2017-06-04 19:16:30

标签: python csv pandas dataframe

首先我知道下面的示例是缩进的,如果有人能指出我正确发布DataFrames的方向,我将非常感激。

现在,我有一个如下数据框:

Ticker_x	Date	Close_x	Ticker_y	Close_y	Lead_Lag	Ticker	15M_Trend	Bollinger_1SD	Bollinger_2SD	Bollinger_and_Trend_1SD	Bollinger_and_Trend_2SD	
ES M7	6/2/2017 3:29	2433.75	VX M7-CF	11.6	N							
ES M7	6/2/2017 4:29	2434.75	VX M7-CF	11.6	Short							
ES M7	6/2/2017 5:29	2435	VX M7-CF	11.55	N							
ES M7	6/2/2017 6:29	2434.25	VX M7-CF	11.57	N							
ES M7	6/2/2017 7:29	2430.25	VX M7-CF	11.7	Short							
ES M7	6/2/2017 8:29	2430.75	VX M7-CF	11.58	Short							
ES M7	6/2/2017 9:29	2433.25	VX M7-CF	11.63	Short							
ES M7	6/2/2017 10:29	2436.75	VX M7-CF	11.61	Short							
ES M7	6/2/2017 11:29	2436.75	VX M7-CF	11.57	Short							
ES M7	6/2/2017 12:29	2435.75	VX M7-CF	11.67	Short							
ES M7	6/2/2017 13:29	2437.75	VX M7-CF	11.64	Short							
ES M7	6/2/2017 14:29	2437.75	VX M7-CF	11.63	Short							

我正在从另一个文件的read_csv函数创建此数据框,并且仅包含下面列出的列。请注意,我正在从'glob'文件夹中获取最新文件:

filenameA = max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv"))

AggregatedA = pd.read_csv(filenameA, usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'])
p2 = r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv"
AggregatedA.to_csv(path_or_buf = p2)

所以新文件'blah2'是上面显示的csv文件。你会注意到,最右边的7或8列是空白的。这些不应该是空白的。它们在我从'filenameA'中选取的原始文件中不是空白的。

即。他们有一堆'Ticker','15M_Trend'和其他列的值。出于某种原因,大熊猫没有拿起那些价值,我不知道为什么。非常感谢帮助!

编辑:这是数据帧的原始行:

Ticker_x	Date	Open_x	High_x	Low_x	Close_x	Volume_x	Open Interest_x	Ticker_y	Open_y	High_y	Low_y	Close_y	Volume_y	Open Interest_y	ES_returns	VX_returns	Beta	Pairs_Spread	zscore	Pairs_Spread_Mean	Pairs_Spread_sdev	ES_percent_change	ES_difference	VX_percent_change	ES_CC	VX_CC	pairs_spread	pairs_zscore	ES_Inverse_price	ES__Inverse_percent_change	Inverse_ES_CC	Inverse_pairs_spread	Inverse_pairs_zscore	Lead_Lag	Ticker	Open	High	Low	Close	Volume	Open Interest	Index_Num	Rolling_OLS_Coefficient	15M_Long	Upper_Sdev_Value	Lower_Sdev_Value	Intercept	Middle	Sdev	Lower_Sdev_value	15M_Trend	Rolling_mean	Rolling_std	Upper_Band	Lower_Band	Upper_Band_2	Lower_Band_2	Bollinger_1SD	Bollinger_2SD	Bollinger_and_Trend_1SD	Bollinger_and_Trend_2SD	Trend_and_LL
ES M7	6/2/2017 3:29	2433.25	2433.75	2433	2433.75	3419	0	VX M7-CF	11.55	11.6	11.53	11.6	253	0	0	0.001727116	-6.946994692					0	0	0.001727116	-0.27013895	-0.27013895	2514.335138	2.733208694	2138.25	0	0.270990395	2218.835138	-1.144384967	N	VX M7-CF	11.55	11.6	11.53	11.6	253	0	1868	-0.006269114		27.09349175		23.18140592	23.10868419	3.984807553	19.12387664	Short	11.7912	0.1573199	11.9485199	11.6338801	12.1058398	11.4765602	Long	N	N	N	N
ES M7	6/2/2017 4:29	2435	2435.5	2434.75	2434.75	2847	0	VX M7-CF	11.59	11.6	11.55	11.6	118	0	0.000410889	0	-6.94464418					0.000410889	1	0	-0.434541317	-0.434541317	2515.307872	2.759879939	2137.25	-0.000467672	0.43443517	2217.807872	-1.197621083	Short	VX M7-CF	11.59	11.6	11.55	11.6	118	0	1869	-0.006344733		27.26010073		23.31610124	23.24250234	4.017598384	19.22490396	Short	11.7868	0.159558575	11.94635857	11.62724143	12.10591715	11.46768285	Long	N	N	N	Short
ES M7	6/2/2017 5:29	2436.25	2436.5	2435	2435	5979	0	VX M7-CF	11.55	11.6	11.5	11.55	716	0	0.00010268	-0.004310345	-6.945304375					0.00010268	0.25	-0.004310345	-0.364155097	-0.364155097	2515.218266	2.75742302	2137	-0.000116973	0.364203518	2217.218266	-1.228176348	N	VX M7-CF	11.55	11.6	11.5	11.55	716	0	1870	-0.006422122		27.43061644		23.45376171	23.37958619	4.051030244	19.32855595	Short	11.7814	0.162944964	11.94434496	11.61845504	12.10728993	11.45551007	Long	N	N	N	N
ES M7	6/2/2017 6:29	2434.5	2435	2433.75	2434.25	11821	0	VX M7-CF	11.5	11.65	11.5	11.57	1919	0	-0.000308008	0.001731602	-6.943255051					-0.000308008	-0.75	0.001731602	-0.391364705	-0.391364705	2514.583461	2.74001741	2137.75	0.000350959	0.391371805	2218.083461	-1.183339232	N	VX M7-CF	11.5	11.65	11.5	11.57	1919	0	1871	-0.006492588		27.58935246		23.57945638	23.50433714	4.08501532	19.41932182	Short	11.7764	0.165551151	11.94195115	11.61084885	12.1075023	11.4452977	Long	N	N	N	N
ES M7	6/2/2017 7:29	2431.25	2431.75	2430	2430.25	16785	0	VX M7-CF	11.6	11.75	11.58	11.7	3737	0	-0.001643217	0.011235955	-6.941089619					-0.001643217	-4	0.011235955	-0.795190709	-0.795190709	2511.460749	2.654396241	2141.75	0.001871126	0.79514286	2222.960749	-0.930583022	Short	VX M7-CF	11.6	11.75	11.58	11.7	3737	0	1872	-0.006552806		27.73075716		23.68800858	23.61134075	4.119416411	19.49192434	Short	11.776	0.165714286	11.94171429	11.61028571	12.10742857	11.44457143	N	N	N	N	Short
ES M7	6/2/2017 8:29	2430.25	2431	2429	2430.75	37511	0	VX M7-CF	11.7	11.8	11.53	11.58	15635	0	0.00020574	-0.01025641	-6.944443722					0.00020574	0.5	-0.01025641	-0.801967629	-0.801967629	2511.166658	2.646332626	2141.25	-0.000233454	0.801808413	2221.666658	-0.997646802	Short	VX M7-CF	11.7	11.8	11.53	11.58	15635	0	1873	-0.006629471		27.90272657		23.82506069	23.74829141	4.154435156	19.59385626	Short	11.773	0.167907287	11.94090729	11.60509271	12.10881457	11.43718543	Long	N	N	N	Short
ES M7	6/2/2017 9:29	2432.25	2433.75	2432.25	2433.25	26047	0	VX M7-CF	11.55	11.7	11.55	11.63	7148	0	0.001028489	0.004317789	-6.902663206					0.001028489	2.5	0.004317789	-0.50559849	-0.50559849	2513.527973	2.711077152	2138.75	-0.001167542	0.506119457	2219.027973	-1.134391675	Short	VX M7-CF	11.55	11.7	11.55	11.63	7148	0	1874	-0.006703266		28.06957526		23.95751485	23.87955587	4.190019391	19.68953648	Short	11.7706	0.169096157	11.93969616	11.60150384	12.10879231	11.43240769	N	N	N	N	Short
ES M7	6/2/2017 10:29	2435.75	2437	2435.5	2436.75	31594	0	VX M7-CF	11.65	11.65	11.55	11.61	5090	0	0.001438405	-0.00171969	-6.862573345					0.001438405	3.5	-0.00171969	-0.494655304	-0.494655304	2516.424477	2.790495933	2135.25	-0.00163647	0.495088995	2214.924477	-1.347047627	Short	VX M7-CF	11.65	11.65	11.55	11.61	5090	0	1875	-0.006773515		28.23118528		24.08370058	24.00506007	4.226125209	19.77893486	Short	11.7608	0.163729992	11.92452999	11.59707001	12.08825998	11.43334002	N	N	N	N	Short
ES M7	6/2/2017 11:29	2436	2437	2435.5	2436.75	25538	0	VX M7-CF	11.65	11.65	11.55	11.57	2278	0	0	-0.003445306	-6.86522263					0	0	-0.003445306	-0.46654779	-0.46654779	2516.180626	2.783809828	2135.25	0	0.466999626	2214.680626	-1.359684729	Short	VX M7-CF	11.65	11.65	11.55	11.57	2278	0	1876	-0.00683482		28.37745101		24.19388406	24.11480519	4.262645818	19.85215938	Short	11.7512	0.160466666	11.91166667	11.59073333	12.07213333	11.43026667	Long	N	N	N	Short
ES M7	6/2/2017 12:29	2437.75	2438.75	2435.5	2435.75	37037	0	VX M7-CF	11.58	11.69	11.55	11.67	4309	0	-0.000410383	0.008643042	-6.890566028					-0.000410383	-1	0.008643042	-0.503091577	-0.503091577	2516.162906	2.783323958	2136.25	0.000468329	0.503595438	2216.662906	-1.25695683	Short	VX M7-CF	11.58	11.69	11.55	11.67	4309	0	1877	-0.006883805		28.50205329		24.28297379	24.20263978	4.299413516	19.90322626	Short	11.7436	0.154929345	11.89852935	11.58867065	12.05345869	11.43374131	N	N	N	N	Short
ES M7	6/2/2017 13:29	2437.75	2438.5	2437	2437.75	26228	0	VX M7-CF	11.68	11.7	11.55	11.64	8570	0	0.000821102	-0.002570694	-6.88244494					0.000821102	2	-0.002570694	-0.520801285	-0.520801285	2517.861659	2.82990182	2134.25	-0.00093622	0.521308056	2214.361659	-1.376214577	Short	VX M7-CF	11.68	11.7	11.55	11.64	8570	0	1878	-0.006929786		28.62236572		24.3666349	24.2859722	4.336393523	19.94957867	Short	11.734	0.145728291	11.87972829	11.58827171	12.02545658	11.44254342	N	N	N	N	Short
ES M7	6/2/2017 14:29	2437.75	2437.75	2437.75	2437.75	1	0	VX M7-CF	11.6	11.7	11.55	11.63	28177	0	0	-0.000859107	-6.904693941					0	0	-0.000859107	-0.516701901	-0.516701901	2518.051591	2.835109521	2134.25	0	0.517232222	2214.551591	-1.36637174	Short	VX M7-CF	11.6	11.7	11.55	11.63	28177	0	1879	-0.006969382		28.73142917		24.4389793	24.35792539	4.37350378	19.98442161	Short	11.724	0.13474087	11.85874087	11.58925913	11.99348174	11.45451826	N	N	N	N	Short

编辑:这是基于以下建议的新代码:

AggregatedA = pd.read_csv(max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv")), usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t')

AggregatedA.to_csv(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv")

出现错误:

ValueError: Usecols do not match names.

编辑:我也尝试使用delimiter =','而不是delimiter ='\ t',它摆脱了错误。但是,右列仍为空白。

1 个答案:

答案 0 :(得分:0)

假设输入是给定的,令人惊讶的是pandas完全解析数据帧,因为应该明确给出分隔符。这样做,代码可以正常工作。如果in.txt包含行,则提供,然后运行

In [26]: AggregatedA = pd.read_csv('in.txt', usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t')

In [25]: AggregatedA.to_csv('out.txt')

将为out.txt提供以下内容:

$ cat out.txt
,Ticker_x,Date,Close_x,Ticker_y,Close_y,Lead_Lag,Ticker,15M_Trend,Bollinger_1SD,Bollinger_2SD,Bollinger_and_Trend_1SD,Bollinger_and_Trend_2SD
0,ES M7,6/2/2017 3:29,2433.75,VX M7-CF,11.6,N,VX M7-CF,Short,Long,N,N,N
1,ES M7,6/2/2017 4:29,2434.75,VX M7-CF,11.6,Short,VX M7-CF,Short,Long,N,N,N
2,ES M7,6/2/2017 5:29,2435.0,VX M7-CF,11.55,N,VX M7-CF,Short,Long,N,N,N
3,ES M7,6/2/2017 6:29,2434.25,VX M7-CF,11.57,N,VX M7-CF,Short,Long,N,N,N
4,ES M7,6/2/2017 7:29,2430.25,VX M7-CF,11.7,Short,VX M7-CF,Short,N,N,N,N
5,ES M7,6/2/2017 8:29,2430.75,VX M7-CF,11.58,Short,VX M7-CF,Short,Long,N,N,N
6,ES M7,6/2/2017 9:29,2433.25,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N
7,ES M7,6/2/2017 10:29,2436.75,VX M7-CF,11.61,Short,VX M7-CF,Short,N,N,N,N
8,ES M7,6/2/2017 11:29,2436.75,VX M7-CF,11.57,Short,VX M7-CF,Short,Long,N,N,N
9,ES M7,6/2/2017 12:29,2435.75,VX M7-CF,11.67,Short,VX M7-CF,Short,N,N,N,N
10,ES M7,6/2/2017 13:29,2437.75,VX M7-CF,11.64,Short,VX M7-CF,Short,N,N,N,N
11,ES M7,6/2/2017 14:29,2437.75,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N