Question

我有这个bash脚本：

#!/bin/bash
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s BTC/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s ETH/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s XRP/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s LTC/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s BCH/USDT

sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 2-dateConvert.py
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 3-concatCrypto.py
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 4-cutDates_1year.py "$(date +%Y-%m-%d -d "1 year ago")" "$(date +'%Y-%m-%d')" < DailyCryptoPricesFull.csv
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 4-cutDates_3months.py "$(date +%Y-%m-%d -d "3 months ago")" "$(date +'%Y-%m-%d')" < DailyCryptoPricesFull.csv

sleep 1
mkdir $(date +"%Y-%m-%d")
sleep 1
mv -- *.csv "$(date +'%Y-%m-%d')"

它提取5个不同的加密时间序列，在dateConvert.py脚本中转换日期格式，在concatCrypto.py脚本中加入所有5个加密，然后生成2个版本的输出，分别为1年和3个月。然后，它将创建一个以当前日期为名称的目录，并将所有文件移入其中。

这一切都工作得很好，但是后来我意识到我需要在venv中完成所有这些工作。我创建了venv并更改了我的python调用（如上所示）以使用venv。

但是现在发生的是第一组命令正确执行（5个数据提取），但是其余的python脚本根本没有执行。（mkdir和mv命令可以正确执行。）

我收到以下错误：

Traceback (most recent call last):
  File "3-concatCrypto.py", line 33, in <module>
    df_merged = reduce(lambda  left,right: pd.merge(left,right,on=['Timestamp'], how='outer'), data_frames).fillna('NULL')
TypeError: reduce() of empty sequence with no initial value
getCrypto.sh: 33: getCrypto.sh: cannot open DailyCryptoPricesFull.csv: No such file
getCrypto.sh: 37: getCrypto.sh: cannot open DailyCryptoPricesFull.csv: No such file

这表明它在concatCrypto.py文件上开始失败，但是我知道在此之前它实际上失败了（真的跳过了脚本），因为日期没有转换。

我猜这与您引用venvs中的路径有关，但是我不确定。

以下是dateConvert.py的内容：

import csv
import datetime
import time
import pandas as pd
import glob

path = "/home/reallymemorable/Documents/scripts/Crypto/*1d.csv"
for fname in glob.glob(path):
    print(fname)
    # Step 1: Convert the dates
    df = pd.read_csv(fname)
    df['Timestamp'] = pd.to_datetime(df['Timestamp'], unit='ms')
    # print(df.head())
    # nameWithDate = str(str(datetime.date.today()) + fname)
    df.to_csv(fname + '-dateFormatted.csv', index=False)

和concatCrypto.py：

import csv
import datetime
import time
import pandas as pd
import glob
from functools import reduce
import os
from datetime import date


data_frames = []

for fname in glob.glob("/home/reallymemorable/Documents/scripts/Crypto/*dateFormatted.csv"):
   data_frames.append(pd.read_table(fname, sep=','))

df_merged = reduce(lambda  left,right: pd.merge(left,right,on=['Timestamp'], how='outer'), data_frames).fillna('NULL')
print(df_merged)
df_sorted = df_merged.sort_values(by='Timestamp')
print(df_sorted)
# df_oneYear = df_sorted[df_sorted['Timestamp'] == '2019-06-11']
# print(df_oneYear)
df_sorted.to_csv('DailyCryptoPricesFull.csv', index=False)

编辑：

错误似乎是glob不在dateConvert.py中运行。

我尝试手动安装它，但出现此错误：

reallymemorable@gort:~/Documents/environments$ python3 --version
Python 3.6.9
reallymemorable@gort:~/Documents/environments$ python --version
Python 2.7.17
reallymemorable@gort:~/Documents/environments$ source mai_trading/bin/activate
(mai_trading) reallymemorable@gort:~/Documents/environments$ python --version
Python 2.7.17
(mai_trading) reallymemorable@gort:~/Documents/environments$ python3 --version
Python 3.6.9
(mai_trading) reallymemorable@gort:~/Documents/environments$ pip3 install glob
Collecting glob
Exception:
Traceback (most recent call last):
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/home/reallymemorable/Documents/environments/trading/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/glob/

Answer 1

编辑： facepalm 我已经更改了目录的名称，并在glob调用中对路径进行了硬编码。

通过venv中的bash运行python脚本时发生意外故障

1 个答案: