Question

我正在编写一个在AzureML中使用的python脚本。我的数据集很大。我有一个名为ID（int）和DataType（text）的数据集。我想将这些值连接成只有一列文本，其中ID和数据类型都用逗号分隔。

如何在执行此操作时避免出错。我的代码中有任何错误吗？

当我运行此代码时，我收到以下错误：

Error 0085: The following error occurred during script evaluation, please view the output log for more information:
---------- Start of error message from Python interpreter ----------
data:text/plain,Caught exception while executing function: Traceback (most recent call last):
File "C:\server\invokepy.py", line 167, in batch
idfs.append(rutils.RUtils.RFileToDataFrame(infile))
File "C:\server\RReader\rutils.py", line 15, in RFileToDataFrame
rreader = RReaderFactory.construct_from_file(filename, compressed)
File "C:\server\RReader\rreaderfactory.py", line 25, in construct_from_file
return _RReaderFactory.construct_from_stream(stream)
File "C:\server\RReader\rreaderfactory.py", line 46, in construct_from_stream
return RReader(BinaryReader(RFactoryConstants.big_endian, stream.read()))
File "C:\pyhome\lib\gzip.py", line 254, in read
self._read(readsize)
File "C:\pyhome\lib\gzip.py", line 313, in _read
self._add_read_data( uncompress )
File "C:\pyhome\lib\gzip.py", line 329, in _add_read_data
self.crc = zlib.crc32(data, self.crc) & 0xffffffffL
OverflowError: size does not fit in an int

我的代码如下：

# The script MUST contain a function named azureml_main
# which is the entry point for this module.
#
# The entry point function can contain up to two input arguments:
#   Param<dataframe1>: a pandas.DataFrame
#   Param<dataframe2>: a pandas.DataFrame

def azureml_main(dataframe1):
import pandas as pd
dataframe1['SignalID,DataType'] = dataframe1['ID'] + " , " + dataframe1['DataType']
dataframe1 = dataframe1.drop('DataType')
dataframe1 = dataframe1.drop('ID')
# Return value must be of a sequence of pandas.DataFrame
return dataframe1

我在AzureML中运行默认的python代码时遇到同样的错误。所以我很确定我的数据不适合数据框。

默认脚本如下：

# The script MUST contain a function named azureml_main
# which is the entry point for this module.
#
# The entry point function can contain up to two input arguments:
#   Param<dataframe1>: a pandas.DataFrame
#   Param<dataframe2>: a pandas.DataFrame
def azureml_main(dataframe1 = None, dataframe2 = None):

    # Execution logic goes here
    print('Input pandas.DataFrame #1:\r\n\r\n{0}'.format(dataframe1))

    # If a zip file is connected to the third input port is connected,
    # it is unzipped under ".\Script Bundle". This directory is added
    # to sys.path. Therefore, if your zip file contains a Python file
    # mymodule.py you can import it using:
    # import mymodule

    # Return value must be of a sequence of pandas.DataFrame
    return dataframe1,

Answer 1

如果您需要将整数ID和字符串DataType列连接到新列SignalID，请使用astype转换。然后，您可以drop列DataType和ID添加参数axis=1：

import pandas as pd

def azureml_main(dataframe1):
    dataframe1['SignalID'] = dataframe1['ID'].astype(str) 
                                      + " , " 
                                      + dataframe1['DataType']

    dataframe1 = dataframe1.drop(['DataType', 'ID'], axis=1)
    # Return value must be of a sequence of pandas.DataFrame
    return dataframe1

OverflowError：大小不适合int

1 个答案: