我想在终端中将文本文件导入MySQL。文本文件太大而无法打开并查看，但我知道它是吧'|'分隔。
我找到了一个博客说这样做：
LOAD DATA INFILE '/tmp/data.txt'
INTO TABLE db1.table1
FIELDS TERMINATED BY '|';
我试过了，但SQL告诉我先创建表，我做了。但它也想知道这些字段名称。在线研究似乎表明，当我制作表格时，我必须指定字段标题及其数据类型。
有解决方法吗？我必须输入大约20个字段名称，我甚至不知道他们的数据类型。
还有其他办法吗？
修改以下评论
我设法使用head -n 3 path / to / file获得头部，结果是：
NIM_EVENT_ID|NIM_EVENT_DATA_TYPE_ID|NIM_EVENT_TYPE_ID|EVENT_TIMESTAMP|EVENT_GMT_TIMESTAMP|PRODUCT_ISSUE_ID|NIM_TITLE_ID|NIM_ISSUE_ID|NIM_USER_ID|RECORD_ID|TRANSACTION_ID|DWELL_MINUTES|FREE_TRIAL_FLAG|UNLIMITED_FLAG|APPLICATION_VERSION_ID|DEVICE_GUID|DEVICE_INFO_ID|DEVICE_BRAND|DEVICE_PRODUCT|DEVICE_TYPE|DEVICE_HARDWARE|DEVICE_MANUFACTURER|DEVICE_MODEL|DEVICE_OS|DEVICE_OS_VERSION|DEVICE_DISPLAY_INFO|DEVICE_DISPLAY_SIZE|DEVICE_DISPLAY_ORIENTATION_ID|EVENT_VERSION|SOURCE_CREATE_TIMESTAMP|SOURCE_LAST_MODIFIED_TIMESTAMP|WORKFLOW_RUN_ID|ETL_CREATE_DATE|ETL_UPDATE_DATE|SOURCE_FILE_ID
103008184|879|883|2013-12-07 00:30:35.027000|2013-12-07 05:30:35.027000|-1|4800010|36000003|57700008|61906389|21385144|0|Y|Y||4c22c4a3-094a-476f-9a48-c67b15c2d3f7|-2|||||||||||887|2|2013-12-06 21:35:39.023000|2013-12-06 21:35:39.023000||2013-12-07 03:31:33.910000|2013-12-07 03:31:33.910000|
98500241|875|880|2013-12-07 15:43:36.984000|2013-12-07 20:43:36.984000|-1|-2|-2|82700012|63226292|||N|N|-i6.1.3-APPLE-3.10.0|45F4BCE1-A6DC-4A66-A071-7D628FC7522E|21|Apple|iPhone OS|iPad|iPad|Apple|iPad|iOS|6.1.3|iPad|DisplayMetrics{width=768.000000, height=1024.000000, scale=2.000000}|887|2|2013-12-07 12:45:50.099000|2013-12-07 12:45:50.099000||2013-12-07 19:31:34.470000|2013-12-07 19:31:34.470000|
我猜我在这里告诉我的是，为了创建.SQL表，要在头部和字段信息中复制一个脚本？这就是我们正在做的事情 - 从头部复制格式以便将表格导入txt文件？
答案 0 :(得分：1)
有很多方法可以解决这个问题，这是一种方法。其中一些步骤可以通过实用程序缩短 - 我假设你没有这些步骤。
编写程序以读取文件，并将每行的每个字段的长度打印到单独的文件中。因此，您将拥有field00.txt，field01.txt等文件。
（注意：您可以使用任何文本编辑器和RegEx轻松完成此操作）
每个文件的内容如下所示：
14
12
15
现在您需要在每个文件中找到最大数量。编写程序或使用文本编辑器对文件进行排序。
（注意：如果你对shell很好，可以使用cut / sort来完成以上所有操作！）
此时，您将知道表中每个字段的最大长度。通过使用预览，您还可以推断出数据类型。根据此信息，为表创建说明。尝试导入数据。如果失败，请参阅错误消息并调整数据类型。再试一次，直到你成功。
编辑：只需阅读您的评论。所以这是使用终端的东西：
终端命令
（警告：我现在没有接到Mac，所以没有测试过这个）
首先，将每列写入自己的文件：
cut -d'|' -f1 bigfile.txt > col01.txt
cut -d'|' -f2 bigfile.txt > col02.txt
cut -d'|' -f3 bigfile.txt > col03.txt
...
cut -d'|' -f20 bigfile.txt > col20.txt
然后找到每列的最大长度：
awk '{ if (length > max) max = length } END { print max } ' col01.txt
awk '{ if (length > max) max = length } END { print max } ' col02.txt
awk '{ if (length > max) max = length } END { print max } ' col03.txt
...
awk '{ if (length > max) max = length } END { print max } ' col20.txt
（注意：由于您使用的是Mac OS X，wc -L将无效。）
检查每列的前几个值，如下所示：
head col01.txt
head col02.txt
...
head col20.txt
现在您知道每列的数据类型和最大长度。基于此，声明/创建您的表。
答案 1 :(得分：1)
我是否必须手动查看输出并尝试确定新SQL表中的每个字段？是否有使用此输出的快捷方式？
优秀的文字编辑是你最好的朋友。这是我在几分钟内用TextMate
得出的CREATE TABLE import_table
(
nim_event_id INT,
nim_event_data_type_id INT,
nim_event_type_id INT,
event_timestamp DATETIME,
event_gmt_timestamp DATETIME,
product_issue_id INT,
nim_title_id INT,
nim_issue_id INT,
nim_user_id INT,
record_id INT,
transaction_id INT,
dwell_minutes INT,
free_trial_flag CHAR(1),
unlimited_flag CHAR(1),
application_version_id VARCHAR(128),
device_guid CHAR(37),
device_info_id VARCHAR(128),
device_brand VARCHAR(128),
device_product VARCHAR(128),
device_type VARCHAR(128),
device_hardware VARCHAR(128),
device_manufacturer VARCHAR(128),
device_model VARCHAR(128),
device_os VARCHAR(128),
device_os_version VARCHAR(128),
device_display_info VARCHAR(128),
device_display_size VARCHAR(128),
device_display_orientation_id INT,
event_version INT,
source_create_timestamp DATETIME,
source_last_modified_timestamp DATETIME,
workflow_run_id INT,
etl_create_date DATETIME,
etl_update_date DATETIME,
source_file_id INT
);
现在，为了能够控制您插入的值，您可以利用SET的会话变量和LOAD DATA INFILE子句。例如。要在表格中将缺席值设置为NULL，您可以
LOAD DATA INFILE '/tmp/data.txt'
INTO TABLE import_table
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n' -- or '\r\n' if the file has been created on a Windows machine
IGNORE 1 LINES -- skip the header line
(
@nim_event_id,
@nim_event_data_type_id,
@nim_event_type_id,
@event_timestamp,
@event_gmt_timestamp,
@product_issue_id,
@nim_title_id,
@nim_issue_id,
@nim_user_id,
@record_id,
@transaction_id,
@dwell_minutes,
@free_trial_flag,
@unlimited_flag,
@application_version_id,
@device_guid,
@device_info_id,
@device_brand,
@device_product,
@device_type,
@device_hardware,
@device_manufacturer,
@device_model,
@device_os,
@device_os_version,
@device_display_info,
@device_display_size,
@device_display_orientation_id,
@event_version,
@source_create_timestamp,
@source_last_modified_timestamp,
@workflow_run_id,
@etl_create_date,
@etl_update_date,
@source_file_id
)
SET
nim_event_id = NULLIF(@nim_event_id, ''),
nim_event_data_type_id = NULLIF(@nim_event_data_type_id, ''),
nim_event_type_id = NULLIF(@nim_event_type_id, ''),
event_timestamp = NULLIF(@event_timestamp, ''),
event_gmt_timestamp = NULLIF(@event_gmt_timestamp, ''),
product_issue_id = NULLIF(@product_issue_id, ''),
nim_title_id = NULLIF(@nim_title_id, ''),
nim_issue_id = NULLIF(@nim_issue_id, ''),
nim_user_id = NULLIF(@nim_user_id, ''),
record_id = NULLIF(@record_id, ''),
transaction_id = NULLIF(@transaction_id, ''),
dwell_minutes = NULLIF(@dwell_minutes, ''),
free_trial_flag = NULLIF(@free_trial_flag, ''),
unlimited_flag = NULLIF(@unlimited_flag, ''),
application_version_id = NULLIF(@application_version_id, ''),
device_guid = NULLIF(@device_guid, ''),
device_info_id = NULLIF(@device_info_id, ''),
device_brand = NULLIF(@device_brand, ''),
device_product = NULLIF(@device_product, ''),
device_type = NULLIF(@device_type, ''),
device_hardware = NULLIF(@device_hardware, ''),
device_manufacturer = NULLIF(@device_manufacturer, ''),
device_model = NULLIF(@device_model, ''),
device_os = NULLIF(@device_os, ''),
device_os_version = NULLIF(@device_os_version, ''),
device_display_info = NULLIF(@device_display_info, ''),
device_display_size = NULLIF(@device_display_size, ''),
device_display_orientation_id = NULLIF(@device_display_orientation_id, ''),
event_version = NULLIF(@event_version, ''),
source_create_timestamp = NULLIF(@source_create_timestamp, ''),
source_last_modified_timestamp = NULLIF(@source_last_modified_timestamp, ''),
workflow_run_id = NULLIF(@workflow_run_id, ''),
etl_create_date = NULLIF(@etl_create_date, ''),
etl_update_date = NULLIF(@etl_update_date, ''),
source_file_id = NULLIF(@source_file_id, '')
;
您可以根据需要进一步调整。上述语句成功加载了您发布的两个样本行
结果如下：
+--------------+------------------------+-------------------+---------------------+---------------------+------------------+--------------+--------------+-------------+-----------+----------------+---------------+-----------------+----------------+------------------------+--------------------------------------+----------------+--------------+----------------+-------------+-----------------+---------------------+--------------+-----------+-------------------+---------------------+----------------------------------------------------------------------+-------------------------------+---------------+-------------------------+--------------------------------+-----------------+---------------------+---------------------+----------------+
| nim_event_id | nim_event_data_type_id | nim_event_type_id | event_timestamp | event_gmt_timestamp | product_issue_id | nim_title_id | nim_issue_id | nim_user_id | record_id | transaction_id | dwell_minutes | free_trial_flag | unlimited_flag | application_version_id | device_guid | device_info_id | device_brand | device_product | device_type | device_hardware | device_manufacturer | device_model | device_os | device_os_version | device_display_info | device_display_size | device_display_orientation_id | event_version | source_create_timestamp | source_last_modified_timestamp | workflow_run_id | etl_create_date | etl_update_date | source_file_id |
+--------------+------------------------+-------------------+---------------------+---------------------+------------------+--------------+--------------+-------------+-----------+----------------+---------------+-----------------+----------------+------------------------+--------------------------------------+----------------+--------------+----------------+-------------+-----------------+---------------------+--------------+-----------+-------------------+---------------------+----------------------------------------------------------------------+-------------------------------+---------------+-------------------------+--------------------------------+-----------------+---------------------+---------------------+----------------+
| 103008184 | 879 | 883 | 2013-12-07 00:30:35 | 2013-12-07 05:30:35 | -1 | 4800010 | 36000003 | 57700008 | 61906389 | 21385144 | 0 | Y | Y | NULL | 4c22c4a3-094a-476f-9a48-c67b15c2d3f7 | -2 | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 887 | 2 | 2013-12-06 21:35:39 | 2013-12-06 21:35:39 | NULL | 2013-12-07 03:31:34 | 2013-12-07 03:31:34 | NULL |
| 98500241 | 875 | 880 | 2013-12-07 15:43:37 | 2013-12-07 20:43:37 | -1 | -2 | -2 | 82700012 | 63226292 | NULL | NULL | N | N | -i6.1.3-APPLE-3.10.0 | 45F4BCE1-A6DC-4A66-A071-7D628FC7522E | 21 | Apple | iPhone OS | iPad | iPad | Apple | iPad | iOS | 6.1.3 | iPad | DisplayMetrics{width=768.000000, height=1024.000000, scale=2.000000} | 887 | 2 | 2013-12-07 12:45:50 | 2013-12-07 12:45:50 | NULL | 2013-12-07 19:31:34 | 2013-12-07 19:31:34 | NULL |
+--------------+------------------------+-------------------+---------------------+---------------------+------------------+--------------+--------------+-------------+-----------+----------------+---------------+-----------------+----------------+------------------------+--------------------------------------+----------------+--------------+----------------+-------------+-----------------+---------------------+--------------+-----------+-------------------+---------------------+----------------------------------------------------------------------+-------------------------------+---------------+-------------------------+--------------------------------+-----------------+---------------------+---------------------+----------------+