从csv文件复制cassandra表

我正在为我的Mac(Mac OS X Yosemite和Oracle jdk1.7.0_55)上的Cassandra,Apache Spark和Flume设置演示环境.景观应作为新分析平台的概念证明,因此我需要在我的cassandra数据库中使用一些测试数据.我正在使用cassandra 2.0.8.

我在excel中创建了一些演示数据并将其导出为CSV文件.结构是这样的:

ProcessUUID;ProcessID;ProcessNumber;ProcessName;ProcessStartTime;ProcessStartTimeUUID;ProcessEndTime;ProcessEndTimeUUID;ProcessStatus;Orderer;VorgangsNummer;VehicleID;FIN;Reference;ReferenceType
0F0D1498-D149-4FCC-87C9-F12783FDF769;AbmeldungKl‰rfall;1;Abmeldung Kl‰rfall;2011-02-03 04:05+0000;;2011-02-17 04:05+0000;;Finished;SIXT;4278;A-XA 1;WAU2345CX67890876;KLA-BR4278;internal

然后我使用以下命令在cqlsh中创建了一个键空间和一个列族:

CREATE KEYSPACE dadcargate 
WITH REPLICATAION  = { 'class' : 'SimpleStrategy', 'replication_factor' : '1' };

use dadcargate;

CREATE COLUMNFAMILY Process (
  ProcessUUID uuid, ProcessID varchar, ProcessNumber bigint, ProcessName varchar, 
  ProcessStartTime timestamp, ProcessStartTimeUUID timeuuid, ProcessEndTime timestamp, 
  ProcessEndTimeUUID timeuuid, ProcessStatus varchar, Orderer varchar,
  VorgangsNummer varchar, VehicleID varchar, FIN varchar, Reference varchar,
  ReferenceType varchar, 
PRIMARY KEY (ProcessUUID))
WITH COMMENT='A process is like a bracket around multiple process steps';

列系列名称及其中的所有列都是使用小写创建的 – 有一天也必须对此进行调查,但目前不太相关.

现在我拿出我的CSV文件,它有大约1600个条目,并希望在我的表名为process的过程中导入它,如下所示:

cqlsh:dadcargate> COPY process (processuuid, processid, processnumber, processname, 
processstarttime, processendtime, processstatus, orderer, vorgangsnummer, vehicleid,
fin, reference, referencetype) 
FROM 'Process_BulkData.csv' WITH DELIMITER = ';' AND HEADER = TRUE;

它给出以下错误:

Record #0 (line 1) has the wrong number of fields (15 instead of 13).
0 rows imported in 0.050 seconds.

这基本上是正确的,因为我的cvs-export中没有timeUUID字段.

如果我尝试没有像这样的显式列名的COPY命令(事实上,我确实错过了两个字段):

cqlsh:dadcargate> COPY process from 'Process_BulkData.csv' 
WITH DELIMITER = ';' AND HEADER = TRUE;

我最后得到了另一个错误:

Bad Request: Input length = 1
Aborting import at record #0 (line 1). Previously-inserted values still present.
0 rows imported in 0.009 seconds.

嗯.有点奇怪,但没关系.也许COPY命令不喜欢缺少两个字段的事实.我仍然认为这很奇怪,因为缺少的字段当然是(从结构的角度来看)但只是空的.

我还有另外一个镜头:我删除了excel中缺少的列,再次将文件作为cvs导出,并尝试在我的csv BUT显式列名中导入WITHOUT标题行,如下所示:

cqlsh:dadcargate> COPY process (processuuid, processid, processnumber, processname, 
processstarttime, processendtime, processstatus, orderer, vorgangsnummer, vehicleid, 
fin, reference, referencetype) 
FROM 'Process_BulkData-2.csv' WITH DELIMITER = ';' AND HEADER = TRUE;

我收到此错误:

Bad Request: Input length = 1
Aborting import at record #0 (line 1). Previously-inserted values still present.
0 rows imported in 0.034 seconds.

任何人都可以告诉我这里我做错了什么吗?根据documentation of copy-command,我设置命令的方式应该适用于其中至少两个.或者我会这么想.

但是,不,我显然在这里缺少一些重要的东西.

cqlsh的COPY命令可能很敏感.但是,在COPY documentation是这一行:

The number of columns in the CSV input is the same as the number of columns in the Cassandra table metadata.

牢记这一点,我确实设法通过命名空字段(分别为processstarttimeuuid和processendtimeuuid)来使用COPY FROM导入数据:

aploetz@cqlsh:stackoverflow> COPY process (processuuid, processid, processnumber, 
processname, processstarttime, processstarttimeuuid, processendtime, 
processendtimeuuid, processstatus, orderer, vorgangsnummer, vehicleid, fin, reference, 
referencetype) FROM 'Process_BulkData.csv' WITH DELIMITER = ';' AND HEADER = TRUE;

1 rows imported in 0.018 seconds.
aploetz@cqlsh:stackoverflow> SELECT * FROM process ;

 processuuid                          | fin               | orderer | processendtime            | processendtimeuuid | processid         | processname        | processnumber | processstarttime          | processstarttimeuuid | processstatus | reference  | referencetype | vehicleid | vorgangsnummer
--------------------------------------+-------------------+---------+---------------------------+--------------------+-------------------+--------------------+---------------+---------------------------+----------------------+---------------+------------+---------------+-----------+----------------
 0f0d1498-d149-4fcc-87c9-f12783fdf769 | WAU2345CX67890876 |    SIXT | 2011-02-16 22:05:00+-0600 |               null | AbmeldungKl‰rfall | Abmeldung Kl‰rfall |             1 | 2011-02-02 22:05:00+-0600 |                 null |      Finished | KLA-BR4278 |      internal |    A-XA 1 |           4278

(1 rows)
翻译自:https://stackoverflow.com/questions/28645815/copy-cassandra-table-from-csv-file

转载注明原文:从csv文件复制cassandra表