如何在Neo4j中创建数十亿个节点？

2 回答

• 5

  那么你要测量的是lucene指数的表现 . 所以不是图形数据库操作 .

  有很多选择：

  neo4j-import

  Neo4j 2.2.0-M03附带neo4j-import，这个工具可以快速，可扩展地将10亿个节点csv导入Neo4j .

  parallel-batch-importer API

  这在Neo4j 2.2中是非常新的

  我使用新的ParallelBatchImporter在5分13秒（53G db）中创建了一个仅有节点的图，其中包含1.000.000.000个节点 . 这使得它约为3.2M节点/秒 .

  代码在这里：https://gist.github.com/jexp/0ff850ab2ce41c9ca5e6

  批量插入器

  您可以使用Neo4j Batch-Inserter-API创建该数据，而无需先创建CSV .

  请参阅此示例，您必须采用该示例才能读取CSV但直接从for循环生成数据：http://jexp.de/blog/2014/10/flexible-neo4j-batch-import-with-groovy/

  Cypher

  如果你想使用Cypher，我建议在 JAVA_OPTS="-Xmx4G -Xms4G" bin/neo4j-shell -path billion.db 中运行类似的东西：

  以下是我在macbook上使用的10M和100M的代码和时序：

  创建一个包含1M行的csv文件

  ruby -e 'File.open("million.csv","w") 
     { |f| (1..1000000).each{|i| f.write(i.to_s + "\n") }  }'
  

  在MacBook Pro上运行的实验Cypher执行是单线程估计大小（15 42）字节*节点计数

  // on my laptop
  // 10M nodes, 1 property, 1 label each in 98228 ms (98s) taking 580 MB on disk
  
  using periodic commit 10000
  load csv from "file:million.csv" as row
  //with row limit 5000
  foreach (x in range(0,9) | create (:Person {id:toInt(row[0])*10+x}));
  
  // on my laptop
  // 100M nodes, 1 property, 1 label each in 1684411 ms (28 mins) taking 6 GB on disk
  
  using periodic commit 1000
  load csv from "file:million.csv" as row
  foreach (x in range(0,99) | create (:Person {id:toInt(row[0])*100+x}));
  
  // on my linux server
  // 1B nodes, 1 property, 1 label each in 10588883 ms (176 min) taking 63 GB on disk
  
  using periodic commit 1000
  load csv from "file:million.csv" as row
  foreach (x in range(0,999) | create (:Person {id:toInt(row[0])*100+x}));
  

  创建索引

  create index on :Person(id);
  schema await
  
  // took about 40 mins and increased the database size to 85 GB
  

  然后我就可以跑了

  match (:Person {id:8005300}) return count(*);
  +----------+
  | count(*) |
  +----------+
  | 1        |
  +----------+
  1 row
  2 ms
  

  回复于 2024-04-26T14:02:18+08:00
• 8

  另一个简单的答案是好的 . 如果你想要更多的东西，Michael Hunger posted a good blog entry on this . 他推荐的东西基本上非常相似，但你也可以循环一些样本数据，并使用随机数 Build 联系 .

  以下是他如何创建100,000个用户和产品并将其链接起来，根据您的需要进行自定义：

  WITH ["Andres","Wes","Rik","Mark","Peter","Kenny","Michael","Stefan","Max","Chris"] AS names
  FOREACH (r IN range(0,100000) | CREATE (:User {id:r, name:names[r % size(names)]+" "+r}));
  
  with ["Mac","iPhone","Das Keyboard","Kymera Wand","HyperJuice Battery",
  "Peachy Printer","HexaAirBot",
  "AR-Drone","Sonic Screwdriver",
  "Zentable","PowerUp"] as names
      foreach (r in range(0,50) | create (:Product {id:r, name:names[r % size(names)]+" "+r}));
  

  让我们不要忘记甜蜜的随机链接：

  match (u:User),(p:Product)
  where rand() < 0.1
  with u,p
  limit 50000
  merge (u)-[:OWN]->(p);
  

  发疯 .

  回复于 2024-04-26T14:02:18+08:00