R - 数据帧 - 转换为稀疏矩阵

4 回答

• 3

  为了避免将所有数据复制到密集矩阵中，可能会有更高的内存效率（但速度更慢）：

  y <- Reduce(cbind2, lapply(x[,-1], Matrix, sparse = TRUE))
  rownames(y) <- x[,1]
  
  #3 x 3 sparse Matrix of class "dgCMatrix"
  #         
  #ABC 1 . .
  #DEF . 1 .
  #GHI . . 1
  

  如果你有足够的内存，你应该使用理查德的答案，即将你的data.frame变成密集矩阵，而不是使用 Matrix .

  回复于 2024-04-25T21:16:52+08:00
• 8

  我一直这样做，这是一个痛苦的屁股，所以我在我的R包中写了一个名为sparsify()的方法 - mltools . 它在 data.table 上运行，这些只是花哨 data.frames .

  ---

  解决你的具体问题......

  Install mltools (or just copy the sparsify() method into your environment)

  Load packages

  library(data.table)
  library(Matrix)
  library(mltools)
  

  Sparsify

  x <- data.table(x)  # convert x to a data.table
  sparseM <- sparsify(x[, !"name"])  # sparsify everything except the name column
  rownames(sparseM) <- x$name  # set the rownames
  
  > sparseM
  3 x 3 sparse Matrix of class "dgCMatrix"
      factor_1 factor_2 factor_3
  ABC        1        .        .
  DEF        .        1        .
  GHI        .        .        1
  

  ---

  通常，sparsify（）方法非常灵活 . 以下是一些如何使用它的示例：

  Make some data. Notice data types and unused factor levels

  dt <- data.table(
    intCol=c(1L, NA_integer_, 3L, 0L),
    realCol=c(NA, 2, NA, NA),
    logCol=c(TRUE, FALSE, TRUE, FALSE),
    ofCol=factor(c("a", "b", NA, "b"), levels=c("a", "b", "c"), ordered=TRUE),
    ufCol=factor(c("a", NA, "c", "b"), ordered=FALSE)
  )
  > dt
     intCol realCol logCol ofCol ufCol
  1:      1      NA   TRUE     a     a
  2:     NA       2  FALSE     b    NA
  3:      3      NA   TRUE    NA     c
  4:      0      NA  FALSE     b     b
  

  Out-Of-The-Box Use

  > sparsify(dt)
  4 x 7 sparse Matrix of class "dgCMatrix"
       intCol realCol logCol ofCol ufCol_a ufCol_b ufCol_c
  [1,]      1      NA      1     1       1       .       .
  [2,]     NA       2      .     2      NA      NA      NA
  [3,]      3      NA      1    NA       .       .       1
  [4,]      .      NA      .     2       .       1       .
  

  Convert NAs to 0s and Sparsify Them

  > sparsify(dt, sparsifyNAs=TRUE)
  4 x 7 sparse Matrix of class "dgCMatrix"
       intCol realCol logCol ofCol ufCol_a ufCol_b ufCol_c
  [1,]      1       .      1     1       1       .       .
  [2,]      .       2      .     2       .       .       .
  [3,]      3       .      1     .       .       .       1
  [4,]      .       .      .     2       .       1       .
  

  Generate Columns That Identify NA Values

  > sparsify(dt[, list(realCol)], naCols="identify")
  4 x 2 sparse Matrix of class "dgCMatrix"
       realCol_NA realCol
  [1,]          1      NA
  [2,]          .       2
  [3,]          1      NA
  [4,]          1      NA
  

  Generate Columns That Identify NA Values In the Most Memory Efficient Manner

  > sparsify(dt[, list(realCol)], naCols="efficient")
  4 x 2 sparse Matrix of class "dgCMatrix"
       realCol_NotNA realCol
  [1,]             .      NA
  [2,]             1       2
  [3,]             .      NA
  [4,]             .      NA
  

  回复于 2024-04-25T21:16:52+08:00
• 5

  您可以将第一列设为行名，然后使用 Matrix 包中的 Matrix .

  rownames(x) <- x$name
  x <- x[-1]
  library(Matrix)
  Matrix(as.matrix(x), sparse = TRUE)
  # 3 x 3 sparse Matrix of class "dtCMatrix"
  #     factor_1 factor_2 factor_3
  # ABC        1        .        .
  # DEF        .        1        .
  # GHI        .        .        1
  

  原始 x 数据框的位置

  x <- structure(list(name = structure(1:3, .Label = c("ABC", "DEF", 
  "GHI"), class = "factor"), factor_1 = c(1L, 0L, 0L), factor_2 = c(0L, 
  1L, 0L), factor_3 = c(0L, 0L, 1L)), .Names = c("name", "factor_1", 
  "factor_2", "factor_3"), class = "data.frame", row.names = c(NA, 
  -3L))
  

  回复于 2024-04-25T21:16:52+08:00
• 3

  你的矩阵有多稀疏？这决定了如何改善它的大小 .

  您的示例矩阵有3 1 s和6 0 s . 有了这个比例，天真地使用Matrix就节省了很多空间 .

  > library('pryr') # for object_size
  > library('Matrix')
  > m <- matrix(rbinom(9e4*1e4, 1, 1/3), ncol = 1e4)
  > object_size(m)
  3.6 GB
  > object_size(Matrix(m, sparse = T))
  3.6 GB
  

  回复于 2024-04-25T21:16:52+08:00