将新的namenode数据目录添加到现有集群

2 回答

• 3

  我相信我可能刚刚回答了我自己的问题 . 我最终将一个现有namenode目录的全部内容复制到新的NFS namenode目录，然后我就可以启动namenode了 . （注意我在复制之前停止了namenode以避免出现问题）

  cp -rp /data/1/dfs/nn /data/nfs/dfs/nn
  

  我想我的假设是namenode会自动将现有的元数据复制到新目录是不正确的 .

  回复于 2024-04-20T05:30:28+08:00
• 0

  在Cloudera CDH 4.5.0中，只有当以下函数（来自 Storage.java ，第418行）返回 NON_EXISTENT 时才会发生该错误 . 在每种情况下都会显示一条警告，其中包含更多详细信息，从 org.apache.hadoop.hdfs.server.common.Storage 查找日志行 .

  简而言之，似乎名称节点认为它不存在，不是目录，不可写或以其他方式抛出 SecurityException .

  /**
   * Check consistency of the storage directory
   * 
   * @param startOpt a startup option.
   *  
   * @return state {@link StorageState} of the storage directory 
   * @throws InconsistentFSStateException if directory state is not 
   * consistent and cannot be recovered.
   * @throws IOException
   */
  public StorageState analyzeStorage(StartupOption startOpt, Storage storage)
      throws IOException {
    assert root != null : "root is null";
    String rootPath = root.getCanonicalPath();
    try { // check that storage exists
      if (!root.exists()) {
        // storage directory does not exist
        if (startOpt != StartupOption.FORMAT) {
          LOG.warn("Storage directory " + rootPath + " does not exist");
          return StorageState.NON_EXISTENT;
        }
        LOG.info(rootPath + " does not exist. Creating ...");
        if (!root.mkdirs())
          throw new IOException("Cannot create directory " + rootPath);
      }
      // or is inaccessible
      if (!root.isDirectory()) {
        LOG.warn(rootPath + "is not a directory");
        return StorageState.NON_EXISTENT;
      }
      if (!root.canWrite()) {
        LOG.warn("Cannot access storage directory " + rootPath);
        return StorageState.NON_EXISTENT;
      }
    } catch(SecurityException ex) {
      LOG.warn("Cannot access storage directory " + rootPath, ex);
      return StorageState.NON_EXISTENT;
    }
  
    this.lock(); // lock storage if it exists
  
    if (startOpt == HdfsServerConstants.StartupOption.FORMAT)
      return StorageState.NOT_FORMATTED;
  
    if (startOpt != HdfsServerConstants.StartupOption.IMPORT) {
      storage.checkOldLayoutStorage(this);
    }
  
    // check whether current directory is valid
    File versionFile = getVersionFile();
    boolean hasCurrent = versionFile.exists();
  
    // check which directories exist
    boolean hasPrevious = getPreviousDir().exists();
    boolean hasPreviousTmp = getPreviousTmp().exists();
    boolean hasRemovedTmp = getRemovedTmp().exists();
    boolean hasFinalizedTmp = getFinalizedTmp().exists();
    boolean hasCheckpointTmp = getLastCheckpointTmp().exists();
  
    if (!(hasPreviousTmp || hasRemovedTmp
        || hasFinalizedTmp || hasCheckpointTmp)) {
      // no temp dirs - no recovery
      if (hasCurrent)
        return StorageState.NORMAL;
      if (hasPrevious)
        throw new InconsistentFSStateException(root,
                            "version file in current directory is missing.");
      return StorageState.NOT_FORMATTED;
    }
  
    if ((hasPreviousTmp?1:0) + (hasRemovedTmp?1:0)
        + (hasFinalizedTmp?1:0) + (hasCheckpointTmp?1:0) > 1)
      // more than one temp dirs
      throw new InconsistentFSStateException(root,
                                             "too many temporary directories.");
  
    // # of temp dirs == 1 should either recover or complete a transition
    if (hasCheckpointTmp) {
      return hasCurrent ? StorageState.COMPLETE_CHECKPOINT
                        : StorageState.RECOVER_CHECKPOINT;
    }
  
    if (hasFinalizedTmp) {
      if (hasPrevious)
        throw new InconsistentFSStateException(root,
                                               STORAGE_DIR_PREVIOUS + " and " + STORAGE_TMP_FINALIZED
                                               + "cannot exist together.");
      return StorageState.COMPLETE_FINALIZE;
    }
  
    if (hasPreviousTmp) {
      if (hasPrevious)
        throw new InconsistentFSStateException(root,
                                               STORAGE_DIR_PREVIOUS + " and " + STORAGE_TMP_PREVIOUS
                                               + " cannot exist together.");
      if (hasCurrent)
        return StorageState.COMPLETE_UPGRADE;
      return StorageState.RECOVER_UPGRADE;
    }
  
    assert hasRemovedTmp : "hasRemovedTmp must be true";
    if (!(hasCurrent ^ hasPrevious))
      throw new InconsistentFSStateException(root,
                                             "one and only one directory " + STORAGE_DIR_CURRENT 
                                             + " or " + STORAGE_DIR_PREVIOUS 
                                             + " must be present when " + STORAGE_TMP_REMOVED
                                             + " exists.");
    if (hasCurrent)
      return StorageState.COMPLETE_ROLLBACK;
    return StorageState.RECOVER_ROLLBACK;
  }
  

  回复于 2024-04-20T05:30:28+08:00