[TRACE] org.apache.hadoop.hbase.util.Merge

org.apache.hadoop.hbase.util.Merge,
是在hbase 0.94版本中用來實現Region合併的工具,
使用方式如下:

$ ./bin/hbase org.apache.hadoop.hbase.util.Merge
Usage: bin/hbase merge <table-name> <region-1> <region-2>

在程式的一開始,
org.apache.hadoop.hbase.util.Merge將會先檢查:
輸入的變數, HDFS的設定, hbase是否關閉:

      if (parseArgs(args) != 0) {
        return -1;
      }
  
      // Verify file system is up.
      FileSystem fs = FileSystem.get(getConf());              // get DFS handle
      LOG.info("Verifying that file system is available...");
      try {
        FSUtils.checkFileSystemAvailable(fs);
      } catch (IOException e) {
        LOG.fatal("File system is not available", e);
        return -1;
      }
  
      // Verify HBase is down
      LOG.info("Verifying that HBase is not running...");
      try {
        HBaseAdmin.checkHBaseAvailable(getConf());
        LOG.fatal("HBase cluster must be off-line, and is not. Aborting.");
       return -1;
     } catch (ZooKeeperConnectionException zkce) {
       // If no zk, presume no master.
     } catch (MasterNotRunningException e) {
       // Expected. Ignore.
     }

確認輸入變數正確(包括table name, region-1, region-2),
HDFS開啟, 且hbase關閉時, 才開始進行merging (透過mergeTwoRegions()).
在mergeTwoRegions()內, 一開始將先去找尋存取.META.表格的RegionServer,
並嘗試根據region-1和region-2的名字, 查詢Region資料, 存入info1和info2中,
確定region-1和region-2的資訊後, 進入merge()程序, 產生合併後的內容,
加入.META.表格當中, mergeTwoRegions()的程式碼如下:

   /*
    * Merges two regions from a user table.
    */
   private void mergeTwoRegions() throws IOException {
     LOG.info("Merging regions " + Bytes.toStringBinary(this.region1) + " and " +
         Bytes.toStringBinary(this.region2) + " in table " + this.tableName);
     HRegion meta = this.utils.getMetaRegion();
     Get get = new Get(region1);
     get.addColumn(HConstants.CATALOG_FAMILY, HConstants.REGIONINFO_QUALIFIER);
     Result result1 =  meta.get(get);
     Preconditions.checkState(!result1.isEmpty(),
         "First region cells can not be null");
     HRegionInfo info1 = HRegionInfo.getHRegionInfo(result1);
     if (info1 == null) {
       throw new NullPointerException("info1 is null using key " +
           Bytes.toStringBinary(region1) + " in " + meta);
     }
     get = new Get(region2);
     get.addColumn(HConstants.CATALOG_FAMILY, HConstants.REGIONINFO_QUALIFIER);
     Result result2 =  meta.get(get);
     Preconditions.checkState(!result2.isEmpty(),
         "Second region cells can not be null");
     HRegionInfo info2 = HRegionInfo.getHRegionInfo(result2);
     if (info2 == null) {
       throw new NullPointerException("info2 is null using key " + meta);
     }
     TableDescriptor htd = FSTableDescriptors.getTableDescriptorFromFs(FileSystem.get(getConf()),
       this.rootdir, this.tableName);
     HRegion merged = merge(htd.getHTableDescriptor(), meta, info1, info2);
 
     LOG.info("Adding " + merged.getRegionInfo() + " to " +
         meta.getRegionInfo());
 
     HRegion.addRegionToMETA(meta, merged);
     merged.close();
   }

在merge()中做了兩件事情,
第一件事情是透過HRegion.merge(r1, r2)將region-1和region-2合併,
第二件事情則是將.META.表中region-1和region-2的資訊刪除,
這樣一來, 在mergeTwoRegions()就只需要把新的資訊加入.META.表中,
merge()的原始碼如下:

   /*
    * Actually merge two regions and update their info in the meta region(s)
    * Returns HRegion object for newly merged region
    */
   private HRegion merge(final HTableDescriptor htd, HRegion meta,
                         HRegionInfo info1, HRegionInfo info2)
   throws IOException {
     if (info1 == null) {
       throw new IOException("Could not find " + Bytes.toStringBinary(region1) + " in " +
           Bytes.toStringBinary(meta.getRegionName()));
     }
     if (info2 == null) {
       throw new IOException("Could not find " + Bytes.toStringBinary(region2) + " in " +
           Bytes.toStringBinary(meta.getRegionName()));
     }
     HRegion merged = null;
     HRegion r1 = HRegion.openHRegion(info1, htd, utils.getLog(info1), getConf());
     try {
       HRegion r2 = HRegion.openHRegion(info2, htd, utils.getLog(info2), getConf());
       try {
         merged = HRegion.merge(r1, r2);
       } finally {
         if (!r2.isClosed()) {
           r2.close();
         }
       }
     } finally {
       if (!r1.isClosed()) {
         r1.close();
       }
     }
 
     // Remove the old regions from meta.
     // HRegion.merge has already deleted their files
 
     removeRegionFromMeta(meta, info1);
     removeRegionFromMeta(meta, info2);
 
     this.mergeInfo = merged.getRegionInfo();
     return merged;
   }

在hbase 0.94的merge工具中, 提供了offline合併Region的工具,
然而, trace程式的結果, 並能夠回答為什麼有offline的限制,
(為什麼不直接online修改.META.表格就好了?)
接下來, 將繼續trace HRegion.merge()這一隻程式,
觀察Region合併的過程.

參考資料:
https://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/util/Merge.html
https://hbase.apache.org/xref/org/apache/hadoop/hbase/util/Merge.html
http://blog.csdn.net/macyang/article/details/6624482