Skip to content

[RFC]: Unload data from in-memory column store(Tianmu_rapid) #1290

@RingsC

Description

@RingsC

1: Overview

This is a submodule of version 2.0, which is used to define the behavior of unload statement.
As the reverse operation of load operation. The unload operation is to unload the loaded data from in-memory column store. Why do we need this operation? The main reseaon is that we have limited main memory. Only part of main memory can be used as IMCS's memory pool.

When it is flushing out the data, there is one thing should be done. That is to check whether some session or queries are using these data. If the data is using by other session or queries. The is operation should not be executed successfully.

2: Implementation

As the opposite operation of secondary_load, the secondary_unload is introudced into in-memory column store to off load all the data from in-memory. The data can be spilled out to disk or discards these data. Which type used is defined by some sytem variables. a var is used to define whether dump to disk or not. If dumps to disk, a variable is used to define where the data is located.

when the data has been unloaded, the meta information of these data also should be unloaded from in-memory column store.

1: User to issue an unload statement, alter table xxx secondary_unload.
2: The sql server will do some checks then invoke Sql_cmd_secondary_load_unload::execute to proceed this statement.
Sql_cmd_secondary_load_unload::mysql_secondary_load_or_unload.
3:The handler handles the unload operation via ha_tianmu_secondary::unload_table.

As the secondary_load did, A subclass of handler was defined, which inherits from handler class. It acts the interface of Tianmu_rapid engine. The definiton of ha_tianmu_secondary is listed as following:

class ha_tianmu_secondary : public handler {
 public:
  ha_tianmu_secondary(handlerton *hton, TABLE_SHARE *table_share);

 private:
  int create(const char *, TABLE *, HA_CREATE_INFO *, dd::Table *) override;

  int open(const char *name, int mode, unsigned int test_if_locked,
           const dd::Table *table_def) override;

  int close() override { return 0; }

  int rnd_init(bool) override { return 0; }

  int rnd_next(unsigned char *) override { return HA_ERR_END_OF_FILE; }

  int rnd_pos(unsigned char *, unsigned char *) override {
    return HA_ERR_WRONG_COMMAND;
  }

  int info(unsigned int) override;

  ha_rows records_in_range(unsigned int index, key_range *min_key,
                           key_range *max_key) override;

  void position(const unsigned char *) override {}

  unsigned long index_flags(unsigned int, unsigned int, bool) const override;

  THR_LOCK_DATA **store_lock(THD *thd, THR_LOCK_DATA **to,
                             thr_lock_type lock_type) override;

  Table_flags table_flags() const override;

  const char *table_type() const override { return "TIANMU_RAPID"; }

  int load_table(THD *thd, const TABLE &table) override;

  int unload_table(THD *thd, const char *db_name, const char *table_name,
                   bool error_if_not_loaded) override;

  THR_LOCK_DATA m_lock;

 private:
  bool primary_inited_;
  ENGINE_TYPE ent_type_;
};

ha_tianmu_secondary::unload_table is used to execute alter table xxx secondary_unload statement. the steps of unloading the data of IMCS is as following: (Taking a chunk as an example)

1: Release the data in a tile. and update the meta data.
2: Release the tiles in a bucket, and update the meta data.
3: Release the buckets in a chunk, and update the meta data.
4: Release the chunks and update the meta data.

3: Unit test

w.i.p

4: Exception handling

Pls refer to issue #1289

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions