Go to file
2021-10-21 16:18:14 +08:00
autoconf [build] Update config.guess and config.sub to version 2021-05-24 2021-06-26 13:38:56 +08:00
base [file_utils] Check the file type to prevent unexpected writes by thin_repair 2021-07-09 01:22:22 +08:00
bin [manpages] Make the footer backward compatible 2021-06-28 08:19:30 +08:00
block-cache [all] Remove unreachable code 2021-06-03 18:54:26 +08:00
caching [cache_repair] Avoid touching the output file by checking input file earlier 2021-09-22 17:07:56 +08:00
contrib [build] Fix customized emitter linkage 2021-06-02 00:51:39 +08:00
dbg-lib Add missing header. 2021-03-04 11:19:04 +00:00
doc [thin_journal_check] Checks journal of block manager activity. 2018-09-24 14:51:46 +01:00
era [btree] Fix ref-counting on overwritten values 2021-09-22 17:05:59 +08:00
features [functional-tests] era_check tests 2017-08-30 14:30:11 +01:00
ft-lib [ft-lib/bcache] rename raise() -> raise_() 2019-06-05 15:02:05 +01:00
functional-tests [tests] Port the remaining cache tests 2021-09-22 17:07:56 +08:00
lib gmock stub 2013-03-22 10:55:55 +00:00
man8 [thin_check] rename --fix-metadata-leaks to --auto-repair 2020-08-12 10:45:20 +01:00
persistent-data [btree] Fix ref-counting on overwritten values 2021-09-22 17:05:59 +08:00
src [era_check (rust)] First code drop 2021-10-21 16:18:14 +08:00
tests [all] Apply cargo fmt, and fix clippy warnings 2021-10-20 20:23:27 +08:00
thin-provisioning [thin] Fix typo 2021-10-20 20:23:27 +08:00
ui [thin_show_metadata] ncurses render 2016-02-26 12:50:17 +00:00
unit-tests [build] Allow running tests without the --enable-testing configuration 2021-06-03 18:54:26 +08:00
xml_metadata add some example metadata dumps 2011-11-16 12:17:23 +00:00
.gitignore [build] Update .gitignore 2021-06-26 00:08:58 +08:00
.pre-commit-config.yaml Add pre-commit hooks 2021-05-11 23:18:43 +08:00
Cargo.lock Merge rust tools into a single pdata_tools exe 2021-10-11 12:07:26 +01:00
Cargo.toml Merge rust tools into a single pdata_tools exe 2021-10-11 12:07:26 +01:00
CHANGES [build] Update CHANGES file 2020-08-12 10:35:03 +01:00
configure.ac version.rs includes VERSION directly. 2021-02-04 10:00:32 +00:00
COPYING Add copyright 2011-12-06 13:43:44 +00:00
get-gmock.sh update get-gmock script. 2016-06-07 11:15:35 +01:00
main.cc [base] introduce a command type that gets registered with the app 2016-01-08 12:51:52 +00:00
Makefile.in [build] Create the destdir for solely installing rust tools 2021-06-28 08:19:30 +08:00
mk_release make mk_release executable 2012-03-05 11:46:20 +00:00
README.md [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. 2020-06-09 09:15:00 +01:00
rustfmt.toml [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. 2020-06-09 09:15:00 +01:00
TODO.org Add a TODO file 2013-04-23 11:18:21 +01:00
VERSION bump version to 0.9.0 2020-08-12 10:46:26 +01:00
version.h.in version.h.in 2011-12-15 14:58:25 +01:00

Introduction

A suite of tools for manipulating the metadata of the dm-thin, dm-cache and dm-era device-mapper targets.

Requirements

A C++ compiler that supports the c++11 standard (eg, g++). The Boost C++ library. The expat xml parser library (version 1). The libaio library (note this is not the same as the aio library that you get by linking -lrt) make, autoconf etc.

A couple of non-essential tools are written in rust, and will require cargo and rustcc to be installed:

thin_metadata_pack
thin_metadata_unpack

There are more requirements for testing, detailed below.

Building

autoconf
./configure
make
sudo make install

Building Rust tools

sudo make install-rust-tools

Quick examples

These tools introduce an xml format for the metadata. This is useful for making backups, or allowing scripting languages to generate or manipulate metadata. A Ruby library for this available; thinp_xml.

To convert the binary metadata format that the kernel uses to this xml format use thin_dump.

thin_dump --format xml /dev/mapper/my_thinp_metadata

To convert xml back to the binary form use thin_restore.

thin_restore -i my_xml -o /dev/mapper/my_thinp_metadata

You should periodically check the health of your metadata, much as you fsck a filesystem. Your volume manager (eg, LVM2) should be doing this for you behind the scenes.

thin_check /dev/mapper/my_thinp_metadata

Checking all the mappings can take some time, you can omit this part of the check if you wish.

thin_check --skip-mappings /dev/mapper/my_thinp_metadata

If your metadata has become corrupt for some reason (device failure, user error, kernel bug), thin_check will tell you what the effects of the corruption are (eg, which thin devices are effected, which mappings).

There are two ways to repair metadata. The simplest is via the thin_repair tool.

thin_repair -i /dev/mapper/broken_metadata_dev -o /dev/mapper/new_metadata_dev

This is a non-destructive operation that writes corrected metadata to a new metadata device.

Alternatively you can go via the xml format (perhaps you want to inspect the repaired metadata before restoring).

thin_dump --repair /dev/mapper/my_metadata > repaired.xml
thin_restore -i repaired.xml -o /dev/mapper/my_metadata

Development

Autoconf

If you've got the source from github you'll need to create the configure script with autoconf. I do this by running:

autoreconf

Enable tests

You will need to enable tests when you configure.

./configure --enable-testing

Unit tests

Unit tests are implemented using the google mock framework. This is a source library that you will have to download. A script is provided to do this for you.

./get-gmock.sh

All tests can be run via:

make unit-test

Alternatively you may want to run a subset of the tests:

make unit-tests/unit_tests
unit-tests/unit_tests --gtest_filter=BtreeTests.*

Functional tests

A bunch of high level tests are implemented in the functional-tests directory. These tests are written in Scheme. To run them you'll need to install chezscheme (http://www.scheme.com/). There is no longer a dependency on the ThunderChez library.

Make sure the tools that you wish to test are in your PATH.

Then,

cd funtional-tests
./run-tests run

Other command are help and list.

The test framework places temporary files under ./test-output/. By default the tests tidy up after themselves, just leaving a log file for each test. You can turn this off by using the --disable-unlink flag if you want all the artifacts left.

Dump Metadata

To dump the metadata of a live thin pool, you must first create a snapshot of the metadata:

$ dmsetup message vg001-mythinpool-tpool 0 reserve_metadata_snap

Extract the metadata:

$ sudo bin/thin_dump -m /dev/mapper/vg001-mythinpool_tmeta
<superblock uuid="" time="1" transaction="2" data_block_size="128"nr_data_blocks="0">
    <device dev_id="1" mapped_blocks="1" transaction="0" creation_time="0" snap_time="1">
        <single_mapping origin_block="0" data_block="0" time="0"/>
    </device>
    <device dev_id="2" mapped_blocks="1" transaction="1" creation_time="1" snap_time="1">
        <single_mapping origin_block="0" data_block="0" time="0"/>
    </device>
</superblock>

Finally, release the root:

$ dmsetup message vg001-mythinpool-tpool 0 release_metadata_snap