emo/thin-provisioning-tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,580 Commits 2 Branches 0 Tags 3.7 MiB
31abc468be
Commit Graph

396 Commits

Author SHA1 Message Date
Ming-Hung Tsai
8fa59553c4 [thin_generate_mappings] Introduce the mapping generation tool 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
621b466333 [all] Eliminate duplicated sector_t typedefs 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
5260a87c0b [base] Move math_utils to namespace base 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
ef8e94b22c [thin_generate_metadata] Support setting transaction id and metadata snapshots 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
524de134a4 [thin_pool] Support creating metadadata snapshot 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
e33c60c7e2 [thin_pool] Add close_thin() 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
89409927b4 [thin_generate_metadata] Support device deletion 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
a5614f7db2 [thin_pool] Complete device deletion 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
4e1ec9326b [thin_pool] Tidying 
- Remove unused headers
- Use predefined types
 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
598696bc32 [thin_pool] Implement synchronous mapping insertion and removal 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
383ae78871 [thin_pool] Determine shared blocks while looking up mappings 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
a842b4ecfd [thin_generate_metadata] Make use of thin_pool.cc 2020-06-19 18:07:55 +08:00
Ming-Hung Tsai
66b519db83 [thin_pool] Fix snapshot creation 2020-06-19 17:22:19 +08:00
Ming-Hung Tsai
2cce8a51a7 [thin_pool] Fix mapped block time 2020-06-19 17:22:19 +08:00
Ming-Hung Tsai
7f7ba950ef [thin_pool] Complete device creation 
- Set up the device details
- Cache device details of opened devices
- Update changed devices at once while committing the pool
 2020-06-19 17:22:19 +08:00
Ming-Hung Tsai
77adb08c3f [thin_pool] Replace the member pointer by references 2020-06-17 19:09:24 +08:00
Ming-Hung Tsai
0b058a2850 [thin_pool] Encapsulate metadata into the pool class 2020-06-17 19:09:24 +08:00
Ming-Hung Tsai
5810846409 [thin] Add more constructors to device_details 2020-06-17 19:09:24 +08:00
Joe Thornber
0b5afc6cb0 [thin_check] fix bugs in thin_check -m 2020-06-12 13:41:47 +01:00
Joe Thornber
86704deacb [thin_check] Add support for --metadata-snap. 
Not tested yet.
 2020-06-09 14:25:19 +01:00
Joe Thornber
01a7462ece Merge branch 'master' into 2020-05-27-check-data-space-map 2020-06-09 13:39:46 +01:00
Joe Thornber
61de3f9287 [thin_metadata_pack/unpack] Replace C++ implementation with a Rust one. 
The Rust implementation is multithreaded, performs better in general and
does custom compression of btree nodes to achieve much better compression
ratios.  unpack also checksums expanded metadata to validate it.

Format version has jumped to 3, no backwards compatibility, but I think
that's ok since we never made a release that contained the C++ version
of these tools.

Benchmarks
==========

On an 8 core, 16 hyperthread machine.

metadata 1G, full:

      Pack size    pack time     unpack time
------------------------------------------------------
C++      193M        50.3s          6.9s (no verify)
Rust      70M         1.4s          1.8s (verify)

metadata 16G, sparse:

       Pack size    pack time     unpack time
------------------------------------------------------
C++      21M          68s           1s   (no verify)
Rust      4M           8.6s         0.5s (verify)
 2020-06-09 09:15:00 +01:00
Joe Thornber
c48851e747 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-06-01 11:14:24 +01:00
Joe Thornber
9f2b747ffa [thin_metadata_pack] Prefetch blocks. 
Doubles performance
 2020-06-01 11:13:51 +01:00
Joe Thornber
16a10d2554 [thin_check] Remove the metadata_checker base class. 
It's really only a single method.
 2020-05-28 14:43:03 +01:00
Joe Thornber
c94f560be8 [thin_check] Check the ref counts in the data space map. 
Hadn't realised this was being done.
 2020-05-28 14:35:51 +01:00
Ming-Hung Tsai
0f2918e989 [thin_generate_metadata] Implement format_metadata and create_thin 2020-05-28 17:30:15 +08:00
Ming-Hung Tsai
0c6d132774 [thin_generate_metadata] Preparation for revising the features 2020-05-28 15:09:20 +08:00
Joe Thornber
e92ae10408 [thin_metadata_pack/unpack] Add version field to header. 2020-05-26 10:59:13 +01:00
Joe Thornber
a934a34d22 [thin_metadata_pack/unpack] Move the file magic nr before the compressed stream. 
Makes it easier to spot whether it's a pack file or not.
 2020-05-26 10:53:14 +01:00
Ming-Hung Tsai
c0106fc0b3 [thin_metadata_pack] Improve block scanning performance 2020-05-23 20:32:09 +08:00
Joe Thornber
0e1700fbe9 [thin_metadata_pack] First pass at pack/unpack 2020-05-22 14:11:48 +01:00
Joe Thornber
b7d20bce48 [dev-tools] Update dev tools to use non-template version of block_manager. 2020-05-01 14:11:33 +01:00
Joe Thornber
4313469475 [all] Switch from boost::shared_ptr -> std::shared_ptr. 
Shared_ptr has moved into the standard library since these tools were
first written.
 2020-04-30 15:02:43 +01:00
Joe Thornber
e801cc607b [block_manager] Hard code block size to 4k. 
We're never going to use anything other than 4k, and by hard coding it we
avoid making block_manager a template.
 2020-04-30 14:30:01 +01:00
Joe Thornber
ad79b627a4 Merge branch 'master' of github.com:jthornber/thin-provisioning-tools 2020-04-08 12:30:26 +01:00
Joe Thornber
0098803345 [thinp_*] Make mtree_ref_counter derive from ref_counter. 
Slight performance hit, but I'd rather everything was derived
from ref_counter.

Also implement it!  Not sure why this wasn't done before.
 2020-04-08 12:24:37 +01:00
Joe Thornber
0ccee1759e [various] Squash a couple of clang warning wrt friend declarations. 2020-04-08 12:18:24 +01:00
Joe Thornber
5e347dddbf [various] remove dead code. 
Squashes warnings from clang++
 2020-04-08 12:12:59 +01:00
Arusekk
3ed3925370 Fix compilation with uClibc 
Closes #12
 2020-03-25 15:58:02 +01:00
Ming-Hung Tsai
955e11bc28 [block-cache] fix potential file descriptor leak 
Encapsulate file descriptor into an object, to ensure that an fd will be
closed properly while exception raised, e.g., the block_cache throws
exception during the block_manager's construction.
 2020-02-23 15:41:16 +08:00
Ming-Hung Tsai
c85ea5ef76 [thin_check] factor out metadata_checker 2020-02-17 11:42:24 +00:00
Joe Thornber
5f2c3bed69 Add some ftests, and fixup whitespace from Nikhil's work 2019-10-28 11:52:21 +00:00
Nikhil Kshirsagar
91b4714e38 Corrupt superblock repair 2019-10-25 15:02:56 +05:30
Joe Thornber
0fc7529c01 [cache_repair, thin_repair] fix bug introduced in recent patch 
I hadn't realised that check_file_exists() also checked that it was
a regular file, which we don't want for the couple of uses I recently
added.

This patch adds an optional arg must_be_regular_file, and defaults
it to true, preserving the original behaviour.  The recent additions
have this set to false.
 2019-10-14 09:21:38 +01:00
Joe Thornber
1dd7b454bb [thin_repair, cache_repair] Check input file exists earlier 
and zero superblock if we fail part way through a repair.

bz1499781
 2019-10-08 14:34:24 +01:00
Joe Thornber
733c7df798 [thin_repair/thin_dump] Fix some more spurious error messages when doing repair. 
Repair was falling back to non-repair behaviour if it thought the roots
were ok.  Now if --repair is specified the same dumping code is always
executed.
 2019-06-03 14:03:24 +01:00
Joe Thornber
e45a211902 [thin/metadata_dumper] Change a couple of confusing class names. 
metadata_emitter is actually a visitor that passes on it's data
to an encapsulated emitter object.

metadata_emitter -> metadata_emit_visitor
metadata_tree_emitter -> metadata_tree_emit_visitor
 2019-06-03 11:07:32 +01:00
Joe Thornber
0624ded67f [thin_repair, thin_dump] Don't look for better roots if the ones in the sb are ok. 
Calculating the most recent trees is inexact, so trees other than the ones
in the superblock, which presumably are the most recent, could be chosen.
 2019-05-27 16:43:32 +01:00
Joe Thornber
5e6ddbf2da [thin_dump, thin_repair] Fix bad metadata_damage exception 
The first pass of the repair process scans all metadata working out the
largest orphan btrees.  This scan doesn't use as much validation as
the btree_walk function which subsequently gets called.

This patch catches any exceptions thrown by the btree walk function
and removes that btree from consideration.
 2019-05-27 16:06:59 +01:00