diff options
Diffstat (limited to 'fs/cramfs/README')
| -rw-r--r-- | fs/cramfs/README | 197 |
1 files changed, 197 insertions, 0 deletions
diff --git a/fs/cramfs/README b/fs/cramfs/README new file mode 100644 index 0000000000..778df5c4d7 --- /dev/null +++ b/fs/cramfs/README @@ -0,0 +1,197 @@ +Notes on Filesystem Layout +-------------------------- + +These notes describe what mkcramfs generates. Kernel requirements are +a bit looser, e.g. it doesn't care if the <file_data> items are +swapped around (though it does care that directory entries (inodes) in +a given directory are contiguous, as this is used by readdir). + +All data is currently in host-endian format; neither mkcramfs nor the +kernel ever do swabbing. (See section `Block Size' below.) + +<filesystem>: + <superblock> + <directory_structure> + <data> + +<superblock>: struct cramfs_super (see cramfs_fs.h). + +<directory_structure>: + For each file: + struct cramfs_inode (see cramfs_fs.h). + Filename. Not generally null-terminated, but it is + null-padded to a multiple of 4 bytes. + +The order of inode traversal is described as "width-first" (not to be +confused with breadth-first); i.e. like depth-first but listing all of +a directory's entries before recursing down its subdirectories: the +same order as `ls -AUR' (but without the /^\..*:$/ directory header +lines); put another way, the same order as `find -type d -exec +ls -AU1 {} \;'. + +Beginning in 2.4.7, directory entries are sorted. This optimization +allows cramfs_lookup to return more quickly when a filename does not +exist, speeds up user-space directory sorts, etc. + +<data>: + One <file_data> for each file that's either a symlink or a + regular file of non-zero st_size. + +<file_data>: + nblocks * <block_pointer> + (where nblocks = (st_size - 1) / blksize + 1) + nblocks * <block> + padding to multiple of 4 bytes + +The i'th <block_pointer> for a file stores the byte offset of the +*end* of the i'th <block> (i.e. one past the last byte, which is the +same as the start of the (i+1)'th <block> if there is one). The first +<block> immediately follows the last <block_pointer> for the file. +<block_pointer>s are each 32 bits long. + +When the CRAMFS_FLAG_EXT_BLOCK_POINTERS capability bit is set, each +<block_pointer>'s top bits may contain special flags as follows: + +CRAMFS_BLK_FLAG_UNCOMPRESSED (bit 31): + The block data is not compressed and should be copied verbatim. + +CRAMFS_BLK_FLAG_DIRECT_PTR (bit 30): + The <block_pointer> stores the actual block start offset and not + its end, shifted right by 2 bits. The block must therefore be + aligned to a 4-byte boundary. The block size is either blksize + if CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified, otherwise + the compressed data length is included in the first 2 bytes of + the block data. This is used to allow discontiguous data layout + and specific data block alignments e.g. for XIP applications. + + +The order of <file_data>'s is a depth-first descent of the directory +tree, i.e. the same order as `find -size +0 \( -type f -o -type l \) +-print'. + + +<block>: The i'th <block> is the output of zlib's compress function +applied to the i'th blksize-sized chunk of the input data if the +corresponding CRAMFS_BLK_FLAG_UNCOMPRESSED <block_ptr> bit is not set, +otherwise it is the input data directly. +(For the last <block> of the file, the input may of course be smaller.) +Each <block> may be a different size. (See <block_pointer> above.) + +<block>s are merely byte-aligned, not generally u32-aligned. + +When CRAMFS_BLK_FLAG_DIRECT_PTR is specified then the corresponding +<block> may be located anywhere and not necessarily contiguous with +the previous/next blocks. In that case it is minimally u32-aligned. +If CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified then the size is always +blksize except for the last block which is limited by the file length. +If CRAMFS_BLK_FLAG_DIRECT_PTR is set and CRAMFS_BLK_FLAG_UNCOMPRESSED +is not set then the first 2 bytes of the block contains the size of the +remaining block data as this cannot be determined from the placement of +logically adjacent blocks. + + +Holes +----- + +This kernel supports cramfs holes (i.e. [efficient representation of] +blocks in uncompressed data consisting entirely of NUL bytes), but by +default mkcramfs doesn't test for & create holes, since cramfs in +kernels up to at least 2.3.39 didn't support holes. Run mkcramfs +with -z if you want it to create files that can have holes in them. + + +Tools +----- + +The cramfs user-space tools, including mkcramfs and cramfsck, are +located at <http://sourceforge.net/projects/cramfs/>. + + +Future Development +================== + +Block Size +---------- + +(Block size in cramfs refers to the size of input data that is +compressed at a time. It's intended to be somewhere around +PAGE_SIZE for cramfs_read_folio's convenience.) + +The superblock ought to indicate the block size that the fs was +written for, since comments in <linux/pagemap.h> indicate that +PAGE_SIZE may grow in future (if I interpret the comment +correctly). + +Currently, mkcramfs #define's PAGE_SIZE as 4096 and uses that +for blksize, whereas Linux-2.3.39 uses its PAGE_SIZE, which in +turn is defined as PAGE_SIZE (which can be as large as 32KB on arm). +This discrepancy is a bug, though it's not clear which should be +changed. + +One option is to change mkcramfs to take its PAGE_SIZE from +<asm/page.h>. Personally I don't like this option, but it does +require the least amount of change: just change `#define +PAGE_SIZE (4096)' to `#include <asm/page.h>'. The disadvantage +is that the generated cramfs cannot always be shared between different +kernels, not even necessarily kernels of the same architecture if +PAGE_SIZE is subject to change between kernel versions +(currently possible with arm and ia64). + +The remaining options try to make cramfs more sharable. + +One part of that is addressing endianness. The two options here are +`always use little-endian' (like ext2fs) or `writer chooses +endianness; kernel adapts at runtime'. Little-endian wins because of +code simplicity and little CPU overhead even on big-endian machines. + +The cost of swabbing is changing the code to use the le32_to_cpu +etc. macros as used by ext2fs. We don't need to swab the compressed +data, only the superblock, inodes and block pointers. + + +The other part of making cramfs more sharable is choosing a block +size. The options are: + + 1. Always 4096 bytes. + + 2. Writer chooses blocksize; kernel adapts but rejects blocksize > + PAGE_SIZE. + + 3. Writer chooses blocksize; kernel adapts even to blocksize > + PAGE_SIZE. + +It's easy enough to change the kernel to use a smaller value than +PAGE_SIZE: just make cramfs_read_folio read multiple blocks. + +The cost of option 1 is that kernels with a larger PAGE_SIZE +value don't get as good compression as they can. + +The cost of option 2 relative to option 1 is that the code uses +variables instead of #define'd constants. The gain is that people +with kernels having larger PAGE_SIZE can make use of that if +they don't mind their cramfs being inaccessible to kernels with +smaller PAGE_SIZE values. + +Option 3 is easy to implement if we don't mind being CPU-inefficient: +e.g. get read_folio to decompress to a buffer of size MAX_BLKSIZE (which +must be no larger than 32KB) and discard what it doesn't need. +Getting read_folio to read into all the covered pages is harder. + +The main advantage of option 3 over 1, 2, is better compression. The +cost is greater complexity. Probably not worth it, but I hope someone +will disagree. (If it is implemented, then I'll re-use that code in +e2compr.) + + +Another cost of 2 and 3 over 1 is making mkcramfs use a different +block size, but that just means adding and parsing a -b option. + + +Inode Size +---------- + +Given that cramfs will probably be used for CDs etc. as well as just +silicon ROMs, it might make sense to expand the inode a little from +its current 12 bytes. Inodes other than the root inode are followed +by filename, so the expansion doesn't even have to be a multiple of 4 +bytes. |