Chapter 51. Getting started with Yaffs

51.1. Licensing considerations

Before you can use Yaffs, you must accept its license. Yaffs is not covered by the standard eCos license. You will be reminded of your license to use Yaffs when you install it.

Most users will only have access to Yaffs under the GNU GPL. This costs nothing to license. However this usually means that if you ever distribute your application, you must do so under the GPL. This requires you to publish or otherwise make all of your application, eCos and everything else you link with it available as source code. For full details refer to the text of the GPL (v2).

If you cannot accept the restrictions and obligations of the GPL, Yaffs for eCos is available under a proprietary license, for a fee. Details are available on request from eCosCentric or Aleph One.

51.2. Installation

Yaffs is included within the standard eCosPro distribution and no additional installation of the GPL-licensed version is required. The proprietary licensed version is supplied as an EPK (eCos Package) file which may be installed alongside the GPL-licensed version. The remainder of this section deals with the installation of the proprietary licensed version.

51.2.1. Installation via the eCos Configuration Tool

  1. Open up the eCos Configuration Tool.
  2. Open up the Administration dialog, from the Tools menu.
  3. Press the Add button.
  4. A file browser windows opens. Navigate to the Yaffs EPK file.
  5. The License screen shows. You must accept the license in order to install the package. Press Yes if you do.

51.2.2. Installing from the command-line

Advanced users may alternatively use the ecosadmin.tcl tool from the command line. You will be prompted to accept the license at the appropriate time during the procedure.

tclsh $ECOS_REPOSITORY/ecosadmin.tcl add yaffs-v1_2_3.epk

51.3. Configuration and Building

After installing the EPK, Yaffs is added to your eCos repository and is configured and built in the normal way.

51.3.1. Package dependencies

To link Yaffs into your application, add CYGPKG_FS_YAFFS to your eCos configuration in the normal way, either using the eCos Configuration Tool (Packages dialog on the Build menu), or the ecosconfig command-line tool.

You will also need to add CYGPKG_IO_NAND and CYGPKG_IO_FILEIO to your configuration if they are not already present. Your platform HAL should supply packages for all NAND device(s) present.

If you started with a smaller template than default, you may also need to add some of the following:

  • CYGPKG_LIBC_STDLIB
  • CYGPKG_LIBC_STRING
  • CYGPKG_MEMALLOC or something else which provides CYGINT_ISO_MALLOC
  • CYGPKG_LIBC_I18N or something else which provides CYGINT_ISO_CTYPE

51.3.2. Configuration options

Yaffs provides a number of package options, including tuning parameters.

CYGPKG_FS_YAFFS_CFLAGS_ADD , CYGPKG_FS_YAFFS_CFLAGS_REMOVE
These settings allow specific build options to be added to or removed from the CFLAGS list when building Yaffs.
CYGSEM_FS_YAFFS_CACHE_SHORT_NAMES
If set, caches files' short names in RAM. This consumes more RAM but improves performance.
CYGPKG_FS_YAFFS_RET_DIRENT_DTYPE
Controls whether Yaffs supports setting the d_type field in a struct dirent. If you don't need this, leave it switched off to save a little code size.
CYGNUM_FS_YAFFS_RESERVED_BLOCKS
The number of blocks to keep in reserve to allow for garbage collection and block failures. The recommended value is 5, but you can tune it for performance. This setting is a global default and may be changed by a mount-time option.
CYGNUM_FS_YAFFS_SHORTOP_CACHES
The number of page cache entries to use. Values of 10 to 20 are recommended; increasing the number consumes more RAM, and 0 disables it altogether. This setting is a global default and may be changed by a mount-time option.
CYGNUM_FS_YAFFS_TRACEMASK
This is a 32-bit bitfield which controls diagnostic output. The bit definitions are found in yportenv.h; they are only useful if you are debugging Yaffs itself.
CYGNUM_FS_YAFFS_TEMP_BUFFERS
Yaffs requires temporary buffers in many places throughout the code. To avoid the overhead of a dynamic malloc every time, a number of buffers are preallocated at mount time. This setting controls how many; should it not prove enough, Yaffs will call malloc on demand as required. The default setting is 6; most users will not need to change it.
CYGSEM_FS_YAFFS_SMALLPAGE_MODE

This option only affects behaviour on so-called "small page" NAND devices (those whose pages are 512 bytes long). Such devices do not have enough space in their Out Of Band area to store a full set of Yaffs metadata tags. There are two ways to work around this:

  • YAFFS2 mode - the default - uses regular tags, but at a price: it steals 16 bytes from the available space per page to store them. This reduces the apparent available size of your filesystem by 1/32!
  • YAFFS1 mode places a smaller tagset in the OOB area, but with a different side-effect: whenever a page is deleted, one byte of the tags area has to be rewritten. Some devices forbid rewrite-without-erase in this way, so it may not be safe for you to use this option. You must refer to the spec sheet for the chip on your board before selecting this option!
CYGSEM_FS_YAFFS_OMIT_YAFFS2_CODE

This causes all YAFFS2 code to be omitted from the build. This option only makes sense when all the devices on which Yaffs is to be used are small-page and operating in YAFFS1 mode.

[Note]Note

There is no corresponding option to omit YAFFS1 code, because that code is only compiled when CYGSEM_FS_YAFFS_SMALLPAGE_MODE is set to YAFFS1.

CYGMEM_FS_YAFFS_REDBOOT_HEAP_REQUIRED

RedBoot carefully controls the amount of memory available for its heap, allocating it from a fixed-size workspace. If Yaffs is being used with RedBoot, the heap space required is likely to go up substantially. The exact amount depends on properties of the filesystem being mounted. Consult the eCosPro Yaffs documentation for more details of Yaffs' memory requirements. This option ensures RedBoot's heap is increased to a more reasonable size, but it has been made an option in order to allow developers to decrease it, if they are sure the filesystem will not require as much memory as this.

[Note]Note

However, note that even this larger amount may not be adequate for some filesystems.

51.4. Using Yaffs

Yaffs appears in the eCos filesystem table. This means that you can mount a filesystem in the standard Unix-like way, then interact with it with calls to open, read, etc.

51.4.1. Mounting a filesystem

Before you can use a filesystem, it must be mounted.

A NAND device is logically organised as one or more partitions, which are usually set up by the relevant platform HAL. You need to tell the mount command which device and partition you wish to access. In the following example, we are mounting partition 0 of the onboard NAND device.

rv = mount("onboard/0", "/nand", "yaffs");
[Note]Note

The device argument to the mount call is a NAND-specific device name, not an entry in /dev. Refer to the documentation for your platform HAL for details of how the NAND device(s) are named, and to the NAND library documentation for details of how partitions are addressed.

You can, if you wish, make the filesystem mounting automatic at static constructor time with the MTAB_ENTRY macro. (Your platform HAL may already do this; check it carefully.)

MTAB_ENTRY(my_nand, "/nand", "yaffs", "onboard/0", "", 0);
[Note]Note

This example is for eCosPro. In eCos, the MTAB_ENTRY macro takes only four arguments.

51.4.1.1. Mount-time options

eCosPro allows various options to be passed to a filesystem at mount time, by combining them with the filesystem argument in a particular format. Yaffs understands the following options:

  • reserved=<int> The number of physical NAND blocks to reserve for garbage collection and block failures (minimum 2). The default is set in CDL as CYGNUM_FS_YAFFS_RESERVED_BLOCKS.
  • caches=<int> The number of page cache entries to use. Values of 10 to 20 are recommended. The default is set in CDL as CYGNUM_FS_YAFFS_SHORTOP_CACHES.
  • skip-checkpoint-read Instructs Yaffs to not attempt to reload the filesystem from a checkpoint, if one exists. In other words, this option forces a full filesystem scan whether or not one is necessary.
  • format Instructs Yaffs to format the filesystem before it mounts it. This deletes its entire contents.

51.4.2. Data flushing

Yaffs operates a cacheing layer in order to save undue wear on the NAND chip if many small writes are performed. Because of this, if you wish to ensure that any data written to a still-open file has been fully flushed, you must make a synchronisation request. This is done with the fsync function, which takes as its argument the file descriptor of the file you wish to synchronise.

51.4.3. Checkpointing

When mounting a filesystem, Yaffs has to scan the NAND chip to recreate its internal state. This can be a slow process, but is made much faster if there is a valid checkpoint.

A checkpoint is written out automatically when you unmount the filesystem. At any other time, you can manually force a checkpoint to be written with one of the following functions:

  • cyg_fs_fssync(mountpoint) synchronises a filesystem (automatically called on umount)
  • sync synchronises all mounted filesystems.
[Note]Note

A checkpoint becomes invalid as soon as there have been any other writes to the filesystem. Finding a good place to sync is necessarily dependent on your application logic.

51.4.4. Limitations

Although Yaffs is a Unix-compatible filesystem, the eCos port does not provide support for the full range of Unix attributes.

  • eCos does not check file or directory permissions; everything it creates is given fixed user and group IDs of zero and standard permissions (files rw-r--r--, directories rwxr-xr-x).
  • It is not currently possible to change file ownership or permissions.
  • It is not currently possible to create symbolic links, FIFOs (named pipes), sockets or device nodes.
  • Hard links to files work in the expected way. Hard links to directories are forbidden.
  • It is not possible to unlink the '.', '..' or 'lost+found' special directories.
[Note]Note

If you will be sharing a Yaffs filesystem between eCos and some other operating system, you are advised to carefully check the other system's definitions of mode (permission) bits and whether any translation may be required.

51.5. Memory requirements

The amount of RAM required by Yaffs to hold its in-memory data structures grows with the number of objects (files and directories) in your filesystem. You are recommended to test your application thoroughly to ensure that sufficient memory exists for Yaffs to operate with the most complicated filesystem it is likely to encounter.

If you wish to estimate your RAM usage, the Yaffs author provides the following calculation:

  • The partition itself requires a yaffs_DeviceStruct of 3608 bytes.
  • If CYGNUM_FS_YAFFS_SHORTOP_CACHES is enabled, each is the size of a NAND page plus 28 bytes.
  • Every object (file, directory or hardlink) in the filesystem takes a yaffs_Object struct, which is 124 bytes.
  • Every page of every file requires a Tnode entry, but they are always allocated in groups of 16 at a time.

    • The size of a single Tnode entry is the number of bits required to number all the pages in the NAND partition Yaffs is using, numbering from one; this is rounded up to a multiple of 2, and has an absolute minimum of sixteen bits.
    • For example: on a partition with 65536 pages, seventeen bits are required for the numbering, which round up to 18. Therefore each file takes 288 bits (36 bytes) per group of Tnodes, and one group of Tnodes will cover up to sixteen pages of data.
[Note]Note

Actual memory consumption will be slightly higher than suggested by the above. This arises from the tree structure holding the Tnodes, overheads from the heap itself, and so on.

Yaffs calls the standard malloc function to allocate memory and free to release it. Normally, the eCos heap occupies all spare RAM not needed for the program, its static data or the stacks. Therefore, most applications will not need to do anything special beyond ensuring there is enough spare RAM available on the platform.

[Tip]Tip

If you wish to experiment with restricted-size heaps to determine much memory your application actually uses under Yaffs, you may find the option CYGSEM_MEMALLOC_INVOKE_OUT_OF_MEMORY of use.

51.5.1. Worked example

Consider a Yaffs filesystem hosted by a NAND partition with 65536 pages, each of size 2k, using the default setting of ten short-op caches. On this filesystem we shall store 10000 files each of size 10240 bytes, hence each requires a single group of Tnodes.

Table 51.1. Yaffs RAM use worked example

ConsumerRAM used (bytes)
Total 1,624,368
yaffs_DeviceStruct 3,608
Short-op caches @ 10 x (2048+28) 20,760
yaffs_Objects @ 10,000 x 124 1,240,000
Tnode groups @ 10,000 x 36 (see above) 360,000

This example is a close match to the actual consumption measured by eCosCentric during testing. (The measured consumption as reported by mallinfo was 1,681,112 bytes, which includes the heap's own overheads.)

51.6. Testing

Yaffs is supplied with a number of test programs, some of which are have been adapted from tests for other filesystems in eCos.

fops

This was the first basic test created for the port of the filesystem. It is believed to exercise all of the code paths (filesystem operations, file operations and directory operations) within the eCos-Yaffs adaptation layer.

This test was originally intended to run on a synthetic NAND filesystem. On real NAND chips, it deliberately omits the more stressful routines to avoid undue wear on the hardware.

yaffs1

A number of filesystem edge-case semantic tests, including file and directory creation and deletion, invalid open and rename operations, and removing nonexistent files and directories.

yaffs2

Concurrent multi-threaded filesystem access and consistency checks.

yaffs4

Semantic and edge-case testing - like yaffs1 - but with long file names.

yaffs5

Tests that file reading and writing works over reasonably large files (up to 1Mbyte) with different I/O chunk sizes. Some operation timings are collected and reported, as is the data rate on large files.

yaffs6

Semantic and edge-case testing - like yaffs1 - but with Cyrillic filenames in order to test UTF-8 correctness.

mounttime

A simple benchmark which repeatedly mounts and unmounts the filesystem and measures how long this takes. You can optionally use the mkfiles routine - also present in the tests directory - to create many short files so you can test performance on a loaded filesystem.

hammer [*]

A stress test designed to shake out corner cases. Repeatedly creates many files of varying sizes from multiple threads until the filesystem fills up, then verifies their contents and removes them. From time to time, all threads pause and the filesystem is unmounted and remounted.

This test is particularly useful when combined with the bad block injection functionality provided by the synthetic NAND device. It has been used in this way by eCosCentric to thoroughly test this package's stability under error conditions.



[*] This test runs forever, until interrupted.