legacy/libisofs/branches/thomas/doc/Tutorial

===============================================================================
                      LIBISOFS DEVELOPMENT TUTORIAL
===============================================================================

Creation date: 2008-Jan-27
Author: Vreixo Formoso
_______________________________________________________________________________

This is a little tutorial of how to use libisofs library for application
development.

Contents:
---------

1. Introduction
   1.1 Library initialization
   1.2 Image context
   1.3 Error reporting
2. Creating an image
   2.1 Image tree manipulation
   2.2 Set the write options
   2.3 Obtaining a burn_source
3. Image growing and modification
   3.1 Growing vs Modification
   3.2 Image import
   3.3 Generating a new image
4. Bootable images
5. Advanced features


-------------------------------------------------------------------------------
1. Introduction
-------------------------------------------------------------------------------

[TODO some lines about refcounts]

-------------------------------------------------------------------------------
1.1. Library initialization

Before any usage of the library, you have to call

	iso_init()

in the same way, when you have finished using the library, you should call

	iso_finish()

to free all resources reserved by the library.

-------------------------------------------------------------------------------
1.2. Image context

Libisofs is image-oriented, the core of libisofs usage is the IsoImage object.
Thus, the first you need to do is to get your own IsoImage object:

	IsoImage *my_image;
	iso_image_new("NEW DISC", &my_image);

An IsoImage is a context for image creation. It holds the files that will be 
added to image, other related information and several options to customize
the behavior of libisofs when working with such Image. i.e., an IsoImage is
a context for libisofs operations. As such, you can work with several image
contexts at a time. 

-------------------------------------------------------------------------------
1.3. Error reporting

In libisofs error reporting is done in two ways: with the return value of
the functions and with the message queue. 

Error codes are negative numbers, defined in "libisofs.h" header. An
error code is associated with a given severity, either "DEBUG", "UPDATE",
"NOTE", "HINT", "WARNING", "SORRY", "FAILURE" and "FATAL". For the meaning
of each severity take a look at private header "libiso_msgs.h". Errors 
reported by function return value are always "FAILURE" or "FATAL". Other kind
of errors are only reported with the message queue. You can get the severity
of any error message with iso_error_get_severity() function.

First of all, most libisofs functions return an integer. If such integer is 
a negative number, it means the function has returned an error. The error code
and its severity is encoded in the return value (take a look at error codes in 
libisofs.h header).

Additionally, libisofs reports most of its errors in a message queue. Error
messages on that queue can be printed directly to stderr or programmatically
retrieved. First of all, you should set the severity threshold over which an
error is printed or enqueued, with function:

	iso_set_msgs_severities()

Errors enqueued can be retrieved with function:

	iso_obtain_msgs()
	
Together with the code error, a text message and its severity, this function
also returns the image id. This is an identifier that uniquely identifies a
given image context. You can get the identifier of each IsoImage with the

	iso_image_get_msg_id()

and that way distinguish what image has issued the message.


-------------------------------------------------------------------------------
2. Creating an Image
-------------------------------------------------------------------------------

An image is built from a set of files that you want to store together in an 
ISO-9660 volume. We call the "iso tree" to the file hierarchy that will be
written to image. The image context, IsoImage, holds that tree, together with
configuration options and other properties of the image, that provide info
about the volume (such as the identifier, author, etc...).

All configuration options and volume properties are set by its corresponding
setters (iso_image_set_volset_id(), iso_image_set_publisher_id()...)

To create an image, you have to follow the following steps:

* Obtain the image context.
    See "1.2 Image context" for details of how to obtain the IsoImage.
* Set the desired properties
* Prepare the iso tree with the files you want to add to image.
    See "2.1 Image tree manipulation" for details
* Select the options for image generation.
    See "2.2 Set the write options"
* Get the burn_source used to actually write the image.


-------------------------------------------------------------------------------
2.1 Image tree manipulation

libisofs maintains in memory a file tree (usually called the iso tree), that 
represents the files and directories that will be written later to image. You
are allowed to make whatever changes you want to that tree, just like you do
to any "real" filesystem, before actually write it to image.

Unlike other ISO-9660 mastering tools, you have full control over the file
hierarchy that will be written to image, via the libisofs API. You can add
new files, create any file in image, change its name, attributes, etc The iso
tree behaves just like any other POSIX filesystem. 

The root of the iso tree is created automatically when the IsoImage is 
allocated, and you can't replace it. To get a reference to it you can use the
function:

	iso_image_get_root()

* Iso tree objects

Each file in the image or iso tree is represented by an IsoNode instance. In 
the same way a POSIX filesystem has several file types (regular files, 
directories, symlinks...), the IsoNode has several subtypes:

                    IsoNode
                       |
        ---------------------------------
        |         |         |           |
      IsoDir   IsoFile  IsoSymlink  IsoSpecial

where

 - IsoDir represents a directory
 - IsoFile represents a regular file
 - IsoSymlink represents a symbolic linke
 - IsoSpecial represents any other POSIX file, i.e. block and character 
   devices, FIFOs, sockets.

You can obtain the concrete type of an IsoNode with the iso_node_get_type()
function.

Many libisofs functions take or return an IsoNode. Many others, however,
require an specific type. You can safety cast any subtype to an IsoNode
object. In the same way, after ensuring you are dealing with the correct
subtype, you can downcast a given IsoNode to the specific subtype.

	IsoDir *dir;
	IsoNode *node;

	node = (IsoNode*) dir;

	if (iso_node_get_type(node) == LIBISO_DIR) {
	    dir = (IsoDir*) node;
	    ...
	}

or with the provided macros:

	IsoDir *dir;
	IsoNode *node;

	node = ISO_NODE(dir);

	if (ISO_NODE_IS_DIR(node)) {
	    dir = ISO_DIR(node);
	    ...
	}

* Adding files to the image

Files can be added to the image or iso tree either as new files or as files
from the filesystem.

In the first case, files are created directly on the image. They do not
correspond to any file in the filesystem. Provided functions are: 

	- iso_tree_add_new_dir()
	- iso_tree_add_new_symlink()
	- iso_tree_add_new_special()

On the other side, you can add local files to the image, either with the

	iso_tree_add_node()

or with

	iso_tree_add_dir_rec().

The first is intended to add a single file, while the last can be used to add,
recursively, a full directory (see below for details).

It is important to note that libisofs doesn't store any kind of link between
the IsoNode and the filesystem file it was created from. The above functions
just initialize a newly created IsoNode with the attributes of a given file in
the filesystem. After that, you can move the original file, change its 
attributes or even delete it. The IsoNode in the image tree remains with the
original attributes. One exception to this rule are the contents of a regular
file. Libisofs does not make any copy of those contents until they're actually
written to image. Thus, you shouldn't modify, move or delete regular files
after adding them to the IsoImage.


* Recursive directory addition.

One common use case is to add a local directory to the image. While this can
be done with iso_tree_add_node(), handling the addition of directory children
in the application, libisofs provides a function suitable for this case:

	iso_tree_add_dir_rec()
	
that takes care of adding all files inside a directory, recursing on directory
children. By default, this function adds all children. However, it is usual 
that you don't want really this. For example, you may want to exclude some
kind of files (backup files, application sockets,...). Libisofs provides
several functions to customize the behavior of that function:

	- iso_tree_set_follow_symlinks()
	- iso_tree_set_ignore_hidden()
	- iso_tree_set_ignore_special()
	- iso_tree_add_exclude()

* Operations on iso tree

[TODO briefly explain how to add node, change attributes, ...]

* Replace mode

[TODO]

-------------------------------------------------------------------------------
2.2 Set the write options

Once you have prepared the iso tree, it is time to select the options for the
image writing. 

These options affect the characteristics of the filesystem to create in the 
image, but also can control how libisofs generates the image.

First of all you have to get an instance of IsoWriteOpts, with the function

	iso_write_opts_new()

The several options available can be classified in:

- Extensions to add to the ISO-9660 image:

	iso_write_opts_set_rockridge()
	iso_write_opts_set_joliet()
	iso_write_opts_set_iso1999()
	
RockRidge is highly recommended, in fact you should use it in all image. Joliet
is needed if you want to use your images in Windows system. Nowadays,
ISO-9660:1999 is no much useful, so in most cases you don't want such 
extension.

- ISO-9660 options:

	iso_write_opts_set_iso_level()
	iso_write_opts_set_omit_version_numbers()
	iso_write_opts_set_allow_deep_paths()
	iso_write_opts_set_allow_longer_paths()
	iso_write_opts_set_max_37_char_filenames()
	iso_write_opts_set_no_force_dots()
	iso_write_opts_set_allow_lowercase()
	iso_write_opts_set_allow_full_ascii()

These control the options for the ISO-9660 filesystem. In most cases you won't
care about them, as it is the RockRidge or Joliet extensions what determine the
properties of the files once the image is mounted.

- File attributes options

	iso_write_opts_set_replace_mode()
	iso_write_opts_set_default_dir_mode()
	iso_write_opts_set_default_file_mode()
	iso_write_opts_set_default_uid()
	iso_write_opts_set_default_gid()
	iso_write_opts_set_replace_timestamps()
	iso_write_opts_set_default_timestamp()
	iso_write_opts_set_always_gmt()
	
They allow to set default attributes for files in image, despite of the real
attributes of the file on the local filesystem.

-------------------------------------------------------------------------------
2.3 Obtaining a burn_source

Finally, you get the burn_source used to write the image with the function:

	iso_image_create_burn_source()
	
The returned burn_source is suitable for using with libburn, to directly burn
the image to a disc. Alternatively, you can use burn_source read() to get
the image contents (for example, to write them to a file, pipe...).

Before creating the burn_source, libisofs computes the size of the image, so
the get_size() function of the burn_source always returns the final image 
size. It also starts a writing thread. All the operations needed to generate
the image are done by this thread, including read the original files contents.
The image is writing to a FIFO buffer, from which the burn_source will read.
The size of the buffer can be set in advanced with a property of the
IsoWriteOpts struct:

	iso_write_opts_set_fifo_size()

You can get the state of the buffer in any moment, with the function:

	iso_ring_buffer_get_status()
	
You can also cancel the writer thread at any time, with the cancel() function
of the burn_source.


-------------------------------------------------------------------------------
3. Image growing and modification
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
3.1 Growing vs Modification

Libisofs is not restricted only to create new images. It can also be used to
modify existing images. It supports two kind of image modifications, that we
have called image growing and image modification:

Image modification consists in generating a new image, based on the contents
of an existing image. In this mode, libisofs takes an image, the users modifies
its contents (adding new files, removing files, changing their names...), and
finally libisofs generates a completely new image.

On the other side, image growing is similar, with the difference that the new
image is dependent on the other, i.e., it refers to files of the other image.
Thus, it can't be mounted without the old image. The purpose of this kind of
images is to increment or add files to a multisession disc. The new image only
contains the new files. Old files are just references to the old image blocks.

The advantage of the growing approach is that the generated image is smaller,
as only the new files are written. This mode is suitable when you just want to
add some files to a very big image, or when dealing with write-once media, such
as CD-R. Both the time and space needed for the modification is much less than
with normal image modify.

The main problem of growing is that the new image needs to be recorded together 
with the old image, in order to be mountable. The total size of the image 
(old + new) is bigger (even much bigger) than a completely new image. So, if 
you plan to distribute an image on Internet, or burn it to a disc, generate a
completely new image is usually a better alternative. 

To be able to mount a grown image, the OS needs to now you have appended new 
data to the original image. In multisession media (such as CD-R), the new data
is appended as a new session, so the OS can identify this and mount the image
propertly. However, when dealing with non-multisession media (such as DVD+RW)
or plain .iso files, the new data is just appended at the end of the old image,
and the OS has no way to know that the appended data is in fact a "new 
session". The method introduced by Andy Polyakov in growisofs can be used in
those cases. It consists in overwrite the volume descriptors of the old image
with a new ones that refer to the newly appended contents.

-------------------------------------------------------------------------------
3.2 Image import

The first thing you need to do in order to modify or grow an image is to import
it, with the function:

	iso_image_import()
	
It takes several arguments. 

First, the image context, an IsoImage previously obtained with iso_image_new().
In most cases you will want to use an empty image. However, if you have already
added files to the image, they will be removed and replaced with the contents
of the image being imported.

The second parameter is an IsoDataSource instance. It abstracts the original 
image, and it is used by libisofs to access its contents. You are free to 
implement your own data source to access image contents. However, libisofs has
a simple implementation suitable for reading images on the local filesystem,
that can be used for import both .iso files and inserted media, via the block
device and POSIX functions. You can get it with 

	iso_data_source_new_from_file()
	
The third parameter of iso_image_import() is a pointer to an IsoReadOpts 
struct. It holds the options for image reading. You get it with:

	iso_read_opts_new()
	
and after calling iso_image_import() you should free it with

	iso_read_opts_free()
	
Some options are related to select what extensions to read. Default options
are suitable for most users.

	iso_read_opts_set_no_rockridge()
	iso_read_opts_set_no_joliet()
	iso_read_opts_set_no_iso1999()
	iso_read_opts_set_preferjoliet()
	
If RockRidge extensions are not present, many files attributes can't be 
obtained. In those cases libisofs uses default values. You have options to
configure what default values to use.

	iso_read_opts_set_default_uid()
	iso_read_opts_set_default_gid()
	iso_read_opts_set_default_permissions()
	
If the original image has been created in another system with a different
charset, you may want to use:

	iso_read_opts_set_input_charset()
	
to specify the encoding of the file names on image.

Finally, to import multisession images, you should tell libisofs that it must
read the last session. For that, you must set the block where the last session
starts:

	iso_read_opts_set_start_block()

The last parameter for iso_image_import(), optional, is a pointer that will
be filled with a library-allocated IsoReadImageFeatures, that lets you access
some information about the image: size, extensions used,...

[TODO: explain that iso_image_import uses dir rec options]

-------------------------------------------------------------------------------
3.3 Generating a new image

After importing the image, the old image tree gets loaded. You are free to 
make any changes to it: add new files, remove files, change names or 
attributes... Refer to "2.1 Image tree manipulation" for details.

When it is ready, you can now create the new image. The process is the same as
explained in "2.2 Set the write options" and "2.3 Obtaining a burn_source".
However, there are some write options that should be taken into account.

First of all, you must select whether you want to grow or modify the image 
(read "3.1 Growing vs Modification" for details). You must call

	iso_write_opts_set_appendable()
	
An appendable image leads to image growing, and a non-appendable image leads
to a completelly new image (modification). An appendable image will be appended
after the old image (in a new session, for example). Thus, in those cases, the 
first block of the image is not 0. You should set the correct lba of the first
block with:

	iso_write_opts_set_ms_block()

That is usually the "Next Writable Address" on a multisession media, and a 
value slightly greater than the old image size on .iso files or DVD+RW media.
You can obtain the old image size with the iso_read_image_features_get_size()
function.

In this last case (i.e., on a non multisession media), you will need to
overwrite the volume descriptors of the old image with the new ones. To do 
this you need:

- Allocate a buffer of at least 64 KiBs.
- Initialize it with the first 64 KiBs of the original image.
- Pass the buffer to libisofs with the iso_write_opts_set_overwrite_buf()
  option.
- After appending the new image, you have to overwrite the first 64 KiBs of 
  the original image with the new content of the buffer.


-------------------------------------------------------------------------------
4. Bootable images
-------------------------------------------------------------------------------



-------------------------------------------------------------------------------
5. Advanced features
-------------------------------------------------------------------------------