Berkeley DB SQL BFILE Extension

1. Introduction
===============
BFILE datatype enables to access binary file that are stored in file systems
outside the database. A BFILE column or attribute stores a BFILE locater, which
serves as a pointer to the binary file. The locater maintains the directory
alias and the filename. You can change the path of BFILE without affecting the
base table by using BFILENAME function.  BFILE is somehow alike BLOB, but it
does not participate in transactions and are not recoverable. Rather, the
underlying operating system provides file integrity and durability.

Applications can load BFILE library libbfile_ext.so if needed.


2. SQL Object and Function
==========================
2.1 DIRECTORY Object

All DIRECTORY objects store in a special table BFILE_DIRECTORY. When loading
BFILE extension, BFILE_DIRECTORY will be created if it does not exist.  It's
strongly not recommend users to manipulate BFILE_DIRECTORY directly.

2.2 SQL Function
- BFILE_CREATE_DIRECTORY(directory, path)
  Create a directory object as path. If directory already exist, set error
massage as “Directory already exist”.

- BFILE_REPLACE_DIRECTORY(directory, path)
  Replace the directory object as path. If directory does not exist, set error
massage as “Directory does not exist”.

- BFILE_DROP_DIRECTORY(directory)
  Drop the directory object. If directory does not exist, set error massage as
“Directory does not exist”.

- BFILE_NAME(directory, filename)
  Returns BFILE locater.

- BFILE_FULLPATH(column)
  Returns the full path.

- BFILE_OPEN(column)
  Extracts directory and filename from BFILE locater and open that file. On
success, BFILE handle is returned. Otherwise 0 is returned.

- BFILE_READ(BFILE handle, amt, offset)
  Reads at most amt data from BFILE handle starting at offset. On success, Data
is returned. Otherwise 0 is returned to indicate no more valid data is
available.

- BFILE_CLOSE(BFILE handle)
  Closes the BFILE handle.

- BFILE_SIZE(column)
  Gets the size of BFILE. On success, the size is returned. Otherwise -1 is
returned.


3. C/C++ API Reference
======================
3.1 Objects
typedef sturct sqlite3_bfile sqlite3_bfile;

3.2 Functions
3.2.1 Result Values From A Query For BFILE
  int sqlite3_column_bfile(sqlite3_stmt *pStmt, int iCol, sqlite3_bfile **ppBfile);

This routines form the "result set query" interface for BFILE.

This routines return information about a single column with BFILE type of the
current result row of a query. The first argument is a pointer to the prepared
statement that is being evaluated (the sqlite3_stmt* that was returned from
sqlite3_prepare_v2() or one of its variants) and the second argument is the
index of the column for which information should be returned. The leftmost
column of the result set has the index 0. The number of columns in the result
can be determined using sqlite3_column_count().

If the SQL statement does not currently point to a valid row, or if the column
index is out of range, the result is undefined. These routines may only be
called when the most recent call to sqlite3_step() has returned SQLITE_ROW and
neither sqlite3_reset() nor sqlite3_finalize() have been called subsequently. If
this routine is called after sqlite3_reset() or sqlite3_finalize() or after
sqlite3_step() has returned something other than SQLITE_ROW, the result is
undefined. If sqlite3_step() or sqlite3_reset() or sqlite3_finalize() are called
from a different thread while this routines is pending, then the result is
undefined.  The pointers *ppBfile is valid until sqlite3_step() or
sqlite3_reset() or sqlite3_finalize() is called. The memory space used to hold
BFILE handle is freed by sqlite3_bfile_final(). Do not pass the pointers
returned into sqlite3_free().

On success, SQLITE_OK is returned and the new BFILE handle is written to
*ppBFile. Otherwise SQLITE_ERROR is returned.

3.2.2 Open A BFILE For Incremental Read
  int sqlite3_bfile_open(sqlite3_bfile *pBfile);

This interfaces opens a handle to the BFILE for read access.

On success, SQLITE_OK is returned. Otherwise SQLITE_ERROR is returned. Note that
the *pBfile variable is always initialized in a way that makes it safe to invoke
sqlite3_bfile_close() on *pBfile regardless of the success or failure of this
routine.  Use the sqlite3_bfile_size() interface to determine the size of the
BFILE.

To avoid a resource leak, every open BFILE handle should eventually be released
by a call to sqlite3_bfile_close().

3.2.3 Close A BFILE Handle
  int sqlite3_bfile_close(sqlite3_bfile *pBfile);

Closes an open BFILE handle.

On success, SQLITE_OK is returned. Otherwise SQLITE_ERROR is returned.

The BFILE is closed unconditionally. Even if this routine returns an error code,
the BFILE is still closed.

Calling this routine with a null pointer (which as would be returned by failed
call to sqlite3_column_bfile()) is a harmless no-op.

3.2.4 Check If BFILE Is Opened
  int sqlite3_bfile_is_open(sqlite3_bfile *pBfile, int *open);

This routine returns information about whether a BFILE handle is opened. The
first argument is a pointer to the BFILE handle and the second argument is a
pointer of integer for which information should be returned.

On success, SQLITE_OK is returned and *open is 1 if the BFILE handle has been
opened, or 0 for not opened. . Otherwise SQLITE_ERROR is returned.

3.2.5 Read Data From A BFILE Incrementally
  int sqlite3_bfile_read(sqlite3_bfile *pBfile, void *z, int nSize, int iOffset, int *nRead);

This function is used to read data from an opened BFILE handle into a
caller-supplied buffer. nSize bytes of data are copied into buffer z from the
open BFILE, starting at offset iOffset.

The size of the BFILE can be determined using the sqlite3_bfile_size()
interface.

On success, SQLITE_OK is returned and *nRead bytes of file data is written to
buffer z. Otherwise, an SQLITE_ERROR is returned.

This routine only works on a BFILE handle which has been created by a prior
successful call to sqlite3_bfile_open() and which has not been closed by
sqlite3_bfile_close(). Passing any other pointer in to this routine results in
undefined and probably undesirable behavior.

3.2.6 Check If A File Exists
  int sqlite3_bfile_file_exists(sqlite3_bfile *pBfile, int *exist);

This routines return information about whether a BFILE exists. The first
argument is a pointer to the BFILE handle and the second argument is a pointer
of integer for which information should be returned.

On success, SQLITE_OK is returned and *exist is 1 if the BFILE exist, or 0 for
not exist. Otherwise SQLITE_ERROR is returned.

3.2.7 Get The Size Of A BFILE
  int sqlite3_bfile_size(sqlite3_bfile *pBfile, off_t *size);

Get the size in bytes of the BFILE.

This routine only works on a BFILE handle which has been created by a prior
successful call to sqlite3_column_bfile() and which has not been finalized by
sqlite3_bfile_final(). Passing any other pointer in to this routine results in
undefined and probably undesirable behavior.

On success, SQLITE_OK is returned and *size is the size of the BFILE. Otherwise
SQLITE_ERROR is returned.

3.2.8 Finalize A BFILE
  int sqlite3_bfile_final(sqlite3_bfile *pBfile);

The sqlite3_bfile_final() function is called to free the BFILE handle. 

On success, SQLITE_OK is returned. Otherwise SQLITE_ERROR is returned.


4. Using The BFILE Module
=========================
Here's a simple example to show how to use BFILE.

Open Database
	rc = sqlite3_open("example.db", &db);
	…
	sqlite3_enable_load_extension(db, 1);
	rc = sqlite3_load_extension(db, "libbfile_ext.so", NULL, &zErrMsg);
Create Directory
	select BFILE_CREATE_DIRECTORY('IMG', './temp');
Create A Table With BFILE Type Column
	create table address_book(name TEXT, address TEXT, photo BFILE);
Insert
	insert into address_book VALUES('Robin', 'CA', BFILE_NAME('IMG','Robin.jpg')); 
	insert into address_book VALUES('Rachel', 'NJ', BFILE_NAME('IMG','Racle.jpg'));
Update BFILE
	update address_book set photo = BFILE_NAME('IMG', 'Robin_new.jpg') 
		where name = 'Robin';
Query With SQL Function
	select name, BFILE_FULLPATH(photo) from address_book;
Query With BFILE APIs
	select name, photo from address_book;

	…
	/* for each row */
	sqlite3_column_bfile(pStmt, 1, &pBfile);
	…
	sqlite3_bfile_open(pBfile);
	…
	sqlite3_bfile_read(pBfile, pUserBuffer, nSize, iOffset, &nRead);
	…
	sqlite3_bfile_close(pBfile);
	…
	sqlite3_bfile_final(pBfile); /* finalize pBflie created by sqlite3_column_bfile */
Query With SQL Function
	Sql_bfile_open = “select name, BFILE_OPEN(photo) from address_book;”;
	Sql_bfile_read = “SELECT BFILE_READ(?, ?, ?);”;
	Sql_bfile_close = “SELECT BFILE_CLOSE(?);”;

	/* BFILE_OPEN */
	rc = sqlite3_prepare(db, sql_bfile_open, -1, &pStmt, 0);
	while (SQLITE_ROW == (rc = sqlite3_step(pStmt))) {
		bHdl = sqlite3_column_int(pStmt, 1);
		if ((void*)bHdl == NULL)  continue;
		
	offset = 0;
	/* BFILE_READ */
	rc2 = sqlite3_prepare(db, sql_bfile_read, -1, &pBfileStmt, 0);
	while (1) {
		rc2 = sqlite3_bind_int(pBfileStmt, 1, bHdl);
		rc2 = sqlite3_bind_int(pBfileStmt, 2, amount);
		rc2 = sqlite3_bind_int(pBfileStmt, 3, offset);
		rc2 = sqlite3_step(pBfileStmt);
		if (rc2 == SQLITE_ROW) {
			photo = (char*)sqlite3_column_blob(pBfileStmt, 0);
			len = sqlite3_column_bytes(pBfileStmt, 0);
			/* EOF */
			if (len == 0)  break;
		}

		sqlite3_reset(pBfileStmt);
		sqlite3_clear_bindings(pBfileStmt);
		offset += amount;
	}

	if (pBfileStmt != NULL) {
		sqlite3_finalize(pBfileStmt);
		pBfileStmt = NULL;
	}

	/* BFILE_CLOSE */
	rc2 = sqlite3_prepare(db, sql_bfile_close, -1, &pBfileStmt, 0);
		rc2 = sqlite3_bind_int(pBfileStmt, 1, bHdl);
		rc2 = sqlite3_step(pBfileStmt);
		sqlite3_finalize(pBfileStmt);
	}
Close Database
	sqlite3_close(db);

